What Information Do I Need To Gather To Allow GSC To Diagnose an HNAS Performance Problem?

A standard set of HNAS diagnostics taken after the event is usually insufficient to diagnose the cause of a HNAS performance problem.  If a HNAS system is having a performance problem the high-level process is the following:

• Understand what options are available for getting help with performance problems.
• Verify that the system in question isn't impacted by any of the common causes of performance problems.
• Performance data collection from the Hitachi storage array.
• Gather a Performance Information Report (PIR) from the HNAS on the impacted file system(s).
• Whilst the PIR is running, gather a short (30sec) packet capture from one of the impacted clients.
• Gather the additional required data.
• Answer the performance problem questionnaire to provide the context for the problem.

 NOTE: It is important that the performance data such as the PIR and packet capture gathered below are collected while the performance problem is happening. 

What help can be provided for the performance problem?

In reality, there is no such thing as a "performance problem," there are only "lack of capacity problems."  As such there are three possible approaches that can be taken to address these:

• Perform a "performance tuning" exercise
• Perform a "system sizing" exercise
• Request a product enhancement to the system so that it has increased capacity without additional hardware by making changes to hardware or software of the product.

• Performance tuning

A performance tuning exercise is something that can be led by Hitachi GSC.  There are three things to bear in mind before going down this path:

1. 1. The purpose of a performance tuning exercise is just to try and determine if there are any changes to the environment which could be made which will increase the capacity available from a given system.  Whether or not any such changes are considered feasible is a customer business decision.
   2. Since capacity is heavily dependent on specific workload, GSC are unable to advise whether any particular load is within the expected capacity range of any particular system configuration (both hardware and system settings/configuration.)
   3. As a result, GSC are unable to guarantee whether it is possible to achieve an acceptable level of performance for any particular workload on any particular hardware configuration.  (Or indeed whether any such configuration may even be possible.)

The way this process proceeds is as follows:

1. 1. Performance data for the HNAS and associated storage is captured that covers a time period when the perceived problem is occurring.
   2. Hitachi GSC evaluate this data and see whether any recommendations can be made for changes to the system which may allow additional load to be sustained on the existing hardware with improved performance.

The changes recommended may be:

• • Simple configuration changes to the file/block storage system (usually bringing the system into "best practice" state.)  It is rare however that these changes will have significant impact, and we would generally expect systems to be setup to best-practice at install time.
  • Changes to the way the file/block storage is used to try and achieve a more efficient use of the resources that are available.  These changes will usually require some data migration moving from the sub-optimal configuration to a hopefully more efficient usage.
  • Suggested changes to the client behavior that would make more efficient use of the available storage resources.

However, is not recommended to propose too many changes at once for the following reasons:

• • Some changes may actually cause the available capacity to decrease due to unanticipated workload related factors,
  • It becomes difficult to tell which changes were useful and which were counter-productive and need to be reversed.

As a result, a performance tuning exercise is usually iterative in nature:

• • Data is gathered and analyzed,
  • Suggested changes are proposed and applied,
  • New data is gathered and analyzed to see whether additional changes may be worthwhile.

The performance tuning loop finishes when:

1. 1. An acceptable level of performance is achieved,
   2. The customer no longer wishes to apply suggested changes,
   3. There are no further suggested changes.

In the event of 2 & 3 this may mean that a system sizing exercise or product enhancement request is then required.