General Troubleshooting Questions

1. What is the current impact?
2. What error(s) is the customer reporting?
3. When did this issue occur and is it ongoing?
4. What troubleshooting steps have been taken so far?
5. What is the currently installed version?

Potential Issue Areas

• Performance issues
• Security permissions issues
• Heap memory issues
• File (NDMP) replication problems
• HNAS resets / HNAS "boot loops"
• NFS/CIFS access issues
• File system down
• AV issues
• DM2C issues
• Gathering packet captures

Performance issues

If a customer is reporting a performance problem refer them to this knowledgebase article:

• What Information Do I Need to Gather to Allow GSC to Diagnose an HNAS Performance Problem

If they are reporting that a node is "hung" refer them to this knowledgebase article:

• What Information Do I Need to Gather to Allow GSC to Diagnose a Hung HNAS System

Security permissions issues

If a customer is reporting a problem with security permissions refer them to this knowledgebase article:

• What Information Do I Need to Gather to Allow GSC to Diagnose a HNAS Security Permissions Issue

Heap memory issues

Events/issues like:

• Failed to allocate panics ("start of "FailedToAllocate" panic"),
• Heap fragmentation ("No big free heap blocks left"),
• Heap related enforced limits:
  • Insufficient free heap:
    • "There is insufficient heap: dropping new connection from"
    • "There is insufficient heap: dropping additional connection request from"
  • Insufficient free heap in big sized heap blocks:
    • "There is insufficient heap: dropping new connection from"
    • "There is insufficient heap: dropping additional connection request from"

Refer to the knowledgebase article What Information Do I Need to Gather to Allow GSC to Diagnose Heap Issues on HNAS.  Once you have the appropriate diagnostics refer to the article How To Diagnose HNAS Heap Memory Issues to continue the diagnosis.

File (NDMP) replication problems

If a customer is reporting a problem with file (NDMP) replication refer them to this knowledgebase article:

• What Information Do I Need to Gather to Allow GSC to Diagnose HNAS File Replication Issues

HNAS resets / HNAS "boot loops"

If a customer (or Remote Ops (Hi-Track)) reports a HNAS reset then you usually just need HNAS managed server diagnostics from after the reset.  If the server has been restarted since the reset then the diagnostics for the reset will no-longer be in the last_dblog.txt.  In that case you will need to ask the customer to get historical logs.  For both diagnostics and historical logs refer to Hitachi NAS (HNAS) Platform Data Collection.

If the HNAS is continually resetting (aka it is stuck in a "boot loop.") refer the customer initially to:

• What Information Do I Need to Gather to Allow GSC to Diagnose a HNAS Boot Loop

Once you have the appropriate diagnostics refer to How To Diagnose HNAS Resets to proceed with the diagnosis.

Remember to check that the system wasn't just being deliberately rebooted.

NFS/CIFS Access

1. Which share/EVS is impacted?
2. Are all clients impacted or just one?
3. Is the EVS/Share pingable by hostname and/or ip address?
4. When did the share become inaccessible?
5. Are there any AV servers attached to the impacted EVS?
6. Any errors reported from the clients?
7. What errors is the HNAS reporting?
8. Which OS is the client(s) running? [e.g. SW version and patch level]
9. Try migrating the impacted EVS to the other node, does the connectivity issue follow the EVS or stick to one node?
10. Collect and upload diags.

File System Down

1. What is the current impact?
2. Which file system is currently down?
3. What time did the file system go offline?
4. Any recent changes in the environment (storage issues, power issues)?
5. Was the customer able to remount the file system? If so what was the result?
6. What is the current state of the underlying storage array?
7. Is the file system full?
8. Any errors reported from the clients?
9. What errors is the HNAS reporting?
10. Please collect and upload diags

AV Server

1. What is the current impact to the HNAS/AV server?
2. Is CIFS access slow or impacted in some way?
3. What is the make and version of the AV server(s)?
4. How many AV servers have been dedicated to each EVS?
5. Are the AV servers shared between the EVS's or dedicated to unique EVS's?
6. Which EVS is impacted?
7. What is the status of the AV server (CPU maxed, hung)?
8. What errors are being seen on the HNAS and/or clients?
9. Are zipped/archived files only impacted or are all file types impacted?

DM2C

1. Which file system is impacted? If multiple, please provide a list.
2. What is the name of the impacted namespace data is being migrated to?
3. Any errors reported from the clients?
4. What errors is the HNAS reporting?
5. What is the current HNAS version?
6. What is the current HCP version?
7. Please collect HCP logs and HNAS diags
8. Which DM2C jobs are impacted? Only one or all?
9. Which HCP account is the HNAS using to interact with the impacted namespace?

Gathering packet captures

Refer the customer to the knowledgebase article:

• How to Collect Packet Captures for Troubleshooting HNAS Problems