How to Replace a HNAS Gateway Node in an HNAS Single Node System without vSMU access

Content

Objective

This procedure will guide the engineer in the replacement of an HNAS Gateway Single Node System.

Environment

Hitachi NAS Gateway Platform
- Hitachi NAS Platform 5300 (HNAS 5300)
- Hitachi NAS Platform 5200 (HNAS 5200)
- Hitachi NAS Platform 4100 (HNAS 4100)
- Hitachi NAS Platform 4080 (HNAS 4080)
- Hitachi NAS Platform 4060 (HNAS 4060)
- Hitachi NAS Platform 4040 (HNAS 4040)

Procedure

Default passwords are no longer documented here. Customers requiring the default passwords should open a GSC case.
Service Partners and Employees may consult Hitachi NAS Upgrade, Deployment, and Procedures Documentation.

Important: When ordering a replacement node for configuration off-site, order 1 of the appropriate Power Supplies for the node.

The Engineer is strongly encouraged to use a serial Putty console session to record all aspects of the system.

Software Tools required on your Laptop before replacement

putty.exe
- A free SSH/Telnet application.
- You will need to configure the new node IP addresses via CLI
WinSCP or pscp
- Used to transfer files between the HNAS system and the Laptop

Pre-requisites Before going On-Site

Obtain the Node Model and Firmware Code Version running on the current Cluster from a current diagnostic in the SR.
1. Have the current code version file available on your laptop.
2. Have the Factory-Reset code that matches the code running on the current Cluster if you must downgrade the replacement node.
Verify that the correct Node has been ordered
1. 4000
  1. SX325141-02 is the replacement for the 4100
  2. SX325140-02 is the replacement for the 4060/4080
  3. SX325143-01 is the replacement for the 4040
2. 5000
  1. SX325151-01 is the replacement for the 5200/5300
Download the required HNAS code version from TISC or Support Connect
Extract the nas-xxxx.xx.tar file from the code .iso to a directory on the laptop
Extract the ssc.exe utility to the same directory as the code on the Laptop
- This utility is located in the latest HNAS GA Code .iso file in the Win64tools directory.

IMPORTANT!!! Obtain new License Key for Replacement Node

Gather Information from Replacement Node

Model, Serial Number, and MacID information are on a label on the Lower Right Front Side of the Chassis.

Request a New License Keys for the Replacement Node

Email TBKeys, cc. your Geo PM requesting:
The License Keys are transfered from Defective Node MacID to the Replcement Node MacID.
A new Permanent License Key for the Replacement Node MacID.
Information to include in both requests
1. Defective Node Model, Serial Number, Firmware Version, and License Key information
2. Replacement Node Model and Serial Number information.

Important Notes:
1) If the Defective Node is a Model 4080, then request a license for the HNAS Model 4080.
2) if the Defective Node is a Model 5300, then request a license for the HNAS Model 5300.

Configuration of Replacement Node (On-Site OR Off-Site)

Note: If configuring Node On-Site, Do NOT plug any Fiber Channel Cables into the node until after the node is upgraded.

Power up the Replacement Node

Plug in a power cord and power up the HNAS Node.
NOTE: Units will NOT boot into Bali until the nas-preconfig script has been run to configure the nodes indicated by a fast-blinking middle led on the back

Configuration of IP Addresses on the Node
Note: Use only one of the two following sections to configure the IP Addresses on the node.

Customer Management Network ONLY

Note: If the system is configured for a Private Management Network, DO NOT USE THIS SECTION.
Go to the Private Management Network ONLY section.

Open a serial PuTTY console session to the Replacement Node.
1. Plug the serial connection into the console port of the node
2. Start a serial PuTTY session (Serial settings (115200, 8, n, 1, n)
3. Set PuTTY to capture all readable output
When prompted, login as Username: root; Enter the current password when prompted.
From the command prompt, type nas-preconfig
NOTE: This script configures the server’s basic network settings and can only be run once.
1. Enter the Admin Service Private (eth1) IP address: <temporary IP address supplied by the customer>
  Note: This must be different than the existing Cluster admin IP address identified in the Gathering Information Section Step 2.c.i.
2. Enter the Admin Service Private (eth1) Netmask: <supplied by the customer>
3. Enter the Optional Admin Service Public (eth0) IP address: <leave blank> or <customer IP address> (if an IP address was identified on eth0 in the Gathering Information Section Step 2.c.ii)
  1. If an IP address was entered in the previous step, then you will be prompted for a Netmask: < Customer Netmask> (This is the customer netmask identified on eth0 in the Gathering Information Section Step 2.c.ii)
4. Enter the Optional Physical Node (eth1) IP address: <IP address of the defective node> (identified in the Gathering Information Section Step 2.d.i)
5. Enter the Physical Node (eth1) Netmask: <Netmask of the defective node> identified in the Gathering Information Section Step 2.d.i
6. Enter the Gateway: <customer network gateway> (identified in the Gathering Information Section Step 2.e.i)
7. Enter the Domain name (without the hostname): Enter the customer network gateway identified in the Gathering Information Section Step 2.f.i
8. Enter the Hostname (without the Domain name): <identified in the Gathering Information Section Step 2.b.i>
9. The script then displays a summary of the above entries and prompts to accept or decline the settings. If any entries are incorrect, answer n, and the script will return to the beginning.
10. Are the above settings correct? [y/n] Y
  1. There will be a message that the mfb.ini has been created
11. Node Reconfiguration
  1. The Node will automatically apply the settings from the mfb.ini file, start the PAPI process and start Bali.
    Note: Wait for the node reconfiguration to complete. This may take 10-20 minutes.
    When the fast-blinking middle led on the back turns to a slow blink, the Node has completed booting into Bali.
  2. On an HNAS 4000 ONLY running pre-14.x code, a reboot may be required if the Node does not boot into Bali.
    1. Enter reboot (if the unit does not automatically boot into Bali)
  3. HNAS 5000: Do NOT reboot the Node under any circumstances.
12. Once the Node has finished the reconfiguration
  1. Logout as root with exit
  2. Log in as manager; Enter the current password when prompted
  3. Confirm that the Bali login is successful
  4. Type ifconfig to verify that the IP addresses were successfully applied
13. Important if the nodes do not boot into Bali
  1. Open a Serial Putty session to the node and enable logging (with all printable output) to a file.
  2. Log in to the node with Username root; Enter the current password when prompted.
  3. Type tail -f /var/log/syslog (allow the output to roll for about 3-5 minutes)
  4. Type tail -f /var/opt/mercury-main/log/dblog (all the output to roll for about 3-5 minutes)
  5. Type getdiagnostic.linux (when complete, the command output will display the name of the diagnostic file)
  6. Prepare a USB stick by formatting it on the laptop for FAT-32; then eject it from the laptop.
  7. Insert the USB stick into the HNAS 5000 USB port (once connected, the device name may be displayed on the screen)
  8. Enter tail -f /var/log/messages to determine the device number from the entries in the log (look for /dev/sdc or /dev/sdc1)
  9. Enter mkdir /tmp/usb
  10. Enter mount /dev/sdc1 /tmp/usb (if this fails then use /dev/sdc)
  11. Enter cp <filename from getdiagnostic.linux> /tmp/usb
  12. Enter ls -l /tmp/usb (to verify that the file was successfully copied to the USB stick
  13. Enter umount /tmp/usb (to unmount the USB Stick)
  14. Insert the USB stick in the laptop and copy the file to the location of the Putty log file.
  15. If not done, open a case and upload the Putty log file and getdiagnostics.linux file to the Case using TUF.
  16. Escalate the Case to GSC-ES-HNAS L2 for analysis
  17. Do NOT manually reboot or power-cycle the node without confirmation from the GSC/Sustaining Engineering.
Change default passwords (If they have been changed from default on the other nodes in the cluster)
1. From the serial console session on the Replacement node
2. Type exit to return to the login prompt
3. When prompted, Log in as root; Enter the current password when prompted.
4. At the prompt, type the following commands:
  1. passwd <Enter>
    1. Enter the new password from the customer for the root user (entered twice - ignore the warning about a dictionary word)
  2. passwd manager <Enter>
    1. Enter the new password from the customer for the manager user (entered twice - ignore the warning about a dictionary word)
  3. passwd supervisor <Enter>
    1. Enter the new password from the customer for the supervisor user (entered twice - ignore the warning about a dictionary word)
5. Type exit to return to the login prompt
6. Log in as manager; Enter the current password when prompted
Verify IP Addresses of the Replacement Node
1. Type ifconfig
  1. Verify that the Cluster and Admin IP addresses are set correctly on eth1

Private Management Network ONLY

Note: If the system is configured for a Customer Management Network, DO NOT USE THIS SECTION.
Go to the Customer Management Network ONLY section.

Open a serial PuTTY console session to the Replacement Node.
1. Plug the serial connection into the console port of the Node
2. Start a serial putty session (Serial settings (115200, 8, n, 1, n)
3. Set putty to capture all readable output
When prompted, log in as root; Enter the current password when prompted.
From the command prompt, type nas-preconfig
NOTE: This script configures the server’s basic network settings and can only be run once.
1. Enter the Admin Service Private (eth1) IP address: 192.0.2.3 (if this conflicts with the existing Cluster admin IP address, the use any address in the 192.0.2.4-9 range)
  Note: This must be different than the existing Cluster admin IP address identified in the Gathering Information Section Step 2.c.i.
2. Enter the Admin Service Private (eth1) Netmask: 255.255.255.0
3. Enter the Optional Admin Service Public (eth0) IP address: <leave blank> or <customer IP address> (if an IP address was identified on eth0 in the Gathering Information Section Step 2.c.ii)
  1. If an IP address was entered in the previous step, then you will be prompted for a Netmask: < Customer Netmask> (This is the customer netmask identified on eth0 in the Gathering Information Section Step 2.c.ii)
4. Enter the Optional Physical Node (eth1) IP address: <IP address of the defective node> (identified in the Gathering Information Section Step 2.d.i)
5. Enter the Physical Node (eth1) Netmask: <Netmask of the defective node> identified in the Gathering Information Section Step 2.d.i
6. Enter the Gateway: <customer network gateway> (identified in the Gathering Information Section Step 2.e.i)
7. Enter the Domain name (without the hostname): Enter the customer network gateway identified in the Gathering Information Section Step 2.f.i
8. Enter the Hostname (without the Domain name): <identified in the Gathering Information Section Step 2.b.i>
9. The script then displays a summary of the above entries and prompts to accept or decline the settings. If any entries are incorrect, answer n, and the script will return to the beginning.
10. Are the above settings correct? [y/n] Y
11. There will be a message that the mfb.ini has been created
12. Node Reconfiguration
  1. The Node will automatically apply the settings from the mfb.ini file, start the PAPI process and start Bali.
    Note: Wait for the node reconfiguration to complete. This may take 10-20 minutes.
    When the fast-blinking middle led on the back turns to a slow blink, the Node has completed booting into Bali.
  2. On an HNAS 4000 ONLY running pre-14.x code, a reboot may be required if the Node does not boot into Bali.
    1. Enter reboot (if the unit does not automatically boot into Bali)
  3. HNAS 5000: Do NOT reboot the Node under any circumstances.
13. Once the Node has finished the reconfiguration
  1. Logout as root with exit
  2. Log in as manager; Enter the current password when prompted
  3. Confirm that the Bali login is successful
  4. Type ifconfig to verify that the IP addresses were successfully applied
14. Important if the nodes do not boot into Bali
  1. Open a Serial Putty session to the node and enable logging (with all printable output) to a file.
  2. Log in to the node with Username root; Enter the current password when prompted.
  3. Type tail -f /var/log/syslog (allow the output to roll for about 3-5 minutes)
  4. Type tail -f /var/opt/mercury-main/log/dblog (all the output to roll for about 3-5 minutes)
  5. Type getdiagnostic.linux (when complete, the command output will display the name of the diagnostic file)
  6. Prepare a USB stick by formatting it on the laptop for FAT-32; then eject it from the laptop.
  7. Insert the USB stick into the HNAS 5000 USB port (once connected, the device name may be displayed on the screen)
  8. Enter tail -f /var/log/messages to determine the device number from the entries in the log (look for /dev/sdc or /dev/sdc1)
  9. Enter mkdir /tmp/usb
  10. Enter mount /dev/sdc1 /tmp/usb (if this fails then use /dev/sdc)
  11. Enter cp <filename from getdiagnostic.linux> /tmp/usb
  12. Enter ls -l /tmp/usb (to verify that the file was successfully copied to the USB stick
  13. Enter umount /tmp/usb (to unmount the USB Stick)
  14. Insert the USB stick in the laptop and copy the file to the location of the Putty log file.
  15. If not done, open a case and upload the Putty log file and getdiagnostics.linux file to the Case using TUF.
  16. Escalate the Case to GSC-ES-HNAS L2 for analysis
  17. Do NOT manually reboot or power-cycle the node without confirmation from the GSC/Sustaining Engineering.
Change default passwords (If they have been changed from default on the other nodes in the cluster)
1. From the serial console session on the Replacement node
2. Type exit to return to the login prompt
3. When prompted, log in as root; Enter the current password when prompted.
4. At the prompt, type the following commands:
  1. passwd <Enter>
    1. Enter the new root password from the customer for the root user (entered twice - ignore the warning about a dictionary word)
  2. passwd manager <Enter>
    1. Enter the new manager password from the customer for the manager user (entered twice - ignore the warning about a dictionary word)
  3. passwd supervisor <Enter>
    1. Enter the new supervisor password from the customer for the supervisor user (entered twice - ignore the warning about a dictionary word)
5. Type exit to return to the login prompt
6. Log in as manager; Enter the current password when prompted.
Verify IP Addresses of the Replacement Node
1. Type ifconfig
  1. Verify that the Cluster and Admin IP addresses are set correctly on eth1

Obtain the Host Port WWNs for the Replacement Node

Note: If working off-site, provide the results of the command to the Customer, prior to your arrival, so enable them to add the Replacement Node to the Switch Zoning and Host Group Security if necessary, see the Customer Actions section below.

From the serial or KVM console session to the replacement node

Discover the WWN for the new Node

Type hport-wwn

pn all hport-wwn
Cluster node x:
Node
WWN : 50:03:01:70:00:09:AA:20

FC Host Port 1
WWN : 50:03:01:70:00:09:AA:11
FC Host Port 2
WWN : 50:03:01:70:00:09:AA:12
FC Host Port 3
WWN : 50:03:01:70:00:09:AA:13
FC Host Port 4
WWN : 50:03:01:70:00:09:AA:14

Customer Actions - Proceed with the replacement if working on-site

Fabric Connection - Switch Zoning

Note: this is normally performed by the Customer

Report the WWNs to the Customer to update the Switch Zoning
1. Add the Replacement Node WWNs
2. Remove the Replaced Node WWNs
Once the Switch Zoning has been updated, proceed to the Storage Reconfiguration.

Storage Reconfiguration - Fabric or Direct Connection

Open Storage Navigator or Storage Navigator Modular 2 (SNM2)
Check if HostGroup Security is enabled on the HostGroups configured for the HNAS nodes.
1. If HostGroup Security is enabled
  1. Add the new host port wwn(s) to each HostGroup as defined in the zoning
  2. Remove the old host port wwns from the corresponding HostGroup for the replaced Node.
2. Verify that HMO 7 & 68 are enabled on each HostGroup

Upgrade/Downgrade the Replacement Node

Verify the code version on the Replacement Node

From the serial console session to the Replacement node
Type ver to obtain the version of code loaded on the replacement node
If the node requires downgrading, use the gateway-reinstall procedure to downgrade the node.
If the node requires upgrading, follow the procedure below

Configure the Laptop for Network connectivity

Configure the laptop network port to an IP address on the same subnet as the node eth1
Connect the Laptop to the node eth1 port

Upgrade and Reboot the Replacement Node

Note: node IP address - this is the Node IP address on eth1 of the replacement node (not the Admin IP address)

Windows
1. Open the HNAS firmware .iso file
2. Navigate to the HiNAS and copy the firmware .tar file corresponding to the cluster version to a folder on the Laptop.
3. Navigate to the Win64tools subdirectory and copy the ssc.exe utility to the laptop folder that contains the .tar file.
4. Open a Command Prompt
5. Use the cd command to change to the folder on your Windows Desktop that contains the ssc.exe utility. (ie. cd C:/HNASFW)
6. Type ssc -u supervisor -p <current supervisor password> <node ip address> package-upgrade <firmware package>.tar
  The package will now upload to the node and be set as default automatically.
7. Type ssc -u supervisor -p <current supervisor password> <node ip address> package-set-default <firmware package>.tar
  Do not skip this step.
8. Type ssc -u supervisor -p <current supervisor password> <node IP address> reboot
  The node will load the new code version
9. Wait approximately 20 minutes for the node to completely boot into Bali
Other OS
1. Open the HNAS firmware .iso file
2. Navigate to the HiNAS and copy the firmware .tar file corresponding to the cluster version to a folder on your home directory.
3. Navigate to the utility directory corresponding to the OS version of the workstation you are operating on and copy the ssc utility to the folder that contains the .tar file.
4. Open a Terminal session.
5. Use the cd command to change to the folder on your home directory that contains the ssc command.
6. Type ./ssc -u supervisor -p <current supervisor password> <node ip address> package-upgrade <firmware package>.tar
  The package will now upload to the node and be set as default automatically.
7. Type ./ssc -u supervisor -p <current supervisor password> <node ip address> package-set-default <firmware package>.tar
  Do not skip this step.
8. Type ./ssc -u supervisor -p <current supervisor password> <node IP address> reboot
  The node will load the new code version
9. Wait approximately 20 minutes for the node to completely boot into Bali
Once the Node had completely booted into Bali
1. Log in as manager; Enter the current password when prompted
2. Type: ver (This will display the version of the new node)
3. Verify the version is correct on the replacement node.

Add the New License Key to the replacement node

Open each of the licensekey files obtained from TBKeys for the Replacement Node.
From the serial console session to the Replacement node
Type licensekey add <New License Key>
Note: This is the New License Key obtained from TBKeys
Repeat for each of the license keys received

If working Off-Site, prepare the replacement node for transit to the site

Shutdown the replacement node
1. Type shutdown -p

On-Site Procedures

Label all Cables on the Defective Node

Label all cables (fiber, Ethernet, Cluster Inter-Connect) connected to the defective node BEFORE you unplug them.

Prep Work on Existing Node (If The Node is NOT DEAD)

Note: if the Existing Node is DEAD, then the configuration data will be obtained from the latest Node Diagnostic.

Gather a Node Registry Backup from the Defective Node

Backup the Server Registry
1. On the laptop
  1. Open a command prompt
  2. Change to the directory of the ssc.exe command
  3. Type ssc -u supervisor -p <current supervisor password> <Node IP address>
  4. Type backupregistry
    When completed, the program saves the registry file to the directory where the ssc.exe is located.
  5. Verify that the archive file opens

Open a Console Session to the Defective Node

Open a console session to the Defective Node
1. Plug the serial connection into the console port of the Node
2. Start a serial putty session (Serial settings (115200, 8, n, 1, n)
3. Set putty to capture all readable output
Log in as root; Enter the current password when prompted.

Collect Diagnostics on Defective Node

From the serial console session to the Defective Node
Type cd /home/manager
Type getdiagnostic.linux
Note: The file will be written to the /home/manager directory of the node , take note of the name of the diagnostic
Type exit (to exit to the login prompt)

Transfer Diagnostic to local Workstation

Open a WinSCP session to the Defective Node
Navigate the Left Pane to the location of the registry backup and ssc.exe
Navigate the Right Pane to /home/manager
Copy the diagnostic taken in the section above from /home/manager to the location on the local workstation

Check for un-mounted file systems on the Defective Node

From the serial console session to the Defective Node
Type filesystem-list
Record any file systems that are un-mounted

Gather Information from the Defective Node

From the serial console session to the Defective Node
Gather Trouble and Settings
1. Type the following commands
  1. trouble
  2. set
Verify the admin IP Addresses of the cluster
1. Type evs list | grep eth[01]
  1. Take note of the node names for each Cluster Node
    Note: Pay particular attention to the node name of the node to be replaced.
2. Type evsipaddr -l | grep eth[01]
  1. Take note of the IP address and Netmask for eth1
  2. Take note of the IP address and Netmask for eth0
3. Type ipaddr
  1. Take note of the IP address and Netmask for eth1
  2. Take note of the IP address and Netmask for eth0
4. Type route
  1. Take note of the default gateway
5. Type dnsdomainname show
  1. Take note of the domain name
Gather the License Key Information
1. Capture the License Key Information
  1. Type licensekey list
Gather the Firmware Version and Model of the Nodes
1. Type: ver | grep -A 5 Model
  $ ver
  Model: HNAS 4100
  Software: 13.8.6320.10 (built 2020-11-28 05:15:19+00:00)
  Hardware: NAS Platform (DM2PEP512F312)
  1. Note the Model number after the label Model:
  2. Note the Firmware Version after the label Software:
  3. Note the Serial number after the label Hardware:
Gather the MacID from the Node
1. Type getmacid
  
  $ getmacid
  MAC ID is 97-E4-D0-2E-C6-0A
Gather the Cluster MacID
1. Type cluster-getmac
  
  $ cluster-getmac
  cluster MAC: 97-E4-D0-2E-C6-0A

Shutdown and Removal of the Defective Node

Open a console session to the Defective Node, If still Online and accessible from the Bali prompt
1. Plug the serial connection into the console port of the node
2. Start a serial PuTTY session (Serial settings (115200, 8, n, 1, n)
3. Set PuTTY to capture all readable output
Log in as manager; Enter the current password when prompted.
Shutdown the Defective node
1. Type shutdown --force --ship --powerdown
Physically remove the defective node from the system

Move Parts from Defective Node to Replacement Node

After the node has powered down
Remove the Battery, Power Supplies, and SFP+ from the old node
Move parts from Defective Node to Replacement Node
1. Battery and Power Supplies
2. 4xx0: 10GbE & 8Gbe SFP+
3. 5x00: 10GbE SFP+

Configuration of the Replacement Node

Installation/Power Up of Replacement Node

Install the Replacement Node in the place of the defective node
Plug in a power cord and power up the node.
Wait approximately 20 minutes for the node to completely boot into Bali

Open a Console Session to the Replacement Node

Open a console session to the Replacement Node
1. Plug the serial connection into the console port of the Node
2. Start a serial putty session (Serial settings (115200, 8, n, 1, n)
3. Set putty to capture all readable output
Log in as manager; Enter the current password when prompted.

Reset the Battery Age (HNAS 4xx0 ONLY)

If the node is running firmware at or above 11.1.3225.02
1. From the serial console session to the Replacement Node
2. Type new-battery-fitted --field --confirm
If the node is running firmware below 11.1.3225.02
1. From the serial console session to the Replacement Node
2. Type new-battery-fitted --field --confirm
3. Type exit to return to the login prompt
4. Log in as root; Enter the current password when prompted.
5. Type /etc/init.d/chassis-monitor restart (to restart the chassis-monitor service)
6. Type exit to return to the login prompt
7. Log in as manager; Enter the current password when prompted.

Verify the Model Number of NEW NODE

From the serial console session to the Replacement Node
Type: ver (This will display the version of the new node)
Model 4000 Only
1. If the existing node is a 4040, the replacement node will be a 4040.
2. If the existing node is a 4080, the replacement node will be a 4060
  Note: The new node will be updated to a 4080 from the installed License Key.
3. If the existing cluster is a 4100, the replacement node will be a 4100.
Model 5000 Only
1. If the existing node is a 5300, the replacement node will be a 5200
  Note: The new node will be updated to a 5300 from the installed License Key.

Optional: Load the SMU Firmware (Internal SMU Only)

Notes: Nodes are shipped from spares without the SMU installed. Use the following procedure to load the SMU firmware on the node

Plugin a USB key with the SMUsetup<filename> on it at the latest version
From the serial console session to the Replacement Node
Type exit to return to the login prompt
Log in as root; Enter the current password when prompted.
Type the following commands
1. mkdir /mnt/usb (if receiving an error that the directory already exists, proceed with the installation)
2. mount /dev/sdc1 /mnt/usb
3. cp /mnt/usb/SMUsetup<tab to complete filename> /home/manager
4. umount /mnt/usb
5. cd /home/manager
6. mount –o loop SMU<tab key to complete> /mnt/usb
7. /mnt/usb/autorun
Type exit to return to the login prompt
Log in as manager; Enter the current password when prompted.

Single Node Cluster Configuration

(Perform only if the original node was configured as a Single Node Cluster)

Note: if the Original Node was configured as a Single Node Cluster, the replacement node must be configured as a Single Node Cluster before restoring the Node Configuration.

From the Command Prompt/Terminal Session established above
1. Type ssc -u supervisor -p supervisor <node IP address> cluster-create -c <cluster name> <QD name/ipaddr>
  
  Cluster name: MY-CLUSTER
  Quorum name: MY-QD
  This command will update the configuration and reboot the system.
  Do you want to proceed? (Y/N)[N]:
  y
2. Wait approximately 20 minutes for the node to completely boot into Bali
3. Once the node has rebooted back into Bali, proceed with the Registry Restore.

Restore Node Configuration

Notes:

If the restore fails and the error includes the following "Failed: restore failed due to cluster mode mismatch", then there is one of two reasons
- The Original Node was configured as a Single Node Cluster, and the steps to configure the Replacement Node as a Single Node Cluster were NOT performed.
- The Original Node was NOT configured as a Single Node Cluster, and the steps above to configure the Replacement node as a Single Node Cluster were performed.
Restoring a configuration from a backup entirely replaces the current configuration with the saved configuration. All configuration settings are replaced with the settings in the backup, such that all configuration changes made since the backup are lost.
After restoring the backup, you will be prompted to reboot the server. The restored configuration will not take effect until the next reboot.
After restoring the backup, the Storage Pools will be displayed as assigned to “another cluster”.

Restore Procedure

The node registry.tgz file should be located in the directory with ssc.exe and the diagnostic files
Windows
1. Open a Command Prompt
2. Use the cd command to change to the folder on your Windows Desktop that contains the ssc command. (ie. cd C:/HNASFW)
3. Type ssc -u supervisor -p supervisor <node ip address> restoreregistry -f <registryfile.tgz>
  Answer Y to the Do you want to continue? (Y/N)[N] question
4. The node will now reboot to make the change active.
5. Wait approximately 20 minutes for the node to completely boot into Bali
Unix/Linux
1. Copy the firmware .tar file corresponding to the cluster version to your home directory containing the ssc.exe file.
2. Open a Terminal session.
3. Use the cd command to change to your home directory that contains the ssc command. (ie. typing cd by itself will change to your home directory)
4. Type ssc -u supervisor -p supervisor <node ip address> restoreregistry -f <registryfile.tgz>
  Answer Y to the Do you want to continue? (Y/N)[N] question
5. The node will now reboot to make the change active.
6. Wait approximately 20 minutes for the node to completely boot into Bali

Verify the Registry has been Restored

From the serial console session to the Replacement Node
Log in as manager; Enter the current password when prompted.
Type evs list (Verify the EVSs exist, do not worry that they are disabled)
Type sd-list (Verify the SDs exist, do not worry that they are NotPres)

Plug in all remaining cables to the new node.

NOTE: All cables should have been labeled in a prerequisite step.

If the customer had a network drop for eth0, plug that into the eth0 port
Reconnect the Cluster Interconnect Cables
1. 4xx0: C1 -> C1 & C2 -> C2
2. 5x00: C1 -> C1 & C3 -> C3
Network Cables
1. 4040: Plugin all customer-facing EVS 1GiB Ethernet Cables exactly as how they came off the old node
2. All: Plugin all customer-facing EVS 10GiB Ethernet Cables exactly as how they came off the old node
Reconnect the Fiber Channel Cables
1. 4xx0: Fiber ports, left to right - 1,2,3,4
2. 5x00: Fiber ports, left to right - 4,3,2,1
Verify that all Management, Cluster Interconnect, Network and Storage Cables to the Replacement node are in place

Storage Configuration

Verify FC Connection Status

From the serial console session to the Replacement Node
Retrieve the status of the FC links
1. Type fc-link-status
Physically observe the fiber channel connection to the fc ports
1. The nodes by default only have fc1 and fc3 enabled
  1. Enable the fc port to match the existing node
    1. Type fc-link x enable
2. If the corresponding ports in the output are not Up
  1. Check that the ports have a link light
  2. HNAS 5x00: Check that the corresponding Storage Ports are set to Point-to-Point
    Note: the HNAS 5x00 does not support Arbitrated Loop

Enable/Disable the HNAS hports to allow login on the storage

From the serial console session to the Replacement Node
1. Type fc-link x disable (repeat for all enabled ports)
2. Type fc-link x enable (repeat for the same ports as above)

Check storage array configuration

From the serial console session to the Replacement Node
Type the following commands
1. pn all scsi-racks
  1. If the output does not show the replacement node and all the ports from the replacement node
    1. Recheck the Storage Reconfiguration has been performed correctly on all HNAS HostGroups.
2. sd-list
  1. If the SDs show Not Pres
    1. Recheck the zoning and the HostGroup wwn settings.
  2. If any of the SDs show Unusbl,
    1. Type scsi-refresh to rescan the scsi bus (this may take a couple of minutes, so be patient)

Verify that the Node can see the storage

From the serial console session to the Replacement Node
Type sd-list
1. If the SDs show Not Pres, recheck the zoning and HostGroup wwn settings, as appropriate.
2. If any of the SDs show Unusbl, type scsi-refresh to rescan the scsi bus (this may take a couple of minutes, so be patient)
Assign the spans to the new node
1. Open a console session to the Node
2. Log in as manager; Enter the current password when prompted.
3. Type span-list (note the names of the spans)
4. Type span-assign-to-cluster <span name>
5. Repeat for any additional spans that exist
Once the SDs show OK
1. Type span-list, check that all spans show Ok
2. Type filesystem-list, check that all file systems are mounted
Enable EVS’s
1. Navigate to Home > Server Settings > EVS Management
  1. Select all EVS
  2. Click <enable>

Check storage configuration

From the serial console session to the Working Node
Type the following commands
1. sd-list
2. scsi-racks (if all ports do not show in the output, check the HostGroup Security settings)
3. fc-host-port-load -v (if SDs do not show balanced, run sdpath --rebalance, unless the storage is a HUS)
4. sdpath
5. fc-host-port-load
6. trouble
Record the printable output from the putty session to a file
Upload the putty output and diagnostics to TUF under the SR that was opened for the replacement.
Check the event log for any obvious serious events.

Final Steps

Have the Customer Collect Diagnostics from the SMU GUI and upload to TUF under the case opened for the replacement.
Upload the PuTTY output to TUF under the case opened for the replacement.
Check the event log for any obvious serious events.
Open an SDM (Service Data Management) case to change the asset serial number from the Replaced Node to the Replacement Node.
1. Serial Number of the replaced node
2. Serial Number of the replacement node

Final Check

Verify that the customer can access their data

CXOne Metadata

Tags: hnasgateway

PageID: 182417

Keywords: https://knowledge.hitachivantara.com/Knowledge/Storage/Network_Attached_Storage/Hitachi_NAS_Platform/How_to_Replace_a_HNAS_Gateway_Node_in_an_HNAS_Single_Node_System_without_vSMU_access