Maintenance of global-active device
You can perform maintenance on external storage while keeping GAD pairs.
Replacing the external storage system while keeping GAD pairs
You can replace the current external storage system with a new one while keeping GAD pairs.
Procedure
Check the pair operation mode for the blocked quorum disk.
Depending on the check results, you might have to split the GAD pair.If the QM column output with pairdisplay -fcxe command is AA, the GAD pair is split.
Go to step 2 if this is the case.If the QM column is other than AA, the GAD pair is not split in most cases. Go to step b.
# pairdisplay -g oraHA -fcxe -d dev0 Group PairVol(L/R) (Port#,TID, LU),Seq#,LDEV#.P/S,Status,Fence,%, P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM oraHA dev0(L) (CL1-C-0, 0, 0) 511111 400.P-VOL PAIR NEVER,100 500 - - 0 1 - - - L/M AA oraHA dev0(R) (CL7-C-0,28, 0) 522222 500.S-VOL PAIR NEVER,100 400 - - 0 1 - - - L/M AA
Split the GAD pair if it is not already split.
pairsplit -g oraHA -IH0
If you split the GAD pair in step 1, resynchronize the GAD pair.
pairresync -g oraHA -IH0
Disconnect the GAD pair from the quorum disk.
Disconnect the primary storage system from the quorum disk.
raidcom disconnect external_grp -ldev_id 0x2045 -IH0
Disconnect the secondary storage system from the quorum disk.
raidcom disconnect external_grp -ldev_id 0x2045 -IH1
Confirm that the GAD pair is disconnected from the quorum disk.
Check if the primary storage system is disconnected from the quorum disk.
(VSP 5000 series)
raidcom get path -path_grp 1 -IH0 PHG GROUP STS CM IF MP# PORT WWN PR LUN PHS Serial# PRODUCT_ID LB PM 1 1-1 NML E D 0 CL5-A 50060e8008000140 1 0 NML 55555 VSP 5000 series N M D
(VSP Gx00 models and VSP Fx00 models)
raidcom get path -path_grp 1 -IH0 PHG GROUP STS CM IF MP# PORT WWN PR LUN PHS Serial# PRODUCT_ID LB PM 1 1-1 DSC E D 0 CL5-A 50060e8007823520 1 0 NML 433333 VSP Gx00 N M
Check if the secondary storage system is disconnected from the quorum disk.
(VSP 5000 series)
raidcom get path -path_grp 1 -IH1 PHG GROUP STS CM IF MP# PORT WWN PR LUN PHS Serial# PRODUCT_ID LB PM 1 1-1 NML E D 0 CL5-C 50060e8008000160 1 0 NML 55555 VSP 5000 series N M D
(VSP Gx00 models and VSP Fx00 models)
raidcom get path -path_grp 1 -IH1 PHG GROUP STS CM IF MP# PORT WWN PR LUN PHS Serial# PRODUCT_ID LB PM 1 1-2 DSC E D 0 CL5-C 50060e8007823521 1 0 NML 433333 VSP Gx00 N M
Prepare a new quorum disk.
Format the disk of a new external storage system.
Map the formatted disk to the primary and secondary storage systems.
Use the same procedure you use for creating a quorum disk. However, you do not need to set an external volume for the quorum disk.
Check the status of the quorum disk.
On the primary storage system, confirm that the status of the quorum disk is BLOCKED.
(VSP 5000 series)
raidcom get quorum -quorum_id 1 -IH0 QRDID : 1 LDEV : 2045 QRP_Serial# : 511111 QRP_ID : R9 Timeout(s) : 30 STS : BLOCKED
(VSP Gx00 models and VSP Fx00 models)
raidcom get quorum -quorum_id 1 -IH0 QRDID : 1 LDEV : 2045 QRP_Serial# : 411111 QRP_ID : M8 Timeout(s) : 30 STS : BLOCKED
On the secondary storage system, confirm that the status of the quorum disk is BLOCKED.
(VSP 5000 series)
raidcom get quorum -quorum_id 1 -IH1 QRDID : 1 LDEV : 2045 QRP_Serial# : 522222 QRP_ID : R9 Timeout(s) : 30 STS : BLOCKED
(VSP Gx00 models and VSP Fx00 models)
raidcom get quorum -quorum_id 1 -IH1 QRDID : 1 LDEV : 2045 QRP_Serial# : 422222 QRP_ID : M8 Timeout(s) : 30 STS : BLOCKED
Replace the old LDEV with a new one.
On the primary storage system, replace the current external volume for the quorum disk with a new one.
raidcom replace quorum -quorum_id 1 -ldev_id 1234 -IH0
On the primary storage system, confirm that the status of the quorum disk is REPLACING.
(VSP 5000 series)
raidcom get quorum -quorum_id 1 -IH0 QRDID : 1 LDEV : 1234 QRP_Serial# : 511111 QRP_ID : R9 Timeout(s) : 30 STS : REPLACING
(VSP Gx00 models and VSP Fx00 models)
raidcom get quorum -quorum_id 1 -IH0 QRDID : 1 LDEV : 1234 QRP_Serial# : 411111 QRP_ID : M8 Timeout(s) : 30 STS : REPLACING
On the secondary storage system, replace the current external volume for the quorum disk with a new one.
raidcom replace quorum -quorum_id 1 -ldev_id 1234 -IH1
On the secondary storage system, confirm that the status of the quorum disk is REPLACING.
(VSP 5000 series)
raidcom get quorum -quorum_id 1 -IH1 QRDID : 1 LDEV : 1234 QRP_Serial# : 522222 QRP_ID : R9 Timeout(s) : 30 STS : REPLACING
(VSP Gx00 models and VSP Fx00 models)
raidcom get quorum -quorum_id 1 -IH1 QRDID : 1 LDEV : 1234 QRP_Serial# : 422222 QRP_ID : M8 Timeout(s) : 30 STS : REPLACING
NoteIf the raidcom replace quorum command is executed normally, the status of the quorum disk changes from BLOCKED to REPLACING in a few seconds. If the status does not change in a few minutes, contact customer support.
Confirm that the status of the quorum disk is NORMAL.
On the primary storage system, confirm that the status of the quorum disk is NORMAL.
(VSP 5000 series)
raidcom get quorum -quorum_id 1 -IH0 QRDID : 1 LDEV : 1234 QRP_Serial# : 511111 QRP_ID : R9 Timeout(s) : 30 STS : NORMAL
(VSP Gx00 models and VSP Fx00 models)
raidcom get quorum -quorum_id 1 -IH0 QRDID : 1 LDEV : 1234 QRP_Serial# : 411111 QRP_ID : M8 Timeout(s) : 30 STS : NORMAL
On the secondary storage system, confirm that the status of the quorum disk is NORMAL.
(VSP 5000 series)
raidcom get quorum -quorum_id 1 -IH1 QRDID : 1 LDEV : 1234 QRP_Serial# : 522222 QRP_ID : R9 Timeout(s) : 30 STS : NORMAL
(VSP Gx00 models and VSP Fx00 models)
raidcom get quorum -quorum_id 1 -IH1 QRDID : 1 LDEV : 1234 QRP_Serial# : 422222 QRP_ID : M8 Timeout(s) : 30 STS : NORMAL
If the raidcom replace quorum command is executed normally, the status of the quorum disk changes from REPLACING to NORMAL in a minute.
If the status does not change in five minutes, check whether remote paths between the storage systems are in normal state.
Confirm that the GAD pair you resynchronized is synchronized normally.
NoteWhen the quorum disk status changes from REPLACING to NORMAL due to a storage system failure, the quorum disk status of a storage system without a failure might not change from REPLACING in five minutes. When this happens, resynchronize the GAD pair after the failure is resolved. By this method, the status changes to NORMAL.
Also, if you do not assign a quorum disk ID to a GAD pair after replacing a quorum disk, the quorum disk status does not change from REPLACING. In this case, we recommend deleting the quorum disk and then creating a new quorum disk before creating a GAD pair. If you want to create a GAD pair without deleting the quorum disk, perform step 8 again after creating a GAD pair.
If the status of the quorum disk does not change after performing these methods, contact customer support.
Note If the status of the quorum disk is FAILED, the primary storage system and the secondary storage system might be connected to different quorum disks. Specify the external volume so that the primary storage system and the secondary storage system are connected to the same quorum disk. After specifying the correct external volume, perform steps 8 through 10.
Expansion of DP-VOL capacity while keeping pair status
You can expand the capacity of a DP-VOL used in the GAD, TrueCopy, Universal Replicator, ShadowImage, or Thin Image pair created between or on the following storage systems while keeping the pair status in each software product.
- VSP 5000 series: Microcode version 90-04-04-00/00 or later
- VSP G/F350, G/F370, G/F700, G/F900: Microcode version 88-06-02-x0/00 or later
- VSP E990: Microcode version 93-02-03-x0/00 or later
Expanding DP-VOL capacity used in GAD pair
When deciding to expand the capacity of a DP-VOL used as a volume in a GAD pair, understand that there are two different procedures, each depending on whether the GAD pair shares the same volumes with other software or does not share the same volumes. This procedure describes expanding DP-VOL capacity when the GAD pair does not share the volume (a GAD pair alone).
Before you begin
- A V-VOL to be expanded is not an external volume
- The LDEV format is being performed on a V-VOL to be expanded
- A pool containing a V-VOL to be expanded meets any of the following
conditions:
- Normal condition
- The used capacity of the pool exceeds the depletion threshold
- The pool is being shrunk
- A V-VOL to be expanded is a volume created on the storage systems described in Expansion of DP-VOL capacity while keeping pair status
- Make sure that there is enough GAD license capacity for the page capacity assigned to the expanded volume capacity before the expansion
Procedure
Verify that both the P-VOL and S-VOL of a GAD pair meet the prerequisites.
Display the detailed information about the GAD pair on CCI.
Verify whether the current differential data management method is the Shared memory (SM) difference management or the hierarchical difference management by a drive. The DM column displayed by the pairdisplay command shows whether the shared memory (S) difference management is applied or the hierarchical (D) difference management is applied.# pairdisplay -g oradb -fe Group PairVol(L/R) (Port#,TID, LU),Seq#, LDEV#.P/S,Status,Fence,Seq#, P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P oradb dev1(L) (CL5-A-0,30, 0) 64568 301.P-VOL PAIR ASYNC,64568 303 - 0 2 1 - - - -/- AA S N oradb dev1(R) (CL5-A-0,30, 2) 64568 303.S-VOL PAIR ASYNC,----- 301 - 0 4 1 - - - -/- AA S N oradb dev2(L) (CL5-A-0,30, 1) 64568 302.P-VOL PAIR ASYNC,64568 304 - 0 3 1 - - - -/- AA D N oradb dev2(R) (CL1-A-0,30, 3) 64568 304.S-VOL PAIR ASYNC,----- 302 - 0 5 1 - - - -/- AA D N
- If the SM difference management is applied, go to Step 3 to temporarily change to hierarchical difference management.
- If the hierarchical difference management is applied, go to Step 4.
- If using HDvM - SN, verify the advanced system settings
No.5 and No.6.
- If both the advanced system settings No.5 and No. 6 are set to Disable, the SM difference management is applied. Go to Step 3.
- If either or both the advanced system settings No.5 and No.6 are set to Enable, the hierarchical difference management is applied. Go to Step 4.
Change the differential data management method to the hierarchical difference management.
Change system settings for hierarchical difference management.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode enable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
- If using HDvM - SN, set the advanced system settings No. 5 to Enable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
Suspend the GAD pair.
Resynchronize the GAD pair.
Display the detailed information about the GAD pair on CCI.
Verify that the current differential data management method is the hierarchical difference management.
Suspend the GAD pair.
Expand the S-VOL capacity in the GAD pair.
- If using CCI,
add -request_id auto option to the raidcom extend
ldev command to specify the asynchronous command
processing.
Command example
The following command expands the capacity of LDEV#44:44 by 10 GB.
raidcom extend ldev -ldev_id 0x4444 -capacity 10G –request_id auto -IH1
- If using HDvM - SN, expand the capacity in the Expand V-VOLs window.
- If using CCI,
add -request_id auto option to the raidcom extend
ldev command to specify the asynchronous command
processing.
Verify that expanding the S-VOL capacity in the GAD pair is finished.
- If using CCI, raidcom get
command_status command to verify that the raidcom extend ldev command is finished processing.
Command example
raidcom get command_status -IH1raidcom get ldev -ldev_id 0x4444 -fx -IH1
Verify the expanded LDEV capacity using the raidcom get ldev command.
- If using HDvM - SN, select Logical devices from the Storage Systems tree, and then display the LDEV tab. Verify that the capacity with the target LDEV number is correct.
- If using CCI, raidcom get
command_status command to verify that the raidcom extend ldev command is finished processing.
Expand the P-VOL capacity in the GAD pair in the same way as the S-VOL capacity.
If the P-VOL capacity in the GAD pair cannot be expanded, see Recovering from failure if one volume capacity in GAD pair cannot be expanded.Verify that expanding the P-VOL capacity in the GAD pair is complete in the same way as the S-VOL capacity.
Resynchronize the GAD pair.
If the resync operation in the GAD pair cannot be performed, see Recovering from failure if one volume capacity in GAD pair cannot be expanded.Verify that the pair statuses of both the P-VOL and S-VOL in the GAD pair have changed to PAIR (Mirror(RL)), and that the volume capacity is not being expanded.
- If using CCI, check that the P column of the
pairdisplay command displays
N.
# pairdisplay -g oradb -fe Group PairVol(L/R) (Port#,TID, LU),Seq#, LDEV#.P/S,Status,Fence,Seq#, P-LDEV# M CTG JID AP EM E-Seq# E-LDEV# R/W QM DM P oradb dev1(L) (CL5-A-0,30, 0) 64568 301.P-VOL PAIR ASYNC,64568 303 - 0 2 1 - - - L/M AA S N oradb dev1(R) (CL5-A-0,30, 2) 64568 303.S-VOL PAIR ASYNC,----- 301 - 0 4 1 - - - L/M AA S N oradb dev2(L) (CL5-A-0,30, 1) 64568 302.P-VOL PAIR ASYNC,64568 304 - 0 3 1 - - - L/M AA D N oradb dev2(R) (CL1-A-0,30, 3) 64568 304.S-VOL PAIR ASYNC,----- 302 - 0 5 1 - - - L/M AA D N
The P column in the
pairdisplay
command displays the status that the volume capacity is being expanded. An N indicates that the volume capacity is not being expanded, an E indicates that the volume capacity is being expanded.The E status is displayed after expanding the P-VOL or S-VOL capacity in the GAD pair, and before performing the pair resync operation. This status changes to N after the resync operation completes.
- If using HDvM - SN, check if the Processing
Status field in the Remote Replication
window is blank.
Expanding is displayed if the volume capacity is being expanded.
- If using CCI, check that the P column of the
pairdisplay command displays
N.
If you temporarily changed the differential data management method from SM difference to hierarchical difference (Step 2), restore to the SM difference management.
To restore to the SM difference management:
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode enable
- If using HDvM - SN, set the advanced system settings No. 5 to Disable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
Suspend the GAD pair.
Resynchronize the GAD pair.
Display the detailed information about the GAD pair on CCI, and then verify that the current differential data management method is the SM difference management.
If using CCI, and the settings of SOMs 1198 and 1199 have been changed, set SOM 1198 to OFF and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
Expansion order policy when GAD pair shares the same volumes with other software
You can expand the capacity of DP-VOLs used in a GAD pair and each software pair when a GAD pair shares the same volumes with Universal Replicator (UR), ShadowImage (SI), and Thin Image (HTI).
When a GAD pair shares the same volumes with other software, the order in which you expand the DP-VOL capacity consists of the following two expansion orders:
- Expansion order in a point to point configuration of the GAD pair
- Expansion order in a point to point configuration of each software product

- When a remote replication (TC/UR/GAD) is included, first expand the S-VOL capacity on the end of the remote replication, and then expand the P-VOL capacity. For more information about the specific expansion procedure, see Expanding DP-VOL capacity when GAD pair is used with Universal Replicator
- When a remote replication (TC/UR/GAD)
shares the same volumes with a local replication (SI/HTI),
first expand the capacity of the S-VOL group in the remote replication. The S-VOL group
consists of the S-VOL in the remote replication and all volumes in the local replication
with which the S-VOL shares the same volume. After expanding the capacity of the S-VOL group
in the remote replication, expand the capacity of the P-VOL group in the remote replication.
The P-VOL group consists of the P-VOL in the remote replication and all volumes in the local
replication with which the P-VOL shares the same volume.
The expansion order in each group follows the expansion order in the local replication. For more information about the specific expansion procedures, see Expanding DP-VOL capacity when GAD pair is used with ShadowImage and Expanding DP-VOL capacity when GAD pair is used with Thin Image.
- In the local replication (SI/HTI), expand the capacity in the order from the highest layer to the lowest. The expansion order is not specified for the volumes in the same layer.
Expanding DP-VOL capacity when GAD pair is used with Universal Replicator
You can expand the capacity of DP-VOLs when pairs are created in a 3DC delta resync configuration with GAD and UR. For the detailed procedures for verifying the differential data management method, expanding the capacity, and verifying the expansion during the following procedure, see Expanding DP-VOL capacity used in GAD pair.
Before you begin
- A V-VOL to be expanded is not an external volume.
- The LDEV format is being performed on a V-VOL to be expanded.
- A pool containing a V-VOL to be expanded meets any of the following
conditions:
- Normal condition
- The used capacity of the pool exceeds the depletion threshold
- The pool is being shrunk
- When using GAD, TC, or DP-VOL, make sure that there is enough software license capacity for the page capacity assigned to the expanded volume capacity before the expansion.
- A V-VOL to be expanded is created on a valid storage system. See Expanding DP-VOL capacity while keeping pair status.
Procedure
Verify that each volume of the P-VOLs and S-VOLs in a GAD pair and DP-VOL pairs with which the GAD pair shares the same volumes meets the prerequisites.
Display the detailed information about the GAD pair, UR pair, and UR delta resync pair on CCI.
Verify whether the current differential data management method is the SM difference management or the hierarchical difference management by a drive.- If the SM difference management is applied, proceed to step 3.
- If the hierarchical difference management is applied, proceed to step 4.
- If using HDvM - SN, verify the advanced system
settings No.5 and No.6.
- If both the advanced system settings No.5 and No. 6 are set to Disable, the SM difference management is applied. Go to Step 3.
- If either or both the advanced system settings No.5 and No.6 are set to Enable, the hierarchical difference management is applied. Go to Step 4.
Change the differential data management method to the hierarchical difference management.
Change system settings for hierarchical difference management.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode enable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
- If using HDvM - SN, set the advanced system settings No. 5 to Enable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
Swap suspend the GAD pair.
The delta resync operation can be performed automatically.Verify the pair statuses of the GAD pair, UR pair, and UR delta resync pair, and that the delta resync operation is complete correctly.
Swap suspend the GAD pair again.
The delta resync operation can be performed automatically.Verify the pair statuses of the GAD pair, UR pair, and UR delta resync pair, and that the delta resync operation is complete correctly.
Display the detailed information about the GAD pair, UR pair, and UR delta resync pair on CCI, and then verify that the current differential data management method is the hierarchical difference management.
Suspend the GAD pair and UR pairs.
Expand each volume capacity, and then verify that the expansion is complete in the following expansion order:
Expand S-VOL in the UR pair (S-VOL in the UR delta resync pair).
If you cannot expanding the S-VOL capacity in the UR pair, expanding the P-VOL capacity in the GAD pair, or resynchronizing the GAD pair or UR pair, make sure to perform the recovery procedure described in Recovering from failure if one volume capacity in GAD pair cannot be expanded.Expand S-VOL in the GAD pair.
Expand P-VOL in the GAD pair.
Resynchronize the GAD pair and UR pair.
Verify that both the pair statuses of the P-VOL and S-VOL in each pair of the GAD pair have changed to PAIR(Mirror(RL)) and in each pair of the UR pair have changed to PAIR.
In addition, verify that each volume capacity in the GAD pair, UR pair, and UR delta resync pair is not being expanded.The following table shows how the expansion process is displayed.
Expansion status CCI HDvM - SN P column in the pairdisplay command Processing Status field in the Remote Replication window Not being expanded N Blank Being expanded E* Expanding* * These statuses have been displayed after expanding the P-VOL or S-VOL capacity in the GAD pair or the UR pair, and before performing the pair resync operation. After the resync operation is complete, the status is migrated to the Not being expanded status. In the UR delta resync pair, the Being expanded status will not be displayed on CCI or HDvM - SN because the expansion in the UR delta resync pair is complete concurrently with the GAD pair and the UR pair. In step 2, if the differential data management method is the SM difference management, restore to the SM difference management using the following procedure:
To restore to the SM difference management:
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode enable
- If using HDvM - SN, set the advanced system settings No. 5 to Disable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
Swap suspend the GAD pair.
The delta resync operation can be performed automatically.Verify the pair statuses of the GAD pair, UR pair, and UR delta resync pair, and that the delta resync operation is complete correctly.
Swap suspend the GAD pair again.
The delta resync operation can be performed automatically.Verify the pair statuses of the GAD pair, UR pair, and UR delta resync pair, and that the delta resync operation is complete correctly.
Display the detailed information about the GAD pair, UR pair, and UR delta resync pair on CCI, and then verify that the current differential data management method is the SM difference management.
If using CCI, and the settings of SOMs 1198 and 1199 have been changed, set SOM 1198 to OFF and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
Expanding DP-VOL capacity when GAD pair is used with ShadowImage
You can expand the capacity of DP-VOLs used as pair volumes when a GAD pair is used with SI. This procedure provides an example for expanding the DP-VOL capacity in a configuration where the SI L1 pair and L2 pair are created for each volume of the P-VOL and S-VOL in the GAD pair. For the detailed procedures for verifying the differential data management method, expanding the capacity, and verifying the expansion during the following procedure, see Expanding DP-VOL capacity used in GAD pair.
Before you begin
- A V-VOL to be expanded is not an external volume.
- The LDEV format is being performed on a V-VOL to be expanded.
- A pool containing a V-VOL to be expanded meets any of the following
conditions:
- Normal condition
- The used capacity of the pool exceeds the depletion threshold
- The pool is being shrunk
- When using GAD, TC, or DP-VOL, make sure that there is enough software license capacity for the page capacity assigned to the expanded volume capacity before the expansion.
- A V-VOL to be expanded is created on a valid storage system. See Expanding DP-VOL capacity while keeping pair status.
- Ensure that the numbers of differential tables and pair tables required after the expansion do not exceed the maximum number allowed on the storage system. For information about calculating the number of tables, see Calculating the numbers of differential and pair tables for each pair.
Procedure
Verify that each volume of the P-VOLs and S-VOLs in a GAD pair and SI pairs with which the GAD pair shares the same volumes meets the prerequisites.
Display the detailed information about the GAD pair on CCI.
Verify whether the current differential data management method is the SM difference management or the hierarchical difference management by a drive.- If the SM difference management is applied, proceed to step 3.
- If the hierarchical difference management is applied, proceed to step 4.
- If using HDvM - SN, verify the advanced system
settings No.5 and No.6.
- If both the advanced system settings No.5 and No. 6 are set to Disable, the SM difference management is applied. Go to Step 3.
- If either or both the advanced system settings No.5 and No.6 are set to Enable, the hierarchical difference management is applied. Go to Step 4.
Change the differential data management method to the hierarchical difference management.
Change system settings for hierarchical difference management.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode enable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
- If using HDvM - SN, set the advanced system settings No. 5 to Enable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
Suspend the GAD pair.
Resynchronize the GAD pair.
Display the detailed information about the GAD pair on CCI, and then verify that the current differential data management method is the hierarchical difference management.
Suspend the GAD pair and all SI pairs.
Verify the pair statuses of the SI pairs.
The SI pair volume capacity can be expanded when the SI pairs are in the PSUS or PSUE status. Verify the current pair statuses of the SI pairs.If the pair statuses are not PSUS or PSUE, perform the pair operations.
Ensure that the pair statuses are migrated to PSUS or PSUE.
Expand each volume capacity, and then verify that the expansion is complete in the expansion order shown in the following figure.
After expanding the first volume capacity (S-VOL in the GAD pair), if you cannot expand the DP-VOL capacity in the SI pair, you cannot expand the P-VOL capacity in the GAD pair, or resynchronizing the GAD pair, make sure to perform the recovery procedure described in Recovering from failure if one volume capacity in GAD pair cannot be expanded.
Resynchronize the GAD pair.
Verify that both the pair statuses of the P-VOL and S-VOL in each pair of the GAD pair have changed to PAIR(Mirror(RL)).
In addition, verify that each volume capacity in the GAD pair and SI pair is not being expanded.The following table shows how the expansion process is displayed.
Expansion status CCI HDvM - SN P column in the pairdisplay command Processing Status field in the Remote Replication or Local Replication window Not being expanded N Blank Being expanded E* Expanding* * These statuses have been displayed after expanding the P-VOL or S-VOL capacity in the GAD pair, and before performing the pair resync operation. After the resync operation is complete, the status is migrated to the Not being expanded status. In the SI pair, the Being expanded status is displayed for a while (30 seconds or so) after the expansion of the P-VOL and S-VOL capacity is complete. After that, the status is migrated to the Not being expanded status. In step 2, if the differential data management method is the SM difference management, restore to the SM difference management using the following procedure:
To restore to the SM difference management:
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode enable
- If using HDvM - SN, set the advanced system settings No. 5 to Disable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
Suspend the GAD pair.
Resynchronize the GAD pair.
Display the detailed information about the GAD pair on CCI, and then verify that the current differential data management method is the SM difference management.
If using CCI, and the settings of SOMs 1198 and 1199 have been changed, set SOM 1198 to OFF and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
Calculating the numbers of differential and pair tables for each pair
If a DP-VOL exceeds 4 TB, the differential tables are placed in the hierarchy memory instead of the shared memory, and the differential tables in the shared memory are not used. Therefore, it is not necessary to calculate the number of differential tables for DP-VOLs over 4 TB.
Procedure
Calculate how many differential tables are required for each pair.
Use the following formulas:number-of-differential-tables-for-expansion = number-of-differential-tables-after-expansion - number-of-differential-tables-before-expansion
number-of-differential-tables-per-pair = ceiling ((volume-capacity [KB] / 256) / 20,448)
Ensure that the ceiling value enclosed in the parenthesis () is rounded up to the next integer.
Calculate how many pair tables are required for each pair.
Use the following formulas:number-of-pair-tables-for-expansion = number-of-pair-tables-after-expansion - number-of-pair-tables-before-expansion
number-of-pair-tables-per-pair = ceiling ((number-of-pair-tables-per-pair / 36)
Ensure that the ceiling value enclosed in the parenthesis () is rounded up to the next integer.
Verify that the following numbers do not exceed the maximum number allowed on the storage system.
- The total numbers of current pair tables and pair tables required for expanding the capacity
- The total numbers of current differential tables and differential tables required for expanding the capacity
You can check the numbers of differential and pair tables in the Local Replication window.
Expanding DP-VOL capacity when GAD pair is used with Thin Image
You can expand the capacity of DP-VOLs used as pair volumes when a GAD pair is used with HTI. This procedure provides an example for expanding the DP-VOL capacity in a configuration where the HTI L1 pair and L2 pair are created for each volume of the P-VOL and S-VOL in the GAD pair. For the detailed procedures for verifying the differential data management method, expanding the capacity, and verifying the expansion during the following procedure, see Expanding DP-VOL capacity used in GAD pair.
Before you begin
- A V-VOL to be expanded is not an external volume.
- The LDEV format is being performed on a V-VOL to be expanded.
- A pool containing a V-VOL to be expanded meets any of the following
conditions:
- Normal condition
- The used capacity of the pool exceeds the depletion threshold
- The pool is being shrunk
- When using GAD, TC, or DP-VOL, make sure that there is enough software license capacity for the page capacity assigned to the expanded volume capacity before the expansion.
- A V-VOL to be expanded is created on a valid storage system. See Expanding DP-VOL capacity while keeping pair status.
- A V-VOL to be expanded is not used in a pair with the clone attribute.
- Ensure that the following numbers required after the expansion do not exceed the
maximum number allowed on the storage system:
- The snapshot estimated manageable capacity
- The number of cache management devices
- The subscription limit of the V-VOL capacity
For information about calculating these numbers, see Calculating the snapshot estimated manageable capacity, the numbers of cache management devices, and the subscription limit of the V-VOL capacity.
Procedure
Verify that each volume of the P-VOLs and S-VOLs in a GAD pair and HTI pairs with which the GAD pair shares the same volumes meets the prerequisites.
Display the detailed information about the GAD pair on CCI.
Verify whether the current differential data management method is the SM difference management or the hierarchical difference management by a drive.- If the SM difference management is applied, proceed to step 3.
- If the hierarchical difference management is applied, proceed to step 4.
- If using HDvM - SN, verify the advanced system
settings No.5 and No.6.
- If both the advanced system settings No.5 and No. 6 are set to Disable, the SM difference management is applied. Go to Step 3.
- If either or both the advanced system settings No.5 and No.6 are set to Enable, the hierarchical difference management is applied. Go to Step 4.
Change the differential data management method to the hierarchical difference management.
Change system settings for hierarchical difference management.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode enable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
- If using HDvM - SN, set the advanced system settings No. 5 to Enable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to ON
and 1199 to OFF by using the
Suspend the GAD pair.
Resynchronize the GAD pair.
Display the detailed information about the GAD pair on CCI, and then verify that the current differential data management method is the hierarchical difference management.
Suspend the GAD pair.
Verify the pair statuses of the HTI pairs.
The HTI pair volume capacity can be expanded when the HTI pairs are in the PAIR, PSUS or PSUE status. Verify the current pair statuses of the HTI pairs.If the pair statuses are not PAIR, PSUS or PSUE, perform the pair operations.
Ensure that the pair statuses are migrated to PAIR or PSUS.
Expand each volume capacity, and then verify that the expansion is complete in the expansion order shown in the following figure.
After expanding the first volume capacity (S-VOL in the GAD pair), if you cannot expand the DP-VOL capacity in the HTI pair, you cannot expand the P-VOL capacity in the GAD pair, or resynchronizing the GAD pair, make sure to perform the recovery procedure described in Recovering from failure if one volume capacity in GAD pair cannot be expanded.
Resynchronize the GAD pair.
Verify that both the pair statuses of the P-VOL and S-VOL in each pair of the GAD pair have changed to PAIR(Mirror(RL)).
In addition, verify that each volume capacity in the GAD pair and HTI pair is not being expanded.The following table shows how the expansion process is displayed.
Expansion status CCI HDvM - SN P column in the pairdisplay or raidcom get snapshot command Processing Status field in the Remote Replication or HTI pairs window Not being expanded N Blank Being expanded E* Expanding* * These statuses have been displayed after expanding the P-VOL or S-VOL capacity in the GAD pair, and before performing the pair resync operation. After the resync operation is complete, the status is migrated to the Not being expanded status. In the HTI pair, the Being expanded status is displayed for a while (30 seconds or so) after the expansion of the P-VOL and S-VOL capacity is complete. After that, the status is migrated to the Not being expanded status. In step 2, if the differential data management method is the SM difference management, restore to the SM difference management using the following procedure:
To restore to the SM difference management:
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode enable
- If using HDvM - SN, set the advanced system settings No. 5 to Disable and No.6 to Disable in the Edit Advanced System Settings window.
- If using CCI, set the system option mode (SOM) 1198 to OFF
and 1199 to ON by using the
Suspend the GAD pair.
Resynchronize the GAD pair.
Display the detailed information about the GAD pair on CCI, and then verify that the current differential data management method is the SM difference management.
If using CCI, and the settings of SOMs 1198 and 1199 have been changed, set SOM 1198 to OFF and 1199 to OFF by using the
raidcom modify system_opt
command.Command example:
raidcom modify system_opt -system_option_mode system -mode_id 1198 -mode disable
raidcom modify system_opt -system_option_mode system -mode_id 1199 -mode disable
Calculating the snapshot estimated manageable capacity, the numbers of cache management devices, and the subscription limit of the V-VOL capacity
Procedure
Calculate the snapshot estimated manageable capacity required for expanding the capacity of a Thin Image root volume.
To expand the capacity of a Thin Image root volume, verify that the snapshot estimated manageable capacity is available. Use the following formulas:Snapshot-estimated-manageable-capacity-for-root-volume-expansion = Snapshot-estimated-manageable-capacity-after-root-volume-expansion - Snapshot-estimated-manageable-capacity-before-root-volume-expansion
Snapshot-estimated-manageable-capacity = (Root-volume-capacity [TB] / 2.6 [TB]) x 3,024 [GB] + (168 [GB] x 2 (consumed-shared-memory [GB]))
Expanding the volumes other than a root volume does not consume the snapshot estimated manageable capacity. You can check the snapshot estimated manageable capacity in the Local Replication window.
Calculate the maximum number of cache management devices required for expanding the capacity of a Thin Image root volume.
To expand the capacity of a Thin Image root volume, verify that the cache management devices are available.Use the following formula:
number-of-cache-management-devices-for-root-volume-expansion = ceiling ((Root-volume-capacity-after-expansion [TB] / 2.6 [TB])) - ceiling ((Root-volume-capacity-before-expansion [TB] / 2.6 [TB]))
Ensure that the ceiling value enclosed in the parenthesis () is rounded up to the next integer.
Expanding the volumes other than a root volume does not consume the cache management devices.
Calculate the V-VOL capacity to be added by expanding the capacity of a Thin Image root volume.
To expand the capacity of a Thin Image root volume, verify that the capacity after the expansion does not exceed the subscription limit of the V-VOL capacity contained in a DP pool.Use the following formula:
V-VOL-capacity-added-by-root-volume-expansion = Thin Image-pair-capacity-calculated-based-on- root-volume-capacity-after-expansion - Thin Image-pair-capacity-calculated-based-on-root-volume-capacity-before-expansion
Thin Image-pair-capacity = Σ ceiling ((Root-volume-capacity [MB] × number-of-pairs* × 2 / 42 [MB])) × 42 [MB]+ Σ ceiling ((Root-volume-capacity [MB] × number-of-pairs* × 2 / 2,921,688 [MB])) × 175,434 [MB]
* Use the maximum number of HTI pairs to be created in the applicable snapshot tree as number-of-pairs instead of the number of pairs currently created in the applicable snapshot tree because the allocated pages are not released if you delete some pairs.
Ensure that the ceiling value enclosed in the parenthesis () is rounded up to the next integer.
Expanding the volumes other than a root volume does not add the V-VOL capacity contained in a DP pool in Thin Image.
Expansion order policy in configuration where P-VOL in remote replication shares the same volume with S-VOL in local replication
Expand the volume capacity in the group that includes the S-VOL in the remote replication in the order shown as 1, 2, 3 in the preceding figure.
Expand the volume capacity in the group that includes the P-VOL in the remote replication in the order shown as 4, 5, 6 in the preceding figure.
Expand the volume capacity in the order from the highest layer to the lowest because the expansion order in the local replication is prioritized. Expand the P-VOL capacity in the local replication first, and then the S-VOL capacity in the local replication that shares the same volume with the P-VOL in the remote replication.For example, when an S-VOL in SI shares the same volume with a P-VOL in TC or UR as shown in the preceding figure, expand the volume capacity in the following order:
- P-VOL in the SI pair shown as 4 in the figure.
- S-VOL in the SI pair (P-VOL in the TC or UR pair) shown as 5 in the figure.
- Volumes in the same layer as the S-VOL in the SI pair.
- Volumes in the lower layer than the S-VOL in the SI pair.
Expansion order policy in 3DC cascade, 3DC multi-target, or 3DC delta resync configuration with remote replications
When a 3DC cascade, 3DC multi-target, or 3DC delta resync configuration using remote replications is created, first expand the S-VOL capacity on the end of the remote replication, and then expand the P-VOL capacity as shown in the following figures.
3DC cascade configuration
3DC multi-target configuration
3DC delta resyc configuration
Expansion order when remote replications in a 3DC cascade configuration share the same volumes with local replications (ShadowImage/Thin Image)
When the remote replications in a 3DC configuration share the same volumes with the local replications (SI/HTI), expand the volume capacity in each following group in the order as shown in the figure.

The expansion order in each group follows the expansion order in the local replication. In a 3DC multi-target or 3DC delta resync configuration, expand the volume capacity in the same way.
Procedure
Expand the S-VOL group that consists of the S-VOL in the remote replication and all volumes in the local replication with which the S-VOL shares the same volume.
Expand the P-VOL group that consists of the P-VOL in the remote replication and all volumes in the local replication with which the P-VOL shares the same volume.
Recovering from failure if one volume capacity in GAD pair cannot be expanded
If only one volume capacity in a GAD pair can be expanded while the other cannot be expanded, the resync operation in the GAD pair cannot be performed after the expansion because the capacity of both of the volumes is not the same.
If a GAD pair is used with Universal Replicator, ShadowImage, or Thin Image and the capacity of the P-VOL and S-VOL in one of the software products fails to expand, it causes the invalid state in which the expansion of the entire volume capacity is not complete. Perform the following operation to recover from the invalid state.
For more information about the I/O operations of the P-VOL and S-VOL in a GAD pair if the resync operation in a GAD pair cannot be performed, see I/O Modes.
Procedure
Verify that all of the prerequisites are met.
If the prerequisites are not met, proceed to step 4.Verify that both the P-VOL and S-VOL in a GAD pair meet the prerequisites for expanding the DP-VOL capacity.
See the prerequisites for Expanding DP-VOL capacity used in GAD pair.
Verify that all of the volumes meet the prerequisites for expanding the capacity in each of the software products.
If a GAD pair is used with UR, SI, or HTI, the capacity of the P-VOLs and S-VOLs in all of the used software products must be expanded.
When the prerequisites for expanding the DP-VOL capacity are met, retry the expansion and make sure that the capacity of the P-VOL and S-VOL in the GAD pair is the same.
If there is not enough free space, increase the free space, and then expand the DP-VOL capacity.
If a GAD pair is used with UR, SI, or HTI, retry the expansion and make sure that the capacity of the P-VOL and S-VOL in each of the software products is the same.
- If you can expand the capacity, go to Step 3.
- If you cannot expand the capacity, go to Step 4.
Resynchronize the GAD pair.
When a GAD pair is used with UR, resynchronize the UR pair. As a result, the recovery procedure from the failure is complete.Delete the GAD pair, expand the volume capacity in the SMPL status, and then re-create the GAD pair.
- If you want to restore to the conditions before the expansion due to a capacity input error and others, delete the GAD pair, and then recreate the LDEV with the correct capacity. After that, recreate the GAD pair.
- If a GAD pair is used with UR, Delete all
pairs, expand the volume capacity in the SMPL status, and then restore the
configurations by following the procedure for creating the
configurations.
If you want to restore to the conditions before the expansion due to a capacity input error and others, delete all pairs, and then recreate the LDEV with the correct capacity. After that, restore the configurations by following the procedure for creating the configurations.
- If a GAD pair is used with SI or HTI, and
the capacity of the P-VOL and S-VOL in a SI pair or HTI
pair is not the same, you can only delete the SI pair or HTI pair. You can read the S-VOL data
in the SI pair or HTI pair before the expansion. If you
want to use the data before the expansion, delete all pairs after the data
reading is complete. Expand the volume capacity in the SMPL status, and then
restore the configurations by following the procedure for creating the
configurations.
If you want to restore to the conditions before the expansion due to a capacity input error and others, delete all pairs, and then recreate the LDEV with the correct capacity. After that, restore the configurations.
Recovering from failure if one volume capacity in GAD pair has been expanded
You can recover from a failure if only one volume capacity in a GAD pair can be expanded, while a failure occurs on the other volume before the expansion.
Procedure
Start to recover from the failure using one of the global-active device disaster recovery procedures.
Depending on the operations during the recovery procedure, one or more of the following actions are required:- If the operation is resynchronizing a GAD pair
The resynch operation cannot be performed because the capacity of the P-VOL and S-VOL in the GAD pair is not the same.
Complete Steps 2 and 3 before the GAD pair resynch operation.
- If the operation is,
- Deleting a GAD pair
- Recovering from a volume failure
- Re-creating a GAD pair
The GAD pair cannot be created because the capacity of the target P-VOL and S-VOL is not the same when re-creating the GAD pair.
Complete Steps 2 and 3 before the GAD pair resynch operation.
- If the operation is,
- Deleting a GAD pair
- Deleting a volume
- Creating a volume
- Re-creating a GAD pair
The GAD pair cannot be created because the capacity of the target P-VOL and S-VOL is not the same when re-creating the GAD pair. Make sure that the capacity of the P-VOL and S-VOL is the same when creating the volumes.
Skip Steps 2 and 3.
- If the operation is resynchronizing a GAD pair
Expand the capacity of volumes whose capacity is not expanded, and then make sure that the capacity of the P-VOL and S-VOL in the GAD pair is the same.
When a GAD pair is used with other software, the capacity of the P-VOLs and S-VOLs in all of the used software products must be expanded. For more information about the order in which the volume capacity is expanded, see Expansion order policy when GAD pair shares the same volumes with other softwareRestart the resync operation and the creation of the GAD pair by following one of the global-active device disaster recovery procedures.