Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Universal Replicator overview

With Hitachi Universal Replicator (UR) you can perform asynchronous replication of data to one or more remote sites.

About Universal Replicator

Hitachi Universal Replicator (UR) presents a solution to avoid cases when a data center is affected by a disaster that stops operations for a long period of time. In the Universal Replicator system, a secondary storage system is located at a remote site from the primary storage system at the main data center, and the data on the primary volumes (P-VOLs) at the primary site is copied to the secondary volumes (S-VOLs) at the remote site asynchronously from the host write operations to the P-VOLs. Journal data is created synchronously with the updates to the P-VOL to provide a copy of the data written to the P-VOL. The journal data is managed at the primary and secondary sites to ensure the consistency of the primary and secondary volumes.

The redundancy provided by the RAID configuration (for example, RAID1 or RAID5) enables recovery from a P-VOL read failure. The primary storage system never reads the Universal Replicator S-VOL for data recovery.

Replication operations

Remote replication for a UR pair is accomplished using the master journal volume on the primary storage system and the restore journal volume on the secondary storage system. As shown in the following figure, the P-VOL data and subsequent updates are transferred to the S-VOL by obtain journal, read journal, and restore journal operations involving the master and restore journal volumes.

GUID-E3793BEB-D88F-45A1-B4F4-5469AC7290DC-low.png
Note
  • Universal Replicator does not have a feature to copy the data from one P-VOL to multiple S-VOLs or to copy the data from multiple P-VOLs to one S-VOL.
  • If the primary system fails reading a P-VOL, the redundancy of RAID-1, RAID-5, or RAID-6 enables the recovery from the failure. The primary system never reads the S-VOL for data recovery.
  • VSP G130 is available only in select markets. Contact your local sales representative for availability.

Obtain journal

Obtain journal operations are performed when the primary storage system writes journal data to the master journal volume. The journal data consists of the P-VOL data to be copied to the S-VOL as well as update sequence information and other metadata. When a pair is created and the P-VOL data is copied for the first time, the journal data copied to the master journal volume is called the base journal data. When the host writes updated data to the P-VOL, normal journal data is copied to the master journal volume. If a P-VOL write operation fails, the primary storage system does not create the journal data.

  • The primary storage system assigns write-sequence numbers to the P-VOL data copied to the master journal volume.
  • The write-sequence numbers and other metadata attached to the journal data ensure the consistency of the data in the P-VOL and S-VOL.

Journal copy

Journal copy operations are performed when journal data is copied from the master journal volume to the restore journal volume on the secondary storage system.

  • When the secondary storage system issues the read-journal command to the primary storage system, the primary storage system transfers the journal data stored in the master journal volume to the restore journal volume in the secondary storage system.
  • Journal copy operations occur on a continual basis unless there is no journal data in the master journal volume. The request for data from the secondary storage system is repeated as soon as the previous read operation is completed.

The UR journal copy operations "pull" data from the primary storage system to the secondary storage system. Journal data is copied to the secondary site only when the secondary storage system issues a read-journal command, with the primary storage system operating as a resource controlled by the secondary storage system's transaction processor.

Restore journal

Restore journal operations are performed when the secondary storage system writes journal data in the restore journal volume to the S-VOL.

  • Journal data in the restore journal volume is copied to the S-VOL according to the write-sequence numbers to ensure data consistency.
  • When a restore journal operation to the S-VOL is completed, the journal data in the restore journal volume is discarded, and the journal data in the master journal volume is also discarded.

The primary storage system discards journal data in the master journal volume when it receives the sequence number from the restore journal, which is attached to the read journal command from the secondary storage system.

System components

The following figure shows the Universal Replicator system components.

GUID-754E4F04-7BC0-45AF-9673-EC1E36F864B4-low.png

The local storage system is a storage system connected to the management client. The remote storage system is a storage system connected to the local storage system. Unless otherwise stated, this document gives explanation on the premise that the local storage system is on the primary site, and the remote storage system is on the secondary site.

UR system components
  • The storage systems at the primary and secondary sites. The primary storage system contains the P-VOLs and master journal volumes, and the secondary storage system contains the S-VOLs and restore journal volumes.
    • The master journal consists of the primary volumes and master journal volumes.
    • The restore journal consists of the secondary volumes and restore journal volumes.
  • The data path connections between the systems. The primary and secondary storage systems are connected usingdedicated Fibre Channel or iSCSI data paths. Data paths are routed from the ports on the primary storage system to the ports on the secondary storage system, and from the secondary storage system to the primary storage system.
  • The UR software on both the primary storage system and the secondary storage system.
  • The hosts connected to the primary and secondary storage systems.
  • The interface tools for configuring and performing UR operations:
    • Device Manager - Storage Navigator graphical user interface (GUI)
    • Command Control Interface command-line interface (CCI)

Storage systems

Universal Replicator operations involve two storage systems. One of the systems is located at the primary site and one is located at the secondary site.

The primary storage system consists of the main control unit and service processor (SVP). The secondary storage system consists of the remote control unit and its SVP.

  • Each storage system can function simultaneously as a primary and secondary storage system.
  • The primary storage system communicates with the secondary storage system over dedicated Fibre Channel or iSCSI remote copy connections.
  • The primary storage system controls the P-VOL and the following operations:
    • Host I/Os to the P-VOL.
    • P-VOL data copy to the master journal.
  • The secondary storage system controls the S-VOL and the following operations:
    • Initial copy and update copy between the P-VOL and the restore journal.
    • Journal commands to the primary storage system.
    • Journal data copy from the master journal to the restore journal.
    • Restore journal data copy to the S-VOL.
    • Pair status management and configuration (for example, rejecting write I/Os to the S-VOLs).

Pair volumes

The P-VOL contains the original data, and the S-VOL is the copy of the P-VOL. The two volumes can be paired, split, resynchronized, and released. When synchronized, the volumes are paired. When the pair volumes are split, updates to the P-VOL are not copied to the S-VOL. When the pair volumes are resynchronized, changed P-VOL data is copied to the S-VOL. If a disaster occurs, production operations can be transferred to the S-VOL. When the primary site is functional again, operations can be transferred and data can be copied back to the P-VOL.

The P-VOL remains available to the host for read and write I/O operations. The secondary storage system rejects write I/Os for the S-VOL by way of command rejection or intervention request, unless the S-VOL write option (Secondary Volume Write) is enabled. If the S-VOL write option is enabled, write I/O is allowed to the S-VOL while the pair is split, and the secondary storage system keeps track of differential data on the S-VOL for use when the pair is resynchronized.

Journal volumes

For Universal Replicator operations, journal volumes are required on the primary and secondary storage systems.

  • Updates to the P-VOL are copied to the master journal volume in the primary storage system. See the illustration in Journals.
  • Master journal data is copied to the restore journal volume on the secondary storage system.
  • Journal volumes can have different volume sizes and different RAID configurations.
  • You can register two journal volumes in a journal in the primary storage system and in the secondary storage system, but the second journal volume becomes a reserve journal volume, which is not used for normal operations. Use one journal volume in each system.

    The reserve journal volume is used for restoring a disk in a pool or pool volume to which the journal volume belongs. For details, see Restoring a pair by adding a reserve journal volume.

For information on planning journal volumes, see Sizing journal volumes .

Journals

Journals enable you to manage data consistency between multiple P-VOLs and S-VOLs. A journal is a group of one or more data volumes and the related journal volume. Like consistency groups, you can use journals to create multiple pairs and to split, resynchronize, and release multiple pairs. Journals are required on the primary and secondary storage systems.

Each data volume and its associated journal volume reside in the same journal.

  • The master journal contains master journal volumes and is associated with the P-VOL.
  • The restore journal contains restore journal volumes and is associated with the S-VOL

Each pair relationship between journals is called a mirror. A mirror ID identifies a pair relationship between journals. When the pair is created, it is assigned a mirror ID.

Mirror

The relationship of a master journal and a restore journal is called a mirror.

A mirror is in Active status when it contains only pairs in COPY status (those not synchronized, with initial copy operation in progress) and pairs in PAIR status (those synchronized, with initial copy operation completed). Usually pairs are in PAIR status and thus mirrors are in Active status.

The mirror status changes when an operation is executed to a mirror. Be sure to check the mirror status to confirm that the operation is completed. The following are a few examples of the mirror statuses. For a complete list of mirror statuses, see Device Manager - Storage Navigator mirror status definitions.

  • Initial Data volumes are not registered in the mirror, or deleting the mirror is completed.
  • Active The mirror has only pairs that are in COPY or PAIR status.
  • Stopped An operation for splitting or deleting the mirror is finished.

The mirror status can be checked only by HDvM - SN or CCI.

TipUniversal Replicator assumes that pair operations are performed in units of mirrors. Be sure to perform pair split and resync operations for each mirror. Requests to perform a split or resync operation for each pair might be rejected.
GUID-63F9388F-DAC2-487F-80C4-CE425F36074F-low.png

Split mirror operations

When you split a mirror, all the pairs in the mirror are split and copy operations of data from the master journal to the restore journal stops. To split a mirror, you must place the mirror in Active status. When mirror splitting is completed, the mirror status becomes Stopped.

Updated data is not reflected to the S-VOL while the pair is split, but only later when the pair is resynchronized. To resynchronize all the pairs in the mirror, resynchronize the mirror itself.

You can select whether or not to enable the S-VOL write option when you split a mirror. If the Secondary Volume Write option is enabled, the host can write the data to the S-VOL while the pair is split. You split a mirror by selecting Secondary Volume Write in HDvM - SN or by the pairsplit -rw command of CCI.

GUID-D53AEADA-C4B7-49FF-B1E5-6D56891BD543-low.png

You can also split a mirror after synchronizing the P-VOL and S-VOL. In that case, select Flush in Split Mode. This allows you to reflect the updated data to the S-VOL when the pair is split. When the secondary storage system accepts the pair split, all the journal data that has been held for the pair is written to the S-VOL. If no journal data (update data) comes to the pair for a fixed period of time, the pair status changes to PSUS. When all pairs in the journal are placed in the PSUS status, volume copying is complete, and the mirror status becomes Stopped.

TipTo create a complete copy of the data volumes in the mirror, you need to stop I/O operations from the host. Creating an instantaneous copy on demand and a copy created in such a way are called point-in-time copy.

Resync mirror operations

When you resynchronize a mirror, data copying from the P-VOL to S-VOL in all pairs in the mirror restarts. To resynchronize a mirror, make sure that the mirror status is Stopped.

NoteYou should resynchronize mirrors while I/O load is low. This is especially important when different types of pairs are intermixed in the same consistency group. Performing a mirror resync operation on a consistency group with different pair types during high I/O load might result in a pair resync failure and pair suspension (PSUE status).

Delete mirror operations

When you delete a mirror, all the pairs in the mirror are deleted, and data copying from the master journal to the restore journal stops.

Specify one of the following modes when deleting a mirror.

  • Normal: The mirror is deleted only when the primary storage system can change the mirror status to Initial.

    In CCI, this operation can be executed by the pairsplit -r command.

  • Force: The mirror is forcibly deleted even when the primary storage system cannot communicate with the secondary storage system.

If the mirror status does not change to Initial after 5 or more minutes from when you start an operation to delete a mirror in Force mode, restart the delete operation in Force mode to ensure all pairs in the mirror are deleted.

After each delete operation in Force mode, wait at least five minutes before you create pairs in the same journal. Otherwise the paircreate operation might fail.

Data path

The physical transmission link between the primary and secondary storage systems is called the data path.

Universal Replicator commands and data are transmitted through the Fibre Channel or iSCSI data path and switches. The data path is the cabling that connects the primary and secondary storage systems. Universal Replicator requires paths in both directions. More specifically, it requires paths with Initiator ports in the primary storage system connected to RCU Target ports in the secondary storage system, and paths with Initiator Ports in the secondary storage system connected to RCU Target ports in the primary storage system.

One data path connection in each direction is required. It is recommended that you use two or more independent connections to provide hardware redundancy. A maximum of eight paths in each direction can be used.

For details, see Planning the data path.

Consistency groups and journals

A consistency group is a group of pairs set in primary storage systems and secondary storage systems. By specifying a CTG ID with a BCM or CCI command, you can perform the same operation on all pairs belonging to the CTG in a batch while keeping data consistency.

A master journal is a group of volumes associated with primary storage system. A restore journal is a group of volumes associated with the secondary storage system. If you set each journal as a consistency group and specify a CTG ID, you can use BCM or CCI commands to operate master journal volumes in a master journal, or restore journal volumes in a restore journal in a batch. That is, you can operate volumes for each consistency group while retaining the data update order between the P-VOL and the S-VOL.

Journals are used in Universal Replicator to guarantee data consistency across multiple pairs. Consistency groups are used in other replication software for the same purpose. The same CTG IDs can be used for Universal Replicator and BCM or CCI. Therefore, when you use BCM or CCI, you can perform UR pair tasks using CTG IDs. For Universal Replicator, consistency group IDs specified by BCM or CCI are assigned to journal IDs. If you use Universal Replicator, it is recommended that you specify the same number for a CTG ID and a journal ID. The journal group number can be different in the primary and secondary storage systems.

The number of consistency groups that can be created depends on the storage system model. Following is the maximum number of consistency groups that can be created for P-VOL's consistency group and S-VOL's consistency group combined. The numbers in parenthesis indicate the range of CTG IDs.

  • VSP E590, VSP E790: 128 (0 to 127)
  • VSP E990: 256 (0 to 255)
  • VSP G350, VSP G370, VSP G700, VSP F350, VSP F370, VSP F700: 128 (0 to 127)
  • VSP G900, VSP F900: 256 (0 to 255)
  • VSP 5000 series: 256 (0 to 255)
GUID-B1F51DA2-1965-4DF7-AC1C-23C919C89B50-low.png
Caution
  • Use the same CTG ID for the P-VOL and S-VOL to be paired.
  • Do not register a journal to multiple CCI CTGs. If you do, time stamps are issued from multiple CCIs, and data consistency in a consistency group cannot be guaranteed.

Device Manager - Storage Navigator

Device Manager - Storage Navigator provides a GUI and command line interface for accessing and managing the storage system, including Universal Replicator.

Device Manager - Storage Navigator communicates with the SVP of each system over defined TCP/IP connections.

The Device Manager - Storage Navigator GUI displays "Local Storage System" for the storage system that you have accessed on the HDvM - SN server or for the storage system and "Remote Storage System" for storage systems that are connected to the storage system you accessed. Therefore, when you access a secondary storage system (contains S-VOLs) at a secondary site, the information for this secondary storage system is displayed under "Local Storage System", and the information for the primary storage system connected to this secondary storage system is displayed under "Remote Storage System".

You must have the Storage Administrator (Remote Copy) roll to perform the Universal Replicator operations. If you do not have the roll, you can refer to the Universal Replicator information only.

When using Virtual storage machine volumes, the LDEV IDs, serial numbers, and virtual information of VSP 5000 series are displayed in Device Manager - Storage Navigator. However, you cannot add a remote connection by specifying the serial number of the virtual storage machine or create a pair by specifying the virtual LDEV ID. You must specify the LDEV IDs and serial numbers of VSP 5000 series to perform these Universal Replicator operations with Device Manager - Storage Navigator.

NoteWhen you create a UR pair by using Device Manager - Storage Navigator, use Device Manager - Storage Navigator to perform the subsequent operations.

Command Control Interface (CCI)

CCI provides a command line interface for accessing and managing the storage system, including Universal Replicator. You can perform the same Universal Replicator operations with CCI as you can with Device Manager - Storage Navigator. In addition, you can automate pair operations using scripts.

When using Virtual storage machine volumes, you can specify the LDEV IDs, serial numbers, and virtural information of VSP 5000 series in the configuration file with CCI. However, when primary storage system is VSP, specify the information on VSP 5000 series in the configuration file.

NoteWhen you create a UR pair by using CCI, use CCI to perform the subsequent operations.

Copy operations

The copy operations for Universal Replicator include initial copy, update copy, and the underlying operations that are executed, such as journal processing and differential data management.

Initial copy operation

When the initial copy operation is executed, all data in the P-VOL is copied in sequence directly to the S-VOL without use of the primary journal volumes. The copied data in the initial copy operation is referred to as base journal data, even though the primary journal volumes are not used.

  • When you create or resynchronize two or more pairs within the same journal, the base journal data is copied to the respective S-VOLs, one at a time. This extends the time required for all the operations to be completed.
  • As an option, you can create a pair and specify that no data is copied during the initial copy operation by specifying None for Initial Copy Type. This option should be specified only when the data in the P-VOL and S-VOL is already identical, or when the P-VOL and S-VOL do not yet contain any data. Use of this option establishes the pair quickly without the delay of the initial copy operation. Once the pair is established (pair status is PAIR), update copy operations are performed to copy all P-VOL updates to the S-VOL.
  • A Universal Replicator pair can also be created using a TrueCopy initial copy operation. This reduces the time to complete the initial copy operation. For details, see Planning pair volumes .

Update copy operation

When a host performs a write I/O operation to a P-VOL of a volume pair, an update copy operation is performed. The updated data is copied to the master journal on the primary storage system along with metadata that includes sequence and other consistency information. The updated data and its metadata are called the journal data. When the secondary storage system issues the read-journal command, which occurs independently of host I/O activity, all journal data in the master journal is sent to the restore journal on the secondary storage system. When the P-VOL updates are successfully duplicated on the S-VOL, the journal data on the primary and secondary storage systems is discarded.

Journal data is transferred using special I/O operations called RIO (remote I/O) that are initiated by the secondary storage system. RIO provides the most efficient type of data transfer. The channel extenders must be capable of supporting RIO.

If an update copy operation fails, the secondary storage system suspends the affected pair or all pairs in the journal, depending on the type of failure. The suspended pair or journal returns to Paired status when the primary and secondary storage systems are resynchronized.

NoteIf the primary storage system receives a request for writing to the P-VOL (update I/O) from a host during initial copy, the primary storage system suppresses initial copy to prioritize the update I/O. However, when a copy pace for mirror option is Faster in HDvM - SN, or when 4 or a larger value is specified by the -copy_size option of the raidcom modify journal command in CCI, the primary storage system does not suppress initial copy. When specifying Faster for a copy pace, be sure not to cause an update I/O during initial copy. If an update I/O occurs, data volume pairs might be split (suspended).

Read and write I/O during remote copy

The primary storage system reads from the P-VOL when it receives a read I/O command. If the read fails, the redundancy provided by RAID 1, RAID 5, or RAID 6 technology recovers the failure. The primary storage system does not read the S-VOL for recovery.

When a primary storage system receives a write I/O command for a P-VOL in PAIR status, the system performs the write operation and performs the update copy operation. The write operation completes independently of the update copy operations on the S-VOL.

The secondary storage system updates the S-VOL according to the write sequence number in the journal data. This maintains data consistency between P-VOL and S-VOL.

If the P-VOL write operation fails, the primary storage system reports a unit check and does not create the journal data for this operation. As mentioned, if the update copy operation fails, the secondary storage system suspends either the affected pair or all Universal Replicator pairs in the journal, depending on the type of failure. When the suspended pair or journal is resynchronized, the primary and secondary storage systems negotiate the resynchronization of the pairs.

In PAIR status, S-VOLs are enabled for read I/O, but disabled for write I/O. The S-VOL write option can be enabled only when you split the pair from the primary storage system.

To reduce the overhead associated with remote copy activities and to maximize rate of data transfer, the storage system uses a special write command for initial and update copy operations. This command transfers the control parameters and the fixed-block architecture (FBA) format data for consecutive updated records in a track using a single write operation. It eliminates the overhead required for performing FBA-to-count-key-data (CKD) and CKD-to-FBA conversions.

NoteIf the I/O data length for the UR P-VOL exceeds 21 MB, the UR pair might be split (suspended). If this happens, adjust the I/O block size for each host platform. For example, if the host platform is Linux, you set 512 as the value for the max_sectors_kb parameter.

Differential data management

Differential data is the data that is changed in the P-VOL and S-VOL (if permitted) while a pair is split or suspended. This changed data is stored in a track bitmap and has not been written to the S-VOL. When the pair is resynchronized, the primary storage system merges the P-VOL and S-VOL bitmaps and copies the differential data to the S-VOL.

The required number of bitmap areas is based on the number of volumes being replicated and the size of the volumes. This affects the maximum number of pairs that can be created in the system.

When you use a DP-VOL in a pair under one of the following conditions, the differential data is managed by the pool to which the UR pair volume is related:

  • You create a UR pair by using a DP-VOL that is larger than 4,194,304 MB (8,589,934,592 blocks). In this case, additional pool capacity (up to 4 pages, depending on the software configuration) is required for each increase of user data size by 4 TB (4,123,168,604,160 bytes).
  • You resynchronize a UR pair that contains a DP-VOL of any size with Advanced System Settings No. 5 and No. 6 enabled.

Advanced System Setting No. 6: Manage differential bitmaps in DP pool at pair create operations for 4 TB or less TC/TCMF/UR/URMF/GAD pairs.

When enabled, the differential data is maintained in a pool with which a DP-VOL that has the user capacity (up to 4,194,304 MB) is linked when a new TC/TCMF, UR/URMF, or GAD pair is created using the DP-VOL. Note that differential data is maintained, regardless of this setting, in a pool linked with the DP-VOL that has the user capacity (greater than 4,194,304 MB) when a new TC/TCMF, UR/URMF, or GAD pair using the DP-VOL is created.

Advanced System Setting No. 5: Manage differential bitmaps in DP pool at pair create and resync operations for 4 TB or less TC/TCMF/UR/URMF/GAD pairs.

When enabled, the differential data is maintained in a pool with which a DP-VOL that has the user capacity (up to 4,194,304 MB) is linked when a new TC/TCMF, UR/URMF, or GAD pair is created using the DP-VOL, or resynchronized. Note that differential data is maintained, regardless of this setting, in a pool linked with the DP-VOL that has the user capacity (greater than 4,194,304 MB) when a new TC/TCMF, UR/URMF, or GAD pair using the DP-VOL is created.

When these Advanced System Settings (6 and 5) are enabled, the differential data is managed by the pool to which the pair volume is related even if the volume is smaller than 4 TB. This enables the total capacity of the pair volumes to be increased over 1.8 PB. If you want the differential data to be managed by the shared memory again after you create pairs whose total capacity is larger than 1.8 PB, you must reduce the total capacity of the pairs that are smaller than 4 TB to 1.8 PB by deleting some pairs. When the differential data is managed by the shared memory, the maximum total capacity of all remote copy pairs (including TC/TCz, UR/URz, and GAD) is 1.8 PB. For example, if the total capacity of the UR pairs is already 1.8 PB, you cannot create any TC pairs.

NoteAfter you create a pair whose differential data is managed in a pool, data management might fail due to insufficient pool capacity. In this case, all of the P-VOL data is copied to the S-VOL in units of tracks when the pair is resynchronized.

S-VOL write option

When splitting a pair, you can set the S-VOL write option (Secondary Volume Write) to allow write I/O to the S-VOL. The S-VOL write option can be specified only when you are connected to the primary storage system, and it applies only to the specified pairs. When you resynchronize a split pair whose S-VOL is write-enabled, the secondary storage system sends the S-VOL track bitmap to the primary storage system, which merges the P-VOL and S-VOL bitmaps to determine the tracks that need to be updated. This ensures proper resynchronization of the pair based on the P-VOL data.

Pair status

The pair status of a volume indicates the synchronization status of the pair to which it belongs. When a pair operation is performed, the pair status of the volumes changes. The pair status of a volume determines the operations that can be performed on the pair. For example, you can only split a pair when the pair status of the volumes is COPY or PAIR.

During normal operations you need to monitor the pair status to ensure that you can perform the desired operations and to verify that operations completed successfully.

Pair status definitions

Device Manager - Storage Navigator displays both the Device Manager - Storage Navigator pair status name and the CCI pair status name, except when the names are the same. When they are the same, the CCI status is not displayed.

The following table lists and describes the pair status names.

HDvM - SN status

CCI status

Description

P-VOL access

S-VOL access

(Unpaired- no HDvM - SN status)

SMPL

The volume is not assigned to a pair and is not in a journal.

Read/Write

Read/Write

GUID-F8E99289-85DC-4357-897A-44672AB815D9-low.png COPY

COPY

The initial copy operation is in progress.

Read/Write

Read Only

GUID-EC986349-8484-4945-8EA5-91705FA7CCCF-low.png PAIR

PAIR

The pair is synchronized. Updates to the P-VOL are duplicated on the S-VOL.

Read/Write

Read Only

HDvM - SN displays this status as GUID-EC986349-8484-4945-8EA5-91705FA7CCCF-low.pngPAIR

PFUL

If data in the journal volume exceeds the threshold (80 %), pair status changes to PFUL. The pair is not suspended; the copy operation continues.

If the journal option, Inflow Control, is set to Enable in HDvM - SN, or the modify journal command is executed in CCI when status changes to PFUL, host I/O is delayed because update I/O to the journal volume is delayed.

Read/Write

Read Only

GUID-E596330E-06D9-43A9-AD3B-01530E3B72A3-low.png PSUS

PSUS

SSUS

The pair has been split by the user and the P-VOL and S-VOL are no longer synchronized.

  • The primary and secondary storage systems keep track of journal data discarded during the split operation.
  • When the operation is performed on the primary storage system, the status of both the P-VOL and S-VOL changes to PSUS.
  • When the operation is performed on the secondary storage system, the status of the S-VOL changes to PSUS, and the primary storage system detects this and changes the P-VOL status to PSUS.

SSUS is the S-VOL’s split status name.

See also UR PSUS types and behaviors.

Read/Write

Read Only, but if you enable the Secondary Volume Write option then split the pair from the primary site, it is Read/Write (Default is Disable).

GUID-4005AB82-28BD-4F76-9932-BEB038E51CAA-low.png PSUE

PSUE

The pair is suspended by the system due to an error; it is not synchronized.

  • The primary and secondary storage systems keep track of journal data discarded during the suspension operation.
  • The primary storage system keeps track of P-VOL tracks that are updated while the pair is suspended.
  • When a UR suspension condition is detected, the primary storage system changes the P-VOL and S-VOL status to PSUE. If the secondary storage system detects the condition, it changes the S-VOL status to PSUE, and the primary storage system detects this and changes the P-VOL status to PSUE.

See also UR PSUS types and behaviors.

HDvM - SN: Read/Write CCI: Read/Write if no error in P-VOL

Read Only

GUID-4005AB82-28BD-4F76-9932-BEB038E51CAA-low.pngSuspending

(No equivalent CCI status)

The pair is not synchronized and in transition from the PAIR or COPY status to PSUS or PSUE status.

Read/Write

Read Only

GUID-D593FB03-7C06-4C64-B561-3D2DAF99569A-low.pngDeleting

(No equivalent CCI status)

The pair relationship is being deleted. When the operation is complete, the status changes to unpaired.

Read/Write

Read Only

GUID-E596330E-06D9-43A9-AD3B-01530E3B72A3-low.png HOLD

(No equivalent CCI status)

  • When used with TC, this status indicates that the creation of a delta resync pair is completed. Delta resync might not be executed depending on the internal process status even if the creation of the delta resync pair is completed. When used with GAD, this status indicates that delta resync can be executed. If the P-VOL status is HOLD, the updated data of the TC or GAD S-VOL is stored in the master journal volume.
  • Operations allowed:

    - Delta resync

    - Pair-delete

    - Change pair options

Read/Write1

Read/Write2

GUID-E596330E-06D9-43A9-AD3B-01530E3B72A3-low.png HOLDING 3

(No equivalent CCI status)

When used with TC, this status indicates that the pair status is changing to HOLD so that the delta resync pair can be created.

Operations allowed on pairs in HOLDING status:

  • Delta resync operation
  • Pair release
  • Change pair options

If the delta resync operation is run on a pair in HOLDING status, the operation could fail. If you specify Entire Copy for the Delta Resync Failure option (Change Journal Option dialog box), differential data is copied to an S-VOL.

When used with GAD, preparing for delta resync. Delta resync execution will fail if the pair status is HOLDING. Specification of Entire Copy is invalid for the Delta Resync Failure option. Resync the UR pair in case of delta resync failure.

Read/Write1

-

GUID-4005AB82-28BD-4F76-9932-BEB038E51CAA-low.png HLDE

(No equivalent CCI status)

  • An error occurred on a pair in HOLD status.
  • When P-VOL status is HLDE, incoming write data for the TC or GAD S-VOL is not stored in the master journal volume.
  • Operations allowed on pairs in HLDE status:

    - Recover pair status to standby (HOLD)

    - Pair delete

    - Change pair options

Read/Write1

-

(No equivalent HDvM - SN status)

PFUS

This status results when the journal volume becomes full and suspends. Pair status changes from COPY, PAIR or PFUL to PFUS. The UR pair is suspended and the copy operation stops. Make sure to review configuration of the remote path and journal volume.

  • If a Dynamic Provisioning virtual volume (DP-VOL) is used as the UR S-VOL, and the capacity of a pool-VOL is nearly full, UR status becomes PFUS and the pair is suspended.
  • If the journal option Inflow Control is set to Yes when status changes to PFUS, the pair is not suspended for the time set in the Data Overflow Watch field in HDvM - SN, or for the time specified by the raidcom modify journal command with the -data_overflow_watch option in CCI, even when the journal volume becomes full. During the time it takes for the journal data area to clear, the response to host I/O is delayed.

    See Changing options used by journals for details about Data Overflow Watch.

Read /Write

Read Only, unless the write option is enabled.

(No equivalent HDvM - SN status)

SSWS

After Takeover, SSWS is the status of the S-VOL. With this status, data can be written to the S-VOL.

The copy flow between P-VOL and S-VOL is reversed by running the CCI horctakeover command or pairsplit -RS. Therefore, the S-VOL is ready for host write operations. During disaster recovery, the host can write data to the S-VOL in SSWS status at the intermediate site or secondary site. However, in a 3DC configuration in which three UR sites are combined, the host cannot write data to the S-VOL if the volume is shared by the mirror that includes the UR delta resync pair.

Read Only

Read/Write

Notes:

  1. Access to the P-VOL depends on the status of the TC or GAD pair that shares the volume.
  2. The UR S-VOL in HOLD status uses two mirrors. In a 3DC configuration combined with TrueCopy, accessing the S-VOL depends on the status of the mirror that is not HOLD. In a 3DC configuration in which three UR sites are combined, the host cannot write data to the S-VOL regardless of the mirror status.
  3. HOLDING indicates either that differential data used for a delta resync operation does not exist or that the system cannot judge whether the delta resync can be performed without the differential data.

    When differential data does not exist in the secondary TC or GAD site, even after the S-VOL is updated, the differential data might be discarded due to the one of the following reasons:

    • The cache or shared memory was added to or removed from the system during maintenance work.
    • The system was powered off during maintenance work.
    • The UR, TC, or GAD pair was suspended and then resynchronized.

    After the system is back to normal, differential data is stored again in the TC S-VOL if you update the data in the primary site.

    The delta resync operation can be executed without differential data if the primary site has not received a data update or if data in the TC or GAD S-VOL and the UR S-VOL is the same.

    Even if the delta resync operation without differential data is executable, the pair status changes to HOLD even when the differential data exists if the conditions for discarding the differential data are satisfied. To change the status of the pair to HOLD, update the data in the primary storage system after the recovery from the condition of the differential data discarding.

    If you cannot connect to the secondary site of TC or GAD due to failures or disasters, the transition from HOLDING to HOLD is not performed correctly. When used with GAD, keep updating I/O from the server to GAD P-VOL or S-VOL for about two minutes.

Additional information on pair status

You should know what happens when you split or suspend a pair.

  • When a pair is split by the user or suspended by the system, the primary storage system notifies the hosts by issuing a service information message (SIM). If SNMP is installed and operational, this SIM results in an SNMP trap indicating the reason for suspension.
  • When you suspend a pair, the status changes to Suspending first, and then to PSUS. When you delete a pair, the status changes to Deleting first, and then to SMPL. However, Suspending and Deleting are not displayed as the pair status in CCI.
  • If you split a pair in Flush mode, it takes some time for the pair status to change to PSUS. To shorten the time, select Purge for Split Mode in HDvM - SN, or execute the pairsplit -P command in CCI before splitting a pair. If a pair is split in Flush mode, the status changes to Suspending first, and then it changes to PSUS when all journals in the master journal are restored to the restore journal. The time for a pair to change to PSUS can be calculated as follows (the actual time might vary depending on the internal processing status of storage systems):

    time-for-Suspending (sec.) = C x U / V

    where:

    • C is the total capacity (GB) of the master journal volume.
    • U is the usage rate of data (%) in the master journal volume.
    • V is the line speed (GB/sec) between the primary and the secondary storage systems.

    The total capacity (GB) can be checked in the Journals tab of the Journals window or using the raidcom get ldev command.

    The data usage rate can be checked in the Performance Monitor window in Performance Monitor or using the raidcom get journal command. For details about Performance Monitor, see the Performance Guide.

    When using CCI to monitor the time until the status changes to PSUS in Flush mode, specify a value equal to or greater than the calculated time (in seconds) for the -t option of the pairsplit command.

UR PSUS types and behaviors

The PSUS status, which indicates that a pair is split or suspended, can be set by the user or by the system from the primary or secondary storage system.

You can check the types of PSUS in Status of the View Pair Properties window. The following table lists and describes the PSUS status types.

PSUS type

Applies to

Description

S-VOL by Operator

P-VOL, S-VOL

  • The user split the pair from the primary or secondary storage system using the S-VOL-write option.
  • CCI displays this PSUS type as SSWS.

By MCU

S-VOL

  • The secondary storage system received a request from the primary storage system to split the pair.
  • The P-VOL PSUS type is S-VOL by Operator.
  • CCI displays this PSUS type as SSWS.

By RCU

P-VOL, S-VOL

  • The primary storage system suspended the pair after detecting an error condition on the secondary storage system.
  • The S-VOL suspend type is S-VOL Failure.
  • CCI displays this suspend type as PSUE.

Pairsplit-S to RCU

P-VOL

  • The primary storage system detected that the S-VOL is unpaired after the user released the pair from the secondary storage system.
  • The pair cannot be resynchronized.

JNL Cache Overflow

P-VOL, S-VOL

  • The pair was suspended because the journal volume was near capacity.
  • CCI displays this PSUS type as SSWS.
  • A pair can be split after the initial copy is complete.
  • A pair must be split to perform maintenance on the P-VOL, or to enable write-access to the S-VOL.
  • After the status changes to PSUS, the primary storage system performs the following tasks:
    • Stops journal-obtain operations
    • Continues to accept write I/Os from hosts to the P-VOL
    • Keeps track of the updated P-VOL tracks
  • When the UR pair is split by the user, the primary storage system and secondary storage system will either execute suspended update copy, synchronize the pair then split or delete suspended update copy then split the pair based on the user specified Split Mode option (Flush or Purge). The journal data that is deleted during UR pair split is stored in the primary and secondary storage systems. When the pair status becomes PSUS, the primary storage system will stop obtaining the journal for the pair and accept write I/O for split UR P-VOL and record the updated P-VOL track during pair split.
  • If you enable the S-VOL write option (Secondary Volume Write) when splitting the pair, the secondary storage system keeps track of updated S-VOL tracks. When the pair is resynchronized, the secondary storage system sends the S-VOL track bitmap to the primary storage system, which then merges P-VOL and S-VOL bitmaps to synchronize the tracks.
  • A split or suspended S-VOL has a separate consistency status, which indicates the S-VOL’s update sequence consistency with respect to the other S-VOLs in the associated journal. The consistency status is displayed only on the secondary storage system. The following table lists and describes the S-VOL consistency statuses.
S-VOL consistency statuses

Consistency status

Description

Volume

  • Only the current pair was split or suspended.
  • Update sequence consistency between this S-VOL and other S-VOLs in the associated journal is not ensured.
  • This S-VOL cannot be used for disaster recovery at the secondary site.
  • This status is indicated when:
    • The pair is split by the user using the Split Pairs window (Suspend Range-Volume (LU) pairsplit option).
    • The pair is suspended due to a failure that did not affect the entire journal.

Mirror

  • The pair was split or suspended along with the other pairs in the associated mirror.
  • Update sequence consistency between this S-VOL and other S-VOLs in this mirror is ensured.
  • This S-VOL can be used for disaster recovery on the secondary storage system.
  • This status is indicated when:
    • The mirror is split using the Split Mirrors window (or by specifying mirror of the CCI pairsplit -r option).
    • All pairs in the associated mirror are suspended due to a failure that affected the entire group, for example, path failure.
    • One pair in the mirror was suspended due to a failure that did not affect the entire group.

UR PSUE types and behaviors

The PSUE status, which indicates that a pair is suspended, can be set only by the primary or secondary storage system. The primary storage system suspends a pair and changes its status to PSUE when it detects any of the following conditions:

  • The user has released the pair on the secondary storage system.
  • An error condition related to the S-VOL or an update copy operation.
  • The primary storage system is not able to communicate with the secondary storage system.
  • A Universal Replicator suspension condition is detected by the system.

The following table lists and describes the PSUE status types.

PSUE Type

Applies to

Description

S-VOL Failure

P-VOL, S-VOL

The primary storage system detected an error during communication with the secondary storage system or during update copying.

S-VOL PSUE type is usually PSUE-S-VOL Failure.

CCI might display this status as SSWS.

MCU IMPL

P-VOL, S-VOL

The primary storage system could not find valid control information in its nonvolatile memory during IMPL (initial microprogram load). This condition occurs if the primary storage system is without power for more than 48 hours (power failure and fully discharged backup batteries).

Initial Copy Failed

P-VOL, S-VOL

The pair was suspended before the initial copy operation was complete. The data on the S-VOL is not identical to the data on the P-VOL.

CCI might display this status as SSWS.

MCU P/S OFF

S-VOL

The primary storage system is powered off.

CCI displays this as SSUS.

When a pair is suspended, the primary storage system stops performing journal-obtain operations for the pair. However, the primary storage system continues the following operations:

  • Continues accepting write I/Os for the suspended P-VOL
  • Keeps track of the P-VOL cylinders/tracks that are updated
  • Keeps track of journal data discarded during the pair suspension. (Both primary and secondary storage systems do this.)

A split or suspended S-VOL has a separate consistency status that indicates its update sequence consistency with respect to the other S-VOLs in the associated journal. Consistency status displays on the secondary storage system only.

When the pair is resynchronized, the primary and secondary storage systems perform the following operations:

  • The secondary storage system sends the S-VOL bitmap to the primary storage system
  • The primary storage system merges the P-VOL and S-VOL bitmaps to synchronize the tracks

These actions ensure that all cylinders/tracks containing journal data discarded on the secondary storage system are resynchronized.

3DC configurations with three UR sites

You create a 3DC multi-target or 3DC cascade configuration using VSP 5100 or VSP 5500 by combining three Universal Replicator sites. VSP 5200 and VSP 5600 do not support a 3DC multi-target or 3DC cascade configuration by combining three Universal Replicator sites. A 3DC multi-target configuration involves one primary site and two secondary sites. A 3DC cascade configuration involves a primary site, an intermediate site, and a secondary site.

NoteA 3DC configuration with three Universal Replicator sites can be used only for configurations with VSP 5000 series (VSP 5100, VSP 5500) as the primary system and VSP G1x00, VSP F1500, or VSP G/F900 as the secondary system.

3DC multi-target configuration with three UR sites

With Universal Replicator, you can set up two secondary sites for one primary site.

The following illustration shows the 3DC multi-target configuration with three UR sites:

  • UR primary site
  • UR secondary site (copies data with a mirror whose ID is an arbitrary number (M in the illustration) from the primary site).
  • UR secondary site (copies data with a mirror whose ID is an arbitrary number (N in the illustration) from the primary site).
GUID-E25C0555-AB86-4174-BB79-A02D6AF30704-low.png
NoteIn a 3DC multi-target configuration using delta resync with three UR sites, both the P-VOL and S-VOL of a delta-resync pair are displayed as secondary volumes in the Remote Replication window (Pair Position column).

3DC multi-target configuration with three UR sites using delta resync

You should create a UR pair that connects the two secondary sites so that the remote copy system created with the host operation site and backup site can be created immediately in the event of a failure or disaster at the primary site. A UR pair that is created to make a triangle-shaped remote copy connection among the three sites is called a UR delta resync pair. By creating a UR delta resync pair in advance, you can transfer the copying operations from between the secondary sites back to from the primary to the secondary site in a short time when the failure is corrected and the primary site is brought back online.

The following illustration shows the 3DC multi-target configuration with three UR sites using delta resync:

GUID-1E9EE0E4-37E5-49CC-AD22-73EBC67245A5-low.png

3DC cascade configuration with three UR sites

With Universal Replicator, you can set up one intermediate site and one secondary site for one primary site. It is advisable that you create a UR pair that connects the primary and secondary sites so that the remote copying system that is created with the host operation site and backup site is configured immediately in the event of a failure or disaster at the intermediate site. A UR pair that is created to make a triangle-shaped remote copy connection among the three sites is called UR delta resync pair. By creating a UR delta resync pair in advance, you can transfer the copying operations from between the primary and secondary sites back to between the intermediate and secondary sites in a short time when the intermediate site failure is corrected and the intermediate site is brought back online.

The following illustration shows the 3DC cascade configuration with three UR sites:

  • UR primary site (copies data with a mirror whose ID is an arbitrary number (M in the illustration) to the intermediate site). Data can be copied with a mirror whose ID is an arbitrary number (P in the illustration) to the secondary site.
  • UR intermediate site (copies data with a mirror whose ID is an arbitrary number (M in the illustration) from the primary site and copies data with a mirror whose ID is an arbitrary number (N in the illustration) to the secondary site). Data can be copied with a mirror whose ID is an arbitrary number (P in the illustration) from the primary site.
  • UR secondary site (copies data with a mirror whose ID is an arbitrary number (N in the illustration) from the intermediate site). Data can be copied with a mirror whose ID is an arbitrary number (P in the illustration) from the primary site.
GUID-C406A23B-A7B9-4613-8799-F7A89196BD86-low.png