Skip to main content
Outside service Partner
Hitachi Vantara Knowledge

Geographically distributed data protection


An individual Hitachi Content Platform (HCP) system maintains a repository of objects in one or more namespaces. To ensure data availability and protect against data corruption, namespaces can be configured (by means of service plans) to store multiple copies of each object. However, storing multiple copies of objects on only one system does not provide protection against the unavailability or loss of that system.

You can create a configuration in which selected tenants and namespaces are maintained on two or more HCP systems and the objects in those namespaces are managed across those systems. This cross-system management helps ensure that data is well-protected against the unavailability or catastrophic failure of a system. Depending on the number of systems and how you configure the relationships between them, data may even be protected against the concurrent unavailability or loss of more than one system.

Typically, the systems involved in cross-system data management are in separate geographic locations and are connected by a high-speed wide area network. This arrangement provides geographically distributed data protection (called geo-protection). You have multiple options for implementing geo-protection, each with its own benefits and drawbacks.

This section of the Help provides an overview of how geo-protection works and how it supports business continuity and disaster recovery. For an introduction to HCP systems and how they work, see Introduction to Hitachi Content Platform.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

About geo-protection


Geo-protection is implemented by the HCP replication service. The replication service is responsible for keeping tenants and namespaces on two or more HCP systems in sync with each other. This process, also called replication, entails sending information from the system where the service is running to one or more other systems. The information sent includes tenant and namespace configuration information, metadata and data for newly created objects, metadata changes for existing objects, and instructions to perform delete operations.

The replication service works on the systems in a replication topology. A replication topology is a configuration of HCP systems wherein each system is related to each other system through a path of one or more replication relationships. The HCP system from which you’re viewing the replication topology is called the local system. Any other system in the replication topology is called a remote system.

You select the tenants to be replicated in a replication topology. For each tenant, the tenant administrator selects the namespaces to be replicated. If the tenant administrator has granted system-level users access to the tenant, you also have the ability to select namespaces for replication.

Geo-protection offers several benefits:

If a system in a replication topology becomes unavailable, a remote system can provide continued data availability. A system is unavailable if it is not running or is not reachable (for example, due to a network issues).

If a system in a replication topology suffers irreparable damage, a remote system can serve as a source for disaster recovery.

If the content verification or protection service discovers objects it cannot repair on a system in a replication topology, the service can use object data and metadata from a remote system to make the needed repairs. For more information on these services, see HCP services.

If an object cannot be read from a system in a replication topology (for example, because a node is unavailable), HCP can try to read it from a remote system. HCP tries to do this only if:

oThe namespace that contains the object is involved in replication.

oThe namespace has the read-from-remote-system feature enabled.

oThe object has already been replicated. Users can check object metadata to determine whether an object has been replicated.

oThe replication networks for the local and remote systems can both use the same IP mode (IPv4 or IPv6).

If a system in a replication topology is unavailable, requests to that system that come through the REST, S3 compatible, or HSwift namespace access protocol can be automatically serviced by a remote system without the client needing to modify the target URL. A remote system can service a request only if:

oThe namespace named in the URL is replicated to the remote system.

oThe namespace named in the URL is configured to accept requests redirected from other HCP systems.

oThe HCP systems involved use a shared DNS for system addressing.

oBoth the DNS and the unavailable system are configured to support DNS failover. (This condition is not a requirement if you’re using a load balancer that can target the requests to a remote system.)

If multiple HCP systems are widely separated geographically, each system may be able to provide faster data access for some applications than the other systems can, depending on where the applications are running.

If an enterprise has several satellite offices, an HCP system at a central facility can consolidate data from the HCP systems at those outlying locations.

For more information on:

Replication topologies, see Replication topologies

The replication service, see Replication service processing

The read-from-remote-system and service-by-remote-system features, see Changing replication options or Managing the Default Tenant and Namespace

Replication networks, see Selecting the network for replication

Configuring HCP in the DNS, see Configuring DNS for HCP

Configuring HCP to support DNS failover, see Managing DNS failover

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Protection types


HCP supports two types of geo-protection: whole-object protection and erasure-coded protection.

Whole-object protection

With whole-object protection, all the data for each object in a replicated namespace is maintained on each HCP system in a replication topology, except on systems where the object is on a metadata-only storage tier. In response to a client request for an object that's on a metadata-only storage tier on the target HCP system, HCP retrieves the object data from another system in the replication topology.

In case of system unavailability or loss, whole-object protection ensures continued data availability when at least one of the systems with the object data remains available. If an object is metadata-only on all the available systems, the object data is inaccessible.

The service plan associated with a namespace determines whether and when objects in that namespace move to a metadata-only storage tier. For information on storage tiers and service plans, see Storage administration.

Erasure-coded protection

With erasure-coded protection, the data for each object in a replicated namespace is subject to erasure coding. With erasure coding, the data is encoded and broken into multiple chunks that are then stored across multiple HCP systems. All but one chunk contains object data. The other chunk contains parity for the object data.

For the purpose of erasure coding, parts of multipart objects are each treated as an individual object. For information on multipart objects, see About multipart uploads.

To implement erasure-coded protection, you create an erasure coding topology. An erasure coding topology is a named replication topology in which object data is protected by erasure coding. The total number of chunks for an erasure-coded object is equal to the number of systems in the erasure coding topology, where each system stores one chunk.

An erasure-coded object can be read from any system in the erasure coding topology. In response to a client read request, HCP reconstructs the object data on the fly by retrieving the chunks stored on the other systems. If any one system in an erasure coding topology is unavailable, the data for an erasure-coded object can be completely reconstructed from the chunks on the remaining systems.

If more than one system in an erasure coding topology is unavailable, HCP cannot reconstruct the erasure-coded objects. In this case, HCP marks the erasure-coded objects as unavailable or irreparable, depending on the reasons why they cannot be reconstructed.

An erasure coding topology does not necessarily include all the HCP tenants that are replicated in the underlying replication topology. You need to explicitly select the tenants to be included. You cannot include the default tenant in an erasure coding topology.

An erasure coding topology has properties that control which objects are erasure coded and when those objects are erasure coded. For information on these and other properties of erasure coding topologies, see Erasure coding topology properties.

Protection types for namespaces

Protection types apply to individual namespaces. Only cloud-optimized namespaces support erasure-coded protection.

When you create a replication-enabled tenant, you choose between:

Allowing erasure-coded protection for all cloud-optimized namespaces owned by the tenant.

Allowing the tenant administrator to allow erasure-coded protection for selected cloud-optimized namespaces.

If the tenant administrator has granted system-level users access to the tenant, you also can allow erasure-coded protection for selected namespaces.

Objects in a replicated namespace that allows erasure-coded protection are erasure coded only if the tenant that owns the namespace is included in an erasure coding topology. If the tenant is not in an erasure coding topology, the namespace uses whole-object protection.

Objects in a replicated namespace that does not allow erasure-coded protection use whole-object protection, even if the tenant that owns the namespace is included in an erasure coding topology.

NoteWebHelp.png

Note:  Regardless of protection type, namespaces obey the DPL specified by their associated service plans. That is, on any given system:

A namespace that uses whole-object protection maintains DPL copies of the complete data for each object

A namespace that uses erasure-coded protection maintains DPL copies of the chunk for each object

For information on service plans and DPL, see Storage tier properties specified in service plans.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Distribution methods for erasure-coded protection


HCP supports two distribution methods for erasure-coded protection: chunk distribution and full-copy distribution.

Chunk distribution

With chunk distribution, after an object is ingested, the chunks for the object are calculated and distributed among the systems in the erasure coding topology. Only the system where the object was ingested has a full copy of the object data.

After all the chunks have been successfully distributed and a specified amount of time has elapsed, the HCP geo-distributed erasure coding service reduces the object on the ingest system to a chunk. The amount of time that the service waits before reducing an object to a chunk is called the erasure coding delay.

For more information on the geo-distributed erasure coding service, see Geo-distributed erasure coding service processing.

Full-copy distribution

With full-copy distribution, after an object is ingested, full copies of the object data are distributed to the systems in the erasure coding topology. After the erasure coding delay expires, the geo-distributed erasure coding service on each system reduces the full copy of the object data on that system to a chunk.

Metadata-only considerations for objects subject to erasure coding

For an erasure-coded object to be properly protected, each system in the erasure coding topology must have a chunk for the object. Therefore, even if the object should be metadata-only on one or more systems, those systems must each store a chunk for the object:

With chunk distribution, each system receives a chunk for an erasure-coded object, regardless of whether the object should be metadata-only on that system.

With full-copy distribution, each system receives a full copy of the object data unless the object should be metadata-only on that system. After the erasure coding delay expires, each system where the object should be metadata-only receives a chunk for the object, and the geo-distributed erasure coding service on each system with a full copy of the object data reduces that full copy to a chunk.

For information on metadata-only objects, see Metadata-only storage tiersMetadata-only storage tiers.

Object counts

An HCP system keeps track of the total number of objects stored on it and, separately, the number of erasure-coded objects stored on it:

The total number of objects stored on a system includes whole objects, metadata-only objects, and objects that are subject to erasure coding, regardless of whether the system currently has a full copy of the object data or a chunk for the object.

The number of erasure-coded objects stored on a system includes only objects for which the system currently has a chunk.

HCP also tracks both of these object counts by tenant and namespace. You can view system-level and tenant-level object counts in the System Management Console, through the management API, and in system-level chargeback reports. For tenants to which you have administrative access, you can also view namespace-level object counts through the management API and in chargeback reports.

Tenant administrators can view total object counts for the applicable tenant and its namespaces in the Tenant Management Console, through the management API, and in tenant-level chargeback reports. Tenant administrators cannot see counts of erasure-coded objects.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Choosing a protection type


The two protection types, whole-object protection and erasure-coded protection, each have their own benefits and limitations. The following sections describe factors you should consider when choosing the protection type for a namespace.

Storage efficiency

With whole-object protection, all the data for an object in a replicated namespace is stored on each system that has that namespace, except on systems where the object is on a metadata-only storage tier. This means that the total storage used for the object data is at most the size of the data times the number of systems in the replication topology. (The storage used can be less due to compression, duplicate elimination, and metadata-only storage tiers.)

NoteWebHelp.png

Note: This discussion assumes a data protection level (DPL) of one. As the DPL increases, the amount of storage needed to store the data for an object also increases, regardless of the protection type. For information on DPL, see Protection service.

For example, in a three-system replication topology in which the object is not metadata-only on any system, the storage used for a 1MB object is at most three MB. Having the object be metadata-only on one or two of the systems increases storage efficiency but decreases the level of data protection.

With erasure-coded protection, each system in the erasure coding topology stores one data or parity chunk for any given erasure-coded object. The size of each chunk for an object is the size of the object data divided by the number of data chunks for the object. This means that the total storage used for an object in a replicated namespace is at most the size of a chunk times the total number of data and parity chunks for the object.

For example, each object in an erasure coding topology with three systems has two data chunks and one parity chunk, so the size of each chunk of a one-megabyte object is .5 MB. As a result, the total storage used for the object is at most three times .5 MB, which is 1.5 MB.

The greater the number of systems in an erasure coding topology, the greater the storage efficiency. The maximum number of systems that can be in an erasure coding topology is six. With six systems, the total storage used for a one-megabyte object is at most 1.2 megabytes.

For information on metadata-only objects, see Metadata-only storage tiers.

Data availability

Whole-object protection can provide better protection from system unavailability than erasure-coded protection provides. With whole-object protection, clients can access objects in a replicated namespace if at least one of the systems where the object data is stored is available. (HCP does not store data for objects on metadata-only storage tiers.)

With erasure-coded protection, all but one of the systems in the erasure coding topology must be available for clients to access objects in a namespace that's replicated on those systems.

Disaster recovery

Both whole-object protection and erasure-coded protection provide support for disaster recovery in case of catastrophic system failure. Disaster recovery entails reprotecting objects in replicated namespaces to their prefailure protection level by recovering them to a replacement system.

With whole-object protection, if a single system fails, the object data can be recovered to the replacement system from any remaining system where the replicated objects are not metadata-only. If two or more systems fail concurrently, whole-object protection can still provide support for disaster recovery. However, as the level of protection increases, the number of systems that must store the complete object data also increases, thereby decreasing storage efficiency.

With erasure-coded protection, if a single system fails, object data can be reconstructed on the replacement system from the chunks on all the remaining systems in the erasure coding topology. If two or more systems fail concurrently, object data cannot be reconstructed and may be permanently lost.

Read performance

With whole-object protection, a client read request for a replicated object results in a single read operation when the request is issued against a system that has the object data. If the object is metadata-only on the target system, that system must retrieve the object data from another system in the replication topology, which may increase the response time.

With erasure-coded protection, a client read request for a single erasure-coded object causes the system that's servicing the request to reconstruct the object from multiple chunks. Only one of those chunks is stored on that system. Therefore, a client read request for an erasure-coded object results in multiple read operations. These operations can be concurrent or sequential, depending on the replication topology on which the erasure coding topology is based. If sequential reads are required, the response time may be longer than if all the reads are concurrent.

NoteWebHelp.png

Note: Rehydration with whole-object protection and restore periods with erasure-coded protection can decrease read response times. For information on rehydration, see Rehydration. For information on restore periods, see Erasure coding topology properties.

These considerations apply to choosing a protection type for a replicated namespace when read performance is a factor:

If objects are likely to be read frequently throughout their lifespans and read performance is more important than storage efficiency, consider using whole-object protection for the namespace.

If objects are unlikely to be read and storage efficiency is important, consider using erasure-coded protection for the namespace.

If objects are likely to be read only for a limited amount of time in the short term and read performance is important, consider using erasure-coded protection with an erasure coding delay that's equal to or greater than the amount of time objects will be read for.

If objects are likely to be read frequently but only for limited periods throughout their lifespans and read performance is important, consider using erasure-coded protection with a restore period. With a restore period, after an object is read, a full copy of the object data is placed on the ingest tier for the containing namespace and is kept there for a configured amount of time.

The ingest tier for a namespace is the storage tier where objects in the namespace are stored when the object data is first written to HCP. The ingest tier is determined by the service plan currently associated with the namespace.

Network bandwidth usage

For namespaces that use whole-object protection, replication entails sending the complete data for an object to each system in the replication topology, except to systems where the object should be metadata-only. Similarly, for namespaces that use erasure-coded protection with full-copy distribution, the complete object data is sent to each system in the erasure coding topology, except to systems where the object should be metadata-only.

For namespaces that use erasure-coded protection with chunk distribution, only object chunks are sent throughout the erasure coding topology. As a result, replication with this type of protection typically requires less network bandwidth than is required for whole-object protection and for erasure-coded protection with full-copy distribution.

On the other hand, with whole-object protection, a client read request against a system that has the object data results in a single data transfer operation. With erasure-coded protection, a client read request results in at least one data transfer operation for each required chunk that's not on the target system in addition to the data transfer operation between the system servicing the request and the client. Therefore, read requests with erasure-coded protection typically use more network bandwidth than is used for read requests with whole-object protection.

Additionally, the chunks for erasure-coded objects are transferred between systems on the replication network. Because servicing read requests takes precedence over replication processing, while any replicated namespaces are experiencing a high read rate, replication may be slow.

NoteWebHelp.png

Note: Rehydration with whole-object protection and restore periods with erasure-coded protection can affect network bandwidth usage for read operations.

PUT copy operations

For a PUT copy operation where the source object is erasure coded, the new object must be reconstructed from the chunks for the source object. For a large object, this reconstruction can take a long time, increasing the potential for the PUT copy operation to time out.

If large objects in a replicated namespace are likely to be sources for PUT copy operations, whole-object protection may be a better choice for the namespace than erasure-coded protection.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

How replication works


To enable replication between two HCP systems, you create a replication link between the two systems. A replication link is a secure, trusted relationship between two systems. The link determines what is replicated and how data is transmitted between the systems.

Link types

A replication link can be active/active or active/passive:

With an active/active link, the HCP tenants and namespaces and default-namespace directories being replicated are read-write on both systems, and data is replicated in both directions between the two systems. Active/active links are designed for use in environments where applications need seamless access to namespaces regardless of where those applications are located. All links in an erasure coding topology must be active/active links.

With an active/active link, HCP gives preference to synchronizing object metadata between the two systems, followed by data synchronization. This preferential treatment enables the fastest access to objects regardless of which system they were created on. If a full copy of the data for an object or a chunk for the object has not yet been replicated to the system targeted by a client request, HCP can read all the object data from a remote system. Additionally, synchronizing metadata first ensures that object names are reserved as quickly as possible.

NoteWebHelp.png

Note: Until a full copy of the data for an object or a chunk for the object is replicated to a given system, the object is metadata-only on that system.

With an active/passive link, the HCP tenants and namespaces and default-namespace directories being replicated are read-write on one system, called the primary system, and read-only on the other system, called the replica. In this case, data is replicated in one direction only — from the primary system to the replica. Active/passive links are designed to support read-only content sharing and disaster recovery.

With an active/passive link, HCP replicates object data and metadata together, thereby ensuring that objects are complete on the replica and can be recovered should that become necessary. However, this approach means that the replica cannot service requests for an object until that object is fully replicated.

To a primary system, an active/passive link is an outbound link; that is, the system is sending data through the link. To a replica, an active/passive link is an inbound link; that is, the system is receiving data through the link. To both systems involved in an active/active link, the link functions as both an outbound link and an inbound link.

Link security

Replication relies on SSL to ensure the integrity and security of transmitted data. Before you can create a replication link, the two systems involved in the link must each have a valid SSL server certificate. Each system must also have the server certificate from the other system installed as a trusted replication server certificate.

Replication priority

Replication is asynchronous with object ingest. An HCP system can, therefore, develop a backlog of objects to be replicated.

You can choose to have the replication service work on objects with the oldest changes first, regardless of which namespaces they’re in. Alternatively, you can have the service balance its processing time evenly across the namespaces being replicated. In this case, the service may replicate recent changes in some namespaces before older changes in others.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

What is replicated


Replication works by tenant. For each replication link, you can select HCP tenants to be replicated on that link. If you have administrative access to the tenant, you can also select the namespaces to include in replication of that tenant. To replicate the default tenant, you select the default-namespace directories to be replicated.

Regardless of the system on which you create the link:

For an active/active link, you can select items to be replicated from both systems involved in the link

For an active/passive link, you can select items to be replicated only from the primary system

You can select the HCP tenants to include in an erasure coding topology from any system in the topology. Each tenant you select is automatically added to every link in the underlying replication topology. You cannot include the default tenant in an erasure coding topology.

What is replicated for each tenant differs between HCP tenants and the default tenant.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication of HCP tenants


When creating a tenant, you specify whether it can be replicated. After creating the tenant, you can change this setting from not allowing replication to allowing replication. You cannot do the reverse.

A replication link or erasure coding topology can include only HCP tenants that are configured to allow replication.

Replicated tenants and namespaces

When an HCP tenant is replicated, the replication service works only on selected namespaces owned by that tenant. The tenant administrator can select and deselect namespaces for replication at any time. If the tenant is being replicated, the tenant and its selected namespaces are replicated on each link that includes the tenant. If the tenant is not being replicated, the selected namespaces are not replicated either.

If an HCP tenant has granted system-level users administrative access to itself, you can select and deselect its namespaces for replication when you create the replication link or by modifying the link at a later time. You should coordinate namespace selection with the tenant administrator to ensure that you both have the same understanding of what should be replicated.

Tenants can be replicated from one HCP system to another regardless of the number of tenants that already exist on the other system. However, you can create a new tenant on a system only if the total number of tenants on the system is less than one thousand.

Similarly, namespaces can be replicated from one HCP system to another regardless of the number of namespaces that already exist on the other system. However, you can create a new namespace on a system only if the total number of namespaces on the system is less than ten thousand.

HCP tenants created on different systems can have the same names as each other. However, every tenant has a unique internal ID, so HCP can distinguish tenants with the same name from one another. You cannot replicate a tenant from one system to another if the second system has a different tenant with the same name. To replicate the tenant in this case, you first need to change the name of the tenant on one of those systems.

Replicated items

The replication service replicates these items for an HCP tenant:

The tenant configuration

The configuration of each namespace being replicated

Existing objects, object creations, object deletions, and metadata changes in the namespaces being replicated

Old versions of objects in the namespaces being replicated

Retention class creations, modifications, and deletions for the namespaces being replicated

Content class and content property creations, modifications, and deletions

All tenant-level log messages

All namespace-level log messages for the namespaces that are being replicated

User accounts defined for the tenant

Group accounts defined for the tenant

Replicated changes

With an active/active link, replicated HCP tenants are read-write on both systems involved in the link. Administrators can make changes to the configuration of the replicated tenants and namespaces on either system, and users and applications can make changes to the content of the replicated namespaces on either system. All changes made on each system are replicated to the other system.

With an active/passive link, replicated HCP tenants are read-write on the primary system and read-only on the replica. Administrators cannot make any configuration changes to the replicated tenants or to any of their namespaces on the replica, nor can users and applications make any changes to the content of the replicated namespaces on the replica. However, all changes made on the primary system are replicated to the replica.

NoteWebHelp.png

Note: Although you cannot make any changes to the configuration of a replicated namespace on a replica, you can reindex the namespace.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication of the default tenant


For the default tenant, the replication service works on one or more selected directories directly under fcfs_data in the default namespace. The service replicates the entire directory hierarchy under each selected directory.

Default namespace properties

Directories in the default namespace on one system involved in a replication link can be replicated only to the default namespace on the other system involved in the link. They cannot be replicated to HCP namespaces.

Additionally:

Before you can include any default-namespace directories in a replication link, the default namespace must exist on both systems involved in the link.

The default namespaces in the two systems must have the same cryptographic hash algorithm and retention mode.

The default namespace can be configured to prevent writes and/or deletes. This configuration can differ between the default namespace on the two systems involved in a link. However, even if the system receiving data is configured to prevent these actions, changes to namespace content on the sending system are replicated.

Replicated directories

You select directories for replication when you create the replication link. You can select additional directories or deselect directories by modifying the link at a later time. A replication link can include at most 256 directories.

NoteWebHelp.png

Note: In releases of HCP earlier than 6.0, a replication link could include more than 256 directories. In a system that was originally upgraded from a release earlier than 6.0, the maximum number of directories that can be included on a preexisting link in which the system participates is equal to either 256 or the number of directories included on the link at the time of the upgrade, whichever is greater.

Directories can be created in the default namespace on both of the systems involved in a replication link, so the two systems can have directories with the same name. You cannot replicate a directory created on one of the systems if a locally created directory with the same name already exists on the other system.

The one exception to this is the directory used for email archiving through SMTP. You can replicate the email directory if the email directory name is the same on both systems.

Replicated changes

With an active/active link, replicated directories are read-write on both systems involved in the link. All changes made to the content of those directories on either system are replicated to the other system.

With an active/passive link, the replicated directories are read-write on the primary system and read-only on the replica. Users and applications cannot make any changes to the content of the replicated directories on the replica. However, all changes made on the primary system are replicated to the replica.

Replicated items

The replication service replicates these items for the default tenant:

Existing objects, object creations, object deletions, and metadata changes in the directories being replicated

Retention class creations, modifications, and deletions for the default namespace

Content class and content property creations, modifications, and deletions

All log messages relating to retention classes and privileged delete operations

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication service processing


With an active/active link, the replication service runs on both HCP systems involved in the link. With an active/passive link, the replication service runs only on the primary system.

Each replication link has a schedule that specifies when the replication service should run. By default, when you create a link, the link schedule has the service running 24 hours a day seven days a week.

The replication service starts sending information on a link the first time you add HCP tenants and/or default-namespace directories to that link, as long as the link schedule has the service running at that time. If the service is not scheduled to run at that time, the service starts sending information as soon as the next scheduled run period starts.

For more information on replication link schedules, see Scheduling activity on a replication link.

Configuration information first

When you first add HCP tenants, namespaces, and/or default-namespace directories to a replication link, the replication service replicates the configuration of those items as soon as possible. The service then starts replicating objects in those namespaces and default-namespace directories. The service replicates objects in chronological order either across all namespaces or within each namespace in turn, depending on the link configuration. The chronological order is oldest-first based on the time of the last metadata change to the object.

Whenever you add a new item to a replication link or change the configuration of an item already included on the link, the replication service replicates the configuration of the new item or the configuration change as soon as possible. This ensures the correct behavior of objects on the receiving system.

Whole-object protection with unavailable systems

With whole object protection, the replication service on the system where an object is ingested sends a full copy of the data for the object to the available systems to which the ingest system is directly connected by a replication link and on which the object is not supposed to be metadata-only. If a system is unavailable, the replication service sends the full copy of the object data to that system when the system becomes available.

Chunk distribution with unavailable systems

For erasure-coded protection with chunk distribution, the replication service on the system where an object is ingested sends a chunk for the object to each available system in the erasure coding topology. When an unavailable system becomes available, the replication service on the ingest system sends a chunk to that system. After all the systems that were unavailable have chunks and the erasure coding delay has expired, the geo-distributed erasure coding service reduces the full copy of the object data on the ingest system to a chunk.

A system in an erasure coding topology can suffer a catastrophic failure after ingesting data while another system in the topology is unavailable. If this happens with a topology that's configured for chunk distribution, the data that was ingested on the failed system while the other system was unavailable is lost.

To minimize the risk of data loss, if an erasure coding topology is configured for chunk distribution:

Any upgrade of a system in the topology should be performed while the system is online

Planned system unavailability (due, for example, to a system shutdown for maintenance or to a loss of connectivity during a network upgrade) should be scheduled for times when data ingest is at a minimum

TipWebHelp.png

Tip: To protect newly ingested objects during a period of planned system unavailability, take these actions before making the system unavailable:

1.For each namespace that's owned by a tenant included in the erasure coding topology and that allows erasure coding, ensure that the service plan associated with that namespace does not include a metadata-only storage tier on at least two of the systems that will remain available. If an applicable service plan includes a metadata-only storage tier, you can temporarily either modify the service plan or associate the namespace with a different service plan.

2.Disable the erasure coding service.

3.Reconfigure the erasure coding topology to use full-copy distribution.

After making the system available again:

1.Reconfigure the erasure coding topology to use chunk distribution.

2.Enable the erasure coding service.

Full-copy distribution with unavailable systems

For erasure-coded protection with full-copy distribution, when an object is ingested, each available system in the erasure coding topology, except systems where the object should be metadata-only storage tier, receives a full-copy of the object data. When an unavailable system becomes available:

If the erasure coding delay has not expired and the object is not supposed to be metadata-only, the system receives a full copy of the object data. If the object should be metadata-only, the system does not receive either a full copy or a chunk.

After all the systems in the topology are available, when the erasure coding delay expires, each system where the object should be metadata-only receives a chunk for the object, and all full copies of the object data are reduced to chunks.

If the erasure coding delay has expired, the system receives a chunk for the object regardless of whether the object should be metadata-only. After all the systems in the topology are available, full copies of the object data are reduced to chunks.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Geo-distributed erasure coding service processing


The geo-distributed erasure coding service is responsible for ensuring that objects that are or have ever been subject to erasure coding are in the correct state at any given time.

On any given HCP system, the geo-distributed erasure coding service runs according to a system-specific schedule. Therefore, the service is not necessarily running at the same time on any set of systems. However, the service on one system can query other HCP systems for the state of the objects on those systems even when the service is not running on those systems.

While running, the geo-distributed erasure coding service examines each object individually. If the system where the service is running has a full copy of the data for an object and should not, the service reduces the data to a chunk. If the system has a chunk for the object and should not, the service restores the chunk to a full copy. If the object is in the state it should be in, the service takes no action on the object.

To have sufficient information to restore a chunk for an object to a full copy of the object data, the geo-distributed erasure coding service can ask other systems to send the chunks they have for that object. Alternatively, if any of those systems has a full copy of the object data, the service can ask for the full copy.

During a scheduled run time, the geo-distributed erasure coding service actually runs only if the system is part of an active erasure coding topology or if the system contains erasure-coded objects. An active erasure coding topology is one that is currently being used for geo-protection. A system can participate in only one active erasure coding topology.

If the geo-distributed erasure coding service doesn't have sufficient run time on a system that's receiving full copies of object data, those full copies may not be reduced to chunks in a timely manner. When scheduling the service, keep in mind that full copies of object data use the full storage required by the object data until they are reduced to chunks.

The geo-distributed erasure coding service cannot restore a chunk for an object to a full copy of the object data if the system where the service is running doesn't have enough space for the complete object data. In this case, the service leaves the chunk on the system as is.

NoteWebHelp.png

Note: The geo-distributed erasure coding service is automatically added to the HCP Default Schedule service schedule when HCP is upgraded to release 8.x from a release earlier than 8.0. You need to schedule the service yourself in any user-created service schedules in which you want the service to have run time.

Determining what the state of an object should be

Several factors determine whether a system should have a chunk or a full copy of the data for a given object:

First, the geo-distributed erasure coding service on the system must determine whether the namespace containing the object has erasure coding allowed.

Then, if erasure coding is allowed, the service must determine whether the tenant that owns the namespace is currently included in an active erasure coding topology.

Finally, if the tenant is included in an active erasure coding topology, the service must determine:

oWhat the state of the object should be with respect to these properties of the erasure coding topology: distribution method, erasure coding delay, restore period, and minimum size for erasure coding. For information on these properties, see Erasure coding topology properties.

oWhat the state of the object is on the other systems in the erasure coding topology.

Keeping objects sufficiently protected

An object that is subject to erasure coding:

Is sufficiently protected as long as all the systems in the erasure coding topology are available or as long as at least two systems in the topology have a full copy of the object data

Is insufficiently protected if the object would become unavailable due to a system in the erasure coding topology becoming unavailable

The geo-distributed erasure coding service never reduces a full copy of the data for an object to a chunk if doing so would cause the object to be insufficiently protected.

Namespace configuration changes

Erasure coding can be disallowed for a namespace that previously had erasure coding allowed. Similarly, replication can be disabled for a namespace that was previously selected for replication.

If erasure coding is disallowed or replication is disabled for a namespace that contains erasure-coded objects, the geo-distributed erasure coding service on each system that contains those objects is responsible for:

Restoring the chunks for the objects to full copies of the object data on HCP systems where the object is not supposed to be metadata-only. The service can restore the object data as long as no more than one system in the erasure coding topology is unavailable or at least one system in the topology has a full copy of the object data.

Deleting chunks for the objects on HCP systems where the object should be metadata-only.

Before disallowing erasure coding or disabling replication for a namespace that contains erasure-coded objects, you need to ensure that each system where the chunks for the objects will be restored to full copies of the object data has sufficient storage to accommodate that data.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication verification service processing


The HCP replication verification service is responsible for checking whether the replication service missed replicating or was unable to replicate any objects that should have been replicated. If the replication verification service encounters such an object, the service tries to replicate the object.

If either the replication service or the replication verification service encounters an object it cannot replicate, the service marks the object as nonreplicating. If the replication verification service replicates a nonreplicating object, the service removes the nonreplicating mark from the object.

The replication verification service is not a scheduled service. The service runs only when you:

Set the service to run continuously while the replication service is running.

Set the service to run once. In this case, if the replication service is running, the replication verification service starts immediately. Otherwise, the replication verification service waits to start until the next time the replication service runs.

You can also disable the service so that it doesn't run at all.

While running, the replication verification service may degrade HCP system performance. Normally, you run the service only when directed to do so by your authorized HCP service provider.

The Overview panel for a replication link indicates whether any tenants on the link have namespaces with nonreplicating objects. For information on viewing a list of the affected tenants, see Link overview.

Tenant administrators can view the list of nonreplicating objects in a namespace in the Tenant Management Console. For more information on this list, see Working with nonreplicating objects.

For instructions on running or disabling the replication verification service, see Managing the replication verification service.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Unavailable and irreparable erasure-coded objects


The status of an erasure-coded object can differ among the systems in an erasure coding topology:

An erasure-coded object is unavailable on a system if all of these are true:

oOne or more systems in the erasure coding topology are inaccessible.

oThe object data cannot be reconstructed from the available chunks.

oNo accessible system has a good full copy of the object data.

An erasure-coded object is irreparable on a system if all of these are true:

oAll systems in the erasure coding topology are accessible.

oThe object data cannot be reconstructed from the available chunks.

oNo system in the topology has a good full copy of the object data.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replicated systems


Replication on each link in a replication topology occurs between two separate HCP systems, each of which is complete in its own right. Because replication is a software function, the two systems can have entirely different hardware configurations, including differing amounts of storage.

The two systems in a replicated pair are connected through the front-end network infrastructure, as shown in the figure below. Replication traffic from each system must be routable to the network selected for replication on the other system.

ReplicatedSystems.jpg

Replication is a network bandwidth-intensive process. To ensure sufficient front-end network capacity when replicating, you need to give extra consideration to planning the front-end infrastructure.

For information on selecting a network for replication, see Selecting the network for replication.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication and Active Directory authentication


An HCP system can be configured to support Windows® Active Directory® (AD) for user authentication. Part of this configuration is the specification of an AD domain. The domain determines the AD groups from which HCP group accounts can be created.

If HCP is configured to support AD, HCP tenants can be configured to allow access by users authenticated by AD. For this access to work, the AD user must belong to one or more AD groups for which corresponding group accounts are defined for the tenant.

For the same AD users to be able to access a given tenant on both systems involved in a replication link, the group accounts on each system must correspond to the same AD groups as they do on the other system. To make this happen, support for AD must be enabled on both systems, and either of these must be true:

The same domain is specified in the AD configuration on both systems.

The domain specified in the AD configuration on one system is trusted by the domain specified in the AD configuration on the other system.

Similarly, for the same AD users to be able to access the default namespace on two systems involved in a replication link, the system-level group accounts on each system must correspond to the same AD groups as they do on the other system.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication and RADIUS authentication


An HCP system can be configured to support RADIUS for user authentication. If HCP is configured this way, HCP tenants can be configured to allow access by RADIUS-authenticated users. Also, if HCP is configured this way, RADIUS-authenticated users with system-level user accounts can access the default namespace.

For the same RADIUS-authenticated users to have access to both systems involved in a replication link, the HCP RADIUS configuration on the two systems must specify the same RADIUS servers.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication topologies


An HCP system can have at most five outbound links and at most five inbound links. An active/active link counts as both an outbound link and an inbound link. The ability for a single system to participate in multiple links enables HCP to support various replication topologies.

Replication can occur:

In two directions on a single link between two HCP systems (active/active replication)

In one direction on a single link between two HCP systems (active/passive replication)

From multiple HCP systems to a single HCP system (many-to-one replication)

From one HCP system to a second HCP system and from that second system to a third HCP system, such that the same HCP tenants and namespaces and default-namespace directories that are replicated to the second system are then replicated to the third system (chained replication)

From one HCP system to multiple other HCP systems (one-to-many replication)

These configurations can be combined to form complex replication topologies.

NoteWebHelp.png

Note: A replication topology can include one or more HCP systems where the occurrences of a given namespace can contain metadata-only objects. If the service plan associated with the namespace does not include a metadata-only storage tier on only one system in the topology and the topology is being used for production purposes, the topology should include a disaster recovery system where the service plan associated with the namespace also does not include a metadata-only storage tier.

For information on metadata-only objects, see Metadata-only storage tiers.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Simple active/active replication


In an simple active/active replication topology, two HCP systems replicate the same HCP tenants and namespaces and default-namespace directories to each other over an active/active link. The items being replicated can be originally created on either system. The items are read-write on the system in which they were created and, after being replicated, are also read-write on the other system.

With active/active replication, client requests can be directed to either system. All changes made on each system, including both configuration changes and changes to namespace content, are replicated to the other system.

What this looks like

The figure below shows a simple active/active replication topology in which two HCP systems (A and B) are replicating to each other.

1_123.jpg

In this figure:

Two of three HCP tenants created locally in system A are being replicated between system A and system B. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

Two tenants created locally in system B are being replicated between system A and system B. In the first tenant, two namespaces are selected for replication. In the second tenant, one of three namespaces is selected for replication.

Uses

The active/active replication topology supports a cloud storage model, where any type of client request can be serviced equally by either HCP system involved in the replication link. With this topology, if one system becomes unavailable (for example, either unexpectedly or for scheduled maintenance), the other system can still provide all the required functionality.

With an active/active replication topology, the processing load can be distributed between the two systems. This distribution can be accomplished in two ways:

By having some applications direct requests to one system and other applications direct requests to the other system. Typically, the assignment of applications to systems is based on geographic location.

With this configuration, a single shared DNS can enable requests to be redirected automatically from one system to the other in case one of the systems becomes unavailable. For more information on this, see Managing DNS failover.

By using a load balancer to divide requests between the two systems. For each client request, the decision as to which system to target can be based on several factors, such as the current load on each system and the geographical distance between the client and each system.

With this configuration, if one system becomes unavailable, the load balancer simply directs all requests to the other system.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Active/active replication ring


In an active/active replication ring topology, four or more HCP systems are connected by active/active links, where:

Each system has exactly two active/active links that connect that system to two other systems in such a way that all the systems are linked in the form of a ring

The same HCP tenants and namespaces and default-namespace directories are replicated on each link

With an active/active replication ring topology, changes made on one system travel around the ring in both directions from that system until they reach a system where those changes have already occurred.

To configure an active/active replication ring topology:

1.Create an active/active link between each pair of systems to be connected in the ring.

2.For each system that has locally created tenants, namespaces, and directories that you want to replicate, add those items to one of the links connected to that system.

3.After those tenants, namespaces, and directories have been replicated, add them to the second link that’s connected to the system to which the items have been replicated.

4.After the items have been replicated to that system, add them to the next link in the ring.

5.Continue adding the replicated items to successive links until you have added them to the second link that connects to the system where the items were originally created.

An active/active replication ring topology that includes four, five, or six HCP systems can be used as the basis for an erasure coding topology. When you add a tenant to the erasure coding topology, if the tenant is not already being replicated in the ring topology, HCP automatically adds that tenant to all the links in the ring topology.

For a comparison of the active/active replication ring topology to the fully connected active/active replication topology, see Fully connected active/active replication.

What this looks like

The figure below shows an active/active replication ring topology that includes four systems (A, B, C, and D).

1_135.jpg

In this figure:

Two of three HCP tenants created locally in system A are being replicated around the ring. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

Two tenants created locally in system D are being replicated around the ring. In the first tenant, two namespaces are selected for replication. In the second tenant, one of three namespaces is selected for replication.

Uses

The active/active replication ring topology supports increasingly large storage clouds. The HCP systems included in the ring are typically in geographically disperse locations, with each system providing the same functionality as all the others. Changes made on any one system are propagated to all the other systems, effectively enabling shared read-write access to the same HCP tenants and namespaces and default-namespace directories regardless of where the client applications are located.

New HCP systems can be added to an active/active replication ring topology at any time. To add a new system (for example, system X) to the topology:

1.Create a new active/active link between any system already in the topology (for example, system B) and system X.

2.Add the tenants, namespaces, and directories already being replicated around the ring to the new link.

3.Create a new link between system X and one of the other systems with which system B participates in a link (for example, system C).

4.Add the tenants, namespaces, and directories already being replicated around the ring to that new link.

5.Delete the link between system B and system C.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Fully connected active/active replication


In a fully connected active/active replication topology, multiple HCP systems are connected by active/active links, where:

Each system in the topology is connected to each other system in the topology by an active/active link

The same HCP tenants and namespaces and default-namespace directories are replicated on each link

Because each system can participate in at most five active/active links, a fully connected active/active replication topology can have at most six systems.

To configure a fully connected active/active replication topology:

1.Create an active/active link between each pair of systems to be included in the topology.

2.For each system that has locally created tenants, namespaces, and directories you want to replicate, add those items to each of the links connected to that system.

3.After those tenants, namespaces, and directories have been replicated, add them to each of the other links in the topology.

A fully connected active/active replication topology can be used as the basis for an erasure coding topology. When you add a tenant to the erasure coding topology, if the tenant is not already being replicated on the links in the fully connected active/active topology, HCP automatically adds that tenant to all the links in the fully connected active/active topology.

What this looks like

The figure below shows a fully connected active/active replication topology that includes four systems (A, B, C, and D).

FullyConnectedActiveActiveReplicationTopology.jpg

In this figure:

Two of three HCP tenants created locally in system A are being replicated throughout the topology. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

Two tenants created locally in system D are being replicated throughout the topology. In the first tenant, two namespaces are selected for replication. In the second tenant, one of three namespaces is selected for replication.

Uses

The fully connected active/active replication topology is an alternative to the active/active ring topology for maintaining the same namespaces on multiple HCP systems in a storage cloud.

A fully connected topology provides better protection from system unavailability and nonfunctioning replication links than a ring topology provides. The network of links in a fully connected topology ensures that each system has multiple paths to each other system, thereby preventing individual systems from becoming isolated from the rest.

For example, suppose:

Namespace NS1 is being replicated in a replication topology that includes systems A, B, C, and D

The service plan associated with NS1 includes a metadata-only storage tier on A and B

Object O1 is metadata-only on both A and B

With a ring topology, if the replication links between B and C and between C and D are broken, system C is isolated. Because no path connects system D to system C, system D cannot retrieve data for O1 from the only other system that has the data (for example, in a situation in which read from remote is required).

ReplicationRingTwoLinksBroken.png

Given the same circumstances with a fully connected topology, system D can still read data for O1 from system C by using the links between D and A and between A and C.

ReplicationFullyConnectedTwoLinksBroken.png

A fully connected topology requires more maintenance than a ring topology. In the worst case, with six systems in a fully connected topology, you need to maintain fifteen replication links. With six systems in a ring topology, you need to maintain only six links.

Additionally, the more systems you have in a fully connected topology, the less the ability to replicate other tenants to systems outside the topology. For example, in a five-system fully connected topology, any system could have one additional link on which to replicate tenants that are not in the topology to a system outside the topology. In a six-system fully connected topology, the systems cannot have any additional links.

A typical use for a fully connected active/active replication topology is as the basis for an erasure coding topology.

Responses to read requests for erasure-coded objects are faster with a fully connected topology than with a ring topology because all chunks not on the target system can be retrieved from directly connected systems. In comparison, with an erasure coding topology based on a ring replication topology, responses to read requests are slower because some chunks can be retrieved only through paths consisting of more than one replication link. Also, the use of the additional links increases network bandwidth usage.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Simple active/passive replication


In a simple active/passive replication topology, one HCP system, the primary system, replicates HCP tenants and namespaces and default-namespace directories to another HCP system, the replica, over an active/passive link. The items being replicated are originally created only in the primary system. The items are read-write on the primary system and, after being replicated, read-only on the replica.

What this looks like

The figure below shows a simple active/passive replication topology in which one HCP system (A) is replicating to a second HCP system (B).

1_129.jpg

In this figure:

From system A, two of three locally created tenants are being replicated to system B. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

In system B, no locally created tenants are being replicated.

Uses

The main purpose of a simple active/passive replication topology is to have a backup of the primary system for use in disaster recovery. However, this topology also supports a scenario in which some applications need read-write access to the replicated items while other applications need only read access. By directing requests from the latter to the replica, you can reduce the load on the primary system.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Active/passive many-to-one replication


In an active/passive many-to-one replication topology, multiple HCP systems replicate to a single other HCP system. Each of the replicating systems is the primary system for an active/passive replication link. The other system is the replica for each of those links.

In this topology, you can think of the replica as being a hub, with the primary systems being at the ends of spokes connected to that hub. The spokes themselves are the replication links. The hub can have at most five spokes.

Each HCP tenant and default-namespace directory selected for replication on a primary system must be unique on the replica. For example, if two of the primary systems each have a tenant named Finance, the Finance tenant can be replicated from only one of the systems. For both tenants to be replicated, they must have different names.

If you want to replicate the email directory in the default namespace on multiple primary systems, the SMTP protocol on each system should specify a different email directory name.

NoteWebHelp.png

Note: You can also create a many-to-one replication topology that consists of a combination of active/passive and active/active links or even only active/active links.

What this looks like

The figure below shows an active/passive many-to-one replication topology in which three primary systems (A, B, and C) are replicating to a single replica (D).

1_27.jpg

In this figure:

From system A, two HCP tenants are being replicated to system D. In the first tenant, two of three namespaces are selected for replication. In the second tenant, one of two namespaces is selected for replication.

From system B, two of three HCP tenants are being replicated to system D. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

From system C, one of two HCP tenants is being replicated to system D. In that tenant, both namespaces are selected for replication.

System D has one HCP tenant of its own that is not the result of replication. This tenant has two namespaces.

Uses

In an active/passive many-to-one replication topology, the replica is typically the largest HCP system. The data center in which it resides is usually fully staffed with IT personnel. The primary systems are typically smaller. Because replication links can be created and managed from either side, the data centers where these systems reside can be minimally staffed with less experienced people.

Active/passive many-to-one replication supports a scenario in which the outlying sites are branch offices of an enterprise that has an HCP system at a primary data center. Applications running at each of those offices connect over a local area network (LAN) to an HCP system at the local data center. Because they are close to the storage they use, the applications perform better than they would if they were using a wide area network (WAN) to connect to the HCP system at the hub. Also, the distribution of the network load among the outlying sites further enhances application performance.

With active/passive many-to-one replication, you can consolidate similar data from multiple sources in a single HCP system. Using a search application, you can then perform federated queries across the replicated namespaces on that system. For example, suppose each of the branch offices of an enterprise stores completed orders in a namespace in its own HCP system. If each of those namespaces is replicated to a single HCP system, you could query that system to find all orders placed for a specific product at any of the branch offices.

Active/passive many-to-one replication enables a single HCP system to be used for backup and recovery of multiple other HCP systems, so the outlying sites don’t need backup systems of their own. For additional data protection, the hub system can be backed up, for example, to tape or to another HCP system (see Many-to-one replication with disaster recovery support).

For information on data recovery in an active/passive many-to-one replication topology, see Failover and failback in an active/passive many-to-one topology with disaster recovery support.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Active/passive chained replication


In an active/passive chained replication topology, HCP tenants and namespaces and default-namespace directories in one HCP system (A) are replicated to a second HCP system (B) and then from the second system to a third HCP system (C). Both the replication link from A to B and the replication link from B to C are active/passive links.

To configure an active/passive chained replication topology:

1.When you create the replication link from A to B, select the tenants, namespaces, and directories you want to replicate. In this link, A is the primary system and B is the replica.

2.When you create the replication link from B to C, select the link from A to B to be included in that link. This causes the HCP tenants, namespaces, and directories replicated on the link from A to B to be replicated again from B to C. In this link, B is the primary system and C is the replica.

The link from B to C can also include tenants, namespaces, and directories that were originally created on B.

You cannot individually select the replicated tenants, namespaces, and directories on B for inclusion in the link from B to C.

In an active/passive chained replication topology with the configuration ABC, B is the replica for the link from A to B. Therefore, in B, the tenants and directories that were replicated from A are read-only, even though B is also the primary system for the link from B to C.

HCP supports replication chains that consist of three HCP systems. Longer chains are not supported.

NoteWebHelp.png

Notes: 

You can also create a chained replication topology in which the first link is active/active and the second link is active/passive. You cannot, however, create a chained replication topology in which the first link is active/passive and the second link is active/active or in which both links are active/active.

In a replication chain ABC in which the objects in a namespace on system B are supposed to be metadata-only, HCP does not remove data from those objects until that data has been replicated to system C. For information on metadata-only objects, see Metadata-only storage tiers.

What this looks like

The figure below shows an active/passive chained replication topology in which system A is replicating to system B and system B is replicating to system C.

1_1.jpg

In this figure:

From system A, two HCP tenants are being replicated to system B. In the first tenant, two of three namespaces are selected for replication. In the second tenant, one of two namespaces is selected for replication.

From system B, the tenants and namespaces created by replication from system A are being replicated to system C because the link from A to B is included in the link from B to C.

One tenant that was originally created on system B is also being replicated to system C. Both the namespaces in that tenant are selected for replication.

Uses

In an active/passive chained replication topology, all three HCP systems are typically located at full-scale data centers. For example, suppose a corporation has three major locations — New York, Los Angeles, and Tokyo. Each of these locations runs an application that is vital to the corporation, but only New York generates the data for these applications. Replicating the data from New York to Los Angeles and from Los Angeles to Tokyo enables applications at each location to access the data through a local area network. Because those applications are not accessing the data on the HCP system in New York, this topology also reduces the load on that system.

Active/passive chained replication helps ensure continuous data availability. If any one or even two of the HCP systems in the chain become unavailable, applications still have access to the stored data.

In an active/passive replication chain, the third HCP system provides disaster recovery functionality for the second system, and the second system provides disaster recovery functionality for the first system. For information on data recovery in an active/passive chained replication topology, see Failover and failback in an active/passive chained topology.

With an active/passive chained replication topology, you can create and manage both replication links from the HCP system in the middle of the chain. To this system, the link from the first system is an inbound link, and the link to the third system is an outbound link.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Active/passive one-to-many replication


In an active/passive one-to-many replication topology, one HCP system replicates to two or more other HCP systems using separate active/passive replication links. The replicating system is the primary system for each link. The other systems are replicas.

Typically, in an active/passive one-to-many replication topology, the replication links include different HCP tenants and namespaces and default-namespace directories.

If three systems need to have the same tenants, namespaces, and directories as each other, consider using a chained topology for them rather than a one-to-many topology. The one-to-many topology puts greater load on the single primary system than does a chained topology, which splits the load between the first and second systems in the chain.

In some cases, however, replicating the same tenants and namespaces to two different replicas in a one-to-many topology may be appropriate. For an example of this, see the description of the second use case in Uses.

NoteWebHelp.png

Note: You can also create a one-to-many replication topology that consists of a combination of active/passive and active/active links or even only active/active links.

What this looks like

The figure below shows an active/passive one-to-many replication topology in which one primary system (A) replicates to two replicas (B and C) over two separate replication links.

1_94.jpg

In this figure:

Two of three HCP tenants in system A are being replicated to system B. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

The third HCP tenant in system A is being replicated to system C. This tenant has two namespaces, both of which are selected for replication.

Uses

An active/passive one-to-many replication topology enables you to replicate different tenants and default-namespace directories from a single primary system to multiple different replicas. You may choose to do this, for example, if the replicas are at sites where users need read access to only a limited number of namespaces. Those sites can then have HCP systems that use smaller amounts of storage.

You can also use an active/passive one-to-many replication topology as a means to upgrade your HCP systems from RAIN to SAIN. This scenario assumes you currently have a RAIN primary system replicating to a RAIN replica on an active/passive link. You want to have a SAIN primary system replicating to a SAIN replica. To make this happen:

1.Create a second active/passive link from the RAIN primary system to the SAIN system that will be the primary system after the upgrade. Use this link to replicate all the HCP tenants, HCP namespaces, and default-namespace directories in the RAIN system.

2.Create a replication link from the new SAIN primary system to the second SAIN system. Include the link created in step 1 above in this link to create a replication chain.

3.When replication is completely up to date, fail over the link from the RAIN system to the SAIN system. To enable replication to become completely up to date, you may need to stop clients from writing to the RAIN system before you fail over the link.

4.Delete the link from the RAIN system to the SAIN system.

5.Decommission the two RAIN systems.

For information on failover, see Failover and failback. For information on data recovery in an active/passive one-to-many replication topology, see Failover and failback in an active/passive one-to-many topology.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Active/active replication with disaster recovery support


You can combine active/active replication with active/passive replication to create a topology in which a third system provides disaster recovery support for the two systems that participate in an active/active link. In this topology, a set of HCP tenants and namespaces and default-namespace directories is replicated on an active/active link between two systems (A and B). Some of those items are also replicated on an active/passive link from A to a third system (C). The rest of the items are also replicated on an active/passive link from system B to system C.

The choice of which items to replicate on each active/passive link is independent of which system the items were initially created on. For example, an HCP tenant created locally on system A could be included on the active/passive link from B to C.

The figure below shows a simple active/active replication topology with disaster recovery support.

1_147.jpg

In this figure:

Two of three HCP tenants created locally in system A are being replicated between system A and system B. In the first tenant being replicated, two of three namespaces are selected for replication. In the second tenant being replicated, one of two namespaces is selected for replication.

Two HCP tenants created locally in system B are being replicated between system A and system B. In the first tenant, two namespaces are selected for replication. In the second tenant, one of three namespaces is selected for replication.

From system A, one HCP tenant initially created on system A and one HCP tenant initially created on system B are being replicated to system C.

From system B, one HCP tenant initially created on system A and one HCP tenant initially created on system B are being replicated to system C.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Many-to-one replication with disaster recovery support


You can combine the many-to-one and chained replication topologies to create a configuration in which multiple HCP systems replicate to a single HCP system (many-to-one), which, in turn, replicates to another HCP system (chained). The last system provides additional assurance of continuous data availability for all the HCP tenants and namespaces and default-namespace directories originally selected for replication. It also provides disaster recovery functionality for the HCP system to which those tenants, namespaces, and directories are initially replicated.

For example, suppose HCP systems A, B, and C replicate to system D, which replicates to system E. To create this combined topology:

1.When you create the replication links from A to D, from B to D, and from C to D, select the tenants, namespaces, and directories you want to replicate.

2.When you create the replication link from D to E, select the links from A to D, from B to D, and from C to D to be included in that link. This causes the tenants, namespaces, and directories replicated on all three of those links to be replicated again from D to E.

In effect, you’re creating three replication chains: ADE, BDE, and CDE.

The figure below shows a many-to-one replication topology with disaster recovery support.

1_53.jpg

For information on data recovery in an active/passive many-to-one replication topology with disaster recovery support, see Failover and failback in an active/passive many-to-one topology with disaster recovery support.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Bidirectional active/passive replication


For any given pair of HCP systems, each system can serve as both a primary system and a replica for the other system, as long as different HCP tenants and different default-namespace directories are being replicated in each direction. A system that’s serving as both a primary system and a replica at the same time has two active/passive links — one outbound and one inbound. This topology is called bidirectional active/passive replication.

What this looks like

The figure below shows a bidirectional active/passive replication topology in which two systems (A and B) are each replicating to the other system.

1_141.jpg

In this figure:

From system A, two locally created tenants are being replicated to system B. In both tenants, all namespaces are selected for replication.

From system B, one locally created tenant is being replicated to system A. In this tenant, all namespaces are selected for replication.

Uses

For each link in a bidirectional active/passive replication topology, the tenants, namespaces, and directories being replicated are read-write on the primary system and read-only on the replica. A bidirectional active/passive replication topology thus supports a scenario like this:

Application 1 running in a data center in New York needs read-write access to Tenant-1, which was created locally in the HCP system (A) in New York.

Application 2 running in a data center in Tokyo needs read-write access to Tenant-2, which was created locally in the HCP system (B) in Tokyo.

Application 1 needs only read access to Tenant-2.

Application 2 needs only read access to Tenant-1

To meet these needs, you could:

Create an active/passive link from system A to system B that includes Tenant-1. System A sends Tenant-1 to system B, where the tenant is in read-only mode. Application 2 can access the tenant more efficiently on system B than on system A.

Create an active/passive link from system B to system A that includes Tenant-2. System B sends Tenant-2 to system A, where the tenant is in read-only mode. Application 1 can access the tenant more efficiently on system A than on system B.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Bidirectional active/passive chained replication


In a bidirectional active/passive chained replication topology, three HCP systems participate in two active/passive replication chains, with each chain going in a different direction. That is, given three HCP systems A, B, and C:

In one chain, system A replicates to system B, which replicates to system C.

In the other chain, system C replicates to system B, which replicates to system A.

What this looks like

The figure below shows a bidirectional active/passive chained replication topology in which the replication chains are ABC and CBA.

1_115.jpg

In this figure:

From system A, Tenant-1 and Tenant-2 are being replicated to system B. In both tenants, all namespaces are selected for replication.

From system C, Tenant-3 is being replicated to system B. This tenant has three namespaces, all of which are selected for replication.

From system B:

oThe tenants and namespaces created by replication from system A (that is, Tenant-1 and Tenant-2 and their namespaces) are being replicated to system C because the link from A to B is included in the link from B to C.

oThe tenant and namespaces created by replication from system C (that is, Tenant-3 and its namespaces) are being replicated to system A because the link from C to B in included in the link from B to A.

Uses

A bidirectional active/passive chained replication topology allows clients to write to selected HCP tenants and namespaces and default-namespace directories on three different systems while reading from all those tenants, namespaces, and directories on all those systems. For example, suppose you have data centers in New York, Los Angeles, and Tokyo, and:

The HCP system in New York has a locally created HCP tenant named Tenant-1

The HCP system in Los Angeles has a locally created HCP tenant named Tenant-2

The HCP system in Tokyo has a locally created HCP tenant named Tenant-3

At each data center, a locally running application needs write access to the namespaces owned by the locally created tenant

Clients at all three locations need read access to the namespaces owned by all three HCP tenants

To meet these needs, you could:

Create an active/passive link (Link-1) from the New York system to the Los Angeles system that includes Tenant-1, which is writable in New York. The New York system sends Tenant-1 to the Los Angeles system, where the tenant is in read-only mode.

Create an active/passive link (Link-2) from the Los Angeles system to the Tokyo system that includes Link-1 and Tenant-2, which is writable in Los Angeles. The Los Angeles system sends both Tenant-1 and Tenant-2 to the Tokyo system, where the tenants are in read-only mode.

Create an active/passive (Link-3) from the Tokyo system to the Los Angeles system that includes Tenant-3, which is writable in Tokyo. The Tokyo system sends Tenant-3 to the Los Angeles system, where the tenant is in read-only mode.

Create an active/passive link (Link-4) from the Los Angeles system to the New York system that includes Link-3 and Tenant-2. The Los Angeles system sends both Tenant-3 and Tenant-2 to the New York system, where the tenants are in read-only mode.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Bidirectional active/passive replication with disaster recovery support


You can combine the bidirectional and one-to-many replication topologies to create a configuration in which each of two systems replicates its locally created HCP tenants and namespaces and default-namespace directories to both the other system and a third system. This third system then provides each of the first two systems with both additional assurance of continuous data availability and disaster recovery functionality.

For example, suppose HCP systems A and B each replicate their locally created HCP tenants and namespaces and default-namespace directories to each other, and you want a third system, C, to provide disaster recovery functionality for each of those systems. To create this combined topology:

Create a link from A to B that includes all the HCP tenants and namespaces and default-namespace directories that were originally created on system A.

Create a link from A to C that includes the same tenants, namespaces, and directories as the link from A to B.

Create a link from B to A that includes all the HCP tenants and namespaces and default-namespace directories that were originally created on system B.

Create a link from B to C that includes the same tenants, namespaces, and directories as the link from B to A.

The figure below shows a bidirectional replication topology with disaster recovery support.

1_88.jpg

You can combine the bidirectional and chained replication topologies to have the same result as the topology described above, which involves the bidirectional and one-to-many topologies. That is, the replicated items in the link from A to B also end up on C, and the replicated items in the link from B to A also end up on C.

In this case, however, instead of the link from A to C replicating the same items as the link from A to B, the link from A to C would include the link from B to A. Likewise, instead of the link from B to C replicating the same items as the link from B to A, the link from B to C would include the link from A to B.

The figure below shows this topology.

1_107.jpg

This topology provides better performance than does the first topology described above. Additionally, if you need to add a tenant to the link from A to B, for example, you don’t need to also add it to the link from A to C. Instead, the tenant is automatically replicated to C because the link from A to B is included in the link from B to C.

On the other hand, the first topology described above is better for assuring continuous data availability in case system A or B fails. For example, if system A is replicating to systems B and C and system B fails, the items being replicated from system A are still making it to system C. In the second topology, if system B fails, the items from system A are not replicated to system C because the replication chain is broken.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Failover and failback


Failover is a process that stops replication on a link and results in a situation in which, for read-write access:

For an active/active link, applications should use only the HCP system to which the link was failed over

For an active/passive link, applications should use only the replica

Typically, you fail over a link when one of the systems involved in the link becomes unavailable. With an active/passive link, this system must be the primary system. You don’t need to fail over the link when the replica fails.

You can fail over a link while both systems are available. You might do this, for example, if you need to shut down one of the systems for maintenance.

Depending on the link configuration, failover either is a manual procedure or occurs automatically. When automatic failover is enabled for a link, the link fails over automatically after the applicable system is unavailable for a specified amount of time.

You enable or disable automatic failover separately for each system involved in an active/active link. For an active/passive link, you enable or disable automatic failover only for the replica.

Failback is the process that restarts replication on a link that has been failed over and returns the HCP systems involved in the link to normal operation. Typically, you fail back a link when an unavailable system becomes available again.

If connectivity was lost between the two systems involved in a failed-over link, before failback can occur, connectivity must be restored. Connectivity exists when the network infrastructure through which the two systems communicate is healthy and the applicable SSL server certificates have been shared between the two systems.

In a disaster recovery situation in which the system that became unavailable has been rebuilt, the link no longer exists on that system. In this case, before failback can occur, the link must be restored to the rebuilt system.

With an active/active link, failback is a manual procedure. With an active/passive link, the failback procedure can be partially automated.

For information on sharing SSL server certificates, see Configuring SSL for replication. For instructions on enabling automatic failover and failback, see Changing automatic failover and failback settings. For instructions on performing failover and failback, see Managing failover and failback.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Failover and failback with active/active links


The effects of failing over and failing back an active/active link differ depending on whether DNS failover was enabled for the system that became unavailable. In all cases, however, when the link fails over, replication on that link stops. When the link fails back, normal replication restarts.

Failover with an active/active link

With an active/active link, failover can occur in either direction between the two systems involved in the link. While the link is failed over, the replicated HCP tenants and namespaces and default-namespace directories remain read-write on both systems. However, because failover normally occurs when one system is unavailable, to avoid wasting resources, neither system tries to read or repair objects from the other system.

With DNS failover enabled, when an active/active link fails over from one of the HCP systems involved in the link (system A) to the other system involved in the link (system B), system B broadcasts a new configuration to the DNS. This new configuration causes client requests targeted to system A by domain name to be redirected to system B when the request is for an HCP namespace or default-namespace directory that was being replicated on the failed-over link.

NoteWebHelp.png

Note: System B can service redirected namespace access requests only if the applicable namespace is configured to allow service by remote systems.

If a client request targeted to system A is for an HCP namespace or default-namespace directory that was not being replicated on the failed-over link, the request is not redirected to system B. Client requests that target system A by IP address are also not redirected to system B.

Client requests that use a domain name to target the Tenant Management Console for a replicated HCP tenant on system A are redirected to system B, but system B cannot process such requests. Instead, system B returns a 403 error code.

While the link is failed over, system A does not broadcast any configuration information to the DNS.

With DNS failover disabled, failing over an active/active link stops replication on the link but does not cause any other changes. Clients can still access system A by domain name (if system A is available).

For more information on DNS failover, see Managing DNS failover.

Failback with an active/active link

Failing back an active/active link entails a single action, fail back. When an active/active link fails back:

Replication immediately restarts in both directions on the link.

With DNS failover enabled, each HCP system involved in the link broadcasts its own configuration to the DNS. From that point on, client requests that target either system by domain name are directed to the specified system.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Failover and failback with active/passive links


The effects of failing over and failing back an active/passive link differ depending on whether DNS failover was enabled for the system that became unavailable. In all cases, however, when an active/passive link fails over, replication on that link stops. When the link fails back, normal replication restarts.

Failover with an active/passive link

Failover with an active/passive link occurs only from the primary system to the replica. When an active/passive link fails over to the replica, the replicated HCP tenants and namespaces and default-namespace directories become read-write on the replica.

With DNS failover enabled, when an active/passive link fails over, the replica broadcasts a new configuration to the DNS. This new configuration causes client requests targeted to the primary system by domain name to be redirected to the replica when the request is for an HCP namespace or default-namespace directory that was being replicated on the failed-over link.

NoteWebHelp.png

Note: A replica can service redirected namespace access requests only if the applicable namespace is configured to allow service by remote systems.

If a client request targeted to the primary system is for an HCP namespace or default-namespace directory that was not being replicated on the failed-over link, the request is not redirected to the replica. Client requests that target the primary system by IP address are also not redirected to the replica.

Client requests that use a domain name to target the Tenant Management Console for a replicated HCP tenant on the primary system are redirected to the replica, but the replica cannot process such requests. Instead, the replica returns a 403 error code.

If the primary system is still available when an active/passive link is failed over and the primary system and the replica can communicate with each other, the replicated HCP tenants and namespaces and default-namespace directories become read-only on the primary system. Also, while the link is failed over, the primary system does not broadcast any configuration information to the DNS.

If the two systems cannot communicate with each other when the link is failed over, the replicated items remain read-write on the primary system, and the primary system continues to broadcast its configuration information to the DNS. If DNS failover is disabled, clients can still access the primary system by domain name.

If clients are allowed to write to both the primary system and the replica while an active/passive link is failed-over, configuration conflicts and conflicts in namespace content may occur when the link is failed back. Although HCP resolves such conflicts in a predictable manner, the recommended practice is to avoid them in the first place. Therefore, when you fail over an active/passive link without DNS failover enabled, you should tell the applicable tenant administrators to direct all client access requests to the replica.

For information on how HCP resolves conflicts resulting from writing to both systems involved in a failed-over active/passive link, see Replication collisions. For more information on DNS failover, see Managing DNS failover.

Failover with bidirectional active/passive links

With bidirectional active/passive links, failover is a independent process for each of the two links. If one of the HCP systems involved in the links becomes unavailable, failover needs to occur only on the link for which that system is the primary system. The link for which that system is the replica does not need to be failed over, and the status of the HCP tenants and namespaces and default-namespace directories on that link does not change.

Failback with an active/passive link

Failing back an active/passive link has two phases, begin recovery and complete recovery. The begin recovery phase is always started manually. The complete recovery phase can be started manually or automatically.

When recovery begins on a link, the replica starts sending configuration changes and changes to namespace content back to the primary system. The replicated HCP tenants and namespaces and default-namespace directories remain read-write on the replica and read-only on the primary system.

The complete recovery phase is designed to allow data recovery to catch up to the current time before normal replication restarts. At the beginning of this phase, the replicated items change to read-only on the replica and remain read-only on the primary system. The replica continues to send data to the primary system until the two HCP systems involved in the link are in sync with each other. Within a minute after that point:

Normal replication restarts on the link.

With DNS failover enabled, each system involved in the link broadcasts its own configuration to the DNS. From that point on, client requests that target either system by domain name are directed to the specified system.

Before starting the complete recovery phase manually, you should wait until data recovery is caught up with the current time. When you configure automatic failback for an active/passive link, you specify how up to date data recovery must be before the complete recovery phase begins.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Replication collisions


With an active/active link, clients can make configuration changes and changes to namespace content on both HCP systems involved in the link. This situation can result in configuration and content collisions.

With an active/passive link, if clients are allowed to make changes on both HCP systems involved in the link while the link is failed over to the replica, configuration and content collisions can occur on failback. Collisions can also occur on failback if changes made on the replica while a link is failed over conflict with configuration or content that was not yet replicated at the time of failover.

The way HCP handles collisions that occur due to replication depends on the type of collision. The general rule for namespace content is that more recent changes have priority over conflicting less recent changes. If conflicting changes occur at exactly the same time, HCP gives priority to the change that occurred on the system where the link was created.

When configuration changes result in a conflict, the replication service pauses replication or recovery of the tenant, as applicable. For more information on conflicting configuration changes, see Automatically paused tenant replication or recovery.

NoteWebHelp.png

Note: If the system time is not in sync on the two systems involved in a replication link, replication collision handling may have unexpected results.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Object content collisions


An object content collision occurs when:

For an HCP namespace without versioning enabled, these events occur in the order shown:

1.An object is created with the same name in that namespace on both systems involved in a replication link, but the object has different content on the two systems.

2.The object on one of the systems is replicated to the other system.

If versioning is enabled for the namespace, no collision occurs. Instead, the less recently created of the two objects becomes an old version of the more recently created object. If the two objects were created at exactly the same time, the object that was created on the system where the link was created is treated as the newer version.

For a default-namespace directory, these events occur in the order shown:

1.An object is created with the same name in that directory on both systems involved in a replication link, but the object has different content on the two systems.

2.The object on one of the systems is replicated to the other system.

When an object content collision occurs, the more recently created object keeps its name and location. The other object is either moved to the .lost+found directory in the same namespace or renamed, depending on the namespace configuration. If the two objects were created at exactly the same time, the object that was created on the system where the link was created keeps its name and location.

When HCP moves an object to the .lost+found directory, the full object path becomes .lost+found/replication/link-id/old-object-path.

When renaming an object due to a content collision, HCP changes the object name to object-name.collision or object-name.version-id.collision, where version-id is the version ID of the object. HCP uses the second format only if versioning has ever been enabled for the namespace that contains the object but is not currently enabled.

If the new name is already in use, HCP changes the object name to object-name.1.collision or object-name.version-id.1.collision, as applicable. If that name is already in use, HCP successively increments the middle integer by one until a unique name is formed.

Objects that have been relocated or renamed due to content collisions are flagged as replication collisions in their system metadata. Clients can use the metadata query API to search for objects that are flagged as replication collisions.

If an object that’s flagged as a replication collision changes (for example, if its retention period is extended), its collision flag is removed. If a client creates a copy of a flagged object with a new name, the collision flag is not set on the copy.

Namespaces can be configured to have the disposition service automatically delete objects that are flagged as replication collisions. When selecting this option for a namespace, the tenant administrator specifies the number of days the disposition service should wait before deleting such an object. The days are counted from the time the collision flag is set. If the collision flag is removed from an object, the object is no longer eligible for deletion by the disposition service.

For information on configuring the method HCP should use to handle object name collisions in a namespace, see Changing replication options or Managing the Default Tenant and Namespace. For information the metadata query API, see Introduction to the HCP metadata query API.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

System metadata collisions


A system metadata collision occurs when these events occur in the order shown:

1.Different changes are made to the system metadata for a given object on each of the two systems involved in a replication link.

2.The changed system metadata on one of the systems is replicated to the other system.

For example, suppose a user on one system changes the shred setting for an object while a user on the other system changes the index setting for the same object. When the object on either system is replicated to the other system, a system metadata collision occurs.

If a collision occurs when changed system metadata for a given object is replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link:

For changed system metadata other than the retention setting and hold status:

oIf the last change made on system A is more recent than the last change made on system B, HCP changes the system metadata on system B to match the system metadata on system A.

oIf the last change on system B is more recent than the last change on system A, HCP does not change the system metadata on system B.

For a changed retention setting:

oIf the retention setting on system A specifies a longer retention period than does the retention setting on system B, HCP changes the retention setting on system B to match the retention setting on system A.

oIf the retention setting on system B specifies a longer retention period than does the retention setting on system A, HCP does not change the retention setting on system B.

For a changed hold status:

oIf the object is on hold on system A but not on system B, HCP places the object on hold on system B.

oIf the object is on hold on system B but not on system A, HCP leaves the object on hold on system B.

Here are some examples of how HCP handles collisions when changed system metadata for a given object is replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link.

Example 1

The object starts out on both system A and system B with these system metadata settings:

Shred: false
Index: false

The table below shows a sequence of events in which the system metadata for the object is changed and the changes are then replicated.

Sequence Event
1 On system A, a client changes the shred setting to true.
2 On system B, a client changes the index setting to true.
3

The changes on system A are replicated to system B. The resulting settings for the object on system B are:

Shred: false
Index: true

Example 2

The object starts out on both system A and system B with these system metadata settings:

Retention: Initial Unspecified
Shred: false
Index: false

The table below shows a sequence of events in which the system metadata for the object is changed and the changes are then replicated.

Sequence Event
1 On system A, a client changes the retention setting to Deletion Prohibited.
2 On system B, a client changes the retention setting to Deletion Allowed.
3 On system B, a client changes the index setting to true.
4 On system A, a client changes the shred setting to true.
5

The changes on system A are replicated to system B. The resulting settings for the object on system B are:

Retention: Deletion Prohibited
Shred: true
Index: false

Example 3

The object starts out on both system A and system B with these system metadata settings:

Retention: Initial Unspecified
Hold: true
Shred: false
Index: false

The table below shows a sequence of events in which the system metadata for the object is changed and the changes are then replicated.

Sequence Event
1

On system A, a client changes the retention setting to Deletion Allowed.

2

On system B, a client changes the retention setting to Deletion Prohibited.

3

On system B, a client changes the index setting to true.

4

On system A, a client changes the shred setting to true.

5

On system A, a client releases the object from hold.

6

The changes on system A are replicated to system B. The resulting settings for the object on system B are:

Retention: Deletion Prohibited
Hold: true
Shred: true
Index: false

7

The changes on system B are replicated to system A. The resulting settings for the object on system A are:

Retention: Deletion Prohibited
Hold: true
Shred: true
Index: false

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Custom metadata collisions


A custom metadata collision occurs when:

For an HCP namespace, these events occur in the order shown:

1.One of these changes occurs:

An annotation is added with the same name to a given object on both systems involved in a replication link, but the annotation has different content on the two systems.

The addition of an annotation to a given object on only one of the systems does not result in a custom metadata collision if the object does not have an annotation with the same name on the other system. In this case, the new annotation is replicated without conflict.

Different changes are made to the content of a given annotation for a given object on each of the two systems involved in a replication link.

A change is made to the content of a given annotation for a given object on one of the systems involved in a replication link, and the same annotation is deleted on the other system involved in the link.

2.The change made on one of the systems is replicated to the other system.

For a default-namespace directory, these events occur in the order shown:

1.One of these changes occurs:

Custom metadata is added to a given object on both systems involved in a replication link, but the added custom metadata is different on the two systems.

The addition of custom metadata to an object on only one of the systems involved in a replication link does not result in a custom metadata collision. Instead, the new custom metadata is replicated from that system to the other system involved in the link without conflict.

The custom metadata for a given object is replaced on both systems involved in a replication link, but the replacement custom metadata is different on the two systems.

The custom metadata for a given object is replaced on one of the systems involved in a replication link, and the same custom metadata is deleted on the other system involved in the link.

2.The change made on one of the systems is replicated to the other system.

If a collision occurs when a custom metadata change for a given object is replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link:

If the last change on system A is more recent than the last change on system B, HCP applies the change from system A to the custom metadata on system B

If the last change on system B is more recent than the last change on system A, HCP does not change the custom metadata on system B

Here are some examples of how HCP handles collisions when custom metadata changes for a given object are replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link.

Example 1

In an HCP namespace, the object starts out with annotations named a1 and a2 on both system A and system B.

The table below shows a sequence of events in which the annotations for the object are changed and the changes are then replicated.

Sequence Event
1

On system B, a client changes the content of a1.

2

On system A, a client makes a different change to the content of a1.

3

On system A, a client adds annotation a3 to the object.

4

On system B, a client adds annotation a3 with different content from the a3 added on system A.

5

The changes on system A are replicated to system B. The resulting annotations for the object on system B are:

a1 with the changed content from system A
a2 (unchanged)
a3 with the content added on system B

6

The changes on system B are replicated to system A. The resulting annotations for the object on system A are:

a1 with the changed content from system A
a2 (unchanged)
a3 with the content added on system B

Example 2

In an HCP namespace, the object starts out with the annotations named a1, a2, and a3 on both system A and system B.

The table below shows a sequence of events in which the annotations for the object are changed and the changes are then replicated.

Sequence Event
1

On system B, a client changes the content of a1.

2

On system A, a client deletes a1.

3

On system A, a client changes the content of a2.

4

On system B, a client changes the content of a2.

5

On system A, a client deletes a3.

6

On system B, a client changes the content of a3.

7

The changes on system A are replicated to system B. The resulting annotations for the object on system B are:

a2 with the changed content from system B
a3 with the changed content from system B

8

The changes on system B are replicated to system A, the resulting annotations for the object on system A are:

a2 with the changed content from system B
a3 with the changed content from system B

Example 3

In a default-namespace directory, the object starts out with same custom metadata on system A and system B.

The table below shows a sequence of events in which the custom metadata for the object is changed and the change is then replicated.

Sequence Event
1

On system B, a client replaces the custom metadata for the object with new custom metadata.

2

On system A, a client replaces the custom metadata for the object with different custom metadata from the custom metadata used on system B.

3

The change on system A is replicated to system B. The resulting custom metadata for the object on system B is the new custom metadata from system A.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Access control list collisions


An access control list (ACL) collision occurs when these events occur in the order shown:

1.Different changes are made to the ACL for a given object on each of the two systems involved in a replication link.

2.The changed ACL on one of the systems is replicated to the other system.

An ACL is treated as a single unit. If a collision occurs when a changed ACL for a given object is replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link:

If the last change to the ACL on system A is more recent than the last change to the ACL on system B, HCP changes the ACL on system B to match the changed ACL on system A

If the last change to the ACL on system B is more recent than the last change to the ACL on system A, HCP does not change the ACL on system B

For example, suppose the ACL for a given object starts out with these grants on both system A and system B:

All users: read
User lgreen: write
User mwhite: write, delete

The table below shows a sequence of events in which the ACL for the object is changed and the change is then replicated.

Sequence Event
1

On system B, a client changes the grants in the ACL to:

All users: read
User lgreen: write, delete
User mwhite: write, delete, read ACL

2

On system A, a client changes the grants in the ACL to:

All users: read
User mwhite: write
User pdgrey: write

3

The changed ACL on system A is replicated to system B. The resulting ACL for the object on system B contains these grants:

All users: read
User mwhite: write
User pdgrey: write

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Configuration collisions


A configuration collision occurs when these events occur in the order shown:

1.Different changes are made to the same configuration property on each of the two systems involved in a replication link.

2.The changed property on one of the systems is replicated to the other system.

Examples of configuration properties are:

The namespace quota for a tenant
The data access permission mask for a tenant
The versioning setting for a namespace
The default shred setting for a namespace
The roles for a user account
The data access permissions a group account has for a namespace
The protocol optimization setting on a namespace

Certain groups of properties are treated as a single unit. Generally, these groups consist of properties that are updated by a single submission in the System or Tenant Management Console. Two notable exceptions to this rule are data access permissions for user accounts and content properties for content classes. In these cases, each set of data access permissions for a namespace and each content property is treated as an individual property.

If a collision occurs when a configuration change is replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link:

If the last change to the configuration on system A is more recent than the last change to the configuration on system B, HCP changes the configuration on system B to match the configuration on system A

If the last change to the configuration on system B is more recent than the last change to the configuration on system A, HCP does not change the configuration on system B

The rules above apply to all configuration collisions except collisions that occur when retention class properties are changed. For information on how HCP handles this type of collision, see Retention class collisions.

Here are two examples of how HCP handles collisions when configuration changes are replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link.

Example 1

A given tenant starts out on both system A and system B with these properties:

Namespace quota: 5
Versioning: disabled

The table below shows a sequence of events in which the tenant configuration is changed and the change is then replicated.

Sequence Event
1

In the System Management Console on system B, an administrator changes the namespace quota for the tenant to ten.

2

In the System Management Console on system A, an administrator enables versioning for the tenant.

3

The change on system A is replicated to system B. Because namespace quota and versioning are properties in the same submission group, the resulting properties for the tenant on system B are:

Namespace quota: 5
Versioning: enabled

Example 2

A given tenant-level user account starts out on both system A and system B with these roles and data access permissions:

Roles: monitor, compliance
Namespace-1 permissions: browse, read, write, delete

The table below shows a sequence of events in which the user account is changed and the changes are then replicated.

Sequence Event
1

In the System Management Console on system B, an administrator changes the namespace quota for the tenant to ten.

2

In the System Management Console on system A, an administrator enables versioning for the tenant.

3

The change on system A is replicated to system B. Because namespace quota and versioning are properties in the same submission group, the resulting properties for the tenant on system B are:

Namespace quota: 5
Versioning: enabled

4

In the Tenant Management Console on system A, an administrator adds the privileged permission to Namespace-1.

5

In the Tenant Management Console on system B, an administrator gives the user account browse and read permissions for Namespace-2.

6

In the Tenant Management Console on system A, an administrator gives the user account browse, read, and write permissions for Namespace-3.

7

The changes on system A are replicated to system B. Because roles and data access permissions are in separate submission groups, the resulting roles and data access permissions for the user account on system B are:

Roles: compliance
Namespace-1 permissions: browse, read, write, delete, privileged
Namespace-2 permissions: browse, read
Namespace-3 permissions: browse, read, write

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Retention class collisions


A retention class collision occurs when these events occur in the order shown:

1.Different changes are made to the same retention class on each of the two systems involved in a replication link.

2.The changed retention class on one of the systems is replicated to the other system.

If a collision occurs when a change to a retention class is replicated from one system (system A) involved in a replication link to the other system (system B) involved in the link:

If the last change to the retention class on system A is more recent than the last change to the class on system B and:

oThe value of the class on system A is greater than the value of the class on system B, HCP changes the value of the class on system B to the value of the class on system A

oThe value of the class on system A is less than the value of the class on system B and:

System B is in enterprise mode, HCP changes the value of the class on system B to the value of the class on system A

NoteWebHelp.png

Note: An exception to this rule is when the value of the class on system A is -2 (Initial Unspecified) and the value of the class on system B is not 0 (Deletion Allowed). In this case, the value of the class on system B does not change.

System B is in compliance mode, HCP does not change the value of the class on system B

If the last change to the retention class on system B is more recent than the last change to the class on system A, HCP does not change the value of the class on system B

NoteWebHelp.png

Note: A retention class value of -1 (Deletion Prohibited) is greater than a value that’s a specific duration. A retention class value of 0 (Deletion Allowed) or -2 (Initial Unspecified) is less than a value that’s a specific duration.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Considerations for cross-release replication


HCP release 8.x systems support replication with other release 8.x systems and with release 7.x systems. HCP does not support replication between 8.x systems and systems earlier than 7.0.

Replication between a release 8.x system and a release 7.x system is called cross-release replication.

Cross-release replication and erasure coding topologies

Erasure coding topologies cannot include HCP systems at a release earlier than 8.0.

Cross-release replication and multipart objects

HCP cannot replicate multipart objects between an 8.x system and a 7.x system. As a result:

When you add a tenant to a cross-release replication link, HCP automatically pauses replication of that tenant if both of the following are true for any of the tenant's namespaces:

oThe namespace has replication enabled.

oThe namespace contains one or more multipart objects.

When you select a namespace for replication, HCP automatically pauses replication of the tenant that owns that namespace if both of the following are true:

oThe replication link on which the tenant is being replicated is a cross-release link.

oThe namespace contains one or more multipart objects.

When a multipart upload is completed on a namespace, HCP automatically pauses replication of the tenant that owns that namespace if both of the following are true:

oThe namespace has replication enabled.

oThe replication link on which the tenant is being replicated is a cross-release link.

For more information on multipart objects and multipart uploads, see About multipart uploads. For more information on automatically paused tenant replication, see Automatically paused tenant replication or recovery.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.