Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

About service plans

Each namespace has a service plan that defines both a storage tiering strategy and a data protection strategy for the objects in that namespace. At any given point in the lifecycle of an object, its storage tiering strategy specifies the types of storage on which copies of that object must be stored, and its data protection strategy specifies the number of object copies that must be stored on each type of storage.

For the purposes of storage tiering and data protection, HCP treats these as individual objects:

  • Parts of multipart objects
  • Chunks for erasure-coded objects
  • Chunks for erasure-coded parts of multipart objects

The service plan for a given namespace defines one or more tiers of storage that can be used to store objects in that namespace. For each object in the namespace, at any given point in the object lifecycle, the service plan specifies the criteria that determine which storage tiers must be used to store copies of that object and the number of copies of that object that must be stored on each tier.

Primary running storage is designed to provide both high data availability and high performance for object data storage and retrieval operations. To optimize data storage price/performance for the objects in a namespace, you can configure the service plan for that namespace to define S Series storage as the ingest tier or create a storage tiering strategy that specifies multiple storage tiers.

By default, HCP stores every object on primary running storage, so every service plan automatically defines primary running storage as the initial storage tier, called the ingest tier. If the HCP system is using S Series storage, this storage can be used as the ingest tier instead of primary running storage. This causes object data to be written directly to the S Series Nodes. Even if you set S Series storage as the ingest tier, metadata is stored on primary running storage.

Storage tiers

The service plan for a given namespace defines one or more storage tiers for that namespace. For each storage tier, the service plan specifies the data protection level (DPL) and the primary running storage metadata protection level (MPL) for each storage tier:

  • The DPL for a given storage tier is the number of copies of the object data that HCP must maintain for each object that’s stored on that tier.
  • The MPL for a given storage tier is the number of copies of the object metadata that HCP must maintain on primary running storage for each object that’s stored on the tier.

For any given storage tier, the primary running storage MPL must be equal to or greater than the DPL.

NoteFor the purpose of storage tiering, HCP treats parts of multipart objects, chunks for erasure-coded objects, and chunks for erasure-coded parts of multipart objects as individual objects.

Every service plan initially defines primary running storage as the initial storage tier, called the ingest tier, and specifies a DPL setting and an MPL setting for that tier. If the HCP system is using S Series storage, you can define S Series storage as a service plan ingest tier to be used as an alternative to primary running storage. If an S Series pool, or pools, is selected as the ingest tier, DPL copies of objects are written directly to S Series storage. MPL copies of object metadata are written to primary running storage.

Tiering objects that are subject to erasure coding

HCP does not tier objects that are subject to erasure coding but not yet erasure coded unless the target tier is primary running storage, an S Series Node, or primary spindown storage. Instead, on any given system in an erasure coding topology, if the system initially receives a full copy of the data for an object that's subject to erasure coding, HCP waits to tier the object until the full copy of the data has been reduced to a chunk.

After the system has a chunk for the object, HCP can move the chunk to the applicable tier, according to the tiering strategy in the service plan associated with the namespace that contains the object. If the object was due to be tiered before the full copy of the data was reduced to chunk, HCP immediately moves the chunk to the applicable tier.

If a system initially receives a chunk for an object, that chunk is immediately subject to the tiering strategy in the applicable service plan.

On any given storage tier, for an object that's subject to erasure coding, HCP keeps DPL copies of the full object data or the chunk for the object, as applicable, where DPL is the DPL specified for that storage tier.

DPL settings for objects and default-namespace directories

Objects have a DPL setting as part of their metadata. This setting is the DPL for the tier where the object data currently should be stored, according the service plan associated with the namespace that contains the object. If the object is not yet on that tier or if the DPL for the tier changes while the object is on that tier, the actual number of copies of the object data may not immediately match the DPL setting for the object.

Directories in the default namespace also have a DPL setting as part of their metadata. This setting is the DPL for the ingest tier in the service plan associated with the default namespace.

Users and applications can see, but not modify, the DPL setting for an object or directory.

Default ingest tier DPL setting

Each HCP system has a default service plan that’s automatically created and configured during the HCP software installation. The default service plan is applied to each tenant and namespace for which another service plan is not explicitly selected.

For any given HCP system, the ingest tier DPL setting that’s initially configured for the default service plan is used as the default ingest tier DPL setting for each new service plan that’s created on the HCP system. When you create a new service plan, you can choose to use the default ingest tier DPL setting or select a different setting. You can also modify any service plan, including the default service plan, to change the ingest tier DPL setting. However, changing the ingest tier DPL setting used for the default service plan does not change the default ingest tier DPL setting that’s used for new service plans.

Typically, the default ingest tier DPL setting used for service plans created on an HCP system is the optimal setting for the type of physical storage that the HCP system uses. For RAIN systems, the default ingest tier DPL is two. For SAIN systems, for which SAN arrays provide a high level of data protection, and for VM systems, for which the virtual environment provides node failover capabilities, the default ingest tier DPL is one.

The default ingest tier DPL setting used for all service plans that are created on an HCP system is the optimal setting for the type of storage used by that system. However, the optimal ingest tier DPL setting that’s configured for the service plan that’s assigned to a specific tenant or namespace is subject to considerations such as whether the tenant or namespace is being replicated and whether the HCP system owner or the tenant administrator has any particular data protection needs.

On HCP RAIN systems, by default, service plans can be configured to set the ingest tier DPL to two, three, or four. To enable HCP RAIN system administrators to configure service plans to set the ingest tier DPL to one, contact your authorized HCP service provider.

Storage tier properties specified in service plans

For each storage tier, including the ingest tier, that’s defined for a given namespace, the service plan specifies:

  • The storage pools that are used to store copies of object data or chunks for erasure-coded objects on the tier. Each storage pool consists of one or more storage components. Each storage component represents a type of primary storage (running or spindown), S Series storage, an extended storage device, or a cloud storage service endpoint.
  • For each object that’s stored on the tier, the number of copies of the object data or of the chunk for an erasure-coded object that HCP must maintain on each storage pool (that is, the DPL).
  • For each object that’s stored on the tier, the number of copies of the object metadata that HCP must maintain on the ingest tier (that is, the MPL).
  • The transition criteria for each tier except for the ingest tier. The transition criteria for a storage tier are the rules that determine when one or more copies of each object in the namespace must be stored on the tier:
    • The object age (number of days since ingest) at which one or more copies of the object data must be moved from the previous tier to this tier.

      The age of an object created by a multipart upload is counted from the time the multipart upload was completed.

    • For service plans that define exactly two tiers, including the ingest tier, whether a threshold will be applied to the second tier, and if so, the percentage of ingest tier storage capacity that must be used (the threshold) before object data can be moved to the second storage tier.
    • For HCP systems with replication enabled, whether objects must be fully replicated before they can be transitioned from the previous tier onto this tier. If replication is disabled for the HCP system, this transition criterion does not appear in the HCP System Management Console.
  • For a namespace that’s currently being replicated to another system, whether the copies of the object that are stored on the tier are to be made metadata-only.

    Regardless of the transition criteria that are specified for a metadata-only tier, objects are moved to such a tier only after they are replicated. When a replicated object is moved to a metadata-only tier, all existing copies of the object data are deleted from the current tier and the ingest tier, and HCP stores the specified number of copies of the object metadata on primary running storage.

    Erasure-coded objects never become metadata-only on any systems in the applicable erasure coding topology. If an erasure-coded object reaches a point in its lifecycle where the object should move to a metadata-only tier, the chunk for the object remains on the last tier before the metadata-only tier, and the DPL for that tier still applies to the chunk.

  • Whether the data for each object stored on the tier is rehydrated (that is, restored on the ingest tier) upon being read from the tier, and if so, the number of days HCP is required to keep a rehydrated copy of object data on the ingest tier.

    Rehydration does not apply to erasure-coded objects. Instead, if the applicable erasure coding topology specifies a restore period, after an erasure-coded object is read, the chunk for the object is restored to a full copy on the ingest tier on the system from which the object was read.

How HCP uses the information in service plans

If the service plan for a given namespace defines only the ingest tier, for each object in that namespace, the Protection service works to ensure that:

  • The correct number of copies of the object data are always stored on the ingest tier, which can be primary running storage or S Series storage
  • The correct number of copies of the object metadata are always stored on primary running storage

If the service plan for a given namespace defines multiple storage tiers, the Storage Tiering service works to ensure that each object in that namespace is stored on the correct tier and the correct number of copies of the object are stored on each tier.

For each object in that namespace, the Storage Tiering service:

  • Moves copies of the object data among the storage tiers that are defined for the namespace to satisfy the transition criteria that are defined for each storage tier
  • Upon moving all existing copies of the data for an object from one tier to another:
    • If the new tier has a different DPL from the previous tier, creates or deletes copies of the object data as required to satisfy the DPL setting for the new tier
    • If the new tier has a different primary running storage metadata protection level (MPL) from the previous tier, creates or deletes copies of object metadata as required to satisfy the MPL setting for the new tier
  • Upon moving a replicated object to a metadata-only tier, deletes all copies of the object data from the previous tier, and if the previous tier is not the ingest tier, deletes any copies of the object data that exist on the ingest tier (primary running storage or S Series storage)
  • Checks to see whether the object data has been read from a data storage tier for which rehydration is enabled, and if so, creates an extra copy of the object data on the ingest tier
  • After moving a replicated object to a metadata-only tier for which rehydration is enabled and making that object metadata-only, checks to see whether that object has been read from a remote system, and if so, restores the data to each copy of the object that’s stored on the ingest tier (primary running storage or S Series storage)

Metadata-only storage tiers

When the Storage Tiering service moves an object off the ingest tier and onto another storage tier, the service removes all copies of the object data from ingest tier and stores the specified number of copies of the object data on the new storage tier. However, at least one copy of the object metadata must always remain on primary running storage. For each storage tier that’s defined for a given namespace, the service plan specifies the number of copies of object data that must be stored on the tier and the number of copies of object metadata that must be stored on primary running storage.

A service plan can include a metadata-only storage tier. For each object that’s stored on a metadata-only tier, the service plan specifies the number of copies of the object metadata that must be stored on primary running storage, but the service plan also specifies that no copies of the data for that object can be stored on any storage tier, including the ingest tier.

Each service plan can be configured to define only one metadata-only storage tier. Additionally, that tier must be the last storage tier that’s defined by the service plan. After objects in a namespace are moved to the metadata-only tier that’s defined for that namespace, the data for those objects is removed from all storage tiers defined for the namespace.

The Storage Tiering service makes an object metadata-only only when all of these conditions are true:

  • The service plan for the namespace that contains the object defines a metadata-only storage tier.
  • The object is on the storage tier that immediately precedes the metadata-only tier defined in the namespace service plan, and the object meets the transition criteria specified for the metadata-only storage tier.
  • A copy of the data for the object exists on at least one other HCP system in the replication topology in which the current system participates. (This is possible because service plans with the same name can have different definitions on different systems.)

When all these conditions are true, the Storage Tiering service deletes all copies of the data for the object from the preceding storage tier. If the preceding storage tier is not the ingest tier, the Storage Tiering service also deletes any copies of the object data that exist on the ingest tier. After deleting all copies of the data for the object, the Storage Tiering service creates or deletes copies of the object metadata on primary running storage, as necessary, to satisfy the primary running storage MPL that’s specified for the metadata-only storage tier.

For the purpose of making objects metadata-only, the Storage Tiering service treats each part of a multipart object as a separate object. The applicable number of copies of the metadata for the part are stored on primary running storage, and no data for the part is stored on any storage tier.

Erasure-coded objects are never made metadata-only on any system in the applicable erasure coding topology. If an erasure-coded object is due to move to a metadata-only storage tier, the chunk for the object remains on the previous tier.

On the other hand, an object that is subject to erasure coding but not yet erasure coded can be made metadata-only on a system if all of these are true:

  • The system does not have a chunk for the object.
  • The erasure coding delay has not yet expired for the object.
  • The object is due to be made metadata-only on that system.
  • At least one other available system in the erasure coding topology has a full copy of the object data.

If rehydration is enabled for a metadata-only storage tier, when rehydrating a replicated object that’s been read from the ingest tier on a remote system, the Storage Tiering service rehydrates all copies of that object on the ingest tier of the local system.

When replicating an object in a namespace to a system on which objects in that namespace can be made metadata-only, HCP replicates only the object metadata if the object is larger than one MB. If the object is smaller than one MB, HCP replicates both the data and metadata.

Here’s a scenario that shows how allowing metadata-only objects can be used to advantage:

  • You have a many-to-one replication topology in which the HCP systems at the outlying sites are much smaller than the central HCP system to which they all replicate. To optimize the use of storage on the outlying systems, you allow the namespaces on those systems to have metadata-only objects while requiring the central system to have the object data. The outlying systems respond to client requests for object data by reading the data from the central system.

In this scenario, the replication topology should include a disaster recovery system (that is, a replica of the central system) to protect against data loss in case of a catastrophic failure of the central system.

ImportantHCP does not prevent you from removing a namespace from a replication topology even if the namespace contains metadata-only objects on one or more systems in that topology. This can result in data for objects in that namespace being permanently inaccessible from those systems. In most cases, HCP warns you if the modification you’re making to a replication link would cause this condition to occur.

For the HDDS search facility to index the data for metadata-only objects, the objects must be rehydrated.

Rehydration

HCP reads objects most efficiently from the ingest tier, which can be either primary running storage or S Series storage, on the system that’s the target of the read request. If you anticipate frequent reads of the same objects in a namespace while the objects are stored on a specific extended storage tier, consider reconfiguring the service plan used by the namespace to enable rehydration on the applicable tier. Similarly, if you anticipate frequent reads of the same objects in a namespace that contains metadata-only objects, consider configuring the service plan used by the namespace to enable rehydration on the metadata-only tier.

Default service plan configuration settings

Each HCP system has a service plan named Default that’s automatically created and configured during the HCP software installation. The Default service plan is applied to each tenant and namespace for which another service plan is not explicitly selected.

When the HCP system is first installed, the Default service plan defines primary running storage (the ingest tier) as the only storage tier. On SAIN and VM systems, by default, the ingest tier DPL and MPL are both set to one. On RAIN systems, by default, the ingest tier DPL and MPL are both set to two.

You can modify the Default service plan at any time to define one or more additional storage tiers or to change the ingest tier DPL and MPL settings.

 

  • Was this article helpful?