Skip to main content
Hitachi Vantara Knowledge

Overview

Introduction to Hitachi Content Platform

Hitachi Content Platform (HCP) is a distributed storage system designed to support large, growing repositories of fixed-content data. HCP stores objects that include both data and metadata that describes the data.

HCP provides access to stored objects through a variety of industry-standard protocols, as well as through various HCP-specific interfaces.

This chapter introduces basic HCP concepts and includes information on what you can do with an HCP namespace.

About Hitachi Content Platform

Hitachi Content Platform is a combination of hardware and software that provides an object-based data storage environment. An HCP repository stores all types of data, from simple text files to medical images to multigigabyte database images.

HCP provides easy access to the repository for adding, retrieving, and deleting data. HCP uses write-once, read-many (WORM) storage technology and a variety of policies and internal processes to ensure the integrity of the stored data and the efficient use of storage capacity

Object-based storage

HCP stores objects in a repository. Each object permanently associates data HCP receives (for example, a document, an image, or a movie) with information about that data, called metadata.

An object encapsulates:

  • Fixed-content data

    An exact digital reproduction of data as it existed before it was stored in HCP. Once it’s in the repository, this fixed-content data cannot be modified.

  • System metadata

    System-managed properties that describe the fixed-content data (for example, its size and creation date). System metadata includes policies, such as retention and data protection level, that influence how transactions and internal processes affect the object.

  • Custom metadata

    Optional metadata that a user or application provides to further describe the object. Custom metadata is specified as one or more annotations, where each annotation is a discrete unit of information about the object. Annotations are typically specified in XML format.

    You can use custom metadata to create self-describing objects. Users and applications can use this metadata to understand and repurpose object content.

  • Access control list (ACL)

    Optional metadata consisting of a set of grants of permissions to perform various operations on the object. Permissions can be granted to individual users or to groups of users.

    ACLs are provided by users or applications and are specified in either XML or JSON format.

HCP can store multiple versions of an object, thus providing a history of how the data has changed over time. Each version is a separate object, with its own system metadata and, optionally, its own custom metadata and ACL.

HCP supports multipart uploads with the Hitachi API for Amazon S3. With a multipart upload, the data for an object is broken into multiple parts that are written to HCP independently of each other. Even though the data is written in multiple parts, the result of a multipart upload is a single object. An object for which the data is stored in multiple parts is called a multipart object.

Namespaces and tenants

An HCP repository is partitioned into namespaces. A namespace is a logical grouping of objects such that the objects in one namespace are not visible in any other namespace.

Namespaces provide a mechanism for separating the data stored for different applications, business units, or customers. For example, you could have one namespace for accounts receivable and another for accounts payable.

Namespaces also enable operations to work against selected subsets of repository objects. For example, you could perform a query that targets the accounts receivable and accounts payable namespaces but not the employees namespace.

Namespaces are owned and managed by administrative entities called tenants. A tenant typically corresponds to an organization such as a company or a division or department within a company.

Namespace access

HCP supports access to namespace content through several namespace access protocols, the HCP Namespace Browser, the HCP metadata query API, the HCP Search Console, and HCP Data Migrator.

Namespace access protocols

HCP supports access to namespace content through several industry-standard protocols:

  • A RESTful HTTP API (called REST in HCP).
  • Hitachi API for Amazon S3, which is a RESTful, HTTP-based S3 compatible API. With the S3 compatible API, namespaces are called buckets.
  • HSwift, which is a RESTful, HTTP-based API that’s compatible with OpenStack Swift. With HSwift, namespaces are called containers.
  • WebDAV.
  • CIFS.
  • NFS.

These protocols support various operations: storing data, creating directories, viewing object data and metadata, viewing directories, modifying certain metadata, and deleting objects. You can use these protocols to access data with a web browser, third-party applications, Windows® Explorer, and other native Windows and Unix tools.

HCP allows special-purpose access to namespaces through the SMTP protocol. This protocol is used only for storing email.

The namespace access protocols are configured separately for each namespace and are enabled or disabled independently of each other.

The HTTP, S3 compatible, HSwift, and CIFS protocols can be configured to require authentication. To use a protocol that requires authentication, users and applications must present valid credentials for access to the namespace.

If the HTTP, S3 compatible, HSwift, or CIFS protocol is enabled but is not configured to require authentication or if the WebDAV or NFS protocol is enabled, users and applications can access the namespace anonymously. You can create a secure namespace by enabling only protocols that require authentication.

HCP Namespace Browser

The HCP Namespace Browser lets you manage content in and view information about namespaces. With the Namespace Browser, you can:

  • List, view, and retrieve objects, including old versions of objects
  • View custom metadata and ACLs for objects, including old versions of objects
  • Store and delete objects
  • Create empty directories
  • Display namespace information, including:
    • The namespaces that you own or can access
    • Retention classes available for a given namespace
    • Permissions for namespace access
    • Namespace statistics such as the number of objects in a given namespace or the total capacity of the namespace
HCP metadata query API

The HCP metadata query API lets you search HCP for objects that meet specified criteria. The API supports two types of queries:

  • Object-based queries search for objects based on object metadata. This includes both system metadata and the content of custom metadata and ACLs. The query criteria can also include the object location (that is, the namespace and/or directory that contains the object). These queries use a robust query language that lets you combine search criteria in multiple ways.

    Object-based queries search only for objects that currently exist in the repository.For objects with multiple versions, object-based queries return only the current version.

  • Operation-based queries search not only for objects currently in the repository but also for information about objects that have been deleted by a user or application or deleted through disposition, purged or pruned. For namespaces that support versioning, operation-based queries can return both current and old version of objects.

    Criteria for operation-based queries can include object status (for example, created or deleted), change time, index setting, and location.

The metadata query API returns object metadata only, not object data. The metadata is returned either in XML format, with each object represented by a separate element, or in JSON format, with each object represented by a separate name/value pair. For queries that return large numbers of objects, you can use paged requests.

HCP Search Console

The HCP Search Console is an easy-to-use web application that lets you search for and manage objects based on specified criteria. For example, you can search for objects that were stored before a certain date or that are larger than a specified size. You can then delete the objects listed in the search results or prevent those objects from being deleted. Similar to the metadata query API, the Search Console returns only object metadata, not object data.

By offering a structured environment for performing searches, the Search Console facilitates e-discovery, namespace analysis, and other activities that require the user to examine the contents of namespaces. From the Search Console, you can:

  • Open objects
  • Perform bulk operations on objects
  • Export search results in standard file formats for use as input to other applications
  • Publish feeds to make search results available to web users

The Search Console works with either of these two search facilities:

  • The HCP metadata query engine

    This facility is integrated with HCP and works internally to perform searches and return results to the Search Console. The metadata query engine is also used by the metadata query API.

    NoteWhen working with the metadata query engine, the Search Console is called the Metadata Query Engine Console.
  • The Hitachi Data Discovery Suite (HDDS) search facility

    This facility interacts with Data Discovery Suite, which performs searches and returns results to the HCP Search Console. Data Discovery Suite is a separate product from HCP.

The Search Console can use only one search facility at any given time. The search facility is selected at the HCP system level. If no facility is selected, the HCP system does not support use of the Search Console to search namespaces.

Each search facility maintains its own index of objects in each search-enabled namespace and uses this index for fast retrieval of search results. The search facilities automatically update their indexes to account for new and deleted objects and changes to object metadata.

NoteNot all namespaces support search. To find out whether a namespace is search-enabled, see your tenant administrator.
HCP Data Migrator

HCP Data Migrator (HCP-DM) is a high-performance, multithreaded, client-side utility for viewing, copying, and deleting data.

With HCP-DM, you can:

  • Copy objects, files, and directories between the local file system, HCP namespaces, default namespaces, and earlier HCAP archives
  • Delete individual objects, files, and directories and perform bulk delete operations
  • View the content of current and old versions of objects and the content of files
  • Purge all versions of an object
  • Rename files and directories on the local file system
  • View object, file, and directory properties
  • Change system metadata for multiple objects in a single operation
  • Add, replace, or delete custom metadata for objects
  • Add, replace, or delete ACLs for objects
  • Create empty directories

HCP-DM has both a graphical user interface (GUI) and a command-line interface (CLI).

HCP nodes

The core hardware for an HCP system consists of servers that are networked together. These servers are called nodes.

When you access an HCP system, your point of access is an individual node. To identify the system, however, you can use either the domain name of the system or the IP address of an individual node. When you use the domain name, HCP selects the access node for you. This helps ensure an even distribution of the processing load.

Permissions

To access a namespace and take action in it, clients must have the necessary permissions. The list below describes the possible permissions and the operations they allow.

  • Browse

    • List directory contents.
    • Check for directory existence.
  • Read

    • Retrieve objects and system metadata.
    • Check for object existence.
    • List annotations.
    • Check for and retrieve annotations.

    Read operations also require browse permission.

  • Read ACL

    Check for and retrieve ACLs.

  • Write

    • Store objects.
    • Create directories.
    • Modify system metadata.
    • Add and replace annotations.
  • Write ACL

    Add, replace, and delete ACLs.

  • Delete

    Delete objects, empty directories, annotations, and ACLs.

  • Purge

    Delete objects and their old versions (also requires delete permission).

  • Privileged

    • Delete or purge objects regardless of retention (also requires delete or purge permissions).
    • Place objects on hold or release objects from hold (also requires write permission).
  • Change owner

    Change object owners.

  • Search

    Search for objects (also requires browse and read permissions).

NoteWhen using the CIFS protocol with a Windows client, you need both read and write permissions to store objects.
Data access permission mask

The operations allowed in a namespace are determined by a data access permission mask for the namespace. Data access permission masks are set at the system, tenant, and namespace levels.

The effective permissions for a namespace are the operations that are allowed by the mask at all three levels. That is, to be in effect for a namespace, a permission must be included in the system-level permission mask, the tenant-level permission mask, and the namespace-level permission mask.

User permissions

To perform an operation in a namespace, the operation must be allowed by the effective permission mask and by your user permissions. The permissions for what you can do in a namespace come from your user account (if you’re an authenticated user), the namespace configuration, and, for individual objects, the object ACL.

NoteACLs are enabled on a per-namespace basis. In namespaces where ACLs are enabled, the namespace can be configured to either enforce or ignore the permissions granted by ACL. To find out the ACLs settings for a namespace, contact your tenant administrator.

Replication

Replication is a process that supports configurations in which selected tenants and namespaces are maintained on two or more HCP systems and the objects in those namespaces are managed across those systems. This cross-system management helps ensure that data is well-protected against the unavailability or catastrophic failure of a system.

A replication topology is a configuration of HCP systems that are related to each other through replication. Typically, the systems in a replication topology are in separate geographic locations and are connected by a high-speed wide area network. This arrangement provides geographically distributed data protection (called geo-protection).

You can read from namespaces on all systems where those namespaces are replicated. The replication topology, which is configured at the system level, determines the systems on which you can write to namespaces.

Replication has several purposes, including:

  • If a system in a replication topology becomes unavailable (for example, due to network issues), another system in the topology can provide continued data availability.
  • If a system in a replication topology suffers irreparable damage, another system in the topology can serve as a source for disaster recovery.
  • If multiple HCP systems are widely separated geographically, each system may be able to provide faster data access for some applications than the other systems can, depending on where the applications are running.
  • If an object cannot be read from one system in a replication topology (for example, because a node is unavailable), HCP can try to read it from another system in the topology. Whether HCP tries to do this depends on the namespace configuration.
  • If a system in a replication topology is unavailable, HTTP requests to that system can be automatically serviced by another system in the topology. Whether HCP tries to do this depends on the namespace configuration.

Operations

You use namespace access protocols to perform operations in a namespace. The operations you can perform depend on the protocol you’re using.

Operation restrictions

The operations you can perform in a namespace are subject to these restrictions:

  • The namespace access protocol must support the operation.
  • If the namespace protocol requires client authentication, you need to provide valid user credentials.
  • The namespace access protocol must be configured to allow access to the namespace from your client IP address.
  • Both the effective namespace permission mask and your permissions must allow the operation.

Supported operations

The table below lists the operations HCP supports and indicates which protocols you can use to perform those operations. Some operations relate to specific types of metadata.

OperationHTTPWebDAVCIFSNFS
Write data from files or memory to a namespace to create an object
Transmit data to and from HCP in gzip-compressed format
Check for object existence
View the content of an object
Copy an object
Store new versions of existing objects
Append to existing objects
Delete an object that’s not under retention
Delete an object that’s under retention if the namespace configuration allows it
View object metadata
View a metafile
Override default index, retention, and shred settings when storing an object
Change the retention setting for an object
Hold or release an object
Enable shredding for an object
Change the index setting for an object
Change object ownership (not related to POSIX UID)
Change the POSIX UID and GID for an object
Change the POSIX permission settings for an object
Change the POSIX atime or mtime value for an object
Store, replace, or delete an annotation for an object
Store, replace, or delete the default annotation for an object
Store or retrieve object data and an annotation in a single operation
Store or retrieve object data and the default annotation in a single operation
Check for annotation existence
Check for existence of the default annotation
List annotations
Read an annotation
Read the default annotation
Store, replace, or delete an ACL for an object
Check for ACL existence
Read ACLs
Create an empty directory in a namespace
View the namespace directory structure, not including metadirectories
View the namespace directory structure, including both directories and metadirectories
Rename an empty directory (unless atime synchronization is enabled)
Delete an empty directory
Create a symbolic link
Read through a symbolic link to an object
Delete a symbolic link
List the namespaces accessible to you
List namespace statistics
List namespace permissions for the user
List the retention classes available in the namespace

These considerations apply to symbolic links:

  • If you use CIFS to create a symbolic link, you can read through the link only with CIFS. You cannot use CIFS to read through a symbolic link created with NFS.
  • HTTP and WebDAV support for reading through symbolic links is limited to retrieving object data. Other HTTP and WebDAV operations on symbolic links may produce unexpected results.
TipYou can use the HCP Search Console to delete, purge, hold, release, change ownership, or modify ACLs on multiple objects with a single operation.

Prohibited operations

HCP never lets you:

  • Rename an object.
  • Rename a nonempty directory.
  • Overwrite a successfully stored object. However, if versioning is enabled, you can write new versions of an object using the REST, S3 compatible, or HSwift API.
  • Modify the fixed-content portion of an object.
  • Delete a directory that contains one or more objects.
  • Shorten the retention period of an object.
  • Store a file (other than a file containing custom metadata), directory, or symbolic link anywhere in the metadata structure.
  • Delete a metafile (other than a metafile containing custom metadata) or metadirectory.
  • Create a hard link.

Object representation

HCP represents objects differently depending on the protocol you’re using:

  • With HTTP, HCP represents each object as a URL.
  • With WebDAV, CIFS, and NFS, the HCP file system (HCP-FS) represents objects using the familiar file and directory structure.

This chapter describes how HCP represents objects through the protocols, including descriptions of the files and directories HCP-FS uses to represent object data and object metadata.

Object representation with HTTP

With HTTP, HCP represents each object as a URL. The root of the object path in the URL is always rest.

Here’s an example of the URL for an object named wind.jpg in the images directory in a namespace named climate in a tenant named geo in an HCP system with the domain name hcp.example.com:

http://climate.geo.hcp.example.com/rest/images/wind.jpg

Users and applications represent system metadata by using query parameters appended to the URLs. HCP returns system metadata in HTTP response headers.

Object representation with WebDAV, CIFS, and NFS

For WebDAV, CIFS, and NFS, HCP includes the HCP file system

(HCP-FS), a standard POSIX file system that represents each object as a set of files. One of these files has the same name as the object. This file contains the fixed-content data. When downloaded or opened, this file has the same content as the originally stored item.

The other files HCP-FS presents contain object metadata. These files, most of which are plain text, are called metafiles. They are in directories called metadirectories.

HCP-FS presents a directory structure with separate branches for files that contain object data and for those that contain object metadata. The structure of the metadata branch parallels that of the data branch.

HCP-FS doesn’t present any metafiles for symbolic links.

Root directories

HCP-FS presents the data files and metafiles for objects in a namespace under two root directories:

  • data

    heads the directory structure containing the data files for all objects. You create the structure under data when you store data and create directories in the namespace. Each data file and directory in this structure has the same name as the object or directory it represents. All object and directory names are user-supplied, with the exception of names for email objects and directories.

  • metadata

    heads the directory structure containing all the metafiles and metadirectories for objects and directories. This structure parallels the file and directory structure under data, excluding symbolic links. HCP-FS creates this structure automatically as you store data and create directories in the namespace.

You cannot change the name of either of these root directories.

Sample data structure

When you view the contents of a namespace, HCP-FS shows each directory and data file under data. The figure below shows a sample directory structure headed by data. It contains one subdirectory, named images. images contains three data files named earth.jpg, wind.jpg, and fire.jpg. These files represent the content of the objects named earth.jpg, wind.jpg, and fire.jpg, respectively.

GUID-6861A7C4-D644-4E80-BA3F-97260A1614BD-low.png

All the data files and directories presented by HCP-FS represent objects that users have stored in the namespace, with one exception. The data directory has a system-generated subdirectory named .lost+found, which is where HCP puts broken objects in the unlikely event it finds any. If you see objects in this directory, tell your tenant administrator.

Metadirectories

Each data file for an object has a corresponding metadirectory with the same name. The location of this metadirectory mirrors the location of the data file. For example, the wind.jpg data file is in the images directory, and the wind.jpg metadirectory is in the images metadirectory. The entire metadirectory structure is under the metadata metadirectory.

The figure below shows the metadirectory structure that corresponds to the data files in the images directory.

GUID-462993AB-6249-4C4F-BA4E-62BC9461EE06-low.png

Metafiles

HCP-FS presents individual metafiles for each piece of HCP-specific object metadata except owner. It doesn’t present individual metafiles for POSIX metadata. However, it does present one additional metafile that summarizes both the HCP-specific and POSIX metadata for an object. Metafiles contain either plain text or XML, so you can read them easily.

You can view and retrieve metafiles through the WebDAV, CIFS, and NFS protocols. You can also use these protocols to overwrite metafiles that contain the HCP-specific metadata you can change. By overwriting a metafile, you change the metadata for the corresponding object.

If an object has a default annotation or an ACL, HCP presents it as a metafile. HCP does not show other annotations as metafiles. You can add, replace, or delete the default annotation only with the HTTP and WebDAV protocols. You can add, replace, or delete other annotations and ACLs only with the HTTP protocols.

For backward compatibility, HCP-FS also presents a metafile named tpof.txt. This metafile is superseded by dpl.txt.

HCP-FS presents these metafiles for objects:

  • created.txt

    Contains the date the object was added to the namespace. This metafile contains two lines:

    • The first line is the date expressed as the number of seconds since January 1, 1970.
    • The second line is the date in this ISO 8601 format:
      yyyy-MM-ddThh:mm:ssZ

    In this format, Z represents the offset from UTC and is specified as:

    (+|-)hhmm

    For example:

    1326978027
    2012-01-19T08:00:27-0500

    You can view the content of this metafile, but you cannot change it.

  • dpl.txt

    Contains the data protection level (DPL) for the object. The DPL is the number of copies of the object HCP must maintain in the repository to ensure the integrity and availability of the object.

    For example:

    2

    You can view the content of this metafile, but you cannot change it.

  • hash.txt

    Contains the name of the cryptographic hash algorithm used to generate the cryptographic hash value for the object, as well as the hash value itself. For example:

    SHA-256
    2BC9AE8640D50145604FB6CFC45A12E5561B40429174CE404A...

    HCP calculates the hash value for an object from the object data.

    You can view the content of this metafile, but you cannot change it.

  • index.txt

    Contains the index setting for the object.

    You can view and change the content of this metafile.

  • replication.txt

    Indicates whether the object is replicated, in this format:

    replicated=true|false

    The value is true only when the object and all its metadata have been replicated. For example, if you add custom metadata to a replicated object, the content of the replicated.txt metafile changes to replicated=false. When the metadata is replicated, the value changes back to replicated=true.

    You can view the content of this metafile, but you cannot change it.

  • retention.txt

    Contains the retention setting for the object.

    You can view and change the content of this metafile.

  • shred.txt

    Contains the shred setting for the object.

    You can view and change the content of this metafile.

  • core-metadata.xml

    Contains a summary of the HCP-specific and POSIX metadata for the object. For example:

    <core-metadata xsi:schemaLocation="http://www.hitachivantara.com
    core-metadata-7_0.xsd">
    <version>3</version>
    <name>/images/wind.jpg</name>
    <name-bytes>2F696D616765732F77696E642E6A7067
    </name-bytes>
    <object-type>File</object-type>
    <creation-time>1326978027</creation-time>
    <update-time>1326978027</update-time>
    <change-time>1326978027</change-time>
    <access-time>1326978027</access-time>
    <uid>17</uid>
    <gid>17</gid>
    <mode>100544</mode>
    <shred>false</shred>
    <index>true</index>
    <retention-value>1462979278</retention-value>
    <retention-string>2016-05-11T11:07:58-0400 (DeptReg223,+7y)
    </retention-string>
    <retention-hold>false</retention-hold>
    <size>238985</size>
    <tpof>1</tpof>
    <dpl>2</dpl>
    <hash-scheme>SHA-256</hash-scheme>
    <hash-value>0B86212A66A792A79D58BB18...</hash-value>
    <retention-class>DeptReg223</retention-class>
    <replicated>true</replicated>
    <replicationCollision>false</replicationCollision>
    <ingestProtocol>CIFS_NFS</ingestProtocol>
    <owner></owner>
    <domain></domain>
    <has-acl>false</has-acl>
    </core-metadata>

    The version element identifies the version of the core-metadata.xml file.

    The ingestProtocol element identifies the protocol through which the object was ingested.

    You can view the content of this metafile, but you cannot change it.

    To see the XML schema for this metafile, use this URL:

    http://tenant-name.hcp-domain-name/static/core-metadata-7_0.xsd
  • custom-metadata.xml

    Contains the custom metadata for the default annotation for the object. This metafile is present only when the object has a default annotation.

  • acl.xml

    Contains the ACL for the object. This metafile is present only when the object has an ACL.

    Objects can have an ACL only if the namespace is configured to allow it.

Complete metadata structure

The figure below shows the complete metadata structure, including the metafiles, generated for the sample data structure. It assumes you’ve added custom metadata and ACLs for each object.

GUID-59CB83BD-44E8-44B5-AF64-65BC6A425AF5-low.png

Object properties

Objects in a namespace have a variety of properties, such as a retention period and an index setting. These properties are defined for each object by the object metadata. HCP maintains both HCP product-specific and POSIX metadata for objects. You can view all the metadata for objects in a namespace and modify some of it.

HCP maintains only POSIX metadata for directories and symbolic links.

The namespace configuration specifies default values for several object properties. When you store an object in the namespace, the object inherits these values. You can override some of these values when using the HTTP protocol to store objects in the namespace.

If only the HTTP protocol is enabled for a namespace, the namespace can be configured to allow objects to have multiple versions.

With the WebDAV, CIFS, and NFS protocols, HCP displays object metadata in metafiles.

For the WebDAV protocol, HCP also provides its own XML namespace with live properties that allow you to store and retrieve HCP system metadata.

Data protection level

The data protection level (DPL) for an object is the number of copies of the object HCP must maintain in the repository to ensure the integrity and availability of the object. Regardless of the DPL, you see each object as a single entity.

Each copy of the data for an object is stored on a different HCP node. Storing the data in different locations helps ensure the availability of data in case one of those locations becomes inaccessible.

The DPL is determined by the namespace configuration. You can view the DPL for an object, but you cannot change it. However, namespace configuration changes can cause the DPL to change.

With HTTP, the data protection level for an object is returned by the X-HCP-DPL response header. With the WebDAV, CIFS, and NFS protocols, you view this value in the dpl.txt metafile.

Cryptographic hash value

The cryptographic hash value for an object is a form of fingerprint. HCP uses a cryptographic hash algorithm to calculate this value from the object data. HCP then uses this hash value to check that object data remains unchanged.

You can view this value, but you cannot change it.

With HTTP, cryptographic hash values are returned by the X-HCP-Hash response header. With the WebDAV, CIFS, and NFS protocols, you view this value in the hash.txt metafile.

HCP does not generate cryptographic hash values for multipart objects. Therefore, the X-HCP-Hash header is not returned for these objects.

Object ingest time and change time

Objects in a namespace have both an ingest time and a change time.

The ingest time is the time when the object or version of the object was first added to the namespace or was deleted. The change time is the time of the most recent of these events:

  • The object was closed after being added to the namespace.
  • The object was deleted.
  • Any metadata, including custom metadata or an ACL, was changed.
  • The object was recovered from a replica.
  • An attempt by the HCP search facility to index the object failed. When this happens, the change time for the object is set to two weeks in the future, at which time the HCP search facility tries again to index it.

Even if an object has not changed since ingestion, the ingest time and change time may not be identical. This is because the ingest time is set when HCP opens the object for write and the change time is set when HCP closes the object after ingestion is complete.

The change time for an object is the same as the POSIX ctime attribute value.

With HTTP, ingest times are returned by the X-HCP-IngestTime header in response to GET and HEAD requests for objects. Change times are returned by the X-HCP-ChangeTimeString and X-HCP-ChangeTimeMilliseconds response headers for the same requests. You can also view ingest and change times for objects in listings of directory contents.

With the WebDAV, CIFS, and NFS protocols, you view ingest time in the created.txt metafile and change time in the core-metadata.xml metafile.

Retention

Objects have a retention property that determines how long the object must remain in the namespace before it can be deleted. This time can range from allowing the object to be deleted at any time to preventing the object from ever being deleted. While an object cannot be deleted due to retention, it is said to be under retention.

If an object is immediately placed under retention when it is stored, it is stored with no POSIX write permissions. When an existing object is placed under retention, its POSIX write permissions are removed.

Object retention settings

Each object in a namespace has a retention setting. Additionally, the namespace is configured with a default retention setting. When an object is stored in the namespace, the object inherits the namespace retention setting. Using HTTP, you can override the default setting by specifying a different setting when you store the object.

NoteIf the default retention setting is in the past, new objects that would otherwise get that setting are added with a setting of Deletion Allowed (0).

Every object, including objects assigned to retention classes, has a specific retention setting that specifies when (or whether) the retention period for the object ends. HCP represents the retention setting in numeric and text formats, as shown in the following table.

Numeric valueText valueMeaning
0Deletion AllowedThe object can be deleted at any time.
-1Deletion ProhibitedThe object cannot be deleted, except by a privileged delete, and the retention setting cannot be changed.
-2Initial UnspecifiedThe object does not yet have a retention setting. An object that has this value cannot be deleted, except by a privileged delete. You can change this retention setting to any other setting.
An integer number of seconds since January 1, 1970, at 00:00:00 UTCA datetime valueThe object cannot be deleted, except by a privileged delete, until the specified date and time.

The format used for retention settings depends on whether you are specifying or retrieving a setting and on the protocol you are using.

Deleting object and versions under retention

If you have privileged permission and the namespace allows it, you can use the HTTP protocol to:

  • Delete an object that is under retention. This action is called privileged delete.
  • Purge all versions of an object that is under retention. This action is called privileged purge. To perform a privileged purge, the namespace must support versioning.

Holding an object

If you have both write and privileged permissions, you can place an object on hold. An object on hold cannot be deleted, even by a privileged delete operation. Also, you cannot store new versions of an object that is on hold. Holding objects is particularly useful when the objects are needed for legal discovery.

While an object is on hold, you can change its retention setting only by increasing the value. You can also change its shred setting and its ACL. And if the namespace is configured to allow changes to custom metadata for objects under retention, you can change its custom metadata.

If you have both write and privileged permissions, you can also release an object from hold. When an object is released, its previous retention setting is again in effect.

TipYou can use the HCP Search Console to place multiple objects on hold or release multiple objects at the same time.

Holding an object using labeled holds

A labeled retention hold, or labeled hold, is a named retention hold that has a unique identifier. The ability to set multiple, independent labeled holds on an object enables multiple organizations or custodians to place a hold on the same object to prevent its deletion or versioning, or metadata modification, until all holds are explicitly released. This feature is supported by both the REST API and Hitachi API for Amazon S3.

A common use of labeled holds is when an object is needed for legal discovery by multiple organizations or custodians. For example, there might be several lawsuits handled by different organizations within a company. The investigations or inquiries have different but overlapping time lines. A file relevant to all of the lawsuits must not be deleted when the first legal hold ends because the other lawsuits need the file as well.

A labeled hold can be placed on a new object at ingest time, or on an existing object in the namespace, provided the client application has write and privileged data access permissions. Similarly, the application can modify a labeled hold on an existing object.

To place a labeled hold on an object at ingest time, use the Hitachi API for Amazon S3 or REST API with the HTTP PUT object method and the X-HCP-LabelRetentionHold header.

To add or remove a labeled hold on an existing object, use the S3 compatible API with the HTTP PUT object copy method and the X-HCP-LabelRetentionHold header, or the REST API HTTP PUT method with the query parameter method to add or remove system metadata.

Retention holds and labeled retention holds on the same object

An object can have both a single hold and one or more labeled holds at the same time. These hold settings are independent of one another. All holds must be released on the object before it can be deleted or have its metadata modified, regardless of the retention setting.

Prerequisite to set privileged data access and write permissions

Before an application can place a labeled hold on an object, the application must have write and privileged data access permissions for the associated namespace.

Privileged permission can be set either programmatically by using the HCP Management API to update the dataAccessPermissions.xml file, or through the Tenant Management Console on the Users page of the Security menu.

Label limits

The following limits apply to the labeled hold feature:

  • Maximum label size

    64 characters

  • Maximum number of labeled holds per object

    100

Retention classes

A retention class is a named retention value that, when used as the retention setting for an object, specifies how long the object must remain in the repository. This value can be:

  • A duration after object creation. For example, a retention class named HlthReg-107 could have a duration of 21 years. All objects that have that class as their retention setting could not be deleted for 21 years after they’re created.
  • One of these special values:
    • Deletion Allowed (0)
    • Deletion Prohibited (-1)
    • Initial Unspecified (-2)

Retention class duration values use this format:

A+yearsy+monthsM+daysd

In this format, A represents the time at which the object was created. For example, this value specifies a retention period of one year, two months, and three days:

A+1y+2M+3d

The duration specification can omit portions with zero values. For example, this value specifies a six-month retention period:

A+6M

You can use retention classes to consistently manage data that must conform to a specific retention rule. For example, if local law requires that medical records be kept for a specific number of years, you can use a retention class to enforce that requirement.

Tenant administrators create retention classes for namespaces. When creating a class, the administrator specifies the class name, the retention setting, and whether HCP can automatically delete objects in the class when the retention period expires.

NoteAutomatic deletion must be enabled for the namespace for objects under retention to be automatically deleted.

Each namespace has its own set of retention classes. You cannot apply a retention class defined in one namespace to an object in a different namespace.

A namespace can be configured to allow administrative users to increase or decrease class retention times and delete classes. Otherwise, the retention time for a class can only be increased, and classes cannot be deleted. In all cases, any change to a retention class affects the retention period of all objects in that class.

If a retention class is deleted, the objects assigned to that class have a retention setting of Deletion Prohibited (-1) and cannot be deleted.

If a new retention class is created with the same name as a deleted retention class, existing objects in the deleted class get the retention setting of the new class.

You can assign a retention class to an existing object using any namespace access protocol. With HTTP, you can assign a retention class to an object when you store the object in the namespace, thereby overriding the default retention setting.

Viewing retention settings

With the HTTP protocol, retention information is returned by HTTP response headers. With the WebDAV, CIFS, and NFS protocols, you view retention information for an object in the retention.txt metafile.

Retention information returned through HTTP

When you use HTTP to retrieve metadata for an object, the returned header information includes four retention-related values, as shown below.

  • X-HCP-Retention

    One of:

    • The retention setting expressed as a number of seconds since January 1, 1970, at 00:00:00 UTC
    • One of the special values 0, -1, or -2
  • X-HCP-RetentionString

    One of:

    • The retention period end time expressed as a datetime value in ISO 8601 format or time in seconds since January 1, 1970
    • One of the special values Deletion Allowed, Deletion Prohibited, or Initial Unspecified
  • X-HCP-RetentionClass

    The retention class of the object or an empty string if the object is not assigned to a retention class. The information is returned in this format:

    (retention-class-name, retention-class-value)

    If the retention class is deleted, retention-class-value is undefined.

  • X-HCP-RetentionHold

    A value of true or false that indicates whether the object is on hold.

retention.txt settings for an object

The table below shows the possible retention settings in the retention.txt metafile for an object.

0

Deletion Allowed

0

Deletion Allowed

Hold

0

Deletion Allowed (retention-class-name, 0)

0

Deletion Allowed (retention-class-name, 0)

Hold

-1

Deletion Prohibited

-1

Deletion Prohibited

Hold

-1

Deletion Prohibited (retention-class-name, -1)

-1

Deletion Prohibited (retention-class-name, -1)

Hold

-2

Initial Unspecified

-2

Initial Unspecified

Hold

-2

Initial Unspecified (retention-class-name, -2)

-2

Initial Unspecified (retention-class-name, -2)

Hold

retention-period-end-seconds-past-1970-1-1retention-period-end-datetimeretention-period-end-seconds-past-1970-1-1retention-period-end-datetimeHold
retention-period-end-seconds-past-1970-1-1retention-period-end-datetime (retention-class-name, retention-class-duration)

retention-period-end-seconds-past-1970-1-1retention-period-end-datetime (retention-class-name, retention-class-duration)

Hold

retention.txt settings for deleted retention classes

If the retention class assigned to an object is deleted, the retention.txt metafile for the object then contains:

  • A retention setting of Deletion Prohibited (-1)
  • The name of the deleted retention class
  • A retention class value of undefined

For example, suppose you assign an object to the HlthReg-107 retention class and then the class is deleted. The retention.txt metafile for the object then contains:

-1
Deletion Prohibited (HlthReg-107, undefined)

Specifying retention settings

You can change the retention setting for an existing object. With HTTP, you can also specify a retention setting when you store an object in the namespace. The following information describes the retention settings you can specify.

With WebDAV, CIFS, or NFS, you change the retention setting on an object by overwriting its retention.txt metafile. In the new file, you specify a single value that tells HCP what change to make. This value must be on a single line. To ensure that HCP processes the value correctly, end the line with a carriage return.

TipWith Windows and Unix, you can also use the echo command to insert the new value into the retention.txt metafile.
Retention setting values

These general rules apply to values you can set:

  • If you’re storing an object in the namespace with HTTP or setting retention for an existing object that is not under retention, you can specify any retention setting — past or present. If you set a time in the past, the object is immediately deletable.
  • If you’re changing the setting for an existing object that is in a retention class, you can replace the class with another class, as long as the duration of the replacement class is not shorter than the duration of the original class. You cannot replace the class with an explicit retention setting, such as Deletion Prohibited or a datetime value.
  • If you’re changing the setting for an existing object that has an explicit datetime retention setting, you can change the setting to lengthen the retention period but not to shorten it.

The list below describes the values you can use to specify the retention setting for an object and contains rules for specifying these values. These values are not case sensitive.

  • 0 (zero) or Deletion Allowed

    Allows the object to be deleted at any time. You can assign this value to an object only when you add it to the namespace or when its retention setting is -2.

    The value -0 is equivalent to 0 (zero).

  • -1 or Deletion Prohibited

    Prevents the object from being deleted and its retention setting from being changed. You can assign this value to an object that is not in a retention class at any time.

    If an object is assigned to a retention class and that class is then deleted, the retention setting for that object changes to -1.

  • -2 or Initial Unspecified

    Specifies that the object does not yet have a retention setting. You can assign this value to an object when you add the object to the namespace. You can also directly change the retention setting for an object from 0 to ‑2.

    You can change ‑2 to any other retention setting.

  • datetime

    Prevents the object from being deleted until the specified date and time. You can assign this value to an object that is not in a retention class if the specified date and time is later than the current retention setting for the object. You cannot assign it to an object for which the current retention setting is -1.

  • offset

    Specifies a period for which to retain the object. You can assign this value to an object that is not in a retention class at any time, except when its current retention setting is -1.

    HCP calculates the retention setting date and time from the offset and one of these:

    • The time the object was added to the namespace
    • The current retention setting
    • The current time
  • C+retention_class_name

    Assigns the object to a retention class.

    You can assign this value to an object if any one of these is true:

    • The original retention period for the object has expired.
    • The original retention period for the object has not expired, and the retention class results in a retention period that is longer than the current retention period.
    • The retention setting for the object is 0 or -2.
    • The retention setting for the object is -1, and the class has a retention setting of -1.
    • The object is in a retention class, and the duration of the new class is not shorter than the duration of the original class.
    • The retention class you assign must already be defined for the namespace.
Specifying a date and time

You can set retention by specifying a date and time in either of these formats:

  • Time in seconds since January 1, 1970, at 00:00:00 UTC. For example:
    1450137600

    The calendar date that corresponds to 1450137600 is Tuesday, December 15, 2015, at 00:00:00 EST.

  • Date and time in this ISO 8601 format:
    yyyy-MM-ddThh:mm:ssZ

    In this format, Z represents the offset from UTC and is specified as:

    (+|-)hhmm

    For example, 2015-11-16T14:27:20-0500 represents the start of the 20th second into 2:27 PM, November 16, 2015, EST.

If you specify certain forms of invalid dates, HCP automatically adjusts the retention setting to make a real date. For example, if you specify 2015-11-33, which is three days past the end of November, HCP changes it to 2015-12-03.

Specifying an offset

You can set retention by specifying an offset from:

  • The time at which the object was added to the namespace
  • The current retention setting for the object
  • The current time

Because you can only extend a retention period, the offset must be a positive value.

Offset syntax

To use an offset as a retention setting, specify a standard expression that conforms to this syntax:

^([RAN])?([+-]\d+y)?([+-]\d+M)?([+-]\d+w)?([+-]\d+d)?([+-]\d+h)?([+-]\d+m)?([+-]\d+s)?

The list below explains this syntax.

  • ^

    Start of the expression

  • ( )

    Sequence of terms treated as a single term

  • ?

    Indicator that the preceding term is optional

  • [ ]

    Group of alternatives, exactly one of which must be used

  • +

    Plus

  • -

    Minus

  • R*

    The current retention setting for the object

  • A*

    The time at which the object was added to the namespace

  • N*

    The current time

  • d+

    An integer in the range 0 (zero) through 9,999

  • y

    Years

  • M

    Months

  • w

    Weeks

  • d

    Days

  • h

    Hours

  • m

    Minutes

  • s

    Seconds

Offset examples

Here are some examples of offset values used to extend a retention period:

  • This value sets the retention value to 100 years past the time when the object was stored:
    A+100y
  • This value sets the end of the retention period to 20 days minus five hours past the current date and time:
    N+20d-5h
  • This value extends the current retention period by two years and one day:
    R+2y+1d

atime synchronization with retention

Some file systems support the use of the POSIX atime attribute to set retention. To take advantage of this existing mechanism, HCP gives you the option of synchronizing atime values with HCP retention settings. When these properties are synchronized, changing one for an object causes an equivalent change in the other.

The tenant administrator enables or disables atime synchronization for a namespace. While atime synchronization is enabled, atime values are automatically synchronized with retention settings for objects subsequently added to the namespace except in these cases:

  • The object is added through NFS with an initial retention setting of Deletion Allowed.
  • The object is added through any protocol with an initial retention setting that is either Initial Unspecified or a retention class.

In these cases, the atime value of an object is set to the time the object is stored.

For any given object, if atime synchronization was not enabled automatically, you can enable it manually.

While atime synchronization is enabled for a namespace, the rules for changing retention settings also apply to changing atime values. You cannot use atime to shorten a retention period, nor can you use it to specify a retention period if the current setting is Deletion Prohibited. Additionally, you cannot change the atime value if the object is on hold.

atime synchronization does not work with objects in retention classes. When you assign an object to a retention class, the atime value for the object does not change, even if the atime value had previously been synchronized with the retention setting. Triggering atime synchronization for an object in a retention class has no effect.

atime synchronization is enabled on a per-namespace basis. To find out whether it’s enabled for the namespace you’re using, contact your tenant administrator.

NoteWith atime synchronization enabled, you cannot rename empty directories. This includes any directories you create using CIFS, which, by default, are named New Folder.
Triggering atime synchronization for existing objects

While atime synchronization is enabled, you can use either the atime attribute or the retention.txt metafile to trigger synchronization for an individual existing object (for which synchronization is not currently in effect):

  • To use the atime attribute to trigger synchronization for an object with retention currently set to a specific date and time in the future, use CIFS or NFS to make a valid change to the value of the atime attribute.
  • To use the atime attribute to trigger synchronization for an object with retention currently set to either Deletion Allowed or Initial Unspecified:

Before you begin

To use the retention.txt metafile to trigger atime synchronization for any object, regardless of its current retention setting, change the retention setting for the object to any valid value except a retention class.

Important
  • Some commands you can use to store objects in the namespace, such as the Unix cp command, have options for preserving the original permissions and timestamps. These commands may modify permissions and atime values in a way that causes the stored object to become read-only, thereby triggering retention. If atime synchronization is enabled, you should review the use of these commands to ensure they do not result in unexpected retention settings.
  • Existing applications ported to HCP may remove write permissions from objects without regard to their atime values. If atime synchronization is enabled for the namespace, this can have the unintended effect of giving some objects infinite retention. Be sure to review ported applications for this behavior before running them.

Procedure

  1. Check that the atime value is the retention setting you want.

    If it isn’t, be sure to change it before performing step 2.
  2. Take one of these actions:

    • If the object has any POSIX write permissions for the owner, owning group, or other, use CIFS or NFS to remove them.
    • If the object has no POSIX write permissions for the owner, owning group, or other, use CIFS or NFS to add at least one and then remove all you added.
    Changing permissions through WebDAV does not trigger atime synchronization.

    Read and execute permissions have no effect on this process.

Results

Triggering atime synchronization for an object creates an association between its atime value and retention setting. Subsequent changes to POSIX permissions do not remove this association.
Removing the association

The association between the atime value and retention setting for an object remains in effect until one of these happens:

  • The retention period for the object expires.
  • You assign the object to a retention class.
  • The tenant administrator disables atime synchronization for the namespace. Any changes to atime attributes or retention settings made after synchronization is disabled are independent of each other. If the tenant administrator subsequently reenables atime synchronization for the namespace, these properties remain independent until you trigger synchronization again for the individual object.
How atime synchronization works

atime values and retention settings have these general correspondences:

  • atime values in the future correspond to specific retention settings in the future.
  • atime values more than 24 hours in the past correspond to retention settings that prevent deletion.
  • atime values 24 hours or less in the past correspond to retention settings that permit immediate deletion.

The table below shows the effects of valid atime changes on retention settings for an object with atime synchronization in effect.

Changing the atime value toWhen the current retention setting isChanges the retention setting to
A time later than the current date and timeInitial Unspecified (-2) or Deletion Allowed (0)The new time of the atime attribute
A time later than the current retention settingA specific date and timeThe new time of the atime attribute
A time more than 24 hours before the current date and time*

Initial Unspecified (-2), Deletion Allowed (0), or

a specific date and time

Deletion Prohibited
A time 24 hours or less before the current date and time*Initial Unspecified (-2) or Deletion Allowed (0)The new time of the atime attribute, which immediately makes the object expired and deletable
*Twenty-four hours is the default setting for this threshold. If you want it changed, please contact your tenant administrator.

The table below shows the effects of valid changes to retention settings on atime values for an object with atime synchronization in effect.

Changing the retention setting toWhen the current retention setting isChanges the atime value to
A time later than the current date and timeInitial Unspecified (-2) or Deletion Allowed (0)The same time as the new retention setting
A time later than the current retention settingA specific date and timeThe same time as the new retention setting
A time before the current date and timeInitial Unspecified (-2) or Deletion Allowed (0)The same time as the new retention setting
Deletion AllowedInitial Unspecified (-2)January 1, 1970 00:00:00 GMT
Deletion Prohibited

Initial Unspecified (-2), Deletion Allowed (0), or

a specific date and time

The current date and time minus the threshold beyond which atime sets retention to Deletion Prohibited*
*Twenty-four hours is the default setting for this threshold. If you want it changed, please contact your tenant administrator.

If atime synchronization has already been triggered for an object and the object is under retention, you cannot use atime to change its retention setting while HCP is configured to disallow permission changes for objects under retention. However, you can modify the setting in retention.txt, and, when you do so, the atime value is synchronized with the new retention setting.

atime synchronization example

The following example shows how to use the atime attribute to trigger retention for the existing wind.jpg object after atime synchronization has been enabled for the namespace; the example uses the NFS protocol:

Procedure

  1. (Optional) Check the current retention setting for the wind.jpg object:

    cat /metadatamount/images/wind.jpg/retention.txt
    0
    Deletion Allowed
  2. (Optional) Check the current permissions for the wind.jpg object:

    ls -l /datamount/images/wind.jpg
    -r--r--r-- 1 root root 23221 Mar 19 09:45 /datamount/images/wind.jpg

    Notice that the object has no write permissions.

  3. Set the atime attribute for the wind.jpg object:

    touch -a -t 201512310000 /datamount/images/wind.jpg
    NoteTo set the value of the atime attribute, you can use the Windows SetFileTime library call, the Unix utime library call, or the Unix touch command.
  4. (Optional) Verify step 3:

    stat /datamount/images/wind.jpg
    File: "/datamount/images/wind.jpg"
    Size: 23221 Blocks: 112 IO Block: 32768 regular file
    Device: 15h/21d Inode: 18 Links: 1
    Access: (0444/-r--r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
    Access: 2015-12-31 00:00:00.000000000 -0500
    Modify: 2012-03-19 09:45:18.000000000 -0400
    Change: 2012-03-23 13:10:17.000000000 -0400
  5. Add write permissions to the wind.jpg object:

    chmod a+w /datamount/images/wind.jpg
  6. Remove all write permissions from the wind.jpg object:

    chmod a-w /datamount/images/wind.jpg
  7. (Optional) Verify that the retention setting has changed to match the atime value:

    cat /metadatamount/images/wind.jpg/retention.txt
    1451520000
    2015-12-31T00:00:00-0500

Shredding

Shredding, also called secure deletion, is the process of deleting an object and overwriting the places where its copies were stored in such a way that none of its data or metadata, including custom metadata, can be reconstructed.

Every object has a shred setting that determines whether it will be shredded when it’s deleted.

With HTTP, the shred settings for an object are returned by the X-HCP-Shred response header. The shred setting values are:

  • false

    Don’t shred.

  • true

    Shred following deletion.

With WebDAV, CIFS, or NFS, you view the shred setting for an object in the shred.txt metafile. The shred setting values in this metafile are:

  • 0

    (zero)

    Don’t shred.

  • 1

    (one)

    Shred following deletion.

Default shred settings

The namespace is configured with a default shred setting. When an object is stored in the namespace, the object inherits this setting. With HTTP, you can override the default shred setting by specifying a different shred setting when you store the object.

Changing shred settings

You can change the shred setting for an existing object from false to true (or 0 to 1) but not from true to false (or 1 to 0).

With WebDAV, CIFS, or NFS, you change the shred setting for an object by overwriting its shred.txt metafile. In the new file, you specify only the new value.

Tip
  • With Windows and Unix, you can also use the echo command to insert the new value into the shred.txt metafile.
  • As a general rule, if you mark an object for shredding, you should mark all other objects with the same content for shredding as well.

Indexing

Each object has an index setting that is either true or false. The setting is present regardless of whether the namespace supports search operations.

The metadata query engine uses the index setting to determine whether to index custom metadata for an object:

  • For objects with an index setting true, the metadata query engine indexes custom metadata.
  • For objects with an index setting false, the metadata query engine does not index custom metadata.
NoteThe namespace configuration can prevent the metadata query engine from indexing a particular annotation.

The HCPsearch facility uses the index setting to determine whether to index an object at all:

  • The HCP search facility indexes objects with an index setting true.
  • The HCP search facility does not index objects with an index setting false.

Metadata query API requests can use this setting as a search criterion. Additionally, third-party applications can use this setting for their own purposes.

With HTTP, index settings are returned by the X-HCP-Index response header. With WebDAV, CIFS, and NFS, you can view the index setting for an object or directory in its index.txt metafile. In this metafile:

  • A value of 1 (one) means true.
  • A value of 0 (zero) means false.
Default index settings

The namespace is configured with a default index setting. When an object is stored in the namespace, the object inherits this setting. With HTTP, you can override the default index setting by specifying a different index setting when you store the object.

Changing index settings

Changing the index setting on an object causes these changes to the indexes maintained by the metadata query engine and the HCP search facility:

  • If you change the index setting of an object from true to false:
    • The metadata query engine removes the custom metadata for the object from the index, but leaves the system metadata and ACL, if any.
    • The HCP search facility completely removes the object from the index.
  • If you change the index setting of an object from false to true:
    • The metadata query engine indexes the custom metadata for the object along with the system metadata and ACL, if any.
    • The HCP search facility indexes the object.

With WebDAV, CIFS, or NFS, you change the index setting on an object by overwriting its index.txt metafile. In the new file, you specify only the new value.

TipWith Windows and Unix, you can also use the echo command to insert the new value into the index.txt metafile.

Ownership

An object in a namespace can be owned by a user or can have no owner. With HTTP, you can set, view, and change the owner of an object. With WebDAV, CIFS, and NFS, you can view this metadata in the core-metadata.xml metafile but you cannot change it.

Ownership is an HCP-specific object property and does not correspond to the POSIX user ID (UID) or group ID (GID) associated with an object.

Ownership for new objects

Ownership for new objects depends on whether the user adding the objects is authenticated:

  • If the user adding an object is authenticated, that user owns the object.
  • If the user adding an object is accessing the namespace anonymously, the object has no owner.
Ownership for objects that existed before release 5.0

Objects that were added to a namespace before the HCP system was upgraded from a release earlier than 5.0 to release 5.x have a special value for object ownership called nobody. These objects effectively have no owner.

You can change such an object to having an owner or having no owner. You cannot change the owner of an object to nobody.

Changing ownership

To change the owner of an object, you use the HTTP protocol.

Note
  • A namespace can be configured to disallow ownership changes for objects that are under retention.
  • Changing the owner of an object that was stored using the NFS protocol removes the POSIX UID and GID values from the object.
  • You can use the HCP Search Console to change ownership of multiple objects with a single operation.

POSIX metadata

To support the NFS protocol, HCP maintains POSIX metadata for objects, directories, and symbolic links. The protocol you’re using to access the namespace determines what actions you can perform on POSIX metadata:

  • With HTTP, you can view some POSIX metadata on objects, but you cannot change it.
  • With WebDAV, you can view all POSIX metadata on objects, directories, and symbolic links, but you cannot change it.
  • With CIFS and NFS, you can view all POSIX metadata on objects, directories, and symbolic links, and you can change some of it.

    A request to change POSIX metadata on a symbolic link changes the metadata on the object that’s the target of the link. Such a request does not change the metadata on the link itself.

The POSIX metadata that HCP maintains for items in a namespace (that is, objects, directories, and symbolic links) depends on the protocol through which the item was added:

  • For items added through NFS, HCP maintains these attributes for each item:
    • A user ID and group ID.
    • A POSIX permissions value.
  • For all items, HCP maintains atime, ctime, and mtime attributes.
NoteMetafiles also have POSIX metadata. With the WebDAV, CIFS, and NFS protocols, you can view the POSIX metadata for metafiles but you cannot explicitly change it.

For example, if you specify an mtime value for an object in a WebDAV command, the mtime values of the corresponding metadirectories equal the specified value. However, the mtime values for the metafiles in these directories reflect the time the request executed. (The atime values of the metafiles equal any specified atime value.)

POSIX time attributes

HCP maintains these POSIX time attributes for all items:

  • atime

    access time

    The time the item was initially added to the namespace. You can change the value of this attribute. For objects, this has no effect unless the atime attribute is synchronized with HCP retention settings.

    HCP does not automatically update the value of this attribute except when atime synchronization is in effect.

  • ctime

    change time

    The time of the last change to the item metadata. The initial value of this attribute is the time the item was added to the namespace. HCP automatically updates the value each time the item metadata changes.

    You cannot change the value of this attribute.

  • mtime

    modify time

    The time the item was initially added to the namespace. You can change the value of this attribute. However, this has no effect on the item.

    HCP does not automatically update the value of this attribute.

POSIX ownership and permissions

For items stored through NFS, HCP maintains these POSIX metadata values:

  • A user ID (UID), which represents the owner, and a group ID (GID), which represents the owning group. UIDs and GIDs are integers greater than or equal to zero.

    UIDs are not related to HCP-specific object ownership.

  • A POSIX permissions value, which is made up of three sets of POSIX permissions — one for the user identified by the POSIX user ID, one for the group identified by the POSIX group ID, and one for all others. A set of permissions is any combination of read, write, or execute, including none.

    POSIX permission values determine the actions users can perform on an item when accessing it on a CIFS or NFS client:

    • For an object:
      • Read permission lets users view and retrieve the object content.
      • Write permission has no effect.
        NoteEven if an object has write permission, its data is secure because WORM semantics prevent it from being modified.
      • Execute permission, which applies only to objects created for executable files, lets users execute the object.
    • For a directory:
      • Read permission lets users see which objects are in the directory.
      • Write permission lets users add and delete objects in the directory or rename empty subdirectories.
      • Execute permission lets users traverse the directory to get to known objects in it, but it does not let users read the directory.
Viewing POSIX permissions

You can view POSIX permission values through the WebDAV, CIFS, and NFS protocols.

With the WebDAV and NFS protocols, permissions are represented by three 3-character strings: one for the owner, one for the owning group, and one for all others. From left to right, the three character positions in each string represent read (r), write (w), and execute (x). Each position has either the character that represents the applicable permission or a hyphen (-), meaning that the permission is denied.

For example, the string below means that the object owner has all permissions for the object, the owning group has read and execute permissions, and all others have only read permission:

-rwxr-xr--

The initial hyphen (-) indicates that this is an object. For a directory, the hyphen is replaced by the letter d. For a symbolic link, it is replaced by the letter l (lower case L).

Windows displays permissions in the Security tab in the Properties window for an item. These permissions don’t map exactly to the POSIX permissions used in a namespace.

Octal permission values

Permissions are also represented by octal values. Each object has an octal permission value that’s the sum of the octal permission values specified for the object owner, the owning group, and all other users not in that group. The table below shows the value that corresponds to each permission.

ReadWriteExecute
Owner400200100
Group040020010
Other004002001

You can represent permissions numerically by combining these values. For example, the octal value 755 represents these permissions:

POSIX owner has read, write, and execute permissions (700).

POSIX group has read and execute permissions (050).

Other has read and execute permissions (005).

POSIX ownership and permissions for new items

For items added through NFS, HCP sets the initial UID, GID, and POSIX permissions to values determined by the NFS client. For items added through HTTP, WebDAV, CIFS, or SMTP, HCP does not set values for these properties.

When you use NFS to view POSIX ownership and permission values for any item for which these values have not been set, HCP displays default values. For UID and GID, these values are determined by the namespace configuration. For POSIX permissions, these values are 555 for objects and 777 for directories.

You can use the CIFS or NFS protocol to change the UID, GID, and permissions for an object.

Changing POSIX ownership and permissions for existing items

If you’re the root user, you can change the UID and GID for an existing item. If you’re the user identified by the UID value for an item or the root user, you can change the permissions for an item. You can make these changes through the CIFS and NFS protocols.

Note
  • The namespace can be configured to disallow POSIX ownership and permission changes for objects that are under retention.
  • If an object has an HCP-specific object owner, you cannot change the UID and GID values for the object.

To change POSIX ownership and permissions through CIFS or NFS, you use the standard technique for that protocol.

Versioning

Versioning is the capability of the namespace to store multiple versions of objects. If the namespace supports versioning, you can save a new version of an object and the previous version will be kept.

Each version is an independent copy of the object that includes both data and metadata. A new version is created each time you save the object, but not when you modify metadata, ACLs, or custom metadata. A version can also be a special entity that represents a deleted object.

NoteIn this book, the term object refers to an object that does not have versions or to the current version of an object with multiple versions. The term old version refers to any version of an object except the current one.

You use HTTP to save a new version of an object. HCP does not support versioning operations through any other protocol.

All objects, including those created when versioning is not enabled, have version IDs. Version IDs are integers. Each time a new version of an existing object is created, it is assigned an ID that is greater than the previous version of the object. The IDs for multiple versions of an object are not necessarily consecutive numbers.

Directories do not normally have versions. One exception exists: if a directory contains or contained one or more versions of an object and also contained, at some time, a subdirectory with the same name as the object, the deleted subdirectory has a version ID.

You can use version IDs and version ID ranges to delete older versions of objects (provided that the versions are not under retention or on hold). Similarly, you can use timestamps and timestamp ranges to delete older versions of objects.

If versioning is enabled for the namespace, pruning may also be enabled. Pruning is the automatic deletion of old versions after a specific length of time. HCP does not prune versions of objects that are on hold.

Each version of an object has its own retention, shred, and index settings and its own custom metadata.

When you store a new version of an object, the retention setting of the old version doesn’t change. However, you cannot explicitly delete old versions of objects under retention, even if their retention periods have expired.

If you specify a retention value for an object by using an offset or retention class, the retention period is relative to the time the current version was stored, not when the first version was stored.

Hold settings apply to all versions of an object; if you put an object on hold, all old versions of the object are also put on hold.

Object owner settings apply to all versions of an object; if you change the owner of an object, that change affects all versions of the object. However, if you restore a deleted object, the object owner of the object may differ between the current version and the old version or versions.

ACLs do not apply to all versions of an object. You can add, replace, or delete an ACL from any specific version of an object or all versions of the object. If you restore a deleted object, the ACL on the object may differ between the current version and the old version or versions.

Tenant administrators can enable, disable, and reenable versioning for the namespace. When versioning is disabled, old versions are not lost. However, while versioning is disabled, you cannot list or access old versions. If versioning is later reenabled, you can again list, retrieve, and purge any preexisting versions (until HCP prunes them because of their age).

Creating versions

If versioning is enabled, you can store new versions of an existing object by storing the object using the same URL you used to create the object originally. HCP retains the previous version until it is pruned or you explicitly purge the object.

Updates to the system metadata, custom metadata, or ACL for an object do not create a new version of the object.

When HCP creates a new version of an object, the previous version becomes an old version:

  • The old version:
    • Includes the object data, system metadata, custom metadata, and ACL. This information, except the hold setting, owner, and ACL, is locked and treated as historic data; it cannot be changed.
    • Can be accessed by using the version ID that was assigned when you first stored it.
  • The new version:
    • Gets a version ID that is greater than the version ID of the previous version.
    • Inherits the system metadata, custom metadata, and ACL from the previous version, unless you specify new metadata when you add the new version to the namespace.
NoteYou cannot create new versions of an object that is under retention or on hold or that has a retention setting of -2 (Initial Unspecified).

Retrieving and listing versions

When you retrieve an object from a namespace that supports versioning, you get the current version by default. You can retrieve an older version of the object by specifying a version ID or ingest time. You can also retrieve multiple object versions by specifying a range of version IDs. To determine the ID to use, you can retrieve an XML listing of the available versions of the object.

When you list versions, the list includes the system metadata values for each version. You can retrieve the system metadata for an individual version by specifying the version ID in the command that checks for object existence. You can retrieve both the custom metadata and system metadata for an individual version by specifying the version ID in the command that retrieves custom metadata. You can retrieve the ACL for an individual version by specifying the version ID in the command that retrieves an ACL.

Deleting objects with versioning enabled

If you delete an object while versioning is enabled, HCP:

  • Retains a copy of the deleted object as an old version.
  • Creates a special version, called a delete marker, as the current version to indicate that the object has been deleted. This version has a version ID but does not include any object data or metadata.

After you delete an object, any attempt to retrieve the object without specifying the version ID of an old version results in an HTTP 404 (Not Found) error code.

HCP keeps all old versions of a deleted object until they are pruned or the object is purged. If shredding is enabled on a version, HCP shreds the version when it is pruned or when the object is purged.

Because HCP keeps deleted objects as old versions, you can retrieve an accidentally deleted object if HCP has not yet pruned it. To do so, request the object, specifying the version ID of the version you want. You can restore the deleted object by storing the retrieved version as a new version.

NoteWhen you restore a deleted object, the new version gets the default metadata values for the namespace (unless you explicitly override them). The new version does not inherit the system metadata, custom metadata, or ACL from the delete marker or any old version of the object.

When you list the versions of an object that has been deleted and restored, the returned XML for the delete marker identifies the version state as deleted, contains the date and time the object was deleted, and contains the version ID for the delete marker. The remaining XML contains information about the object that was deleted.

Purging objects

You cannot delete specific old versions of an object, but if you have purge permission, you can purge the object to delete all its versions.

If the current version of an object is under retention, you cannot purge the object. However, if you have both purge and privileged permissions, you can perform a privileged purge to delete all versions of the object.

Custom metadata

Custom metadata is user-supplied descriptive information about an object. Custom metadata is specified as annotations, where each annotation is a discrete unit of information about the object.

You can add, replace, or delete an annotation in its entirety. You cannot modify it in place.

Each object can have zero or more annotations, including a special one named default.

Annotations let you compartmentalize custom metadata by logical units, such as organization or application. They can also help prevent collisions between two users or applications trying to update custom metadata at the same time. You can create an annotation that has no content.

Each annotation has a name. The default annotation can be created or accessed without specifying a name or by using the name default. Annotation names must be one through 32 characters long. They must contain at least one alphanumeric character and can contain periods (.), underscores (_), and hyphens (-). Annotation names are case sensitive.

Annotations are typically specified in XML format, but this is not necessarily required. The namespace configuration determines whether HCP checks that annotations are well-formed XML. While checking is enabled, if you try to store an annotation that is not well-formed XML, HCP rejects it.

The following list shows limits that apply to custom metadata.

  • Maximum number of annotations per object

    10

  • Maximum annotation size

    1 GB

  • Largest annotation that HCP indexes

    1 MB

  • Largest nondefault annotation that HCP checks for well-formed XML

    1 MB

  • Largest default annotation that HCP checks for well-formed XML

    1 GB

  • Maximum level of nested XML elements

    100

  • Maximum number of XML elements in an annotation

    10,000

As specified above, the largest annotation that HCP indexes is 1 MB. This means that the cumulative size of all indexed annotations for the object cannot exceed 1 MB total. For example, if you have four separate 215 KB annotations with a cumulative size of 860 KB, HCP indexes all annotations. If you add a fifth 215 KB annotation, the cumulative size exceeds the 1 MB limit and HCP does not index any of the annotations for the object.

To add, replace, or delete the default annotation, you can use the HTTP or WebDAV protocol. To add, replace, or delete any other annotation, you need to use HTTP.

With HTTP, you can use a single request to store or retrieve an annotation together with the object data. With WebDAV, you store and retrieve annotations separately from object data.

With WebDAV, you store the default annotation in the custom-metadata.xml metafile. You can also use the custom-metadata.xml metafile to store dead properties.

With HTTP, you use a GET request to retrieve an annotation for an object. With WebDAV, CIFS, and NFS, you retrieve the default annotation for an object from the custom-metadata.xml metafile.

The namespace configuration determines what you can do with custom metadata for objects that are under retention. The namespace can be set to:

  • Allow all annotation operations for objects under retention
  • Allow only the addition of new annotations for objects under retention and disallow replacement or deletion of existing annotations
  • Disallow all annotation operations for objects under retention

Whether annotations are indexed and how they are indexed depend on system-level and namespace-level settings.

Access control lists

An access control list (ACL) grants permissions to perform operations on an individual object to specified users or groups of users. An ACL can be specified as either XML or JSON. You add, replace, or delete an ACL in its entirety. You cannot modify it in place.

An ACL contains up to one thousand access control entries (ACEs). Each ACE specifies one user or one group of users and the permissions granted to that user or group. In the ACL body, an ACE is represented by the grant entry.

NoteThis book uses the term entry to refer to an XML element and the equivalent JSON object and the term property for an XML attribute or the equivalent JSON name/value pair.

When you specify an ACL for an object, you can grant only the permissions you already have. That is, you cannot use an ACL to grant permissions that exceed your own.

To add, replace, or delete an ACL, you use HTTP.

With HTTP, you use a GET request to retrieve an ACL for an object. With WebDAV, CIFS, and NFS, you view the ACL for an object in the acl.xml metafile.

HCP provides two predefined ACLs that you can specify when storing an object:

  • all_read

    Allows any user, authenticated or anonymous, to view and retrieve the object

  • auth_read

    Allows any authenticated user to view and retrieve the object

The use of ACLs is enabled on a per-namespace basis. In namespaces where ACLs are enabled, the namespace can be configured to either enforce or ignore the permissions granted by ACLs. To find out the ACL settings for a namespace, contact your tenant administrator.

ACL permissions

The following describes the permissions that can be granted through an ACL.

  • Read

    • Retrieve objects and system metadata
    • Check for object existence
    • List annotations
    • Check for and retrieve custom metadata
  • Read_ACL

    Check for and retrieve ACLs

  • Write

    • Store objects
    • Create directories
    • Set and change system and custom metadata
  • Write_ACL

    Set and change ACLs

  • Delete

    Delete objects, custom metadata, and ACLs

NoteWith the CIFS or NFS protocol:
  • If you are using an AD user account, when you try to delete an object for which you have been granted delete permission by an ACL, the delete operation fails if the object is located in the root data directory.
  • To read or retrieve an object for which you have been granted read permission by an ACL, you also need browse permission for the namespace.

ACL body

The body of an ACL consists of entries in XML or JSON format.

XML format

The XML ACL body has the format shown below. Elements at each hierarchical level can be in any order.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<accessControlList>
    <grant>
        <grantee>
            <type>(user|group)</type>
            <name>(hcp-username
            |active-directory-username
            |active-directory-group
                  |all_users
            |authenticated)
            </name>
            If the name entry specifies an Active Directory user or
            group, include the domain element
            <domain>active-directory-domain</domain>
        </grantee>
        <permissions>
            Any combination of the following
            <permission>READ</permission>
            <permission>READ_ACL</permission>
            <permission>WRITE</permission>
            <permission>WRITE_ACL</permission>
            <permission>DELETE</permission>
        </permissions>
    </grant>
    Up to 999 additional grant elements
</accessControlList>
JSON format

The JSON ACL body has the format shown below. Entries at each hierarchical level can be in any order.

{
    "grant": [{
        "grantee": {
            "type":"(user|group)",
            "name":"(hcp-username
            |active-directory-username
            |active-directory-group
            |all_users
            |authenticated)"[,]
            If the name entry specifies an Active Directory             user or group,
            include the domain entry
            "domain":"active-directory-domain"
        },
        "permissions": {
            "permission":[["READ"[,]|"READ_ACL"[,]|"WRITE"[,]
            |"WRITE_ACL"[,]|"DELETE"]]
        }
    }]
    Up to 999 additional grant entries
}
ACL contents

XML has a single top-level accessControlList element. JSON has a corresponding unnamed top-level object. All ACLs must contain this entry in their body. The top-level entry contains the entries listed in the table below.

EntryValid valuesDescription
grantN/A

Container for grantee and permissions entries. Identifies one user or one group of users and the permissions granted to that user or group.

An ACL can contain up to one thousand grant entries.

granteeN/AChild of grant entry. Container for name, type, and domain entries.
name

One of:

  • The username of a user that’s defined in HCP.
  • The username of an Active Directory user account. This can be either the user principal name or the Security Accounts Manager (SAM) account name for the AD user account.
  • The name of an Active Directory group.
  • all_users
  • authenticated

Specifies the user or group of users to which the ACL grants permissions.

HCP has two special groups that you can specify in an ACL:

  • all_users

    Grants permissions to all users

  • authenticated

    Grants permissions to all authenticated users

To grant permissions to one of these special groups, specify group in the type entry and omit the domain entry.

HCP returns an HTTP 400 (Bad Request) error code if a user or group is specified in more than one name entry.

type

One of:

  • user

    The name entry specifies an HCP or Active Directory user account

  • group

    The name entry specifies an Active Directory group, all_users, or authenticated

Specifies the type of the value specified in the name entry.

HCP returns an HTTP 400 (Bad Request) error code if the value of the type entry doesn’t correspond to the value of the name entry.

domainThe name of an Active Directory domain

Specifies the Active Directory domain that contains the user account or group specified in the name entry.

This entry is required if the name entry specifies an Active Directory user account or group.

permissionsN/AContainer for any combination of permission entries.
permission

Any of:

  • READ
  • WRITE
  • READ_ACL
  • WRITE_ACL
  • DELETE
Child of permissions entry. Specifies the permissions granted to the user or group specified in the name entry.

ACL examples

This section contains examples of an ACL in both XML and JSON formats. This ACL grants:

  • Read permission to all users
  • Write permission to the AD user lgreen that is defined in the ad.example.com domain
  • Write and delete permission to HCP user pdgrey
XML format
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<accessControlList>
    <grant>
        <grantee>
            <name>all_users</name>
            <type>group</type>
        </grantee>
        <permissions>
            <permission>READ</permission>
        </permissions>
    </grant>
    <grant>
        <grantee>
            <name>lgreen</name>
            <type>user</type>
            <domain>ad.example.com</domain>
        </grantee>
        <permissions>
            <permission>WRITE</permission>
        </permissions>
    </grant>
    <grant>
        <grantee>
            <name>pdgrey</name>
            <type>user</type>
        </grantee>
        <permissions>
            <permission>WRITE</permission>
            <permission>DELETE</permission>
        </permissions>
    </grant>
</accessControlList>
JSON format
{
    "grant": [{
        "grantee": {
            "name":"all_users",
            "type":"group"
        },
        "permissions": {
            "permission":["READ"]
        }
    }]
    "grant": [{
        "grantee": {
            "name":"lgreen",
            "type":"user",
            "domain":"ad.example.com"
        },
        "permissions": {
            "permission":["WRITE"]
        }
    }]
    "grant": [{
        "grantee": {
            "name":"pdgrey",
            "type":"user"
        },
        "permissions": {
            "permission":["WRITE","DELETE"]
        }
    }]
}

ETags

An ETag is an identifier for the content of an object. HCPgenerates an ETag for each object or version stored in a namespace. HCP returns the ETag for an object or version in an ETag header in the response to a GET, HEAD, or PUT request for that object or version. If HCP has not yet generated the ETag for an object or version, the ETag header is not included in the response.

You can use ETags in conditional operations. For example, you could use an If-None-Match header to store a new version of an object only if the content of that new version does not match the content of the current version.

Replication collision handling

If clients can write to multiple systems in a replication topology, collisions can occur when different changes are made to the same objects on different systems. Similarly, if you can make configuration changes to the tenant and its namespaces on multiple systems in a replication topology, configuration collisions can occur.

The way HCP handles collisions that occur due to replication depends on the type of collision. However, the general rule is that more recent changes have priority over conflicting less recent changes.

Object content collisions

An object content collision occurs when, for a namespace without versioning enabled, these events occur in the order shown:

  1. An object is created with the same name in that namespace on two systems in a replication topology, but the object has different content on the two systems.
  2. The object on one of the systems is replicated to the other system.

If versioning is enabled, no collision occurs. Instead, the less recently created of the two objects becomes an old version of the more recently created object.

When an object content collision occurs, the more recently created object keeps its name and location. The other object is either moved to the .lost+found directory in the same namespace or renamed, depending on the namespace configuration.

When HCP moves an object to the .lost+found directory, the full object path becomes .lost+found/replication/system-generated-directory/ old-object-path.

When renaming an object due to a content collision, HCP changes the object name to object-name.collision or object-name.version-id.collision, where version-id is the version ID of the object. HCP uses the second format only if versioning has ever been enabled for the namespace that contains the object but is not currently enabled.

If the new name is already in use, HCP changes the object name to object-name.1.collision or object-name.version-id.1.collision, as applicable. If that name is already in use, HCP successively increments the middle integer by one until a unique name is formed.

Objects that have been relocated or renamed due to content collisions are flagged as replication collisions in their system metadata. Clients can use the metadata query API to search for objects that are flagged as replication collisions.

If an object that’s flagged as a replication collision changes (for example, if its retention period is extended), its collision flag is removed. If a client creates a copy of a flagged object with a new name, the collision flag is not set on the copy.

You can configure namespaces to have the DIsposition service automatically delete objects that are flagged as replication collisions. When selecting this option for a namespace, you specify the number of days the DIsposition service should wait before deleting such an object. The days are counted from the time the collision flag is set. If the collision flag is removed from an object, the object is no longer eligible for deletion by the DIsposition service.

System metadata collisions

A system metadata collision occurs when these events occur in the order shown:

  1. Different changes are made to the system metadata for a given object on each of two systems in a replication topology.
  2. The changed system metadata on one of the systems is replicated to the other system.
For example, suppose a user on one system changes the shred setting for an object while a user on the other system changes the index setting for the same object. When the object on either system is replicated to the other system, a system metadata collision occurs.

If a collision occurs when changed system metadata for a given object is replicated from one system (system A) in a replication topology to another system (system B) in the topology:

  • For changed system metadata other than the retention setting and hold status:
    • If the last change made on system A is more recent than the last change made on system B, HCP changes the system metadata on system B to match the system metadata on system A.
    • If the last change on system B is more recent than the last change on system A, HCP does not change the system metadata on system B.
  • For a changed retention setting:
    • If the retention setting on system A specifies a longer retention period than does the retention setting on system B, HCP changes the retention setting on system B to match the retention setting on system A.
    • If the retention setting on system B specifies a longer retention period than does the retention setting on system A, HCP does not change the retention setting on system B.
  • For a changed hold status:
    • If the object is on hold on system A but not on system B, HCP places the object on hold on system B.
    • If the object is on hold on system B but not on system A, HCP leaves the object on hold on system B.
Here are some examples of how HCP handles collisions when changed system metadata for a given object is replicated from one system (system A) in a replication topology to another system (system B) in the topology.
Example 1

The object starts out on both system A and system B with these system metadata settings:

  • Shred: false
  • Index: false

The list below shows a sequence of events in which the system metadata for the object is changed and the changes are then replicated.

  1. On system A, a client changes the shred setting to true.
  2. On system B, a client changes the index setting to true.
  3. The changes on system A are replicated to system B. The resulting settings for the object on system B are:
    • Shred: false
    • Index: true
Example 2

The object starts out on both system A and system B with these system metadata settings:

  • Retention: Initial Unspecified
  • Shred: false
  • Index: false

The list below shows a sequence of events in which the system metadata for the object is changed and the changes are then replicated.

  1. On system A, a client changes the retention setting to Deletion Prohibited.
  2. On system B, a client changes the retention setting to Deletion Allowed.
  3. On system B, a client changes the index setting to true.
  4. On system A, a client changes the shred setting to true.
  5. The changes on system A are replicated to system B. The resulting settings for the object on system B are:
    • Retention: Deletion Prohibited
    • Shred: true
    • Index: false
Example 3

The object starts out on both system A and system B with these system metadata settings:

  • Retention: Initial Unspecified
  • Hold: true
  • Shred: false
  • Index: false

The list below shows a sequence of events in which the system metadata for the object is changed and the changes are then replicated.

  1. On system A, a client changes the retention setting to Deletion Allowed.
  2. On system B, a client changes the retention setting to Deletion Prohibited.
  3. On system B, a client changes the index setting to true.
  4. On system A, a client changes the shred setting to true.
  5. On system A, a client releases the object from hold.
  6. The changes on system A are replicated to system B. The resulting settings for the object on system B are:
    • Retention: Deletion Prohibited
    • Hold: true
    • Shred: true
    • Index: false
  7. The changes on system B are replicated to system A. The resulting settings for the object on system A are:
    • Retention: Deletion Prohibited
    • Hold: true
    • Shred: true
    • Index: false

Custom metadata collisions

A custom metadata collision occurs when these events occur in the order shown:

  1. One of these changes occurs:
    • An annotation is added with the same name to a given object on each of two systems in a replication topology, but the annotation has different content on the two systems.

      The addition of an annotation to a given object on only one of the systems does not result in a custom metadata collision if the object does not have an annotation with the same name on the other system. In this case, the new annotation is replicated without conflict.

    • Different changes are made to the content of a given annotation for a given object on each of the two systems in a replication topology.
    • A change is made to the content of a given annotation for a given object on one system in a replication topology, and the same annotation is deleted on another system in the topology.
  2. The change made on one of the systems is replicated to the other system.

If a collision occurs when a custom metadata change for a given object is replicated from one system (system A) in a replication topology to another system (system B) in the topology:

  • If the last change on system A is more recent than the last change on system B, HCP applies the change from system A to the custom metadata on system B
  • If the last change on system B is more recent than the last change on system A, HCP does not change the custom metadata on system B
Here are two examples of how HCP handles collisions when custom metadata changes for a given object are replicated from one system (system A) in a replication topology to another system (system B) in the topology.
Example 1

The object starts out with annotations named a1 and a2 on both system A and system B.

The list below shows a sequence of events in which the annotations for the object are changed and the changes are then replicated.

  1. On system B, a client changes the content of a1.
  2. On system A, a client makes a different change to the content of a1.
  3. On system A, a client adds annotation a3 to the object.
  4. On system B, a client adds annotation a3 with different content from the a3 added on system A.
  5. The changes on system A are replicated to system B. The resulting annotations for the object on system B are:
    • a1 with the changed content from system A
    • a2 (unchanged)
    • a3 with the content added on system B
  6. The changes on system B are replicated to system A. The resulting annotations for the object on system A are:
    • a1 with the changed content from system A
    • a2 (unchanged)
    • a3 with the content added on system B
Example 2

The object starts out with the annotations named a1, a2, and a3 on both system A and system B.

The list below shows a sequence of events in which the annotations for the object are changed and the changes are then replicated.

  1. On system B, a client changes the content of a1.
  2. On system A, a client deletes a1.
  3. On system A, a client changes the content of a2.
  4. On system B, a client changes the content of a2.
  5. On system A, a client deletes a3.
  6. On system B, a client changes the content of a3.
  7. The changes on system A are replicated to system B. The resulting annotations for the object on system B are:
    • a2 with the changed content from system B
    • a3 with the changed content from system B
  8. The changes on system B are replicated to system A, the resulting annotations for the object on system A are:
    • a2 with the changed content from system B
    • a3 with the changed content from system B

Access control list collisions

An ACL collision occurs when these events occur in the order shown:

  1. Different changes are made to the ACL for a given object on each of two systems in a replication topology.
  2. The changed ACL on one of the systems is replicated to the other system.

An ACL is treated as a single unit. If a collision occurs when a changed ACL for a given object is replicated from one system (system A) in a replication topology to another system (system B) in the topology:

  • If the last change to the ACL on system A is more recent than the last change to the ACL on system B, HCP changes the ACL on system B to match the changed ACL on system A
  • If the last change to the ACL on system B is more recent than the last change to the ACL on system A, HCP does not change the ACL on system B
For example, suppose the ACL for a given object starts out with these grants on both system A and system B:
  • All users: read
  • User lgreen: write
  • User mwhite: write, delete

The list below shows a sequence of events in which the ACL for the object is changed and the change is then replicated.

  1. On system B, a client changes the grants in the ACL to:
    • All users: read
    • User lgreen: write, delete
    • User pdgrey: write
  2. On system A, a client changes the grants in the ACL to:
    • All users: read
    • User mwhite: write
    • User pdgrey: write
  3. The changed ACL on system A is replicated to system B. The resulting ACL for the object on system B contains these grants:
    • All users: read
    • User mwhite: write
    • User pdgrey: write

 

  • Was this article helpful?