Working with objects

You use the HTTP PUT method to store an object in a bucket. To store an object, you need write permission for the bucket.

For a request to store an object, the request body consists of the data in a specified file. This data becomes the object content.

When you store an object, you specify a name for it. The object name does not need to be the same as the name of the file containing the original data. For information on naming objects, see Object names.

If versioning is enabled and you try to store an object with the same name as an existing object, HCP creates a new version of the object. If versioning is disabled and you try to store an object with the same name as an existing object, HCP returns a 409 (Conflict) status code and does not store the object. For information on versioning, see Versioning.

You can add custom metadata to an object in the same request as you use to store the object. To do this, you use x-amz-meta- headers. For information on custom metadata, see Custom metadata.

You can specify an ACL for an object in the same request as you use to store the object. To do this, you need to use ACL headers. You cannot use an ACL request body when storing an object. For information on ACLs, see Access control lists.

If the ACL you specify in a request to store an object is invalid, HCP returns a 400 (Bad Request) or 501 (Not Implemented) status code and does not store the object.

If you’re an authenticated user, when you store an object, you become the object owner. If you’re accessing the bucket anonymously, the new object has no owner. For information on object ownership, see Object owners.

ETags

When you store an object, HCP returns the ETag for the object in the ETag response header. An ETag is an identifier for the content of an object. ETags are useful for making object-level operations conditional based on the object content. Operations that can be made conditional are checking the existence of an object (Checking the existence of an object or folder), copying an object (Copying an object), and retrieving an object (Retrieving an object).

Ensuring object integrity

When you store an object, you can use the Content-MD5 request header to ensure the integrity of the object data. The value you specify for this header must be the Base64-encoded MD5 hash of the original file data.

When you include the Content-MD5 header in a request to store an object, HCP calculates the Base64-encoded MD5 hash of the data it receives and compares that to the header value. If the values don’t match, HCP returns a 400 (Bad Request) status code and does not store the object.

Object encryption

When you store an object, you can use the x-amz-server-side-encryption request header to determine whether objects stored in HCP are encrypted. If stored objects are encrypted, the response headers include x-amz-server-side-encryption with a value representing the encryption algorithm and key length HCP is using. If stored objects are not encrypted, the value of the x-amz-server-side-encryption response header is None.

You use the HTTP PUT method to create a folder in a bucket. To create a folder, you need write permission for the bucket.

To tell HCP to create a folder instead of an object in response to a PUT request, you can do either these:

•Include a forward slash (/) or the percent-encoded equivalent (%2F) after the folder name in the request.

•Include the Content-Type header in the request with a value of x-directory.

In either case, you also need to include the Content-Length header in the request. However, HCP ignores any request body.

When you create a folder, you specify a name for it. The rules for folder names are the same as the rules for object names. For more information on object names, see Object names.

You use the HTTP HEAD method to check the existence of an object or folder in a bucket. To check the existence of an object, you need read permission for the bucket containing the object or for the object itself. To check the existence of a folder, you need read permission for the bucket.

In response to a request to check the existence of an object or folder, HCP returns a 200 (OK) status code if the object or folder exists and a 404 (Not Found) status code if the object or folder doesn’t exist. If you don’t have read permission for the bucket or object, HCP returns a 403 (Forbidden) status code.

Checking the existence of an object version

By default, a HEAD request to check the existence of an object checks the existence of the current version of the object. However, while versioning is enabled for the target bucket, you can use the versionId query parameter to check the existence of a specific version of an object.

If the version identified by the versionId parameter does not exist, HCP returns a 404 (Not Found) status code. If the version identified by the versionId parameter is a delete marker, HCP returns a 200 (OK) status code.

Valid values for the versionId parameter are integers greater than or equal to zero. Normally, the value is the version ID of a version of the object specified in the request.

The versionId query parameter is not case sensitive.

If you include the versionId parameter in a HEAD request while versioning is disabled, the parameter is ignored.

For information on object versions, see Versioning. For information on delete markers, see Deleting an object or folder.

Object information

If the status code returned in response to a request to check the existence of an object, object version, or folder is 200 (OK), the response headers include this information:

•The ETag for the specified item (ETag header). For information on ETags, see Storing an object.

•While versioning is enabled for the target bucket, the version ID of the specified item (x-amz-version-id header).

•The date the specified item was last modified (Last-Modified header).

•The Internet media type of the specified item (Content-Type header).

•The size of the specified item, in bytes (Content-Length header).

•For an object or object version, any custom metadata stored in the .metapairs annotation for that item (x-amz-meta- headers). For information on x-amz-meta- headers, see Custom metadata.

Conditionally checking the existence of an object, object version, or folder

You can use the If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since request headers to make HEAD requests conditional:

•The If-Match and If-None-Match headers compare the ETag for the specified item to one or more values that you specify. Typically, each value is the ETag for an object or object version of interest.

•The If-Modified-Since and If-Unmodified-Since headers compare the date and time the specified item was last modified to a date and time that you specify. The comparison is at the level of seconds, so you cannot use these headers to differentiate between changes that happened at different milliseconds within the same second.

If the specified item meets all the conditions specified by the conditional headers included in the request, HCP returns a 200 (OK) status code. If the specified item does not meet the condition specified by:

•An If-Match or If-Unmodified-Since header, HCP returns a 412 (Precondition Failed) status code

•An If-None-Match or If-Modified-Since header, HCP returns a 304 (Not Modified) status code

If a request includes multiple different conditional headers, HCP processes any If-Match and If-None-Match headers before any If-Modified-Since or If-Unmodified-Since headers. If a request includes more than one of any given header, HCP processes only the first one of those headers and ignores the others.

You use the HTTP PUT method with the acl query parameter to add an ACL to an existing object. Adding an ACL to an object replaces any existing ACL in its entirety. You cannot modify an existing ACL in place.

To add an ACL to an object, you need write ACL permission for the bucket containing the object or for the object itself.

You can add an ACL only to the current version of an object. However, the ACL you add applies to all versions of the object.

To add an ACL to an object, you can use either request headers or an ACL request body. You cannot use ACL headers and an ACL request body in the same request.

With ACL headers, you can specify either a canned ACL or individual x-amz-grant- headers. You cannot specify both a canned ACL and an x-amz-grant- header in the same request.

You can use an ACL request body to change the owner of an object. You cannot use ACL headers to do this. To change the owner of an object, you need both write ACL permission for the bucket or object and change owner permission for the bucket.

If you try to add an ACL that specifies a user account that does not exist, HCP returns a 400 (Bad Request) status code and does not add the ACL to the object.

For an introduction to ACLs and information on how to specify them, see Access control lists.

You use the HTTP GET method with the acl query parameter to retrieve the ACL for an object. To retrieve the ACL for an object, you need read ACL permission for the bucket containing the object or for the object itself.

The object ACL is returned in an XML response body. The format of the response body is the same as the format you use for the ACL request body when you add an ACL to a bucket. For the format of the ACL request body, see Specifying an ACL in the request body.

For an introduction to ACLs, see Access control lists.

You use the HTTP PUT method with the x-amz-copy-source header to copy an object from one location to another. The source and target locations can be two different buckets within the same tenant, or they can be the same bucket. The source object is the object you’re copying. The target object is the object that results from the copy operation.

To copy an object, you need read permission for the bucket containing the source object or for the source object itself and write permission for the target bucket.

When copying an object, you can specify a name for the target object that’s different from the name of the source object.

By default, a copy operation copies the current version of the source object specified in the request. However, while versioning is enabled for the source bucket, you can use the versionId query parameter with the source object specification to copy a specific version of the object.

If the version identified by the versionId parameter is a delete marker, HCP returns a 503 (Service Unavailable) status code. If the current version of the source object is a delete marker and you don't specify a version ID, HCP returns a 404 (Not Found) status code. Fo r information on delete markers, see Deleting an object or folder.

HCP does not copy version IDs with objects. The object created by a copy operation has its own version ID.

By default, HCP copies any custom metadata for the source object to the target object. However, in the copy request, you can specify replacement custom metadata to be used for the target object. To apply this custom metadata to the target object, you need to include the x-amz-metadata-directive header with a value of REPLACE in the copy request.

HCP does not copy ACLs with objects. However, in the copy request, you can specify an ACL for the target object. To do this, you need to use ACL headers. You cannot use an ACL request body when copying an object. For information on ACLs, see Access control lists.

If the ACL you specify in a request to copy an object is invalid, HCP returns a 400 (Bad Request) or 501 (Not Implemented) status code and does not copy the object.

If you’re an authenticated user, when you copy an object, you become the owner of the target object. If you’re accessing the bucket anonymously, the target object has no owner. For information on object ownership, see Object owners.

In response to a request to copy an object, HCP returns an XML response body containing the ETag and last modification date of the target object. For the format of this response body, see Response body (PUT object copy).

Note: Depending on the replication configuration, copy operations may be slow and, for large objects, may time out.

Copying an object to itself

If the target object you specify in a copy request identifies an existing object and versioning is enabled, HCP creates a new version of the existing object.

If the target object you specify in a copy request identifies an existing object and versioning is disabled:

•If the value of the x-amz-metadata-directive header is REPLACE, HCP replaces all custom metadata for the existing object with the custom metadata specified in the copy request. If the request doesn’t specify any custom metadata, HCP removes all custom metadata from the existing object.

When you replace or remove the custom metadata for an object in this way, the version ID of the object does not change.

•If the value of the x-amz-metadata-directive header is COPY or if the request does not include the x-amz-metadata-directive header, HCP returns a 400 (Bad Request) status code and does not replace the custom metadata for the object.

In no case does HCP replace the content of an existing object.

Conditionally copying an object

You can use the x-amz-copy-source-if-match, x-amz-copy-source-if-none-match, x-amz-copy-source-if-modified-since, and x-amz-copy-source-if-unmodified-since request headers to make copy requests conditional:

•The x-amz-copy-source-if-match and x-amz-copy-source-if-none-match headers compare the ETag of the source object or object version to one or more values that you specify. Typically, each value is the ETag for an object or object version of interest.

•The x-amz-copy-source-if-modified-since and x-amz-copy-source-if-unmodified-since headers compare the date and time the source object or object version was last modified to a date and time that you specify.

If the source object or object version:

•Meets all the conditions specified by the conditional headers included in the request, HCP performs the copy operation

•Does not meet all the conditions specified by the conditional headers included in the request, HCP returns a 412 (Precondition Failed) status code and does not copy the object

If a request includes multiple different conditional headers, HCP processes any x-amz-copy-source-if-match and x-amz-copy-source-if-none-match headers before any x-amz-copy-source-if-modified-since or x-amz-copy-source-if-unmodified-since headers. If a request includes more than one of any given header, HCP processes only the first one of those headers and ignores the others.

Note: The x-amz-copy-source-if-modified-since and x-amz-copy-source-if-unmodified-since request headers are not compatible with s3curl.

Object encryption

When you copy an object, you can use the x-amz-server-side-encryption request header to determine whether objects stored in HCP are encrypted. If stored objects are encrypted, the response headers include x-amz-server-side-encryption with a value representing the encryption algorithm and key length HCP is using. If stored objects are not encrypted, the value of the x-amz-server-side-encryption response header is None.

Client timeouts

Copying a large object can take some time. If a client times out because a copy operation is taking too long, HCP continues the operation in the background.

Because the connection to the client is broken, HCP cannot report the completion of the copy operation to the client. Until the operation is complete, HCP returns a 404 Not Found status code in response to HEAD requests for the object being created by the copy operation.

After a copy operation finishes successfully, HCP returns information about the new object in response to HEAD requests for the object. If the operation does not succeed, HCP continues to return a 404 Not Found status code in response to HEAD requests for the object.

If copy operations are causing a client to time out, consider increasing the client timeout interval.

You use the HTTP GET method to retrieve an object from a bucket. Retrieving an object means retrieving the object data. To retrieve an object, you need read permission for the bucket containing the object or for the object itself.

Retrieving an object version

By default, a request to retrieve an object retrieves the current version of the object. However, while versioning is enabled for the target bucket, you can use the versionId query parameter to retrieve a specific version of an object.

Valid values for the versionId parameter are integers greater than or equal to zero. Normally, the value is the version ID of a version of the object specified in the request.

The versionId query parameter is not case sensitive.

If you include the versionId parameter in a GET request while versioning is disabled, the parameter is ignored.

For information on object versions, see Versioning.

Objects with custom metadata

If a retrieved object or object version has custom metadata, the headers returned in response to the request include the applicable x-amz-meta- headers. For information on x-amz-meta- headers, see Custom metadata.

Retrieving part of an object

You can use the Range header in a GET request to retrieve only part of an object or object version. By using the Range header, you can limit the amount of data returned, even when you don’t know the size of the object.

The value of the Range header is the range of bytes you want to retrieve. The first byte of the data for an object is in position 0 (zero), so a range of 1-5 specifies the second through sixth bytes, not the first through fifth.

To specify a byte range in a range header, you use this format:

Range: bytes=byte-range

The table below shows the valid values for byte-range.

Value	Description	Example
start-position–end-position	Bytes in start-position through end-position, inclusive. If end-position is greater than the size of the object data, HCP returns the bytes in start-position through the end of the data. Valid values for start-position and end-position are integers greater than or equal to zero. For the specified range to be valid, end-position must be greater than or equal to start-position.	Five hundred bytes beginning with the two-hundred-first: bytes=200-699
start-position–	Bytes in start-position through the end of the object data. Valid values for start-position are integers greater than or equal to zero.	All the bytes beginning with the seventy-sixth and continuing through the end of the object data: bytes=75-
–offset-from-end	Bytes in the offset-from-end position, counted back from the last position in the object data, through the end of the object data. If offset-from-end is greater than the size of the object data, HCP returns the complete object data. Valid values for offset-from-end are integers greater than or equal to zero.	The last 25 bytes of the object data: -bytes=-25

These considerations apply to Range header values:

•If you specify a valid range in which the start position is less than the size of the object data, HCP returns the requested range of data with a 206 (Partial Content) status code.

•If you specify a valid range in which the start position is greater than or equal to the size of the object data, HCP returns a 416 (Requested Range Not Satisfiable) status code and does not return any data.

•If you specify an offset of zero, HCP returns a 416 (Requested Range Not Satisfiable) status code and does not return any data.

•If you specify an invalid value (for example, a value in which the start position is greater than the end position, HCP ignores the Range header and returns the complete object data with a status code of 200 (OK).

A GET request to retrieve a range of bytes in a multipart object is most efficient when the start and end bytes for the range are aligned with the start and end bytes for one or more of the parts uploaded to create the object.

You cannot retrieve a part of an in-progress multipart upload. For more information on multipart objects, see Working with multipart uploads.

Conditionally retrieving an object

You can choose to retrieve an object or object version only if its ETag and/or last modification date and time meet certain criteria. You might do this, for example, in an application that maintains a local cache of frequently used objects. With such an application, you can reduce the load on HCP and the network by retrieving objects only if they have changed since they were cached.

You use the If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since request headers to make GET requests conditional:

•The If-Match and If-None-Match headers compare the ETag for the requested object or object version to one or more values that you specify. Typically, each value is the ETag for an object or object version of interest.

•The If-Modified-Since and If-Unmodified-Since headers compare the date and time the requested object or object version was last modified to a date and time that you specify.

If the requested object or object version meets all the conditions specified by the conditional headers included in the request, HCP returns the object data. If the requested object or object version does not meet the condition specified by:

•An If-Match or If-Unmodified-Since header, HCP returns a 412 (Precondition Failed) status code and does not return the object data

•An If-None-Match or If-Modified-Since header, HCP returns a 304 (Not Modified) status code and does not return the object data

If a request includes multiple different conditional headers, HCP processes any If-Match and If-None-Match headers before any If-Modified-Since or If-Unmodified-Since headers. If a request includes more than one of any given header, HCP processes only the first one of those headers and ignores the rest.

Overriding response headers

In a request to retrieve an object or object version, you can specify values to be returned in certain response headers. The values you specify in the request override any values that might otherwise be returned for those headers. The headers you can override are returned only in response to a successful request.

To specify response header values, you can use the request headers listed in the table below. The valid values for each request header are the valid values for the corresponding response header.

Request header	Response header
response-cache-control	Cache-Control
response-content-disposition	Content-Disposition
response-content-encoding	Content-Encoding
response content-language	Content-Language
response-content-type	Content-Type
response-expires	Expires

This book does not describe the response headers listed above, with the exception of Content-Type. For information on the other response headers, see http://www.w3.org/Protocols/rfc2616/rfc2616.html.

You use the HTTP DELETE method to delete an object or folder in a bucket. To delete an object, you need delete permission for the bucket containing the object or for the object itself. To delete a folder, you need delete permission for the bucket.

Objects under retention or on hold

You cannot delete an object or any version of that object if the object is under retention or on hold.

Deleting an object while versioning is enabled

When you delete an object while versioning is enabled, HCP:

•Retains the current version of the object as an old version. The version ID does not change.

•Creates a delete marker as the new current version of the object. A delete marker is a special version of an object that indicates that a version of the object has been deleted.

A delete marker has a version ID but does not have any data or metadata. When you delete an object while versioning is enabled, the version ID of the delete marker is different from the version ID of the object you deleted.

The example below shows what happens when you delete object Obj1 while versioning is enabled. Obj1 has multiple versions, which are listed below with the current version at the top.

Before deletion	After deletion
Obj1 — version ID 87288727469829 Obj1 — version ID 87288727469825 Obj1 — version ID 87288727469816	Obj1 — version ID 87288727469833 (delete marker) Obj1 — version ID 87288727469829 (old version) Obj1 — version ID 87288727469825 Obj1 — version ID 87288727469816

Deleting an object while versioning is disabled

When you delete an object while versioning is disabled, HCP changes the current version of the object to a delete marker and does not change the version ID.

The example below shows what happens when you delete object Obj1 while versioning is disabled. Obj1 has multiple versions that were created while versioning was enabled. The versions are listed below with the current version at the top.

Before deletion	After deletion
Obj1 — version ID 87288727469829 Obj1 — version ID 87288727469825 Obj1 — version ID 87288727469816	Obj1 — version ID 87288727469829 (delete marker) Obj1 — version ID 87288727469825 Obj1 — version ID 87288727469816

Deleting a deleted object

If you try to delete an object where the current version is a delete marker, HCP returns a 204 (No Content) status code.

Recovering a deleted object

If you inadvertently delete an object, you can recover it from an old version. You can recover an object only while versioning is enabled.

To recover an object, use a PUT request to copy the object to itself. Use the versionId query parameter with the source object specification to specify which version of the object you want to use for recovery.

You cannot use a delete marker to recover an object.

You cannot recover an object from an old version that has been pruned. For information on pruning, see Versioning.

For more information on using the PUT method to copy an object, see Copying an object.

You use the HTTP POST method to delete multiple objects in a bucket. To delete multiple objects, you need delete permission for the bucket containing the objects or for the objects themselves. You cannot use this method with the S3 compatible API to delete:

•Objects that are under retention.

•Objects that are on hold.

•Delete markers of objects.

•Multiple versions of objects.

Since the S3 compatible API currently does not support deleting multiple objects based on version ID, if you try to delete objects with multiple versions, only the current versions of the objects are deleted. For more information on what you can and cannot delete, see Deleting an object or folder.