Support for Amazon S3 API
HCP for cloud scale is compatible with the Amazon Simple Storage Service (Amazon S3) REST API, which allows clients to store objects in containers called buckets. A bucket is a collection of objects and has its own individual settings, such as ownership and lifecycle. Using HCP for cloud scale, you can perform common read and write operations on objects and buckets, and manage ACL settings through the client access data service.
For information about using Amazon S3, see the Amazon S3 API documentation.
For information about obtaining S3 user credentials, see S3 User Credentials.
The following tables list the supported Amazon S3 API features and describes any implementation differences between Amazon and HCP for cloud scale S3 APIs.
Authentication and addressing operations
| Feature | Implementation differences |
| Authentication with AWS Signature Version 4 | Fully implemented |
| Addressing virtual host (like http://bucket.server/object) | Fully implemented |
| Addressing Path style (like http://server/bucket/object ) | Fully implemented |
| Signed/Unsigned payload | Fully implemented |
| Chunked request | Fully implemented |
| Pre-signed URL | Fully implemented |
Service operations
| Feature | Implementation differences |
| GET service (list buckets) | Fully implemented |
Bucket operations
| Feature | Implementation differences |
| GET Bucket (list objects) V1 | Fully implemented |
| GET Bucket (list objects) V2 | Fully implemented |
| PUT Bucket | When anonymous requests to create or delete a bucket use an invalid bucket name, Amazon S3 performs an access check first and returns 403. HCP for cloud scale returns 400 if the bucket name validation check fails. |
| DELETE Bucket | |
| HEAD Bucket | |
| PUT Bucket ACL | ACL email address grantee types are not supported. In AWS each grantee is specified as a type=value pair, where the type is one of the following:
id and uri. |
| GET Bucket ACL | |
| List Multipart Uploads | Fully implemented |
| GET Bucket Lifecycle (except transition action) | HCP for cloud scale does not support Object Transition actions. If these actions are included it will throw a Malformed XML exception. |
| PUT Bucket Lifecycle (except transition action) | |
| DELETE Bucket Lifecycle (except transition action) | |
| GET Bucket Versioning | Version Listing Requests do not strictly comply to documented behavior for NextKeyMarker/NextVersionIdMarker. S3 documentation currently states that these values "specifies the first key not returned that satisfies the search criteria." However, HCP for cloud scale specifies the last key returned in the current response. S3 V1 object listings do not call out as specific a requirement and V2 object listings utilize a continuation token (opaque to the caller); internally, HCP for cloud scale shares the same listing logic across all three listing types. |
| GET Bucket Object Versions | Fully implemented |
Object operations
| Feature | Implementation differences |
| GET Object | If a lifecycle policy is configured for a bucket, HCP for cloud scale displays the expiration date of an object (in the x-amz-expiration header) fetched using the ?versionId subresource. Amazon only displays this when performing unversioned GET requests. |
| HEAD Object | If a lifecycle policy is configured for a bucket, HCP for cloud scale displays the expiration date of an object (in the x-amz-expiration header) fetched using the ?versionId subresource. Amazon only displays this when performing unversioned HEAD requests. |
| PUT Object | Content-Type Validations: Amazon is extremely liberal in what is accepted for the Content-Type of an object. HCP for cloud scale adds additional checks for what is allowed. |
| Object and Version Encoding | Amazon AWSS3 Object and Version listing documentation mentions the ability to pass an encoding parameter (url). This is so the object name XML in the response to the client can be escaped to avoid names containing invalid XML characters. This encoding is only documented as applied to object names and not Owner/DisplayNames. Additionally, there is no mention of escaping for Bucket Listing requests. The Owner/DisplayName is a concern as there is a possibility that user display names may not be able to contain characters that could cause XML parsing issues. Amazon may be able to restrict this, though it does not currently return a display name for all regions. HCP for cloud scale utilizes Foundry IDPs, thus controlling restriction is not in the realm of HCP for cloud scale. Bucket name restrictions should prevent problematic bucket names from being created. For security, HCP for cloud scale passes the user display name through a uri encoder before returning it in XML responses. |
| Object tagging | Amazon wraps eTags in double-quotes. For XML listings (v1 object, v2 object, version) it escapes these, for example:
It's not necessary for HCP for cloud scale to perform this because double-quotes do not need to be escaped within content, only attributes. |
| Expiration Date URL Encoding (x-amz-expiration header) The RuleID portion of the x-amz-expiration header is URL-encoded by HCP for cloud scale using the same encoding strategy that Amazon suggests for V4 authentication. This may result in encoded strings that do not exactly match how Amazon encodes RuleIDs in general. However, decoding them should always return the original string. | |
| GET Object ACL | ACL email address grantee types are not supported. In AWS each grantee is specified as a type=value pair, where the type is one of the following:
|
| PUT Object ACL | |
| DELETE Multiple Objects | Fully implemented |
| POST Object | Fully implemented |
| Initiate/Complete/Abort Multipart Upload | Fully implemented |
| Upload Part | Fully implemented |
| List Multipart Uploads | Fully implemented |
Unsupported S3 APIs
The following lists are the unsupported Amazon S3 API features.
Authentication API
- Authentication v2 (deprecated by AWS)
Bucket APIs
- GET/PUT/DELETE Bucket Website
- GET/PUT/DELETE Bucket Policy
- GET/PUT/DELETE Bucket Tagging
- GET/PUT/DELETE Bucket CORS (cross-origin resource sharing)
- GET Bucket Location
- PUT Bucket Versioning (versioning is always On)
- GET/PUT Bucket Logging
- GET Bucket Notification
- GET/PUT Bucket requestPayment
- GET/PUT/DELETE Bucket Inventory
- List Bucket Inventory Configurations
- GET/PUT/DELETE Bucket Replication
- GET/DELETE Bucket Metrics
- List Bucket Metrics Configurations
- GET/PUT/DELETE Bucket Analytics
- List Bucket Analytics Configurations
- PUT/GET Bucket Accelerate
- Server-Side Encryption with Customer-Provided Encryption Keys (SSE-C)
- Server-Side Encryption with Storage-Managed Encryption Keys (SSE-S3)
Object APIs
- PUT Object (Copy)
- Options Object
- GET/POST Object Torrent
- SELECT Object Content (SQL)
- Upload Part - Copy