Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Working with POST object uploads

With the Hitachi API for Amazon S3, you can accept objects uploaded directly to HCP from web browser forms (that is, a POST form request). This process is called a POST object upload.

This section of the Help starts with general information about and considerations for working with POST object uploads.

About POST object uploads

The POST request is an alternative to a PUT request. A POST object upload is the process of accepting objects from an HTTP form sent by a web browser. When a user submits the form, the browser uploads its contents, which can include data such as a text field or a file, to HCP. Any parameters are passed, not within the header as in a PUT request, but as form fields within the body of the request. The web page can be hosted on the same domain as HCP or a different domain. A user must have write access to a bucket to add an object to it. The POST request can upload a security policy with the request using form fields in the multipart/form-data encoded message body, so a bucket administrator can create preset conditions under which the upload is allowed.

Because the request is authenticated without passing data through a secure intermediary node that protects credentials, POST object uploads can reduce latency. Because an object is uploaded in a single operation, an HTTP success response indicates that the entire object has been stored.

POST object upload authentication

A POST object upload request can be authenticated in these ways:

  • Header authentication

    The header can include AWS Signature Version 2 authorization information. Header authentication supports only AWS local user account credentials. Use a header with a value in the format Authorization: AWS all_users: for anonymous access, or Authorization: AWS access-key:signature for authenticated access.

    NoteActive Directory, SPNEGO, cookie, and HCP authentication are not supported.
  • In-form authentication

    The POST form can contain authentication-related fields. In-form authentication supports only AWS local user account credentials. All authorization-related form fields must be presented for in-form authentication to succeed. If both V2 and V4 authentication form fields are presented, V2 authentication information is used.

  • Security policy validation

    A policy is included in the request. Required if in-form authentication is used. If the policy is missing, HCP returns a status of 400.

For a namespace that does not permit anonymous access (that is, a non-public bucket), either header or in-form authentication is required. (Both can be provided.)

Security policies

The POST security policy is part of the authentication of a POST object upload request. The policy consists of base-64 encoded JSON code which is submitted as a form field in the request.

If a form contains fields that don't meet the policy's requirements, the request is rejected with the status 403.

Example

Here's an example of a POST policy using AWS Signature Version 4 conditions:

{ "expiration": "2020-11-02T123:01:00.000Z",
    "conditions": [
        {"bucket": "Finance"},
        ["starts-with", "$key", "user/user1/"],
        {"acl": "public-read"},
        {"success_action_redirect": "http://www.company.com/success"},
        {"x-amz-meta-uuid": "14365123651274"},
        {"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
        {"x-amz-algorithm": "AWS4-HMAC-SHA256"},
        {"x-amz-date": "20201102T000000Z" }
    ]
}

A security policy always contains the elements expiration and conditions. The example policy uses two condition matching types, exact matching and starts-with matching. The following sections describe these elements.

Expiration

The element expiration specifies the expiration date and time of the security policy in ISO8601 GMT date format. For example, 2019-08-10T12:00:00.000Z specifies that the POST policy is not valid after midnight GMT on August 10, 2019.

For AWS Signature Version 4 authentication, if the field x-amz-date contains an expired date, the policy is considered expired.

Condition matching

The table below describes condition matching types that you can use to specify POST security policy conditions. You must specify at least one condition for each field that you specify. You can create more complex matching criteria by specifying multiple conditions for a given form field. If there are multiple conditions for a field, then all the conditions must match.

  • Exact Matches

    The form field value must match the value specified. This example indicates that the ACL must be set to public-read:

    {"acl": "public-read" }

    This example is an alternate way to indicate that the ACL must be set to public-read:

    [ "eq", "$acl", "public-read" ]
  • Starts With

    The value must start with the specified value. This example indicates that the object key must start with user/user1:

    ["starts-with", "$key", "user/user1/"]
  • Matching Any Content

    To configure the POST security policy to allow any content within a form field, use starts-with with an empty value (""). This example allows any value for success_action_redirect:

    ["starts-with", "$success_action_redirect", ""]
  • Specifying Ranges

    For form fields that accept a range, separate the upper and lower limit with a comma. This example allows a file size from 1 to 10 MB:

    ["content-length-range", 1048579, 10485760]
Conditions

The conditions in a POST policy are an array of objects, any of which you can use to validate the request. You can use these conditions to restrict what is allowed in the request. For example, the policy conditions in the preceding example require the following conditions:

  • The request must specify the bucket name Finance.
  • The object key name must have the prefix user/user1.
  • The object ACL must be set to public-read.

Each field that you specify in a form (except x-amz-signature, AccessKeyId, file, policy, and field names that have the prefix x-ignore-) must appear in the list of conditions.

Note

All variables within the form are expanded before validating the security policy. Therefore, all condition matching is executed against the expanded form fields. For example, suppose that you want to restrict your object key name to a specific prefix (user/user1). In this case, you set the key form field to user/user1/${filename}. Your POST policy should be [ "starts-with", "$key", "user/user1/" ] (do not enter [ "starts-with", "$key", "user/user1/${filename}" ]).

The list below describes supported conditions.

  • acl

    Specifies the ACL value that must be used in the submitted form.

  • bucket

    Specifies the acceptable bucket name.

  • content-length-range

    Specifies the minimum and maximum allowable size for the uploaded content.

    If the object is compressed and the value of content-encoding is gzip, the size limit is applied to the size of the uncompressed object.

  • key

    Specifies the acceptable key name or a prefix of the uploaded object.

  • success_action_ redirect

    redirect

    Specifies the URL to which the client is redirected upon successful upload.

  • success_action_status

    If you don't specify a value for success_action_redirect, this element specifies the status code returned to the client when the upload succeeds.

  • x-amz-algorithm

    (AWS Signature Version 4 authentication only field)

    Specifies the signing algorithm that must be used during signature calculation. The value is AWS4-HMAC-SHA256.

  • x-amz-credential

    (AWS Signature Version 4 authentication only field)

    Specifies the credentials that you used to calculate the signature. Required if you include a POST policy document with the request. Use this format:

    access_key_id/date/aws_region/aws_service/aws4_request

    For Amazon S3, the value of aws_service is s3.

    For example:

    AKIAIOSFODNN7EXAMPLE/20191024/us-east-1/s3/aws4_request
  • x-amz-date

    (AWS Signature Version 4 authentication only field)

    Specifies a date value in ISO8601 format. Required if you include a POST policy document with the request.

    For example:

    20191024T000000Z
  • x-amz-meta-*

    User-specified metadata.

  • x-amz-*

    Other x-amz- headers.

  • x-hcp-retention

    Specifies the retention value for the object being stored. This value can be a fixed date, an offset, a retention class, or a special value.

  • x-hcp-retentionhold

    Specifies whether the object is on hold. This value can be either true or false.

  • x-ignore-*

    Users can specify additional form fields with this header that should be ignored.

Character escaping

The table below describes characters that must be escaped within a security policy.

Escape sequenceDescription
\\Backslash
\bBackspace
\fForm feed
\nNew line
\rCarriage return
\tHorizontal tab
\vVertical tab
\unnnnAll Unicode characters

Executing a POST object upload

When you execute a POST object upload, you specify a name for the object you're creating.

In the upload request, you can specify custom metadata to be added to the object when the upload is completed. To do this, you use the field x-amz-meta- within the post form.

In the upload request, you can specify an ACL to be added to the object when the upload is completed. To do this, you use the field acl within the post form.

If the ACL you specify in a request to execute an upload is invalid, HCP returns the status code 400 (Bad Request) or 501 (Not Implemented) and does not execute the upload.

When you complete an upload, you become the owner of the resulting object.

Required headers

The list below describes the headers you can use in a request to execute a POST object upload.

  • Authorization

    Specifies user credentials or requests anonymous access.

  • Host

    Specifies the hostname for the request. The host name identifies either a tenant or a bucket.

    For a tenant, use this format:

    tenant-name.hcp-domain-name

    For a bucket, use this format:

    bucket-name.tenant-name.hcp-domain-name
Optional headers
  • x-amz-server-side-encryption

    Requests that the response headers include x-amz-server-side-encryption, which indicates whether objects stored in HCP are encrypted. The value of the x-amz-server-side-encryption request header can be any character string.

    Data is only encrypted when Encryption for Primary Running Storage is configured.

Required form fields

The content of a POST object upload request contains a form with fields that are passed to HCP as the contents of a Content-Disposition header. The list below describes the fields you can use in a POST object upload.

  • AWSAccessKeyId

    (AWS Signature Version 2 authentication only field)

    Required if policy is sent with request

    Specifies the owner of the bucket who grants an anonymous user access for a request that satisfies the constraints in the policy.

    A Base64-encoded username for a user account.

  • Signature

    (AWS Signature Version 2 authentication only field)

    Required if policy is sent with request

    Specifies a value calculated using the secret key and the policy string, as "string to sign."

  • file

    Contains file or text content. Can be used with a File or a Textarea form element.

    HCP ignores any fields that appear after this field.

    You can only upload one file at a time.

  • key

    Specifies the name of the uploaded key.

    To use the file name provided by the user, use the variable ${filename}. For example, if the user Greta uploads the file new_hq.jpg and you specify /user/greta/${filename}, the key name is /user/greta/new_hq.jpg.

  • policy

    Required if bucket is not publicly writable

    Specifies a security policy describing what is permitted in the request. Requests without a security policy are considered anonymous and work only on publicly writable buckets.

  • x-amz-algorithm

    Required if policy is sent with request

    (AWS Signature Version 4 authentication only field)

    Specifies the signing algorithm that must be used during signature calculation. The value is AWS4-HMAC-SHA256.

  • x-amz-date

    Required if policy is sent with request

    (AWS Signature Version 4 authentication only field)

    Specifies the date and time at which the request is being made according to the requester. Normally, this is the current date and time.

  • x-amz-signature

    Required if policy is sent with request

    (AWS Signature Version 4 authentication only field)

    Specifies a value calculated using the secret key and specific elements of the request, including the policy string, as "string to sign."

  • x-amz-credential

    Required if using V4 authentication and policy is sent with request

    (AWS Signature Version 4 authentication only field)

    Specifies the credentials that you used to calculate the signature. It provides access key ID and scope information identifying the region and service for which the signature is valid. This should be the same scope you used to calculate the signing key.

    Use this format:

    access-key-id/date/aws-region/aws-service/aws4_request

    For example:

    AKIAIOSFODNN7EXAMPLE/20191024/us-east-1/s3/aws4_request

    For Amazon S3, the aws-service string is s3.

Optional form fields
  • acl

    Adds a canned ACL to the resulting object when the upload is completed. The default is private.

  • Content-Encoding

    REST-specific heades. You can use the values in a policy; they are otherwise treated as follows:

    If value is gzip and file is gzipped, HCP decodes the file.

  • Expires

    REST-specific headers. You can use the values in a policy; they are otherwise treated as follows:

    If present, HCP returns the value in the response header.

  • success_action_ redirect

    redirect

    Specifies a URL to which the client is redirected upon successful upload.

    If not specified, HCP returns the empty document type specified in the field success_action_status.

    If HCP cannot interpret the URL, it acts as if the field is not present.

    If the upload fails, HCP displays an error and does not redirect the user to a URL.

    NoteThe field redirect is deprecated.
  • success_action_status

    If you don't specify success_action_redirect, this status code is returned to the client when the upload succeeds.

    Accepts the values 200, 201, or 204 (the default).

    If set to 200 or 204, HCP returns an empty document with a 200 or 204 status code.

    If set to 201, HCP returns an XML document with a 201 status code.

    If not set or set to an invalid value, HCP returns an empty document with a 204 status code.

    NoteSome versions of Adobe Flash Player do not properly handle HTTP responses with an empty body. To support uploads through Adobe Flash, set to 201.
  • x-amz-meta-*

    Adds customer metadata, stored as key-value pairs, to the resulting object when the upload is completed.

  • x-hcp-retention

    Specifies the retention value for the object being stored. This value can be a fixed date, an offset, a retention class, or a special value

  • x-hcp-retentionhold

    Specifies whether the object is on hold. This value can be either true or false.

Response headers

The list below describes the headers returned in response to a successful POST object upload request.

  • Date

    The date and time when HCP responded to the request, in Greenwich Mean Time (GMT). The date and time are returned in this format:

    DDD dd MMM yyyy HH:mm:ss GMT

    For example:

    Thu, 23 Mar 2017 14:27:05 GMT
  • Location

    Specifies the URL to which the client is redirected if the upload is successful.

  • x-amz-server-side-encryption

    Specifies whether objects stored in HCP are encrypted. Possible values are:

    • If objects are encrypted, AES256
    • If objects are not encrypted, None

    This header is returned only if the request headers include x-amz-server-side-encryption.

  • x-amz-version-id

    Specifies the version ID of the object. This header is returned only while versioning is enabled for the bucket.

Response body

If success_action_code is set to 201, HCP returns information about the object that results from a successful upload request in an XML response body.

The list below describes the XML elements in the response body returned in response to a request to complete a POST object upload. The elements are listed in alphabetical order.

  • Bucket

    Child of the PostResponse element.

    The Bucket element specifies the bucket where the object was created.

  • ETag

    Child of the PostResponse element.

    The ETag or entity tag element specifies the MD5 hash of the object. The element reflects the contents of an object, not its metadata.

    You can do a conditional GET operation by combining the ETag tag with an If-Modified tag.

  • Key

    Child of the PostResponse element.

    The Key element specifies the name of the object.

  • Location

    Child of the PostResponse element.

    The Location element specifies the complete URL for the object.

Return codes

The table below describes HTTP status codes that can be returned in response to a request to complete a POST object upload.

CodeMeaningDescription
200OKHCP successfully processed the request.
201OKHCP successfully processed the request. Returns an XML document with a 201 status code. Recommended for uploads through Adobe Flash.
204OKHCP successfully processed the request. (Default success response.)
400Bad Request

Possible reasons include:

  • The request does not include a Content-Type header.
  • The security policy is missing or malformed.
403Forbidden

Possible reasons include:

  • The request specifies anonymous access. Only an authenticated user can complete an upload.
  • The credentials provided with the request are invalid.
  • The credentials provided with the request do not match the credentials used to initiate the upload.
  • You do not have permission to complete uploads in the specified bucket.
  • The S3 compatible API is currently disabled for the specified bucket.
  • The request uses HCP, AD, SPENEGO, or cookie authentication.
500Internal Server Error

An internal error occurred.

If this error persists, contact your tenant administrator.

503Service Unavailable

HCP is temporarily unable to handle the request, probably due to system overload, maintenance, or upgrade. Try the request again, gradually increasing the delay between each successive attempt.

If this error persists, contact your tenant administrator.

Example: POST object upload

Here’s a sample POST object upload request that initiates an object upload for an object named acctg/Checklist.txt in the bucket named finance with AWS Signature Version 2 in-form authentication.

Request headers and form

<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    </head>
    <body>

    <form action="https://finance.europe.hcp.example.com" method="post" enctype="multipart/form-data">
        <input type="input" name="key" value="acctg/${filename}" /><br />
        <input type="hidden" name="AWSAccessKeyId" value="bGdyZWVu" />
         <input type="hidden" name="Signature" value="yAk4cp0LQzNo9HMxXsiM9NNSjVs=" />
        <input type="hidden" name="Policy" value='eyJleHBpcmF0aW9uIjoiMjAxOS0xMi0zMFQxMjowMDowMC4wMDBaLdwiY29uZGl0aW9ucyI6W3siYnVja2V0IjogIm5zIn0sWyJzdGFydHMtd2l0aCIsICIka2V5IiwgImRpci9kaXIxLyJdLFsic3RhcnRzLXdpdGgiLCAiJHN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyIsICIiXSx7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LHsieC1hbXotbWV0YS1jbGFzcyI6ICJidXNpbmVzcyJ9LFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLCAwLCAxMDQ4NTc2MF1dfQ==
/>
        <input type="hidden" name="success_action_status" value="201"/>
        <input type="hidden" name="acl" value="public-read"/>
        <input type="hidden" name="x-amz-meta-class" value="business"/>
        <input type="file" name="file" /> <br />

         <!-- The elements after this will be ignored -->
         <input type="submit" name="submit" value="Upload to HCP" />
    </form>

</html>

Security policy

{ "expiration": "2019-12-31T12:00:00.000Z",
      "conditions": [
        {"bucket": "finance"},
        ["starts-with", "$key", "acctg/"],
        {"acl": "public-read"},
        {"x-amz-meta-class": "business"},
        ["content-length-range", 0, 10485760]
    ]
}

Response headers

Cache-Control: no-cache,no-store,must-revalidate
Content-Encoding: gzip
Content-Security-Policy: default-src 'self'; script-src…elf'; frame-ancestors 'self';
Content-Type: application/xml;charset=utf-8
Date: Thu, 11 Apr 2019 20:10:25 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
Transfer-Encoding: chunked
Vary: Origin, Access-Control-Request…Access-Control-Request-Method
Vary: Accept-Encoding, User-Agent
x-amz-version-id: 99520854223169
X-Content-Type-Options: nosniff
X-DNS-Prefetch-Control: off
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block

Response body

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <PostResponse>
    <Location>https://finance.europe.hcp.example.com/hs3/acctg%2FChecklist.txt</Location>
    <Bucket>finance</Bucket>
     <Key>acctg/Checklist.txt</Key>
    <ETag>"fc7abcc28fdf00ef400d7240546de4b9"</ETag>
</PostResponse>

 

  • Was this article helpful?