Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Working with POST object uploads


With the Hitachi API for Amazon S3, you can accept objects uploaded directly to HCP from web browser forms (that is, a POST form request). This process is called a POST object upload.

This section of the Help starts with general information about and considerations for working with POST object uploads. Then, for each POST object upload operation, this section:

Describes the operation

Shows the request line for the operation

Describes the request headers for the operation

Describes the response headers returned for a successful execution of the requested operation

Shows the format of the request or response body, where applicable

Explains the HTTP status codes that can be returned in response to requests for the operation

Presents one or more examples of requests for the operation

For general information about S3 compatible requests and HCP responses to those requests, see Requests and responses.

For information about the examples, see Examples in this help.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

About POST object uploads


The POST request is an alternative to a PUT request. A POST object upload is the process of accepting objects from an HTTP form sent by a web browser. When a user submits the form, the browser uploads its contents, which can include data such as a text field or a file, to HCP. Any parameters are passed, not within the header as in a PUT request, but as form fields within the body of the request. The web page can be hosted on the same domain as HCP or a different domain. A user must have write access to a bucket to add an object to it. The POST request can upload a security policy with the request using form fields in the multipart/form-data encoded message body, so a bucket administrator can create preset conditions under which the upload is allowed.

Because the request is authenticated without passing data through a secure intermediary node that protects credentials, POST object uploads can reduce latency. Because an object is uploaded in a single operation, an HTTP success response indicates that the entire object has been stored.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

POST object upload authentication


A POST object upload request can be authenticated in these ways:

Header authentication: The header can include AWS Signature Version 2 authorization information. Header authentication supports only AWS local user account credentials. Use a header with a value in the format Authorization: AWS all_users: for anonymous access, or Authorization: AWS access-key:signature for authenticated access.

NoteWebHelp.png

Note: Active Directory, SPNEGO, cookie, and HCP authentication are not supported.

In-form authentication: The POST form can contain authentication-related fields. In-form authentication supports only AWS local user account credentials. All authorization-related form fields must be presented for in-form authentication to succeed. If both V2 and V4 authentication form fields are presented, V2 authentication information is used.

Security policy validation: A policy is included in the request. Required if in-form authentication is used. If the policy is missing, HCP returns a status of 400.

For a namespace that does not permit anonymous access (that is, a non-public bucket), either header or in-form authentication is required. (Both can be provided.)

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Security policies


The POST security policy is part of the authentication of a POST object upload request. The policy consists of base-64 encoded JSON code which is submitted as a form field in the request.

If a form contains fields that don't meet the policy's requirements, the request is rejected with the status 403.

Example

Here's an example of a POST policy using AWS Signature Version 4 conditions:

{ "expiration": "2020-11-02T123:01:00.000Z",
    "conditions": [
        {"bucket": "Finance"},
        ["starts-with", "$key", "user/user1/"],
        {"acl": "public-read"},
        {"success_action_redirect": "http://www.company.com/success"},
        {"x-amz-meta-uuid": "14365123651274"},
        {"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
        {"x-amz-algorithm": "AWS4-HMAC-SHA256"},
        {"x-amz-date": "20201102T000000Z" }
    ]
}

A security policy always contains the elements expiration and conditions. The example policy uses two condition matching types, exact matching and starts-with matching. The following sections describe these elements.

Expiration

The element expiration specifies the expiration date and time of the security policy in ISO8601 GMT date format. For example, 2019-08-10T12:00:00.000Z specifies that the POST policy is not valid after midnight GMT on August 10, 2019.

NoteWebHelp.png

Note:  For AWS Signature Version 4 authentication, if the field x-amz-date contains an expired date, the policy is considered expired.

Condition matching

The table below describes condition matching types that you can use to specify POST security policy conditions. You must specify at least one condition for each field that you specify. You can create more complex matching criteria by specifying multiple conditions for a given form field. If there are multiple conditions for a field, then all the conditions must match.

Condition match type Description
Exact Matches

The form field value must match the value specified. This example indicates that the ACL must be set to public-read:

{"acl": "public-read" }

This example is an alternate way to indicate that the ACL must be set to public-read:

[ "eq", "$acl", "public-read" ]

Starts With

The value must start with the specified value. This example indicates that the object key must start with user/user1:

["starts-with", "$key", "user/user1/"]

Matching Any Content

To configure the POST security policy to allow any content within a form field, use starts-with with an empty value (""). This example allows any value for success_action_redirect:

["starts-with", "$success_action_redirect", ""]

Specifying Ranges

For form fields that accept a range, separate the upper and lower limit with a comma. This example allows a file size from 1 to 10 MB:

["content-length-range", 1048579, 10485760]

Conditions

The conditions in a POST policy are an array of objects, any of which you can use to validate the request. You can use these conditions to restrict what is allowed in the request. For example, the policy conditions in the preceding example require the following conditions:

The request must specify the bucket name Finance.

The object key name must have the prefix user/user1.

The object ACL must be set to public-read.

Each field that you specify in a form (except x-amz-signature, AccessKeyId, file, policy, and field names that have the prefix x-ignore-) must appear in the list of conditions.

NoteWebHelp.png

Note:  All variables within the form are expanded before validating the security policy. Therefore, all condition matching is executed against the expanded form fields. For example, suppose that you want to restrict your object key name to a specific prefix (user/user1). In this case, you set the key form field to user/user1/${filename}. Your POST policy should be [ "starts-with", "$key", "user/user1/" ] (do not enter [ "starts-with", "$key", "user/user1/${filename}" ]).

The table below describes supported conditions.

 
Element name Description
acl

Specifies the ACL value that must be used in the submitted form.

bucket Specifies the acceptable bucket name.
content-length-range

Specifies the minimum and maximum allowable size for the uploaded content.

Note: If the object is compressed and the value of content-encoding is gzip, the size limit is applied to the size of the uncompressed object.

key

Specifies the acceptable key name or a prefix of the uploaded object.

success_action_  redirect,
redirect

Specifies the URL to which the client is redirected upon successful upload.

success_action_status

If you don't specify a value for success_action_redirect, this element specifies the status code returned to the client when the upload succeeds.

x-amz-algorithm

(AWS Signature Version 4 authentication only field)

Specifies the signing algorithm that must be used during signature calculation. The value is AWS4-HMAC-SHA256.

x-amz-credential

(AWS Signature Version 4 authentication only field)

Specifies the credentials that you used to calculate the signature. Required if you include a POST policy document with the request. Use this format:

access_key_id/date/aws_region/aws_service/aws4_request

For Amazon S3, the value of aws_service is s3.

For example:

AKIAIOSFODNN7EXAMPLE/20191024/us-east-1/s3/aws4_request

x-amz-date

(AWS Signature Version 4 authentication only field)

Specifies a date value in ISO8601 format. Required if you include a POST policy document with the request.

 

For example:

20191024T000000Z

x-amz-meta-*

User-specified metadata.

x-amz-* Other x-amz- headers.
x-hcp-retention Specifies the retention value for the object being stored. This value can be a fixed date, an offset, a retention class, or a special value.
x-hcp-retentionhold Specifies whether the object is on hold. This value can be either true or false.
x-ignore-* Users can specify additional form fields with this header that should be ignored.

Character escaping

The table below describes characters that must be escaped within a security policy.

Escape sequence Description
\\

Backslash

\b Backspace
\f

Form feed

\n

New line

\r

Carriage return

\t

Horizontal tab

\v

Vertical tab

\unnnn

All Unicode characters

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Executing a POST object upload


When you execute a POST object upload, you specify a name for the object you're creating. For information about naming objects, see Object names.

In the upload request, you can specify custom metadata to be added to the object when the upload is completed. To do this, you use the field x-amz-meta- within the post form. For information about custom metadata, see Custom metadata.

In the upload request, you can specify an ACL to be added to the object when the upload is completed. To do this, you use the field acl within the post form. For information about ACLs, see Access control lists.

If the ACL you specify in a request to execute an upload is invalid, HCP returns the status code 400 (Bad Request) or 501 (Not Implemented) and does not execute the upload.

When you complete an upload, you become the owner of the resulting object. For information about object ownership, see Object owners.

Object encryption

The response headers returned in response to a successful upload request include the header x-amz-server-side-encryption, which indicates whether objects stored in HCP are encrypted. If stored objects are encrypted, the value of this header represents the encryption algorithm and key length HCP is using. If stored objects are not encrypted, the value of this header is None.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Request headers (POST object upload)


The table below describes the headers you can use in a request to execute a POST object upload.

Request header Description Required
Authorization See Authentication. Yes
Host See Common request headers. Yes
x-amz-server-side-encryption

Requests that the response headers include x-amz-server-side-encryption, which indicates whether objects stored in HCP are encrypted. The value of the x-amz-server-side-encryption request header can be any character string.

Data is only encrypted when Encryption for Primary Running Storage is configured.

No

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Form fields (POST object upload)


The content of a POST object upload request contains a form with fields that are passed to HCP as the contents of a Content-Disposition header. The table below describes the fields you can use in a POST object upload.

Form field Description Required

AWSAccessKeyId

(AWS Signature Version 2 authentication only field)

Specifies the owner of the bucket who grants an anonymous user access for a request that satisfies the constraints in the policy.

A Base64-encoded username for a user account.

For more information see Authentication.

Yes, if policy is sent with request

Signature

(AWS Signature Version 2 authentication only field)

Specifies a value calculated using the secret key and the policy string, as "string to sign."

For more information see Authentication.

Yes, if policy is sent with request
acl

Adds a canned ACL to the resulting object when the upload is completed. The default is private.

For valid values for this field, see Canned ACLs.

No
Content-Encoding,
Expires

REST-specific headers. You can use the values in a policy; they are otherwise treated as follows:

Content-Encoding: If value is gzip and file is gzipped, HCP decodes the file.

Expires: If present, HCP returns the value in the response header.

No
file

Contains file or text content. Can be used with a File or a Textarea form element.

HCP ignores any fields that appear after this field.

You can only upload one file at a time.

Yes
key

Specifies the name of the uploaded key.

To use the file name provided by the user, use the variable ${filename}. For example, if the user Greta uploads the file new_hq.jpg and you specify /user/greta/${filename}, the key name is /user/greta/new_hq.jpg.

Yes
policy

Specifies a security policy describing what is permitted in the request. Requests without a security policy are considered anonymous and work only on publicly writable buckets.

For more information see Security policies.

Yes, if bucket is not publicly writable
success_action_  redirect,
redirect

Specifies a URL to which the client is redirected upon successful upload.

If not specified, HCP returns the empty document type specified in the field success_action_status.

If HCP cannot interpret the URL, it acts as if the field is not present.

If the upload fails, HCP displays an error and does not redirect the user to a URL.

Note: The field redirect is deprecated.

No
success_action_status

If you don't specify success_action_redirect, this status code is returned to the client when the upload succeeds.

Accepts the values 200, 201, or 204(the default).

If set to 200 or 204, HCP returns an empty document with a 200 or 204 status code.

If set to 201, HCP returns an XML document with a 201 status code.

If not set or set to an invalid value, HCP returns an empty document with a 204 status code.

Note: Some versions of Adobe Flash Player do not properly handle HTTP responses with an empty body. To support uploads through Adobe Flash, set to 201.

No
x-amz-meta-*

Adds customer metadata, stored as key-value pairs, to the resulting object when the upload is completed.

No

x-amz-algorithm

(AWS Signature Version 4 authentication only field)

Specifies the signing algorithm that must be used during signature calculation. The value is AWS4-HMAC-SHA256. Yes, if policy is sent with request

x-amz-date

(AWS Signature Version 4 authentication only field)

For more information see Common request headers. Yes, if policy is sent with request

x-amz-signature

(AWS Signature Version 4 authentication only field)

Specifies a value calculated using the secret key and specific elements of the request, including the policy string, as "string to sign."

For more information see Authentication.

Yes, if policy is sent with request

x-amz-credential

(AWS Signature Version 4 authentication only field)

Specifies the credentials that you used to calculate the signature. It provides access key ID and scope information identifying the region and service for which the signature is valid. This should be the same scope you used to calculate the signing key.

Use this format:

access-key-id/date/aws-region/aws-service/aws4_request

For example:

AKIAIOSFODNN7EXAMPLE/20191024/us-east-1/s3/aws4_request

For Amazon S3, the aws-service string is s3.

Yes, if using V4 authentication and policy is sent with request
x-hcp-retention Specifies the retention value for the object being stored. This value can be a fixed date, an offset, a retention class, or a special value No
x-hcp-retentionhold Specifies whether the object is on hold. This value can be either true or false. No

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Response headers (POST object upload)


The table below describes the headers returned in response to a successful POST object upload request.

Response header Description
Date See Common response headers.
Location Specifies the URL to which the client is redirected if the upload is successful.
x-amz-server-side-
encryption

Specifies whether objects stored in HCP are encrypted. Possible values are:

If objects are encrypted, AES256

If objects are not encrypted, None

x-amz-version-id Specifies a version ID for the uploaded object. This header is returned only if bucket versioning is enabled.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Response body (POST object upload)


If success_action_code is set to 201, HCP returns information about the object that results from a successful upload request in an XML response body, in this format:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <PostResponse xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Location>url</Location>
    <Bucket>bucket</Bucket>
    <Key>key</Key>
    <ETag>"eTag"</ETag>
</PostResponse>

The table below describes the XML elements in the response body returned in response to a request to complete a POST object upload. The elements are listed in alphabetical order.

Element Description
Bucket

Child of the PostResponse element.

The Bucket element specifies the bucket where the object was created.

ETag

Child of the PostResponse element.

The ETag or entity tag element specifies the MD5 hash of the object. The element reflects the contents of an object, not its metadata.

You can do a conditional GET operation by combining the ETag tag with an If-Modified tag.

Key

Child of the PostResponse element.

The Key element specifies the name of the object.

Location

Child of the PostResponse element.

The Location element specifies the complete URL for the object.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Status codes (POST object upload)


The table below describes HTTP status codes that can be returned in response to a request to complete a POST object upload. For more information about HTTP status codes and the error codes that can accompany them, see Error codes.

Code Meaning Description
200 OK

HCP successfully processed the request.

201 OK HCP successfully processed the request. Returns an XML document with a 201 status code. Recommended for uploads through Adobe Flash.
204 OK HCP successfully processed the request. (Default success response.)
400 Bad Request

Possible reasons include:

The request does not include a Content-Type header.

The security policy is missing or malformed.

403 Forbidden

Possible reasons include:

The request specifies anonymous access. Only an authenticated user can complete an upload.

The credentials provided with the request are invalid.

The credentials provided with the request do not match the credentials used to initiate the upload.

You do not have permission to complete uploads in the specified bucket.

The S3 compatible API is currently disabled for the specified bucket.

The request uses HCP, AD, SPENEGO, or cookie authentication.

500 Internal Server Error

An internal error occurred.

If this error persists, contact your tenant administrator.

503 Service Unavailable

HCP is temporarily unable to handle the request, probably due to system overload, maintenance, or upgrade. Try the request again, gradually increasing the delay between each successive attempt.

If this error persists, contact your tenant administrator.

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

Example: POST object upload


Here’s a sample POST object upload request that initiates an object upload for an object named acctg/Checklist.txt in the bucket named finance with AWS Signature Version 2 in-form authentication.

Request headers and form

<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    </head>
    <body>

    <form action="https://finance.europe.hcp.example.com" method="post" enctype="multipart/form-data">
        <input type="input" name="key" value="acctg/${filename}" /><br />
        <input type="hidden" name="AWSAccessKeyId" value="bGdyZWVu" />
         <input type="hidden" name="Signature" value="yAk4cp0LQzNo9HMxXsiM9NNSjVs=" />
        <input type="hidden" name="Policy" value='eyJleHBpcmF0aW9uIjoiMjAxOS0xMi0zMFQxMjowMDowMC4wMDBaLdwiY29uZGl0aW9ucyI6W3siYnVja2V0IjogIm5zIn0sWyJzdGFydHMtd2l0aCIsICIka2V5IiwgImRpci9kaXIxLyJdLFsic3RhcnRzLXdpdGgiLCAiJHN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyIsICIiXSx7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LHsieC1hbXotbWV0YS1jbGFzcyI6ICJidXNpbmVzcyJ9LFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLCAwLCAxMDQ4NTc2MF1dfQ==
/>
        <input type="hidden" name="success_action_status" value="201"/>
        <input type="hidden" name="acl" value="public-read"/>
        <input type="hidden" name="x-amz-meta-class" value="business"/>
        <input type="file" name="file" /> <br />

         <!-- The elements after this will be ignored -->
         <input type="submit" name="submit" value="Upload to HCP" />
    </form>

</html>

Security policy

{ "expiration": "2019-12-31T12:00:00.000Z",
      "conditions": [
        {"bucket": "finance"},
        ["starts-with", "$key", "acctg/"],
        {"acl": "public-read"},
        {"x-amz-meta-class": "business"},
        ["content-length-range", 0, 10485760]
    ]
}

Response headers

Cache-Control: no-cache,no-store,must-revalidate
Content-Encoding: gzip
Content-Security-Policy: default-src 'self'; script-src…elf'; frame-ancestors 'self';
Content-Type: application/xml;charset=utf-8
Date: Thu, 11 Apr 2019 20:10:25 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
Transfer-Encoding: chunked
Vary: Origin, Access-Control-Request…Access-Control-Request-Method
Vary: Accept-Encoding, User-Agent
x-amz-version-id: 99520854223169
X-Content-Type-Options: nosniff
X-DNS-Prefetch-Control: off
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block

Response body

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <PostResponse>
    <Location>https://finance.europe.hcp.example.com/hs3/acctg%2FChecklist.txt</Location>
    <Bucket>finance</Bucket>
     <Key>acctg/Checklist.txt</Key>
    <ETag>"fc7abcc28fdf00ef400d7240546de4b9"</ETag>
</PostResponse>

© 2015, 2019 Hitachi Vantara Corporation. All rights reserved.

 

  • Was this article helpful?