Hitachi Content Platform for Cloud Scale v1.1.0 Release Notes

Port 8000 for System Management App GUI

Port 9084 for MAPI and Object Storage Management App GUI

An object POST request made using the python library requests may fail and return a 400 HTTP code.

Specify bucket names using a virtual hosted style URL (for example, https://bucketname.hostname).

If an invalid PUT request is sent to the parent URL (that is, a hostname without the bucketname), the request is rejected, but with an HTTP code 501 ("Not Implemented") instead of 405 ("Method Not Allowed"), which varies from Amazon S3 behavior.

Resolution

This issue is resolved. The request now returns HTTP code 405.

Amazon S3 and management API requests are traced, and traces are stored in the background and indexed in a tracing database. Open indexes consume system resources, and closed indexes consume disk space.

Resolution

This issue is resolved. A scheduled task, Clean old jaeger indexes, runs each day to close open indexes after 30 days (the default) and delete indexes after 60 days (the default). You can configure these retention periods as service options in the Tracing Collector service:

• Max open Index age: How long to keep tracing indexes open in the database, in days. Enter a value from 1 to 365 days inclusive; the default is 30 days
• Max Index age: How long to keep tracing indexes in the database, in days. Enter a value from 1 to 365 days inclusive; the default is 60 days

Job configuration is not supported in this release.

Resolution

This issue is resolved.

Using an authenticated proxy when configuring a storage component fails. Attempting to activate the storage component fails and the storage component remains in the state INACTIVE.

Resolution

This issue is resolved. Proxy authentication is supported in this release.

In Firefox, the dashboard may not display the Active Objects graph.

Resolution

This issue is resolved. This is a known Firefox issue. In Firefox, do this:

1. Select Preferences.
2. Click Privacy & Security.
3. In the Certificates section, click View Certificates.
4. Click the Servers tab.
5. Click Add Exception.
6. Add the Prometheus host and port (by default, 9191).
7. Click Get Certificate and add the exception.
8. Select Permanently store this exception and click Confirm Security Exception.
9. Click OK.
10. Reload the Dashboard page.

Using the graphical user interface, there is no function to set a storage component to the state read-only.

Resolution

This issue is resolved. The function is supported in this release.

The Dashboard page sometimes does not properly display all the elements on it.

Resolution

This issue is resolved.

The MAPI endpoint /storage_component/update reverts the values of https, readOnly, usePathStyleAlways, and isProxy back to their default values if you don't set them in the request. If your existing values match the default values, you won't be affected. However if your existing values for those configurations are not the default values, they are overwritten.

Resolution

This issue is resolved.

When using the graphical user interface to reconfigure a storage component, if you try to set the values Region, Label, Proxy Host, or Proxy Port to a null string, the change is not retained.

Resolution

This issue is resolved.

If you change or add an SSL client certificate using the System Management application, you cannot use it to add a new storage component, and access to storage components that use the old certificate will fail when the certificate expires.

Workaround

Use the System Management MAPI endpoint certificates/refresh.

After changing the cluster hostname, the change is not automatically made externally visible.

Workaround

Restart the S3 Gateway:

1. In the System Management application, select Services.
2. Select the S3-Gateway service.
3. Click REPAIR.

Newly created buckets and objects are not immediately visible. Some client applications (such as Cloudberry Explorer) immediately retrieve the list of buckets to display the new bucket or object, which is not visible.

Workaround

If the new bucket or object is not immediately visible, refresh the list manually.

When manually deploying a four-node, multi-instance system, the Tracing Agent service returns an alert that the service is below the recommended instance count even when the correct number of service instances are deployed.

Workaround

If you have deployed the correct number of instances you can safely ignore this alert.

Amazon S3 POST redirection is not supported in this release.

The following services must be deployed with their default port values:

• Metadata Cache
• Tracing Agent
• Tracing Collector
• Tracing Query

A number of API endpoints refer to jobs. Jobs are not supported in this release.

The following scenario can cause data corruption:

1. A PUT operation is executed on an object or object part
2. The call specifies a Content-Length header that is larger than the actual size of the object
3. The call does not specify a Content-MD5 header

This scenario is not recommended.

Workaround

Check that the Etag received in the PUT response is as expected.

Authentication and authorization use a system management authorization client service which has a different timeout interval. If a management API authorization or authentication request times out but the underlying client service doesn't, the thread is blocked.

Workaround

Stop and restart the MAPI Gateway service container.

The MAPI schema includes public APIs, which do not require OAuth tokens.

Workaround

None needed. The public APIs do not require OAuth tokens.

If you're configuring a storage component with multiple retries and long timeouts and the endpoint for the storage component is unreachable, and if as a result you send multiple verification or activation requests, the management APIs can become unresponsive.

Workaround

If MAPI becomes unresponsive, use the System Management Services function Repair.

The storage component to which data is written is selected at random. If a filled storage component is selected, the write may fail.

Workaround

Use the MAPI endpoint PATCH /storage_component/update to set the filled storage component's readOnly flag to true.

Bucket Lifecycle Version 1 is not supported in this release.

A Metadata Gateway service instance becomes unavailable if the container or physical node hosting it fails. This can cause underprotection.

Workaround

If the System Management application indicates that a Metadata Gateway service is unavailable, remove the Metadata Gateway instance. This starts the service on another node instance.

If there no instances available to host a Metadata Gateway, the System Management application doesn't remove the instance and instead displays an error message. In this situation, use the System Management application to do this:

1. Add a new Metadata Gateway instance.
2. Remove the unavailable Metadata Gateway instance.

You can create a bucket with a name that includes capital letters. However, the bucket does not appear in listings, and you cannot put objects into the bucket.

Workaround

Don't name buckets with capital letters.

A snapshot provides the current state of the state machine from a leader node (service instance) to a follower service instance that is out of synch. If a leader node runs out of space to store snapshots and can't send out its latest snapshot, the follower node will be unable to resynchronize.

Workaround

Bring down the leader service instance, increase its storage space, and restart the service.

It is possible for installation not to finish because the metadata-cache service fails to deploy during installation.

Workaround

Reinstall the system.

The Active Objects graph is refreshed every ten seconds. During metadata balancing, metrics gauges can momentarily return inaccurate counts. Therefore, the Active Objects graph can show incorrect object counts at some times.

Workaround

Ignore short-term variations in the graph, which do not reflect the actual object count.

Searching the Object Storage Management online help for strings containing non-alphanumeric characters such as a hyphen can cause the browser to become unresponsive and the search to time out. (For example, the Chrome browser displays an error page labeled "Aw, Snap!").

Workaround

If a search times out, reload the page and enter the search again without hyphens or other nonalphanumeric characters.

The Refresh Token Timeout configuration value in the System Management application (Configuration > Security > Settings) has no effect.

The online Help available for the System Management application describes the generic application functions, not the functions pertaining to HCP for cloud scale.

Workaround

For information about the HCP for cloud scale System Management application, go to https://knowledge.hitachivantara.com/Documents/Storage/HCP_for_Cloud_Scale/1.1.0/System_Management.

Rarely, a system deployment, service management operation, or system update fails because a service fails to start. When this happens, the System Management application is inaccessible from the instance where the failure occurred.

The logs in the watchdog-service log directory contain this error:

Workaround

Restart the Docker service on the instance where the service failed to start. For example, if you are using systemd to run Docker, run:

After restarting Docker, retry the system deployment, service management operation, or system update.

ENS-7922

Occasionally, package installation may report failure with this error:

Also, uploading an update package may fail with this error: Error loading update package from data connection

However, your uploads may have actually succeeded.

Workaround

Try waiting for a while, refreshing the System Management application in your web browser, and then checking the relevant System Management application pages again.

ENS-6276

ENS-7924

Services may fail to deploy when both of the following are true:

• Your instances have large numbers of CPUs
• Your instances have systemd installed

{:timestamp=>"2017-09-14T17:13:43.824000+0000", :message=>"The error reported is: \n unable to create new native thread"}

And:

Failed to launch container: Failed to run 'docker -H unix:///var/run/docker.sock pull com.hds.ensemble/cassandra:25.0.0.1523': exited with status 2; stderr='runtime/cgo: pthread_create failed: Resource temporarily unavailable

Workaround

On each instance:

• If the instance is running a version of systemd prior to version 231, edit the setting DefaultTasksMax in the file /etc/systemd/system.conf to have a value of 4096:

  DefaultTasksMax=4096

• On the line OPTIONS in the file /etc/sysconfig/docker, edit the setting --default-ulimit to have a value of nofile=65536:1048576:

  OPTIONS="--default-ulimit nofile=65536:1048576"

Before starting an update, you are prompted to specify the network configuration for any new services included in the version that you're updating to. However, you can specify only the port numbers for the new service. You cannot specify the network type (that is, internal or external) for the service to use. Each new service gets the default network type, which is determined by the service itself.

If a system instance enters the Down state, when you attempt to upload an update package, the upload fails. However, after the system instance recovers, when you try again to upload an update package, the upload again fails, even though the system is in a healthy state.

Workaround

1. In the System Management application, go to the page Monitoring > Processes and for the task Upload Plugin Bundle click Retry Task.
2. Upload the update package again.

During installation, you can configure volumes for system services by specifying different values in the volume.config file on each system instance. Each volume is correctly configured with the settings you specify, but the page Monitoring > Services > Service Details in the System Management application incorrectly shows each volume as having identical configurations.

The Cassandra service can fail to deploy with the error Could not contact node over JMX. The log file on the node running the service instance includes the following entry: java.lang.RuntimeException: A node required to move the data consistently is down (/nnn.nnn.nnn.nnn). If you wish to move the data from a potentially inconsistent replica, restart the node with -Dcassandra.consistent.rangemovement=false

Workaround

1. Restart the Cassandra container running on that node.
2. Redeploy the service.

Services and jobs can be configured to use the Docker local volume driver for storing data on NFS servers. Such services and jobs may fail to deploy when the following are true:

• System instances are running a version of Docker later than 1.12.6
• SELinux is enabled

Workarounds

Do one of the following:

• Update Docker to Community Edition (ce) version 18.03 or later
• Disable SELinux

Validation of volume configuration values is not honored by the upgrade process. As a result, invalid configuration values are passed to Docker.

Workaround

Use caution when specifying volume values.

HCP for cloud scale uses a System Management function to validate tokens. The function does not time out. If the identity provider is unavailable, the requesting thread is blocked.