Hitachi Content Intelligence v2.2.0 Release Notes
About this document
This document provides information about Hitachi Content Intelligence (HCI) software release v2.2.0.8, including new features, important enhancements, product requirements, as well as fixed and known issues.
Intended audience
This document is intended for system administrators, Hitachi Vantara representatives, and authorized service providers who configure and operate the HCI software.
About this release
This release of Hitachi Content Intelligence (HCI) includes new features, such as Solr aliases and HCP for Cloud Scale Bucket Indexing, as well as password policy enhancements and multiple bug, security, and CVE fixes.
Major features
General updates
To set additional safeguards to your local account(s) (both admin and secondary admin users), new password settings can be implemented to enhance their security.
Found under the Configuration > Security > Password Settings tab in the Admin App, administrators can now apply any or all of the following policies:- Password Expiration (Days): The number of days until a password needs to be reset.
- Minimum Password Age (Days): The number of days until a password can be changed again. Set this value to 0 to ignore it.
- Password History: The number of unique new passwords that must be associated with an account before an old password can be resused. Set this value to 0 to ignore it.
- Allowed Login Attempts: The number of failed login attempts before a user is locked out of the system. Set this value to 0 to ignore it.
- Lockout Duration Mode: The amount of time (in minutes) that a user is locked out of the system after reaching the maximum Allowed Login Attempts.
For users with multiple Solr index collections, Solr aliases can help provide federated, sorted search results to make all of the accompanying documents more manageable to search through. The aliases you create will appear as a selectable index value in the Search App.
The results can then be queried, refined, and sorted based upon the settings you implement for the alias in Workflow Designer, and bulk actions can also be applied.
The new Solr aliases features appears as a new panel within the Workflow Designer App.
- Aliases cannot be used to execute actions in a pipeline or as workflow outputs.
- Aliases cannot be backed up or restored.
- Aliases will only function with internal Solr 8 indexes.
Due to an OpenSSL security issue regarding CVE-2022-1292, the Dashboard service has been officially removed from HCI.
This service was previously undeployed by default. If you currently have it deployed on your system, it needs to be scaled down prior to an upgrade to version 2.2, at which point it will then be removed from the product.
For more information about this CVE and the others fixed in this release, see Resolved CVEs.
HCP for Cloud Scale Bucket Indexing allows HCP for cloud scale (HCP-CS) users to index and search the objects contained within the buckets of their S3 Console software through HCI. The auto-generated workflow creates two data connectors and an associated pipeline to seamlessly process objects between your HCI and HCP-CS systems.
HCP for Cloud Scale Bucket Indexing is a premium feature that is disabled by default. To enable it, contact your Hitachi Vantara representative.
Resolved issues
|
Issue |
Area affected |
Description | Outcome |
| HCI-6762 | Update | A job stuck in the DELETE_READY state causes updates to fail at the Cancel all jobs step. | Jobs in the DELETE_READY state no longer impact update completion. |
| HCI-6778 | Workflow Designer | Historical metrics are not being being deleted correctly in accordance with the value set in their Days to keep logs config property. | Historical metrics are now deleted correctly. |
| HCI-6877 | Solr | Solr shards are left in the recovery_failed state after a terminated shard split operation occurs. | Solr shards left from an terminated shard split operation are now correctly deleted. |
| HCI-6880 | Solr | Solr shard splitting may leave multiple replicas of the same shard on the same node. | Duplicate Solr shard replicas are now balanced and moved to separate nodes. |
| HCI-6957 | UI | HTTP Strict Transport Security (HSTS) is not currently implemented for the HCI/HCM UI. HSTS instructs the browser to only use HTTPS. | As of release v2.2, the HCI/HCM user interface now enforces HSTS. |
| HCI-7016 | Solr | When changing a field type in the schema, a message prompt appears letting the user know that the change will only affect documents which are indexed moving forward. This is incorrect, as changes to the docValues field result with no additional documents being indexed at all. | A message now appears which correctly explains the potential impact of their changes. |
| HCI-7103 | Connectors | The MQE connector incorrectly references an object's filename in the HCI_path and HCI_relativePath fields if it includes a plus sign. | The HCI_path and HCI_relativePath fields now list the correct values. |
| HCI-7149 | Content Monitor | HCM generates 404 errors on HCP MAPI, which causes them to incorrectly appear in visualizations. | The 404 error messages no longer appear in visualizations. |
| HCI-7229 | Admin App | If a Solr index is in an unhealthy state, where no replicas of any particular shard are showing as a leader, a change (specifically a reduction) to the IPL could potentially cause loss of data. | An error message now appears alerting users of their unhealthy state and an IPL change is prevented. |
| HCI-7309 | Workflow Designer | The Text and Metadata Extraction stage fails to process certain EML files based on their encoding. | EML files are now detected and processed correctly. |
| HCI-7725 | Update | Updates from HCI 1.10.0 or earlier do not remove the Hadoop Distributed File System (HDFS) plugin. | The HDFS plugin is now correctly removed upon updating from 1.10.0 or earlier. |
| HCI-7881 | Security | As a result of vulnerabilities stemming from CVE-2022-22965, depencies of Spring Framework v4.3.6 need to be removed from the Analytics service. | The Analytics service no longer references Spring Framework v4.3.6. |
| HCI-7975 | Security | HCI uses SSL v1.0.2k and v1.1.1 and as a result, is vulnerable to exploits found in CVE-2022-1292 and CVE-2022-0778. | SSL has been updated to v1.1.1o and the associated vulnerabilities no longer exist. |
| HCI-8020 | Security | Apache Log4j 1 no longer provides security updates due to its end-of-life status. To better secure HCI, Log4j 1 needs to be updated to Log4j 2 in both the Apache Kafka and Spark services. | Log4j 1 has been updated to Log4j v2.17.2 in both the Spark and Kafka services. |
| HCI-8094 | Security | The HCI code still references vulnerable Solr 8 versions, even though their underyling functionality was removed from the product. | Solr 8 has been updated to Log4j v2.17.2 and references to previously vulnerable versions of Solr no longer appear. |
| HCI-8105 | Security | The HCI code still references vulnerable Solr 6 versions, even though their underyling functionality was removed from the product. | Solr 6 has been updated to Log4j v2.17.2 and references to the vulnerable versions no longer appear. |
| HCI-8107 | Security | In order to maintain consistency around our security, the Apache Elastic and Logstash services should be updated alongside Spark and Kafka (HCI-8020). | The Elastic and Logstash services have been updated to Log4j v2.17.2. |
| HCI-8109 | Security | The Solr 8 JAR files need to be updated to maintain consistency with HCI-8904. | Solr 8 has been updated to Log4j v2.17.2 and references to previously vulnerable versions of Solr no longer appear. |
| HCI-8240 | Import | Importing a package that contains data connectors as dependencies fails. | Packages containing data connectors as dependencies are now imported correctly. |
Known issues
|
Issue |
Area affected |
Description | Outcome/Workaround |
| HCI-286 | Workflow Designer App | Using Retry Failed Documents manually in a workflow that has failures doesn't add to the output metrics after it has successfully indexed those files. | The results of the Retry Failed Documents setting do not affect output metrics and the files are still indexed successfully. |
| HCI-353 | Job Driver | When attempting to migrate data to a namespace that has already hit its hard quota, 413 errors are received without any failure notifications in the workflow. The failures are instead reported in the advanced historic metrics. | N/A |
| HCI-1047 | Content Monitor App | Replication links with forward slashes (/) do not appear in the Replication metrics. | Replace the forward slashes in the Replication link name with any other ASCII character (space, %, etc.). |
| HCI-1737 | Workflow Designer | Enabling Process all documents ignores the HCP and Sharepoint connector's directories when a workflow is resumed. | N/A |
| HCI-1918 | Metrics | Metrics service cannot be scaled to twice the current number of instances. |
|
| HCI-6103 | Workflow Designer App |
On a multi-node cluster, if one of the nodes goes down, the workflow halts with a task error. The task error states that the driver heap limit is too low for the workflow, but raising this value in the workflow settings does not restart the workflow. |
The workflow-agent job type is configured to run on all nodes in a cluster by default. Scaling the workflow-agent off of the bad node will allow workflows to resume. |
| HCI-6109 | Admin App | HCI enters a bad state after rebooting 2 of the 3 master nodes available on a 4-node cluster. | Reboot all HCI master nodes at the same time and the clusters will return normally. |
| HCI-6128 | Metrics | Metrics from historical logs are ignored after an update and present an OOM message. |
If you notice your imported logs missing after update, reimport them and the metrics will display correctly. |
| HCI-6304 | Workflow Designer App | Testing a data connection accessible by proxy shows a blank certificate and cancelling it makes the system seem unresponsive. |
If your data connection can only be accessed by proxy and requires a certificate, download the certificate from the other system and manually add it to the Admin App To do this, navigate to Configuration > Certificates > Client and click UPLOAD CLIENT CERTIFICATE. |
| HCI-6540 | Solr | During an update, Solr shards appear in the Gone state if Marathon references the old port number and node name pairing when starting the Solr service. |
Delete the Gone shards:
|
| HCI-6857 | Search App | Search App exclusive users with bulk action permissions are able to see Workflow Designer as an option in their SSO menu. Clicking it presents a message telling the user the page cannot be displayed. | N/A |
| HCI-7020 | Admin App | When updating from HCI 1.6.x to 1.10.1, the doc folder is owned by root on several nodes in the cluster, causing an update failure. | Changing the user from root to hci and retrying the failure resolves the issue. |
| HCI-7341 | Solr | When attempting to recreate a delete Solr collection with a different initial schema, the old schema is still present in the UI. This is an internal HCI ticket to track the progress of SOLR-15674, which was filed directly with Apache. | N/A |
| HCI-7366 | Search App | The autocomplete functionality of the search bar in Search App does not work for file names in Chinese after reindexing text field changes. | N/A |
| HCI-7368 | Solr | If an index is created without using a SolrCloud connection URL, attempts to create bulk actions using the index will fail. | N/A |
| HCI-7369 | Solr | When attempting to remove a copy field from an index, the delete fails with a "Collection not found" error and the index is unable to be written to. | N/A |
| HCI-7370 | Solr | Solr can only use comparatives with small numbers and returns 0 results (in error) when using larger ones. | N/A |
| HCI-8242 | Import | Importing a Solr alias bundle containing an index associated with HCP for Cloud Scale Bucket Indexing results in an error. |
|
| HCI-8243 | Import | After importing specific components from a bundle, an error message is received when attempting to add additional components. | Import the bundle in its entirety. If the error persists, contact your Hitachi Vantara representative. |
Resolved CVEs
This table lists the high and critical Common Vulnerabilities and Exposures (CVEs) that no longer affect HCI as of v2.2. For more information, refer to https://nvd.nist.gov.
|
Issue |
Area affected |
Description |
| CVE-2022-1292 | OpenSSL |
OpenSSL vulnerability The c_rehash script does not properly sanitise shell metacharacters to prevent command injection. This script is distributed by some operating systems in a manner where it is automatically executed. On such operating systems, an attacker could execute arbitrary commands with the privileges of the script. Use of the c_rehash script is considered obsolete and should be replaced by the OpenSSL rehash command line tool. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2022-1292. Resolution HCI has upgraded their OpenSSL version to 1.1.1o, which addresses this issue. Additionally, the Dashboard service was removed from the product. |
| CVE-2022-0778 | OpenSSL |
OpenSSL vulnerability The BN_mod_sqrt() function, which is used when parsing certificates, can be used to trigger an infinite loop by crafting a certificate that has invalid explicit curve parameters. Since certificate parsing happens prior to verification of the certificate signature, any process that parses an externally supplied certificate may thus be subject to a denial of service attack. In particular, the attacker can use a self-signed certificate to trigger the loop during verification of the certificate signature. This issue affects OpenSSL versions 1.0.2, 1.1.1 and 3.0. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2022-0778. Resolution HCI has upgraded their OpenSSL version to v1.1.1o, which addresses this issue. |
| CVE-2019-17571 | Log4j |
Log4j vulnerability Included in Log4j 1.2 is a SocketServer class that is vulnerable to deserialization of untrusted data, which can be exploited to remotely execute arbitrary code when combined with a deserialization gadget when listening to untrusted network traffic for log data. This affects Log4j versions 1.2 up to 1.2.17. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2019-17571. Resolution HCI has upgraded their Solr 6 and Solr 8 Log4j versions to 2.17.2, which addresses this issue. |
| CVE-2020-9493 | Log4j |
Log4j vulnerability CVE-2020-9493 identified a deserialization issue that was present in Apache Chainsaw. Prior to Chainsaw version 2.0, Chainsaw was a component of Apache Log4j 1.2.x, where the same issue exists. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2020-9493. Resolution HCI has upgraded their Solr 6 and Solr 8 Log4j versions to 2.17.2, which addresses this issue. |
| CVE-2022-22965 | Java | Java Development Kit (JDK) vulnerability A Spring MVC or Spring WebFlux application running on JDK 9+ may be vulnerable to remote code execution (RCE) via data binding.For more information on this CVE, see https://nvd.nist.gov/vuln/detail/cve-2022-22965. Resolution HCI has removed Spring Framework version 4.3.6 from its Analytics service. |
| CVE-2002-23302 | Log4j |
Log4j vulnerability JMSSink in all versions of Log4j 1.x is vulnerable to deserialization of untrusted data when the attacker has write access to the Log4j configuration, or if the configuration references an LDAP service the attacker has access to. The attacker can provide a TopicConnectionFactoryBindingName configuration causing JMSSink to perform JNDI requests that result in remote code execution in a similar fashion to CVE-2021-4104. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2022-23302. Resolution HCI has upgraded their Solr 6 and Solr 8 Log4j versions to 2.17.2, which addresses this issue. |
| CVE-2002-23305 | Log4j |
Log4j vulnerability By design, the JDBCAppender in Log4j 1.2.x accepts an SQL statement as a configuration parameter where the values to be inserted are converters from PatternLayout. The message converter, %m, is likely to always be included. This allows attackers to manipulate the SQL by entering crafted strings into input fields or headers of an application that are logged allowing unintended SQL queries to be executed. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2022-23305. Resolution HCI has upgraded their Solr 6 and Solr 8 Log4j versions to 2.17.2, which addresses this issue. |
| CVE-2002-23307 | Log4j |
Log4j vulnerability CVE-2020-9493 identified a deserialization issue that was present in Apache Chainsaw. Prior to Chainsaw v2.0, Chainsaw was a component of Apache Log4j 1.2.x, where the same issue exists. For more information on this CVE, see https://nvd.nist.gov/vuln/detail/CVE-2022-23307. Resolution HCI has upgraded their Solr 6 and Solr 8 Log4j versions to 2.17.2, which addresses this issue. |
System requirements
This section lists the hardware, networking, and operating system requirements for running an HCI system with one or more instances.
Hardware requirements
This table shows the minimum and recommended hardware requirements for each instance in an HCI running Hitachi Content Search.
|
Resource |
Minimum |
Recommended |
|
RAM |
16 GB |
32 GB |
|
CPU |
4-core |
8-core |
|
Available disk space |
50 GB |
500 GB |
Software requirements
The following table shows the minimum requirements and best-practice software configurations for each instance in an HCI system.
| Resource | Minimum | Best |
| IP addresses | (1) static | (2) static |
| Firewall Port Access | Port 443 for SSL traffic Port 8000 for System Management App GUI Port 8888 for Content Search App GUI | Same |
| Network Time | IP address of time service (NTP) | Same |
Operating system and Docker minimum requirements
Each server or virtual machine you provide must have the following:
- A 64-bit Linux distribution
- Docker version 1.13.1 or later installed
- IP and DNS addresses configured
Additionally, you should install all relevant patches on the operating system and perform appropriate security hardening tasks.
- Install the current Docker version suggested by your operating system, unless that version is earlier than 1.13.1. The system cannot run with Docker versions prior to 1.13.1.
- HCI will not function on an operating system that uses cgroups v2. If your system currently utilizes it, you must downgrade to cgroups v1 prior to installation.
Operating system and Docker qualified versions
This table shows the operating systems, as well as the Docker and SELinux configurations, on which this HCI release was qualified. It acts a point of reference for our customers to better share how we operate within our internal environment and does not represent any requirements that need to be followed within your own.
| Operating system | Docker version | Docker storage configuration | SELinux setting |
| CentOS 7.6 | Docker 18.03.1-ce | device-mapper | Enforcing |
| CentOS 8.1 | Docker 19.03.13 | overlay2 | Enforcing |
| Red Hat Enterprise Linux 8.1 | Docker 20.10.14 | overlay2 | Enforcing |
| Ubuntu 18.04.4 LTS | Docker 18.03.1-ce | overlay2 | Not Installed |
Docker considerations
The Docker installation folder on each instance must have at least 20 GB available for storing the HCI Docker images.
Make sure that the Docker storage driver is configured correctly on each instance before installing HCI. To view the current Docker storage driver on an instance, run docker info.
If you are using the Docker devicemapper storage driver:
- Make sure that there's at least 40 GB of Docker metadata storage space available on each instance. HCI needs 20 GB to install successfully and an additional 20 GB to successfully update to a later version. To view Docker metadata storage usage on an instance, run docker info.
- On a production system, do not run
devicemapperinloop-lvmmode. This can cause slow performance or, on certain Linux distributions, HCI might not have enough space to run.
SELinux considerations
You should decide whether you want to run SELinux on system instances and enable or disable it before installing HCI. To enable or disable SELinux on an instance, you must restart the instance. To view whether SELinux is enabled on an instance, run: sestatus
To enable SELinux on the system instances, use a Docker storage driver that supports it. The storage drivers that SELinux supports differ depending on the Linux distribution you're using. For more information, see the Docker documentation.
Time source requirements
If you are installing a multi-instance system, each instance should run NTP (network time protocol) and use the same external time source. For information, see support.ntp.org.
Supported browsers
The following browsers are qualified for use with HCI software. Other browsers or versions might also work.
- Google Chrome (latest version as of the date of this publication)
- Microsoft Edge (latest version as of the date of this publication)
- Mozilla Firefox (latest version as of the date of this publication)
Documentation set
Along with your release notes, the following guides and documentation comprise the full set of HCI reference documentation:
- Hitachi Content Intelligence Installing Hitachi Content Intelligence
- Hitachi Content Intelligence Deploying the HCI Example OVF
- Hitachi Content Intelligence (HCI) Getting Started Guide
- Hitachi Content Intelligence Workflow Designer Help
- Hitachi Content Intelligence Search Help
- Hitachi Content Intelligence Content Monitor Help
- Hitachi Content Intelligence Administrator Help
To learn more, visit the HCI Knowledge page.