Streaming Data Platform SDK v3.2.0-02 Release Notes
About this document
This document (RN-93HSDPSDK001-10, November 2018) provides late-breaking information about the Hitachi Streaming Data Platform software development kit (HSDP SDK) version 3.2.0-02. It includes information that was not available at the time the technical documentation for this product was published, as well as a list of known problems and solutions.
Intended audience
This document is intended for customers and Hitachi Vantara partners who license and use the Hitachi Streaming Data Platform software development kit.
Accessing product downloads
Product software, drivers, and firmware downloads are available on Hitachi Vantara Support Connect: https://support.hitachivantara.com/.
Log in and select Product Downloads to access the most current downloads, including important updates that may have been made after the release of the product.
Product package contents
See Available documents for the documentation provided for this release.
| Medium | DVD-ROM | Revision | Release Type |
| Software | Hitachi Streaming Data Platform Software Development Kit | 3.2.0-02 | Full Package |
New features and important enhancements
· HSDP now provides a mechanism by which the external adapter of the data input source is notified of any data clogging in stream data processing.
System requirements
Operating systems
| Operating Systems* | Version | Architecture |
| Red Hat Enterprise Linux Server | 6.5 or later | EM64T |
| AMD64 | ||
| 7.1 or later | EM64T | |
| AMD64 | ||
| SUSE Linux Enterprise Server | 11 SP2 | EM64T |
| AMD64 | ||
| 11 SP3 | EM64T | |
| AMD64 | ||
| 12 | EM64T | |
| AMD64 |
(*) Also runs under Kernel-based Virtual Machine (KVM) on the same OS and version.
Virtualization software
| Virtualization software | Version | Update | Architecture |
| VMware vSphere® ESXi™ | 5.1 | 1 | EM64T |
| AMD64 | |||
| 5.5 | 1 | EM64T | |
| AMD64 | |||
| 6.0 | None | EM64T | |
| AMD64 | |||
| 6.5 | None | EM64T | |
| AMD64 |
Prerequisite programs
Hitachi Streaming Data Platform v3.2.0-02 is required to run the SDK v3.2.0-02.
Memory and disk space requirements
Disk Space Requirements
The disk space requirement shown here applies when HSDP-SDK runs in the following configuration.
· Install directory (HSDP-SDK)
2,746 KB
Memory Requirements
The following is the minimum memory requirement for running each tool of HSDP-SDK in a sample query. (For information about sample queries, see "Example of related external adaptor" in Hitachi Streaming Data Platform Release Notes). Information about the memory required by HSDP when the tools are run can also be found in Hitachi Streaming Data Platform Release Notes.
CQL
Note that the memory requirements for HSDP-SDK might increase depending on the following factors.
· The number of registered query groups
· The number of streams in a query
· The data size of a tuple (This depends on the number of columns and the data type)
Resolved problems
| # | Corrected Problems | Applicable products | Applicable OS |
| #1 | Sending a large number of control tuples from an external adapter will affect the performance of stream data processing. | Hitachi Streaming Data Platform Software Development Kit | Red Hat Enterprise Linux Server SUSE Linux Enterprise Server |
| #2 | A BufferOverFlowException occurs at the putControl method of the external adaptor API if UTF-16 (or UTF-16BE or UTF-16LE) is specified as the character encoding and then data is transmitted by specifying the type BigDecimal as an argument of the control tuple. | Hitachi Streaming Data Platform Software Development Kit | Red Hat Enterprise Linux Server SUSE Linux Enterprise Server |
Known problems
Restriction on this release
(1) HSDP SDK operation is not guaranteed if an invalid property value is specified in the query group property file, the SDP manager definition file, and the external adaptor definition file, or if any of the HSDP files are named with invalid characters.
Installing HSDP SDK 3.2.0-02
Installation
(1) Log on to the installation-target machine as a root user.
(2) Mount the downloaded HSDP SDK disk image as a directory.
(3) Move to the directory where the HSDP SDK installer is placed:
# cd <CD-ROM-mount-directory>/HSDP-SDK/
(4) Run the following command:
# install.sh
When the installation is complete, the HSDP SDK is installed in the following directory: /opt/hitachi/hsdp/
Uninstallation
Procedure
(1) Log on to the target machine where you want to uninstall the product as a root user.
(2) Move to a directory other than /opt/hitachi/hsdp/sdk/.
(3) Run the following command:
# /opt/hitachi/hsdp/sdk/uninstall/uninstall.sh
Port numbers
HSDP uses 8 consecutive ports, from 20425 to 20432, by default. To use ports from p to p+7, where p is the initial port number, specify p by using the –port option of the hsdpsetup command. (hsdpsetup –port p ...).
Usage precautions
Notes on the virtualization platform environment
HSDP is designed to run on a standard or non-virtual machine. However, you may choose to deploy it on a virtual machine. Although the software has gone through rigorous testing, running in a virtualized environment can lead to performance issues that maybe a result of virtualizing hardware resources and sharing I/O. In addition, the functions that use CPU time might not work properly.
For this reason, a full operation evaluation and performance verification of the system is necessary. Perform checks after deployment, and confirm that HSDP uses appropriate hardware, virtualization software, guest OS, and applications.
Notes on designing the external adapter
Using the putControl method to send a large number of control tuples in a short time will affect the performance of stream data processing.
For this reason, a good practice is to design the external adapter with a system configuration such that the system sends individual control tuples at adequate intervals.
Documentation
Available documents
| Document name | Document number | Published date |
| Hitachi Streaming Data Platform Application Development Guide | MK-93HSDP001-06 | July, 2018 |
Documentation errata
The following table describes the amended content for the APIs for an external adapter.
| Interface / Class | Method | Changes |
| HSDPStreamOutput interface | · register(HSDPEventListener listener) | · Deleted the following sentence from "Explanation": "Even if the input stream cannot be connected within hsdp.tcp.connect.timeout seconds, the method finishes correctly." · Added the following sentence to the "Exceptions" table: "Failed to connect the output-stream". |
| HSDPAdaptorManager class | · openStreamInput(java.lang.String targetName) method | · Deleted the following sentence from "Explanation": "Even if the input stream cannot be connected within hsdp.tcp.connect.timeout seconds, the method finishes correctly." · Added the following sentence to the "Exceptions" table: "Failed to connect the output-stream". |
| HSDPAdaptorManager class | · openStreamOutput(java.lang.String targetName) method | · Changed a sentence in "Explanation" from "Opens the output stream" to "Searches the output stream". |
APIs for external adaptor
- register(HSDPEventListener listener) method
- Explanation
- Registers callback objects.
- Based on the result of the search using the openStreamOutput() method of the HSDPAdaptorManager class, establishes the connection with the output stream.
- Waits for the connection with the output stream for the maximum of hsdp.tcp.connect.timeout seconds. When a connection succeeds, the method finishes correctly.
- If you register multiple callback objects to one stream, the received data is notified to the respective callback objects.
- The same callback object cannot be registered for multiple times in one stream.
- If you register the same callback object to multiple streams, the callback object must be thread-safe, because the same callback objects are executed concurrently.
-
- Exceptions
| Exception | Generation Condition |
| HSDPAdaptorException | · Closed. · The specified callback has already been registered. · Failed to connect the output stream. |
- openStreamInput(java.lang.String targetName) method
- Explanation
- Opens the input stream.
- Searches for the input stream that corresponds with the specified destination name from the SDP broker, and establish the connection with the input stream based on the search result.
- Waits for the connection with the input stream for the maximum of hsdp.tcp.connect.timeout seconds. When a connection succeeds, the method finishes correctly.
- Exceptions
| Exception | Generation Condition |
| HSDPAdaptorException | · Stopped. · The format of the specified argument is incorrect. · The specified stream is not an input stream. · The number of trials to connect with the SDP broker has reached the maximum. · A method argument is specified with multiple streams, including identical streams. · The information does not match between the streams specified for an argument. · Failed to connect the input stream. |
- openStreamOutput(java.lang.String targetName) method
- Explanation
- Searches the output stream.
- Search for the output stream that corresponds with the specified destination name from the SDP broker.
- If the output stream is searched successfully, the method finishes correctly.