Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Logs and diagnostic information

Each service maintains its own set of logs. By default, log files are maintained in the folder install_path/hcpcs/log on each instance in the system. During installation, you can configure each service to store its logs in a different (that is, non-default) location.

Log levels

The following table lists the available log levels.

NoteRaising the log level (for example, from WARN to INFO) results in writing more data to the log file, but the file size increases more rapidly. Lowering the log level (for example, from WARN to ERROR) results in the file size increasing more slowly, but results in writing less data to the log file.
LevelLevels included
ALLFATAL, ERROR, WARN, INFO, DEBUG, TRACE
TRACEFATAL, ERROR, WARN, INFO, DEBUG, TRACE
DEBUGFATAL, ERROR, WARN, INFO, DEBUG
INFOFATAL, ERROR, WARN, INFO
WARNFATAL, ERROR, WARN (default)
ERRORFATAL, ERROR
FATALFATAL
OFFNone

Log management

You can manage any of the log files yourself. That is, you can delete or archive them as necessary.

CautionDeleting log files can make it more difficult for support personnel to resolve issues you might encounter.

System logs are managed automatically in these ways:

  • Retirement: All log files are periodically added to a compressed file and moved to install_path/hcpcs/retired/. This occurs at least once a day, but can also occur:
    • Whenever you run the log_download script.
    • Hourly, if the system instance's disk space is more than 60% full.
    • At the optimum time for a specific service.
  • Rotation: When a log file grows larger than 10MB in size, the system stops writing to that file, renames it, and begins writing to a new file. For example, if the file exampleService.log.0 grows to 10 MB, it is renamed to exampleService.log.1 and the system creates a new file named exampleService.log.0 to write to.
  • Removal: When a log file becomes older than 90 days, it is removed. If the system instance's disk space ois more than 70% full, log files are deleted when they become older than one day.
  • When an optimum number of log files for a specific service is reached, the system can overwrite the oldest file. For example, if a service is limited to 20 log files, when the file exampleService.log.19 is filled, the system overwrites the file named exampleService.log.0.

Retrieving logs and diagnostic information

The tool log_download lets you easily retrieve logs and diagnostic information from all instances in the system. This tool is located at this path on each instance:

install_path/hcpcs/bin/log_download

For information about running the tool, use this command:

install_path/hcpcs/bin/log_download -h

Note
  • When using the tool log_download, if you specify the option --output, do not specify an output path that contains colons, spaces, or symbolic links. If you omit the option --output, you cannot run the script from within a folder path that contains colons, spaces, or symbolic links.
  • When you run the script log_download, all log files are automatically compressed and moved to the folder install_path/hcpcs/retired/.
  • If an instance is down, you need to specify the option --offline to collect the logs from that instance. If your whole system is down, you need to run the script log_download with the option --offline on each instance.

Default log locations

Default log locations

By default, each service stores its logs on each instance on which the service instance runs, in its own folder at this path:

install_path/hcpcs/log

This table shows the default log folder names for each service. Depending on how your system was configured when first deployed, your system's logs might not be stored in these folders.

ServiceDefault log folder nameContains information about
Admin-Appcom.hds.ensemble.plugins.service.​adminAppThe System Management application.
Databasecom.hds.ensemble.plugins.service.​cassandra
  • System configuration data.
  • Document fields and values.
Schedulingcom.hds.ensemble.plugins.service.​chronosWorkflow task scheduling.
N/Acom.hds.ensemble.plugins.service.​containerActionCreated by custom actions run by service plugins.
Metricscom.hds.ensemble.plugins.service.​​elasticsearchThe storage and indexing of:
  • System events
  • Performance and failure metrics for workflow tasks
Network-Proxycom.hds.ensemble.plugins.service.​haproxyNetwork requests between instances.
Message Queuecom.hds.ensemble.plugins.service.​kafkaThe transmission of data between instances.
Loggingcom.hds.ensemble.plugins.service.​logstashThe transport of system events and workflow task metrics to the Metrics service.
Service-Deploymentcom.hds.ensemble.plugins.service.marathonThe deployment of high-level services across system instances. High-level services are the ones that you can move and configure, not the services grouped under System Services.
Cluster-Workercom.hds.ensemble.plugins.service.​​mesosAgentThe work ordered by the Cluster-Coordination service.
Cluster-Coordinationcom.hds.ensemble.plugins.service.​mesosMasterHardware resource allocation.
Watchdogcom.hds.ensemble.plugins.service.remoteActionInternal system processes.
Sentinelcom.hds.ensemble.plugins.service.​sentinelThe internal system processes.
Watchdogcom.hds.ensemble.plugins.service.​watchdogGeneral diagnostic information.
Synchronizationcom.hds.ensemble.plugins.service.​zookeeperThe coordination of actions and database activities across instances.
S3-Gatewaycom.hitachi.aspen.foundry.service.​clientaccess.​dataThe client access data service.
Tracing-Agentcom.hitachi.aspen.foundry.service.​jaeger.​​agentThe tracing agent service.
Tracing-Collectorcom.hitachi.aspen.foundry.service.jaeger.​collectorThe tracing collector service.
Tracing-Querycom.hitachi.aspen.foundry.service.​jaeger.​​queryThe tracing query service.
MAPI-Gatewaycom.hitachi.aspen.foundry.service.​mapi.​gatewayThe management API gateway.
Policy-Enginecom.hitachi.aspen.foundry.service.metadata.async.​policy.​engineThe metadata asynchronous policy engine.
Metadata-Cachecom.hitachi.aspen.foundry.service.metadata.cacheThe metadata cache.
Metadata-Coordinationcom.hitachi.aspen.foundry.service.metadata.coordinationMetadata coordination.
Metadata-Gatewaycom.hitachi.aspen.foundry.service.metadata.gatewayThe metadata gateway.
Telemetry-Servicecom.hitachi.aspen.foundry.service.metrics.​prometheusTelemetry.
Message-Queuecom.hitachi.aspen.foundry.service.rabbitmq.serverThe message broker.
Key-Management-Servercom.hitachi.aspen.foundry.service.vault.vaultThe key management server.