Logs and diagnostic information
Each service maintains its own set of logs. By default, the logs are maintained in the
install_path/hci/log folder on each instance in the system. During installation, you can configure each service to store its logs in a different, non-default location.
You can manage any of the HCI log files yourself, deleting or archiving them as necessary.
System logs are managed automatically in these ways:
- All log files are periodically added to a compressed file and moved to install_path/hci/retired/. This occurs at least one time a day, but can also occur:
- Whenever you run the
- Hourly, if the system instance's disk space is more than 60% full.
- Whenever you run the
- After a log file grows larger than 10MB in size, the system stops writing to that file, renames it, and begins writing to a new file. For example, if
exampleService.log.0grows too large, it is renamed to
exampleService.log.1and the system creates a new
exampleService.log.0to write to.
log_download tool lets you easily retrieve logs and diagnostic
information from all instances in the system. This tool is located at this path on each
For information about running the tool, use this command:
- When using the
log_downloadtool, if you specify the
--outputoption, do not specify an output path that contains colons, spaces, or symbolic links. If you omit the
--outputoption, you cannot run the script from within a folder path that contains colons, spaces, or symbolic links.
- When you run the
log_downloadscript, all log files are automatically compressed and moved to the folder install_path/retired/.
- If an instance is down, you need to specify the option
--offlineto collect the logs from that instance. If your whole system is down, you need to run the script log_download with the option
--offlineon each instance.
By default, each service stores its logs in its own folder at this path:
This table shows the default log folder names for each service. Depending on how your system was configured when first deployed, your system's logs might not be stored in these folders.
|Related service||Default log directory name||Contains information about|
|Admin-App||com.hds.ensemble.plugins.service.adminApp||The System Management application.|
|Cluster-Coordination||com.hds.ensemble.plugins.service.mesosMaster||Hardware resource allocation.|
|Cluster-Worker||com.hds.ensemble.plugins.service.mesosAgent||Work ordered by the Cluster-Coordination service.|
|Index||com.hds.ensemble.plugins.service.solr||Index collections and search indexes.|
|Logging||com.hds.ensemble.plugins.service.logstash||The transport of system events and workflow task metrics to the Metrics service.|
|Metrics||com.hds.ensemble.plugins.service.elasticsearch||The storage and indexing of:|
|Message Queue||com.hds.ensemble.plugins.service.kafka||Transmission of data between instances.|
|Monitor-App||com.hds.ensemble.plugins.service.mapApp||The HCM application.|
|Network-Proxy||com.hds.ensemble.plugins.service.haproxy||Network requests between instances.|
|Scheduling||com.hds.ensemble.plugins.service.chronos||Workflow task scheduling.|
|Search-App||com.hds.ensemble.plugins.service.searchApp||The Search App|
|Sentinel||com.hds.ensemble.plugins.service.sentinel||Internal system processes.|
|Service-Deployment||com.hds.ensemble.plugins.service.marathon||The deployment of high-level services across system instances. High-level services are the ones that you can move and configure (such as Index), not the services grouped under System Services.|
|Synchronization||com.hds.ensemble.plugins.service.zookeeper||Coordination of actions and database operations across instances.|
|Watchdog||com.hds.ensemble.plugins.service.remoteAction||Internal system processes|
|Watchdog||com.hds.ensemble.plugins.service.watchdog||General diagnostic information.|