Service list

Database

http://cassandra.apache.org/

Decentralized database that can be scaled across large numbers of hardware nodes.

How it's used

Stores system configuration data. Also stores document discovery and failure data for workflow tasks.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance. Valid values are decimal numbers.

Service-Specific Options

• Max Heap Size: Maximum amount of memory to allocate to the Java heap for each instance of the service. Valid values for this setting are integers representing a number of bytes. You can optionally specify suffixes of k (for kilobytes), m (for megabytes), or g (for gigabytes). The default is 1800m.
• Heap New Size: The size of the young generation within the Java heap for the service. Valid values for this setting are integers representing a number of bytes. You can optionally specify suffixes of k (for kilobytes), m (for megabytes), or g (for gigabytes). The default is 512m.
  NoteThe Heap New Size value should be approximately 1/4th of the Max Heap Size's value.

Advanced Options

• Compaction frequency: How often database compaction operations are run. The options are Weekly (default), Daily, or Disabled.
• Compaction Throughput (MB/s): Throttles compaction to the given total throughput across the system. Setting this to 0 disables throttling. The default is 16 MB/s.
• Stream Throughput (Mb/s): Throttles all outbound streaming file transfers on this node. Setting this to 0 disables throttling. The default is 200 Mb/s.
• Read timeout (in milliseconds): The length of time allotted to database queries. The default is 12000 (12 seconds).

• 1 Required
• 3 Recommended

Index

http://lucene.apache.org/solr/

Data indexing and search platform.

How it's used

The search engine that manages all internal search indexes.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance. Valid values are decimal numbers.

Service-Specific Options

Solr Health Monitoring Options: The config options used to monitor the Solr service.

• Enable Shard Monitoring: If enabled, triggers an alert when a shard is unhealthy for a configurable time period.
• Shard Alert Time: After a shard is unhealthy for the configured time period (in minutes), an alert is generated.

Settings for each index:

• Rebalance shards: If enabled, the index shards are automatically rebalanced across the system instances that run the Index service.
• Index protection level: The total number of copies to create for the index. A value of 1 means the index has no replicas, and therefore no redundancy. The maximum value you can specify is equal to the number of instances that run the Index service.

• 0 Required
• 3 Recommended

Logging

https://www.elastic.co/products/logstash

Collection engine for event data. Can perform transformations on the data it collects and then send that data to a number of outputs.

How it's used

Transports system logs and metrics data to the Metrics service.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance. Valid values are decimal numbers.

Service-Specific Options

• Heap settings: The amount of memory to allocate for the Java heap for each instance of the service. Valid values for this setting are integers representing a number of bytes. You can optionally specify suffixes of k (for kilobytes), m (for megabytes), or g (for gigabytes). The default is 512m.

• 1 Required
• 1 Recommended

Message Queue

https://kafka.apache.org/

Stream processing platform for handling real-time data streams.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance. Valid values are decimal numbers.

• Heap settings: The amount of memory to allocate for the Java heap for each instance of the service. Valid values for this setting are any valid Java heap settings. The default is:

  -Xmx1800m -Xms512m

• 1 Required
• 3 Recommended

Metrics

https://www.elastic.co/

Data indexing and search platform.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance.

  Valid values are decimal numbers.

Service-Specific Options

• Heap size: The amount of memory to allocate for the Java heap for each instance of the service. Valid values for this setting are integers representing a number of bytes. You can optionally specify suffixes of k (for kilobytes), m (for megabytes), or g (for gigabytes). The default is 1024m.
• For each internal index that this service manages, you can configure the Index protection level. This is the number of additional replicas to maintain for each index shard. The total number of replicas for each shard is equal to this setting plus 1. For example, a value of 3 means that the index has 4 total copies of each shard.

  The maximum value you can specify is equal to the number of instances in the system minus 1.

• 1 Required
• 3 Recommended

Monitor-App

Powers the Monitor App.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance.

  Valid values are decimal numbers.

Service-Specific Options

Max Heap Size: The amount of memory to allocate for the Java heap for each instance of the service.

Valid values for this setting are integers representing a number of bytes. You can optionally specify suffixes of k (for kilobytes), m (for megabytes), or g (for gigabytes). The default is 256m.

Syslog Queue Settings

The Monitor-App service uses an Apache Kafka message queue to collect syslog messages sent by the sources that it monitors. Syslog messages in this queue are processed, indexed, and displayed in visualizations in HCM.

Messages are automatically deleted from the queue if they grow too old or the queue too large.

You can configure these settings for the queue:

Retention days: The maximum number of days that a message can remain in the queue. The service automatically removes messages older than this limit. The default is 30 days.

Max queue size: The maximum size that the queue can grow to, in megabytes. When the queue reaches this size, old messages are automatically removed. The default is -1 (no size limit).

Metric Settings

monitorAccessLogIndex retention days: The amount of time to retain the indexed metrics in monitorAccessLogIndex, used for generating visualizations. That is, the Monitor App cannot display information about the systems it monitors if that information is older than this limit. The default is 30 days.

monitorMetricsIndex retention days: The amount of time to retain the indexed metrics in monitorMetricsIndex, used for generating visualizations. That is, the Monitor App cannot display information about the systems it monitors if that information is older than this limit. The default is 365 days.

monitorEventsIndex retention days: The amount of time to retain the indexed metrics in monitorEventsIndex, used for generating visualizations. That is, the Monitor App cannot display information about the systems it monitors if that information is older than this limit. The default is 365 days.

prometheus index Retention Days: The amount of time to retain the indexed metrics in the Prometheus index, used for generating visualizations of an HCP for Cloud Scale system. Metrics older than this limit are deleted in the Monitor App. The default is 30 days.

Anomaly Detection

Anomaly Detection Run Interval: How often your system runs the Anomaly Detection algorithm, measured in hours. The default is 2 hours.

Anomaly Detection Model Archive Interval: How often your system archives Anomaly Detection models, measured in hourshours. The default is 5 hourshours.

Anomaly Detection Model Retention: How long your system keeps old Anomaly Detection models before deleting them, measured in days. The default is 1 day.

Anomaly Detection Training Days: How often your system retains its data, measured in days. The default is 30 days.

Forecasting

Forecasting Job Interval: How often your system runs the Forecasting job, measured in hours. The default is 24 hours.

Forecasting Model Archive Interval: How often your system archives Forecasting models, measured in hours. The default is 5 hours.

Forecasting Model Retention: How long your system keeps old Forecasting models before deleting them, measured in days. The default is 1 day.

Forecasting Model Max Training: The maximum number of months of historical data to use when training Forecasting models. The default is 36 months.

Forecasting Horizon: How far into the future the Forecasting model predicts, measured in days. The default is 365 days.

Forecasting Confidence Cutoff: How wide the cone of certainty can be, measured in points of standard deviation. The default is 6 points.

Forecasting Historical Value Type: Choose to have the system store the mean of all values for each day, the actual value, or none of the values. The default is Mean.

• 0 Required
• For HCM, 1 Recommended

Scheduling

https://mesos.github.io/chronos/

Job scheduler for Apache Mesos.

Docker Container Options

Each service runs within its own Docker container. You can configure these settings for this service's container:

• Container Memory: The hard memory limit for the service's Docker container, in megabytes (MB).
• CPU: The relative CPU usage weight for the service's Docker container. Generally, a higher value means that the container receives more CPU resources than other processes (including other service Docker containers) running on the instance. Valid values are decimal numbers.

Service-Specific Options

• Heap settings: The amount of memory to allocate for the Java heap for each instance of the service. Valid values for this setting are integers representing a number of bytes. You can optionally specify suffixes of k (for kilobytes), m (for megabytes), or g (for gigabytes). The default value is 512m.

• 1 Required
• 1 Recommended

Admin-App

Runs the Admin App.

Cluster-Coordination

Mesos (master) - mesos.apache.org

Hardware resource management solution for distributed systems.

Cluster-Worker

Mesos (slave) - mesos.apache.org

Hardware resource management solution for distributed systems.

Network-Proxy

HAProxy - haproxy.org

Load balancer for TCP and HTTP-based applications.

Sentinel

Runs internal system processes and monitors the health of the other services.

Service-Deployment

Marathon - https://mesosphere.github.io/marathon/

Orchestration platform for Mesos applications.

Synchronization

Apache Zookeeper - https://zookeeper.apache.org/

Coordinates configuration settings and other information between a number of distributed services.

Watchdog

Monitors the other System Services and restarts them if necessary. Also responsible for initial system startup.