Skip to main content
Hitachi Vantara Knowledge

S Series Balancing service

The S Series Balancing service is a configurable service that balances object data across S Series Nodes in the same storage pool. The service ensures that the percent of space used across S Series Nodes in a pool remains roughly equivalent. The service is particularly useful when one or more S Series Nodes in the same storage pool is added to, removed from, or retired from an HCP system.

When the S Series Balancing service runs, it evaluates the storage level of each node in a pool. If the levels differ by a wide margin, the service moves objects around to bring the levels closer to a balanced state.

For the purpose of S Series balancing, HCP treats the following items as individual objects:

  • Parts of multipart objects
  • Parts of in-progress multipart uploads
  • Chunks for erasure-coded objects
  • Chunks for erasure-coded parts of multipart objects

The S Series Balancing service is included in the HCP Default Schedule. However, it remains idle until storage pools are configured to take advantage of the service.

To ensure optimal system performance in deployments with S Series storage pools that include multiple S Series Nodes, you need to add the service to your active service schedule.

New installations and upgrades will benefit from the S Series Balancing feature only when an S Series pool with multiple S Series Nodes is configured to use the service.

S Series Balancing service processing

The S Series Balancing service detects and repairs imbalances in storage availability across S Series Nodes in the same storage pool by comparing storage usage statistics.

If the service determines that storage usage is imbalanced across S Series Nodes in the same pool:

  • The service determines whether each S Series Node in the pool is a source of objects to move or a target to move them to.
  • From the S Series Node source, the service moves objects individually to the S Series Node target provided the storage managed by the target node does not have a copy of the object to be moved.

    When selecting objects to move, the S Series Balancing service considers the size of the object data and any custom metadata that the object includes, and the age of the object. If an S Series Node is retiring, the balancing service excludes the node in its balancing algorithm.

    At any point in time, HCP is unlikely to be in a perfectly balanced state for the following reasons:

    • The S Series Balancing service is designed to stop balancing when the nodes are within a 10% difference threshold.
    • Object additions and deletions to and from the system do not trigger the S Series Balancing service to run.
    • When all the objects in a directory have been deleted, the empty directory remains in the namespace. Directories in the default namespace, whether empty or not, have metadata that takes up space.

Best practices in configuring S Series Balancing

The S Series Balancing service balances object data across S Series Nodes in the same storage pool to spread the storage load.

When the S Series Balancing service runs, balancing is initiated if S Series Nodes in the same storage pool have a percent-used disparity greater than 10%. Balancing is relative to the size of each storage node, that is, percent capacity.

Best practices

To optimize balancing in a deployment with multiple attached S Series Nodes, if storage pools are sharing S Series Nodes, they should either share all their nodes with one another, or share none.

The practice of sharing all S Series Nodes allows data from any pool to be stored on any node, thereby maximizing the balancing algorithm and making use of all available storage capacity. Each pool must also have balancing enabled. If one pool does not enable balancing, balancing for the other pools might not complete.

Similarly, the practice of pools sharing none of their S Series Nodes ensures that each pool that participates in balancing can always move data.

The following figure shows a best-practice configuration where Pools A and B share all the same S Series Nodes, in the example, Nodes 1 and 2.

best practice configuration - Pools A and B share all the same S Series Nodes

Because there is a greater than 10% disparity in the used capacity between Nodes 1 and 2 (Node 1 = 36% used, Node 2 = 18% used), the S Series Balancing service will have work to do. Each time the service runs, data will be moved from Node 1 to Node 2 until there is a less than 10% disparity in used capacity. For example, if Node 1 reaches 31% used capacity and Node 2 reaches 23% used capacity, the service will stop.

The following figure shows a best-practice configuration where Pools A and B each have their own set of S Series Nodes. There is no node sharing between the pools.

best practice configuration - Pools A and B each have their own set of S Series Nodes

For Pool A, because there is a greater than 10% disparity in the used capacity between Nodes 1 and 2 (Note 1 = 36% used, Node 2 = 18% used), the S Series Balancing service will have work to do when it runs. For Pool B, because there is a less than 10% disparity in the used capacity between Nodes 3 and 4 (Node 3 = 36% used, Node 4 = 32% used), the S Series Balancing service will not have any work to perform.

Poor practices

There are some configuration scenarios that can prevent the S Series Balancing service from moving data, or can lead to less-than-optimal performance. When pools share some, but not all, of the same S Series Nodes, an object imbalance results across nodes. Data from any pool cannot be placed on any node, which prevents the S Series Balancing service algorithm from using all available capacity. In some instances, capacity on one S Series Node in a pool should be moved to another node in the pool, but there is insufficient data to move to balance the nodes.

The following figure shows a configuration where Pools A and B share Node 1 but not Node 2.

Poor practice - Pools A and B share Node 1 but not Node 2

Because there is a greater than 10% disparity in the used capacity between Nodes 1 and 2 (Node 1= 41% used, Node 2 = 9% used), the S Series Balancing service has work to do for Pool B. When the service runs, data will be moved from Node 1 to Node 2. Data movement will occur each time the S Series Balancing service is scheduled to run, or when the service is initiated manually from the Overview page of the System Management Console.

However, only the data available to Pool B (in Bucket 2) will be moved. As mentioned earlier, the S Series Balancing service completes when there is a less than 10% disparity in the used capacity between the nodes that are participating in balancing. In this scenario, even after the 5 TB in Bucket 2 is moved from Node 1 to Node 2, there will still be a greater than 10% balance disparity. The S Series Balancing service will continue to report that it has work to do, but it will be unable to address the disparity.

To correct this poorly performing configuration, you could create another bucket on Node 2, and add the bucket to Pool A.

Best practice - add bucket to Pool A

S Series Balancing service monitoring

The HCP System Management Console provides several ways to monitor various aspects of S Series Balancing service processing.

  • The System Events page in the Console displays S Series Balancing service event status and metrics about each event. Metrics include the number of objects examined, the number of bytes examined, the number of objects moved to another S Series Node, and the number of bytes moved.
  • The S Series Balancing page in the Console displays balancing status and metrics for each pool configured to use the service. Metrics include the pool balancing status (balanced, balancing, paused, unscheduled, or unavailable) and the percent balanced.
    NoteThe S Series Balancing service reports that a pool is balanced when the S Series Node in the pool have a percent-used disparity of less than 10%.
  • The Overview, Services, and Monitoring System Events pages in the Console show S Series Balancing service status.

 

  • Was this article helpful?