Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Migration service

The Migration service migrates data off selected storage nodes in either an HCP RAIN or SAIN system or off selected storage arrays in an HCP SAIN system in preparation for retiring those devices. During a data migration, the service copies objects and, if applicable, the metadata query engine index from the selected devices to free storage on the remaining devices. Before you start a data migration, you need to ensure that those devices have enough unused capacity to hold the data to be migrated.

After copying an object, the service deletes it from the source device. Once the migration is complete, you can submit a request to your authorized HCP service provider to finalize the migration and remove the retired devices from the system.

For the purpose of data migration, HCP treats these as individual objects:

  • Parts of multipart objects
  • Parts of in-progress multipart uploads
  • Chunks for erasure-coded objects
  • Chunks for erasure-coded parts of multipart objects
Important

After a data migration off a storage node in an HCP system is finalized, the system can never again include a node with the same fourth octet in its back-end IP address as that node had.

Retiring a node is not part of the normal procedure for replacing a node that has failed or for upgrading to newer hardware. In these cases, the new node can use the same back-end IP address as the one being replaced.

The Migration service runs only when you explicitly start a data migration. When the migration is complete, the service stops automatically.

When you start a data migration, the selected nodes or storage arrays automatically become read-only (except for allowing the Migration service to delete objects). After the migration is complete, they remain read-only.

When you start a migration of data off selected nodes in an HCP system, HCP automatically removes any NFS volumes from those nodes and associates those volumes with other nodes in the system.

Typically, when migrating data off nodes, before starting a data migration, you submit a request to your authorized HCP service provider to add new nodes to the HCP system in order to maintain (or increase) the system storage capacity. However, if the nodes not selected for migration have sufficient free space to accommodate all the data to be migrated, adding new nodes before the data migration is not required.

For a SAIN system, before starting a data migration off storage arrays, your SAN storage administrator, working in conjunction with your authorized HCP service provider, needs to add logical volumes (LUNs) from new or existing storage arrays to any nodes on which all the existing LUNs on all the existing arrays are being retired. Migrated data, however, can be written to any node, and does not necessarily have to be written to the same node from which the data is being migrated.

The HCP system cannot be upgraded while a data migration is in progress. Before the system can be upgraded, you need to either allow the migration to finish or cancel the migration. If you cancel the migration, you can configure a new migration of data off the same devices after the system is upgraded.

ImportantTo prevent data loss in namespaces that are not being replicated and that have service plans that set the ingest tier DPL to 1, always migrate data off a device before submitting a request to your authorized HCP service provider to remove the device from the HCP system.

Considerations for migrations on RAIN systems

Using the Migration service to retire nodes in a RAIN system entails removing nodes from the system and, optionally, adding new nodes. After any new nodes are added to the HCP system but before you begin the data migration, you need to:

  • For each combination of domain and network configured in the DNS, remove the IP addresses of the nodes being retired and add the IP addresses of any new nodes
  • For each replication link that identifies the HCP system by its IP addresses, remove from the link configuration the IP addresses of the nodes that are being retired and add the IP addresses of any new nodes

Target storage requirements for SAIN systems

The information in this section is intended for your SAN storage administrator. It outlines storage requirements that, if not met, prevent a data migration from being started.

Each node in an HCP SAIN system must have one OS LUN and at least two data LUNs. If a LUN being migrated is the OS LUN for a node, a replacement for that LUN must be added to the node before the data migration can occur. If the existing LUN is number zero, the new LUN must be number 128. If the existing LUN is number 128, the new LUN must be number zero. Additionally, the new LUN must have a capacity of at least 30 GB.

Migration procedure

The complete procedure for retiring a device is:

  1. Take one of these actions:
    • Optionally, for a RAIN system, submit a request to your authorized HCP service provider to add one or more storage nodes to the system. After a data migration is finalized, the HCP system must still have at least four storage nodes.
    • For a SAIN system, either submit a request to your SAN storage administrator and your authorized service provider to add one or more storage nodes to the system or work together with your authorized service provider to add LUNs to the nodes on which all of the existing LUNs are on the storage arrays that you’re retiring.
  2. For a RAIN system, update the DNS and any replication links as needed.
  3. Configure the data migration by selecting the devices to be retired. HCP can perform only one data migration at a time. Therefore, you should select all of the devices that you want to retire so that you don’t have to run multiple sequential data migrations.

    Certain hardware errors, such as a degraded RAID group on a source or target node, prevent you from configuring a data migration. In such cases, you need to fix the problem before you can continue.

  4. Review the configuration of the data migration.

    If the migration configuration is not acceptable, HCP provides detailed information about the problems.

  5. Submit requests to your authorized HCP service provider and/or your SAN administrator (if you’re migrating storage off a SAIN system), as necessary, to fix any reported problems.
  6. Optionally, enter a description for the data migration and/or change the performance level for the Migration service.
  7. Ensure that all of the nodes in the HCP system are running and healthy.
  8. Start the data migration.

    If any nodes become unavailable while the Migration service is running, the service stops migrating data. When those nodes become available, the service automatically starts migrating data again.

  9. Monitor the data migration and manage it by changing the performance level or pausing the migration, as needed. You can also modify the migration description at any time (for example, to record when and how long the migration was paused).
  10. When the data migration is complete (that is, the migration status is Migrated):
    • If a migration report is available, review it. This report identifies tenants that own namespaces containing unacknowledged irreparable objects. For the default tenant and for HCP tenants that are configured to allow system-level users to manage them, the report also lists the unacknowledged irreparable objects in those namespaces.

      If the Migration service encounters one or more objects that it cannot migrate, it marks those objects as irreparable (if they weren’t already marked that way).

    • If the data migration statistics show that not all objects were migrated, contact your authorized service provider for help.

Migration page

The Migration page in the HCP System Management Console lets you configure, monitor, and manage data migrations.

NoteTo monitor data migrations, you need the monitor or administrator role. To configure and manage data migrations, you need the administrator role.

To display the Migration page, in the top-level menu of the System Management Console, select Services Migration.

You can also perform a migration by using the Retire Primary Storage wizard. You can access the wizard from the Retirement pane on the Storage page in the System Management Console.

Configuring a data migration

When configuring a data migration on a RAIN system, you select nodes to be retired. When configuring a data migration on a SAIN system, you can either select nodes or storage arrays for retirement.

Configuring a migration on a RAIN system

To configure a data migration on a RAIN system:

  1. On the left side of the Migration page, click Configuration.

    The Configuration panel displays step 1 (one) of the migration configuration (Choose items for migration). The Select Hardware for Retirement section lists the storage nodes in the HCP system.

  2. Select the nodes from which you want to migrate the data.

    To clear your selections and start over, click Cancel.

  3. Click Next.

    The Configuration panel displays step two of the migration configuration (Review configuration summary and confirm). The Configuration Summary section in this panel indicates whether the migration configuration is acceptable.

    NoteWhen you click Next, HCP checks that the system is in a valid state to perform the migration. This includes checking for degraded RAID groups. This check can take up to 90 seconds.

    If the configuration is not acceptable, you can click view details in the Configuration Summary section to display the specific reasons why. You can also click Configuration Report to download the configuration summary and details to a file. The default name for this file is Configuration-Report.txt.

    The Configuration Details section in the step-two panel lists the nodes selected for migration:

    • To change the migration configuration, click Modify Configuration. The Configuration panel redisplays step 1, which shows your current selections.
    • To restart the migration configuration, click Cancel. The Configuration panel redisplays step 1 with all selections cleared.
  4. Optionally, add a description of the data migration and/or change the performance level for the Migration service:
    • To add a description:
      1. Click Add description.
      2. In the text box that opens, type a description of the migration. This text can be up to 1,024 characters long and can contain any valid UTF-8 characters, including white space.
    • To change the performance level, in the Performance Level field, select Low, Medium, or High. The higher the performance level, the greater the load on the HCP system.
  5. Click Start Migration.

    The Migration service begins preparing for the data migration, and the Migration page switches to the Overview panel.

Configuring a migration on a SAIN system

To configure a data migration on a SAIN system:

  1. On the left side of the Migration page, click Configuration.

    The Configuration panel displays step 1 (one) of the migration configuration (Choose items for migration). From this panel you can select to either retire storage arrays or hardware nodes.

  2. Take one of these actions:
    • Retire Storage Arrays:
      1. Select Retire Entire Array.

        The Select Hardware for Retirement section lists the storage systems used by the HCP system. Each array is assigned a number, starting from zero. Below this list, the section shows the number of LUNs currently selected for migration out of the total number of LUNs for each node.

        To view additional details about the LUNs, click in the row for the node you’re interested in or click expand all to see details about the LUNs on all nodes. After displaying details for all the LUNs, you can click collapse all to hide the details.

        The details shown for each LUN are:

        • The number of the array the LUN comes from.
        • The LUN number.
        • The worldwide identification number (WWID) for the LUN.
        • The type of LUN (OS, data, or standby). Standby means that the LUN mapping provides zero-copy-failover support for a data LUN on a different node.
        • The LUN size.
      2. Select the storage arrays or storage nodes from which you want to migrate the data.

        When you select an array for migration, all HCP LUNs on the array are selected automatically. You cannot select or deselect the LUNs individually.

        To clear your selections and start over, click Cancel.

    • Retire Hardware Nodes:
      1. Select Retire Specific Hardware Nodes.
      2. Select the nodes from which you want to migrate the data.

        To clear your selections and start over, click Cancel.

  3. Click Next.

    The Configuration panel displays step two of the migration configuration (Review configuration summary and confirm). The Configuration Summary section in this panel indicates whether the migration configuration is acceptable.

    NoteWhen you click Next, HCP checks that the system is in a valid state to perform the migration. This check can take up to 90 seconds.

    If the configuration is not acceptable, you can click view details in the Configuration Summary section to display the specific reasons why. You can also click Configuration Report to download the configuration summary and details to a file. The default name for this file is Configuration-Report.txt. You can send this file to your SAN storage administrator, who can then correct the problems.

    The Configuration Details section in the step-two panel lists the devices or nodes selected for migration. It also shows the number of LUNs currently selected for migration out of the total number of LUNs for each node. As in the step-1 panel, you can view details about the selected LUNs. In this case, the details have an additional column, Migration Status, that indicates whether the data on the LUN can (Ready) or cannot (Not Ready) be successfully migrated.

    To change the migration configuration, click Modify Configuration. The Configuration panel redisplays step 1, which shows your current selections.

    To restart the migration configuration, click Cancel. The Configuration panel redisplays step 1 with all selections cleared.

  4. Optionally, add a description of the data migration and/or change the performance level for the Migration service:
    • To add a description:
      1. Click Add description.
      2. In the text box that opens, type a description of the migration. This text can be up to 1,024 characters long and can contain any valid UTF-8 characters, including white space.
    • To change the performance level, in the Performance Level field, select Low, Medium, or High. The higher the performance level, the greater the load on the HCP system.
  5. Click Start Migration.

    The Migration service begins preparing for the data migration, and the Migration page switches to the Overview panel.

Monitoring a data migration

To monitor a data migration, you use the Overview pane on the Migration page. This page shows information about the current data migration and the last completed or canceled data migration.

Information about the current data migration

The top of the Overview pane displays information about the current data migration and the Migration service.

  • The current status of the Migration service:
    • Not Migrating

      The Migration service is not running. No migration is in progress.

    • Starting Migration

      The Migration service is preparing for the data migration. This includes determining the number of objects to be migrated and the size of the data to be migrated. It also includes changing the HCP system configuration to prevent data from being written to the selected devices.

    • Migrating

      The Migration service is actively migrating data off the selected devices.

    • Paused

      A data migration is in progress, but the Migration service is not actively migrating data at this time.

    • Completing Migration

      The Migration service is verifying that the migration was successful and waiting while HCP rebalances metadata.

    • Migrated

      The Migration service has finished migrating data off the selected devices and is no longer running.

      If the Overview panel displays a Migration Report link, click the link to download the migration report to a file. The default name for this file is Migration-number-Report.txt, where number is the number automatically assigned to the migration when the copying process started.

      Be sure to review the migration report before having your authorized service provider finalize the migration.

  • The estimated amount of time remaining to complete the current data migration.
  • The amount of time the Migration service has been running. This value does not include any time during which the service was paused.
  • The time the Migration service started.
  • The total number of items that have been migrated so far out of the total number of items to be migrated, along with a progress bar and text indicating the percent of items migrated. Items are any of these:
    • Whole objects
    • Parts of multipart objects
    • Parts of in-progress multipart uploads
    • Chunks for erasure-coded objects
    • Chunks for erasure-coded parts of multipart objects
  • The amount of data migrated so far, in KB, out of the total amount of data to be migrated, along with a progress bar and text indicating the percent of data migrated.
  • The current performance-level setting for the Migration service.
  • The description of the current data migration.

    To modify the description:

    1. Click Edit description.
    2. In the text box that opens, edit the migration description.
    3. Click Submit.

To view the configuration of the current data migration:

  1. Click View details.

    The Current Migration Details window opens. This window shows the same Configuration Summary and Configuration Details sections as step two of the Configuration pane.

  2. After viewing the migration configuration, click Close.

Information about the last data migration

The Migration History section in the Overview pane displays information about the last completed or canceled data migration.

  • The time at which the data migration was completed or canceled.
  • The total amount of time the Migration service took to perform the data migration. This value does not include any time during which the service was paused.
  • The number of items that were migrated out of the total number of these items that were to be migrated, along with a progress bar and text indicating the percent of items migrated. Items are any of these:
    • Whole objects
    • Parts of multipart objects
    • Parts of in-progress multipart uploads
    • Chunks for erasure-coded objects
    • Chunks for erasure-coded parts of multipart objects
  • The amount of data migrated, in KB, out of the total amount of data that was to be migrated, along with a progress bar and text indicating the percent of data migrated.
  • The description of the data migration.

    To modify the description:

    1. Click Edit description.
    2. In the text box that opens, edit the migration description.
    3. Click Submit.

To view the configuration of the data migration:

  1. Click View details.

    The Previous Migration Details window opens. This window shows the same Configuration Summary and Configuration Details sections as step-two of the Configuration pane.

  2. After reviewing the migration configuration, click Close.

Managing a data migration

While the Migration service is migrating data, you can use the Management panel on the Migration page to:

  • Change the performance level of the Migration service
  • Change the description of the data migration
  • Pause or resume the data migration
  • Cancel the data migration

Changing the performance level

The performance level determines how much load the Migration service puts on the HCP system. If the system load from other activities is heavy, you can lower the performance level for the Migration service to make more system resources available to those activities. If the system load from other activities is light, you can increase the performance level for the Migration service, thereby allowing the service to use more system resources.

You can change the performance level for the Migration service, in the Management panel.

Before you begin

To monitor data migrations, you need the monitor or administrator role. To configure and manage data migrations, you need the administrator role.

Procedure

  1. In the Performance Level field, select Low, Medium, or High.

  2. Click Update Settings.

  3. In response to the confirming message, click Update Settings.

    HCP pauses the data migration and changes the performance level.
  4. Click Resume.

Pausing or resuming a migration

You can pause or resume the data migration at any time while the Migration service is copying objects. You would do this, for example, if you need to make changes to HCP networking or during periods of heavy namespace activity. While the migration is paused, the selected devices remain read-only.

To pause or resume a data migration, in the Management panel, click Pause or Resume, as applicable.

Canceling a migration

You can cancel the data migration at any time while the Migration service is copying objects or while the migration is paused. When you do this, the Migration service stops and the selected devices become read-write. Any data that was already migrated remains in its new location. Additionally, information about the data migration moves to the Migration History section in the Overview panel.

Before you begin

To monitor data migrations, you need the monitor or administrator role. To configure and manage data migrations, you need the administrator role.

Procedure

  1. Click Cancel.

  2. In response to the confirming message, click Cancel Migration.

 

  • Was this article helpful?