Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Viewing data migration policies and schedules

Having created both data migration paths and data migration rules, data migration policies can now be created. Policies assign a rule or set of rules to a specific data migration path. They also define the conditions that initiate data migrations.

Procedure

  1. To view data migration policies, navigate to Home Storage Management Data Migration to display a list of policies and schedules.

    GUID-6C86887C-9F95-4BAD-AEA2-BA80E04A12AA-low.png

    The following tables describe the fields on this page:

    Item/Field for Policies Description
    Name Name of a data migration policy.
    EVS

    Primary EVS from which the migration originates.

    Primary File System

    Files in the primary file system or virtual volume that will be migrated.

    Secondary File System

    Secondary file system, to which all data will be migrated to.

    NoteIf the path to the secondary file system is an external path, the name or IP address of the server hosting the secondary file system is also displayed in parentheses. The displayed server name/IP address is a link, and you can click the link to display the full path.
    Rule

    Rules that can be triggered in this migration policy.

    details Displays the details for the selected policy.
    add Advances to the Add Data Migration Policy page.
    remove Deletes the selected migration policy.
    Data Migration Rules Advances to a page where all the configured data migration rules will be displayed. New rules can be created here as well.
    Data Migration Cloud Accounts Advances to the Cloud Accounts and Destinations page.
    Data Migration Paths Displays the configured data migration paths. New paths can be created as well.
    NDMP Configuration Advances to a page where the NDMP can be configured by entering the username, password, and NDMP version number.
    Item/Field for Schedules Description

    Policy Name/Schedule ID

    Displays the name given to the Data Migration Policy
    EVS Displays the primary EVS from where the migration is scheduled to originate.
    Next Run Displays the month, date, year and time for the next scheduled data migration run for this policy.
    Migration Type

    Displays the type of data migration that has been scheduled to run:

    • Migrate files (followed by only once, daily, or weekly depending on the selected scheduled type).
    • Simulate migration - Generates a one-time report of files that would be migrated. Does not actually migrate files.
    • Report migrated files - Generates a one-time report with a list of previously migrated files. This migration type only applies to WFS/HNAS and not Data Migrator to Cloud.
    Last Status Displays the final status of the last run of the data migration operation.
    details Displays the current settings for the selected schedule.
    add Advances to the Add Data Migration Schedule page.
    remove Deletes the selected migration policy.
    run now Starts the scheduled data migration immediately.
    Stop Migration(s) Click the Stop Migrations link to stop a selected, in-process migration. Only migrations in progress can be stopped.
    Data Migration Status & Reports

    Advances to a page where all the completed migration runs will be listed in detail.

Adding a WFS/HNAS data migration policy

You can add a WFS/HNAS data migration policy in the NAS Manager.

  1. Navigate to and then click add under the Policies section to display the Add Data Migration Policy page.

    GUID-A06CEFBE-42AD-4508-9915-6F2F604BA0CE-low.png

    The following table describes the fields in this page:

    Item Description
    Policy Identification This section allows you to specify a name to identify the migration policy.
    Name Name for the new data migration policy.
    Migration Path This section allows you to specify the data migration path to be used by the migration (or reverse migration) policy.
    Migrate Data When creating a migration policy, this section displays the data migration source and target information:
    • Primary EVS/File System: Displays the name of the EVS and file system for primary storage (the migration source).
    • Virtual Volumes: If a virtual volume has been selected as primary storage, the virtual volume will be displayed.
    • Secondary Target Type: Displays WFS/HNAS to represent the migration type.
    • Secondary File System: Displays the name of the file system on secondary storage that will host the migrated data (the migration target).
    Reverse Migrate

    When creating a reverse migration policy, this section displays the destination for the reverse migration (the original data migration source and target information). Files that have been migrated from the original source to secondary storage will be returned to this EVS and File System when the reverse migration policy criteria are met.

    change... Opens the Select a Path page, which enables you to select a different path.
    Data Migration Paths Opens the Data Migration Paths page, which allows you to create data migration paths.
    Pre-Conditions

    This section allows you to specify the rules (the criteria) that the files must match/meet in order to be migrated (or reverse migrated) according to this policy.

    Available WFS/HNAS Rules Rules with specific threshold limits are displayed here. This list of rules define the set of conditions which trigger the migration/reverse migration. You can:
    • Add a Pre-condition to the Selected Rules list by selecting it and clicking the right arrow (>).
    • Remove a rule from the Selected Rules list by selecting it and clicking the left arrow (<).

    This policy will be run either by a defined schedule, or when started manually. Once the policy is run, the threshold specifies when the selected rules are applied. You can specify that the selected rules will be applied when either of the following conditions are met:

    • When the primary file system's free space falls below X% (set the percentage level for the condition).
    • When other conditions are not met. These conditions are defined in the selected rule(s).

    After selecting rules and the threshold, save the policy by clicking OK.

    Selected WFS/HNAS Rules

    Displays the rules containing the criteria/conditions to be used to determine if a file should be migrated. The criteria in the rules are applied when the threshold (the when condition specified in the Available Rules section) is met.

    Data Migration Rules Opens the Data Migration Rules page, which allows you to create rules.
  2. Add a descriptive name for the policy. The name should make it easy to recognize the purpose of the policy.

  3. Select a migration path. You can specify that the policy:

    • Migrate Data: To move the data from the primary file system to the secondary file system when the criteria defined in the Selected Rules are met.

    • Reverse Migrate: To move the data from the secondary file system back to the primary file system when the criteria defined in the Selected Rules are met.
  4. Select one of the migration paths you created on the Data Migration Paths page. The path specifies the source and the destination of the migration. For a:

    • Migration operation, the source is the primary file system and the destination is the secondary file system.

    • Reverse migration operation, the source is any secondary file system and the destination of the revers migration is the primary file system (the original source).

      When a file is migrated, a CVL (cross volume link) or an XVL (external volume link) is left in the primary file system to point to the file in the secondary file system.

    If the currently displayed path is not the one you want this policy to use, click change to display the Select a Path page:

    Select the path you want the policy to use, then click OK.

    NoteIf you are adding a policy to perform a test migration, a valid migration path is required, but it may be a real path involving primary and secondary file systems, or a “test-only” path in which a secondary file system does not exist.
  5. Select the rules and specify when the rules will be applied. Using the Pre-conditions section, specify the rules that contain the criteria that must be met in order to migrate the file. Also, choose the threshold that triggers the migration of the file.

  6. Verify your settings, then click OK to save the policy as defined, or cancel to return to the Data Migration page.

Using Pre-Conditions

When a migration policy is scheduled to run, it evaluates the percentage of available free space in the Policy's primary storage. Based on this analysis, one rule may be triggered to define the data set subject to migration. Migrations of data from primary storage then occurs based on the statements in the rule that was triggered. Only a single rule will be engaged during any particular migration operation.

When defining pre-conditions, customer support recommends aggressive tiering; specifically, it may be desirable to migrate .mp3 files and the contents of the directory /tmp regardless of the available free space. Then, if free space on primary storage is reduced to less than 50%, also to migrate all files not accessed within the last sixty days. Finally, if available free space is reduced to less than 15%, also to migrate the contents of users' home directories.

The following rules illustrate this scenario:

Rule Statement
Rule 1: INCLUDE (<FILENAME *.mp3>) OR <PATH /tmp/*)
Rule 2: INCLUDE (<FILENAME *.mp3>) OR <PATH /tmp/*)
INCLUDE (<INACTIVE_OVER 60>)
Rule 3: INCLUDE (<FILENAME *.mp3>) OR <PATH /tmp/*)
INCLUDE (<INACTIVE_OVER 60>)
INCLUDE (<PATH /home/*>)
Related pre-conditions

  • Rule 3 if free space is less than 15%.
  • Rule 2 if free space is less than 50%.
  • Rule 1 if no other condition applies.

When the migration policy is scheduled to run, different rules may be triggered based on the available free space on primary storage. When a migration policy is engaged, only a single rule will be triggered to run.

For example:

  • If free space is at 80%, then Rule 1 will be used.
  • If free space is at 40%, then Rule 2 will be used.
  • If free space is at 10%, then Rule 3 will be used.

When percentage thresholds are specified, they are evaluated based on whole number percentages. This means that if two rules are specified, one that will take effect at 8% of free space and one at 9% of free space, if the file system has 8.5% free space available, then the rule with the 8% pre-condition will apply.

Note If the primary storage defined in the migration path is a virtual volume, free space will be based on the limit defined by the virtual volume quota. If this quota has not been defined, then free space available will be based on the free space of the file system hosting the virtual volume.
Connection Errors

When attempting to add a new migration policy, a connection error may be indicated by a message saying "Unable to connect to <IP address>" or "Error accessing <source/destination> server".

The "Unable to connect to" message means one of the following:

  • The server is not currently powered up or is temporarily disconnected from the network. The server must be available and properly connected when creating a migration policy.
  • The NDMP service may be disabled. The migration uses the NDMP service which must be enabled when adding or running replications. Use the NDMP configuration page (or the ndmp-status command) to enable and start the NDMP service.
  • The Gigabit Ethernet port providing access to the EVS which hosts the File System is not accessible from the SMU. This may be the case if the network is set up with private subnetworks as commonly used with VLANs. In this case, the server may have been configured so that SMU access is through the management ports instead of the ports set using the ndmp-management-ports-set command.

The "Error accessing server" message may occur as a result of restricting NDMP access using the ndmp-option command. The allowip and blockip options can be set such that the SMU is not allowed to access the NDMP services via the standard routes. If the NDMP connection restrictions are definitely required, change the configuration of the server to allow SMU access via the management ports using the ndmp-management-ports-set command. The SMU connections then bypass the allowip/blockip checks.

Modifying a data migration policy

You can modify a data migration policy in the NAS Manager.

  1. Navigate to Home Storage Management Data Migration.

  2. Select the policy to modify and then click details to display the Modify Data Migration Policy page.

    The following table describes the fields:

    Item Description
    Policy Identification Name of the data migration policy.
    Migration Path The specified data migration path to be used by the migration (or reverse migration) policy.
    Pre-Conditions

    This section allows you to modify the rules (the criteria) that the files must match/meet in order to be migrated (or reverse migrated) according to this policy.

    Available Rules Rules with specific threshold limits are displayed here. This list of rules define the set of conditions which trigger the migration/reverse migration. You can:
    • Add a Pre-condition to the Selected Rules list by selecting it and clicking the right arrow (>).
    • Remove a rule from the Selected Rules list by selecting it and clicking the left arrow (<).

    This policy will be run either according to a defined schedule, or when started manually. Once the policy is run, the threshold specifies when the selected rules are applied. You can specify that the selected rules will be applied when either of the following conditions are met:

    • When the primary file systems free space falls below X% (set the percentage level for the condition).
    • When other conditions are not met. These conditions are defined in the selected rule(s).

    After selecting rules and the threshold, save the policy by clicking OK.

    Selected Rules

    Displays the rules containing the criteria/conditions to be used to determine if a file should be migrated. The criteria in the rules are applied when the threshold (when condition specified in the Available Rules section) is met.

    (To create rules, see Data Migration Rules) Advances to the Data Migration Rules page, which allows you to create rules.
  3. Make updates as necessary.

  4. Click OK.

About the ndmp-management-ports-set command

The NAS Manager replication and data migration features use the NDMP service on the NAS server. The NDMP service is usually accessed via the IP address of the EVS which hosts the file system, this access usually happens through a Gigabit Ethernet port. In some cases, the IP address is within a private subnetwork and is not accessible from the NAS Manager. When this is the case, the ndmp-management-ports-set command can be used to request that the NAS Manager access goes through the management ports and is then relayed to the NDMP service.

The ndmp-management-ports-set command takes two parameters which are the TCP ports. One is used to accept the incoming connection on the management port and one is used to pass the requests to the NDMP code. These must be ports that are not in use by any other service. In particular, these ports must not be the standard NDMP service port. The port numbers 10001 and 10002 usually work and, being next to the standard NDMP port 10000, can be useful in identifying the port usage.

Having set up the NDMP management ports this way, all NAS Manager replication and data migration NDMP accesses will be routed via the management port. Note that the actual data transfer connections involved are between the NAS server EVSs and do not run over the management connections. In particular, a replication between two NAS servers passes the data over a TCP connection between EVS IP addresses through the Gigabit Ethernet ports. Therefore, the two EVSs must have IP addresses that can communicate with each other.

Migration schedules

After a data migration policy has been defined, it must be scheduled. The decision on how often to run a policy may be affected by the rules selected in this policy. For example:

  • A policy with a single rule to migrate all .mp3 files may be scheduled to run once every month.
  • Another policy, used to archive a working /project directory once the project is complete, may be scheduled as a Once Only Schedule.
  • Other policies which migrate based on various Pre-conditions, and are triggered on available free space, may be scheduled to run every week.

When planning migration schedules, schedule migrations during off-peak times, such as evenings and weekends.

After a data migration has begun, additional data migrations for the same policy cannot be started until the current one has completed. However, it is possible to start multiple concurrent data migrations if each have its own policy.

Adding a Data Migration schedule

You can add a data migration schedule in the NAS Manager.

NoteYou must create a migration policy before you can schedule a migration.

Procedure

  1. Navigate to Home Storage Management Data Migration.

  2. Click add in the Schedule section of the page to display the Add Data Migration Schedule page:

    GUID-880CEEE9-ED2E-4322-9BA7-C012B3B5F040-low.png

    The following table describes the fields on this page:

    Field/Item Description
    Data Migration Policy Select a migration policy from the list.
    Migration Type

    Choose from the following migration type options:

    Migrate Files: Select this option and then choose only once, daily, or weekly, from the list. Selecting the Once Only option causes the policy to run only once, at the specified date and time.

    Simulate Migration: Select this option to generate a report of files that would be migrated. Does not actually migrate files. Only run once.

    Report Migrated Files: Select this option to generate a report with a list of previously migrated files. Only run once. This option only applies to WFS/HNAS and does not apply to Data Migrator to Cloud.

    Date and Time to Start

    Specifies when the policy will run.

    From the calendar next to the field, select the start date for the policy's initial run. The selected date appears on the field.

    Enter the scheduled run time in a 24 hour setting (for example, 11:59 PM will be entered as 23:59). The current NAS Manager date and time are provided below for reference.

    Duration Type Choose from the following duration types:

    Run until migration completes indicates that the scheduled policy should run until it has completed.

    Suspend migration after x Hours:Minutes. Resume when the next schedule starts indicates the scheduled policy should be suspended after the time specified and resumed at the next scheduled interval. Note that this option only applies to Data Migrator to Cloud and not to WFS/HNAS.

  3. Verify your settings. Then click OK to save or cancel to decline.

Modifying a schedule

Once defined, schedules can be easily modified to meet the changing requirements of data migration policies. When modifying a schedule, the scheduled date and time, as well as the interval in which the schedule will run can be changed.

Procedure

  1. Navigate to Home Storage Management Data Migration.

  2. Under the Schedules section of the page, fill the check box next to the schedule to modify and then click details.

    The following table describes the fields on this page:

    Field/Item Description
    Data Modify Policy Name of the schedule. This is a read-only field.
    Data Migration Type The type of migration: WFS/HNAS, External, or Cloud.
    Migration Type Displays the current option.

    Migrate files - options are:

    • only once
    • daily
    • weekly

    Simulate Migration - Generates a report of files that would be migrated. Does not actually migrate files. Only run once.

    Report Migrated Files - Generates a report with a list of previously migrated files. Only run once. This option applies to WFS/HNAS and External data migration types and does not apply to Data Migrator to Cloud type.

    Next Run Date and time of next scheduled run.
    Initial Run Date and time of initial run.
    Reschedule To change this, fill in the check box and enter the new date and time.
    Duration Type
    • Run until job completes indicates that the scheduled policy should run until it has completed
    • Suspend migration after x Hours:Minutes. Resume when the next schedule starts indicates the scheduled policy should be suspended after the time specified and resume at the next scheduled interval. Note that this option only applies to the Data Migrator to Cloud.
  3. Make any modifications as necessary.

  4. Click OK.

 

  • Was this article helpful?