Creating specific and detailed migration rules
Before building migration rules, refer to the following information regarding syntax, keywords, connectors, conditionals, and statement order. The following example provides a three-step process for assembling simple, yet specific and detailed rules:
- Start with a simple INCLUDE statement that is specific about what should be migrated, such as:
INCLUDE (<PATH /Temp> AND <FILENAME *.mp3>)
- Refine the INCLUDE statement by adding exceptions to the rule with restrictive EXCLUDE statements. Add these EXCLUDE statements above the INCLUDE, such as:
EXCLUDE (<FILE_SIZE_UNDER 2MB>)
EXCLUDE (<ACTIVE_WITHIN 14>)
- The rule should finally appear this way:
EXCLUDE (<FILE_SIZE_UNDER 2MB>)
EXCLUDE (<ACTIVE_WITHIN 14>)
INCLUDE (<PATH /Temp> AND <FILENAME *.mp3>)
Click on a topic at the left for more information about a specific rule.
Rules syntax
Data migration rules compare a series of INCLUDE and EXCLUDE statements, each qualified by expressions stating the criteria for data migration. The following guidelines govern rule building:
-
At least one INCLUDE or EXCLUDE. Each rule must contain at least one INCLUDE or EXCLUDE statement. Rules consisting only of EXCLUDE statements imply that everything on primary storage should be migrated except what has been specifically excluded.
Note If a rule contains only INCLUDE statements, all items not specified by the INCLUDE statements are excluded.
- Wildcards. The asterisk (*) can be used as a wildcard character to qualify PATH and FILENAME values.
- When used in a PATH value, “*” is only treated as a wildcard if it appears at the end of a value; for example, <PATH /tmp*>.
- In a FILENAME value, a single “*” can appear either at the beginning or the end of the value.
- Multiple instances of the wildcard character are not supported and additional instances in a value definition will be treated as literal characters.
- Bracketed keyword/value pairs. Expressions identifying migration criteria should be enclosed in brackets. All criteria contain a keyword, defining the condition for data migration, followed by a single value of a list of values; for example, <FILENAME *.doc>.
- Evaluation of statement sequence. When using multiple INCLUDE or EXCLUDE statements, they are evaluated using top-down ordering.
- Grouping criteria within statements. Parentheses are used to group the criteria in INCLUDE and EXCLUDE statements; for example, INCLUDE (<PATH /Temp/*>).
- Number of INCLUDE or EXCLUDE statements per line. When using multiple INCLUDE or EXCLUDE statements in a rule, each INCLUDE or EXCLUDE statement must be placed on its own line (multiple INCLUDE and/or EXCLUDE statements may not be put on the same line).
- Separating multiple listed values. When defining multiple values in a FILENAME list, use a comma to separate values; for example, INCLUDE (<FILENAME *.mp3,*.wav,*.wmv>).
-
Characters requiring escaping. the following characters need to be escaped with a backslash (\) when used as a part of PATH or FILENAME values: \ (backslash), > (greater than), and , (comma); for example, INCLUDE (<FILENAME *a\,b> OR <PATH /tmp/\>ab>).
Note Backslashes used when specifying a domain and user name should not be escaped (double backslashes are not required when specifying domain_name\user_name).
- Forward slash (/) reserved as a path separator. The forward slash (/) is used as a path separator; as such, it must not be used in a FILENAME list.
- Evaluation of absent PATH. If a PATH element is not specified in a statement, the statement will apply to the entire file system or virtual volume defined in the data migration path.
- Quotation mark usage. Quotation marks (“) are not allowed around a FILENAME or PATH list.
Keywords
The following table describes the keywords and their related values that can be used to build rule statements. Each keyword can be defined in the rule with an INCLUDE or EXCLUDE statement to indicate how the keyword values are to be applied.
|
Keyword | Value(s) |
| FILENAME |
Names and types of files contained in the rule. Separate multiple names by commas. FILENAME values may start or end with a “*” wildcard character to indicate all files starting/finishing with specific characters. Usage: FILENAME will often be used with an INCLUDE statement to ensure that non-essential files are migrated to secondary storage. It can also be used with an EXCLUDE statement to prevent specific important data sets from being migrated. For example: <FILENAME *.mp3,*.txt,*filename, filename*> |
| PATH | Specifies literal paths to which a rule applies. Values must be full paths, starting with a forward slash (/). Separate multiple paths by commas. PATH values may end with a “*” wildcard character to indicate all subdirectories under the specified path.
Usage: When used in an INCLUDE statement, PATH specifies directories to migrate. This is useful when migrating less-critical directories such as temp or home directories. When used in an EXCLUDE statement, directories can be excluded from migration, leaving all the files within on primary storage. For example: <PATH /temp/*,/home*,/other/dir*> |
| USERNAME |
Specifies user names to which a rule applies. Values must be valid Windows or UNIX user names. Separate multiple names by commas. Usage: When used in an INCLUDE statement, USERNAME specifies the user name of file owners whose files are to be migrated. This is useful when migrating files owned by a particular user. When used in an EXCLUDE statement, users can be excluded from migration, leaving all the files owned by the specified user(s) on primary storage. Windows user names are specified in the form domain\username. Note Backslashes
in user names should not be escaped (double backslashes are
not required). For example: jjames, myco\smithr, myco\wsmith Note The USERNAME keyword is not supported for cloud
data migration rules.
|
| FILE_SIZE_OVER |
Identifies a subset of files in a rule with sizes crossing an upper threshold. The threshold value is appended to the keyword and defined by the threshold size in B, KB, MB, or GB. Usage: This will likely be used with INCLUDE statements to ensure files of very large sizes are migrated to secondary storage. For example: <FILE_SIZE_OVER 4GB> |
| FILE_SIZE_UNDER |
Identifies a subset of files in a rule with sizes crossing a lower threshold. The threshold value is appended to the keyword and is defined by the threshold size in B, KB, MB, or GB. Usage: This will usually be used in an EXCLUDE statement to ensure that very small files are not migrated en masse. Migrating small files that take up little space provides minimal value in extending the efficiency of primary storage. For example: <FILE_SIZE_UNDER 10KB> |
| OLDER_THAN |
Identifies files that were created more than a specified number of days in the past (files older than x days). The value is appended to the keyword and defines the number of days within which the activity has occurred from the current date and time Usage: Used primarily in INCLUDE statements to ensure that older files are migrated. For example: <OLDER_THAN 28> |
| NEWER_THAN |
Identifies files that were created less than a specified number of days in the past (files newer than x days). The value is appended to the keyword and defines the number of days within which the activity has occurred from the current date and time. Usage: Used primarily in EXCLUDE statements to ensure that newer files are not migrated. For example: <NEWER_THAN 14> |
| INACTIVE_OVER |
Identifies files that have not been accessed within a specific number of days. A file’s last access time is updated whenever the file is read or modified. The value is appended to the keyword and defines the number of days of inactivity from the current date and time. Usage: Used primarily in INCLUDE statements to ensure that older, less frequently used files are migrated. For example: <INACTIVE_OVER 21> |
| ACTIVE_WITHIN |
Identifies files accessed within a specific number of previous days. A file’s last access time is updated whenever the file is read or modified. The value is appended to the keyword and defines the number of days of inactivity from the current date and time. Usage: Used primarily in EXCLUDE statements to prevent actively used files from being migrated. For example: <ACTIVE_WITHIN 30> |
| UNCHANGED_OVER |
Identifies files not modified within a specific number of previous days. A file’s modification time is updated whenever the file’s contents have been changed. The value is appended to the keyword and defines the number of days of inactivity from the current date and time. Usage: Used primarily in INCLUDE statements to ensure that older, less frequently used files are migrated. For example: <UNCHANGED_OVER 14> |
| CHANGED_SINCE |
Identifies files modified within a specific number of previous days. A file’s last access time is updated whenever the file’s contents have been changed. The value is appended to the keyword and defines the number of days of inactivity from the current date and time. Usage: Used primarily in EXCLUDE statements to prevent actively used files from being migrated. For example: <CHANGED_SINCE 7> |
Connectors
Statements can combine multiple criteria, as follows:
-
AND indicates that both statements must be satisfied. For example, in the statement:
INCLUDE (<FILENAME *.mp3> AND <FILE_SIZE_OVER 5GB>)
both conditions must be true in order for the statement to be true.
-
OR indicates that only one statement needs to be satisfied. For example, for the same statement, replacing AND with OR:
INCLUDE (<FILENAME *.mp3> OR <FILE_SIZE_OVER 5GB>)
only one condition needs to be true for the statement to be true.
While AND requires both conditions to be true, OR only requires that either condition to be true.
Conditionals
The following table shows a set of rules with explanations. The syntax can easily be broken down into cause and effect statements, identified by IF and THEN connectors.
| Rule | Description |
| INCLUDE (<FILENAME *.doc>) | IF the file is a .doc file, THEN include it for migration. |
| EXCLUDE (<PATH /mydir/*>) | IF the path is the /mydir directory THEN exclude it from migration. |
| INCLUDE (<FILENAME *.prj> AND <FILE_SIZE_OVER 4GB>) | IF the file is a .prj file AND the .prj file is over 4 GB in size, THEN include it for migration. |
| INCLUDE (<PATH /unimportant>) | IF the path is the /unimportant directory THEN include it for migration. |
| EXCLUDE (<FILE_SIZE_OVER 100GB>) INCLUDE (<FILE_SIZE_OVER 12GB>) | IF files are larger than 12 GB but smaller than 100 GB in size, THEN include them for migration. |
Statement order
Statement order is critical. Statements are evaluated top-down, starting with the first statement defined. Therefore, as the following examples illustrate best practice usually specifies EXCLUDE statements at the top of the rule.
Rule scenario A:
INCLUDE (<PATH /Temp> AND <FILENAME *.mp3>)
EXCLUDE (<ACTIVE_WITHIN 14>)
EXCLUDE (<FILE_SIZE_UNDER 2MB>)
The above rule is interpreted as:
- IF path name includes /Temp AND file name is *.mp3 THEN MIGRATE.
- IF file is active less than 14 days AND less than 2 MB in size THEN EXCLUDE.
In scenario A, all the .mp3 files under /Temp will be migrated based on the first INCLUDE statement. Statements 2 and 3 are disregarded since they are evaluated after the more inclusive INCLUDE statement that has already added what rules 2 and 3 are trying to exclude.
Rule scenario B:
If the same rules were ordered differently:
EXCLUDE (<FILE_SIZE_UNDER 2MB>)
EXCLUDE (<ACTIVE_WITHIN 14>)
INCLUDE (<PATH /Temp> AND <FILENAME *.mp3>)
The above rule is interpreted as:
- IF file is less than 2 MB in size OR active less than 14 days THEN EXCLUDE.
- IF path name includes /Temp AND file name is *.mp3 THEN MIGRATE.
In this scenario, only .mp3 files greater than 2 MB in size that have been inactive for greater than 14 days will be migrated.