Skip to main content
Close

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Using deduplication file system

  1. Last updated
  2. Save as PDF

This section describes how to use deduplication on a file system.

Parent Topic
  • File Services Administration Guide

    This guide explains about file system formats, and provides information about creating and managing file systems, and enabling and configuring file services (file service protocols).

Child Topics

Determining sufficient space for dedupe conversion
File systems that have not been formatted with dedupe support must be converted before they will support deduplication. However, the conversion process is an offline process which can sometimes fail because there is not sufficient scratch or free space available. In this scenario, you must expand the file system and then re-attempt the conversion.
Note File systems formatted or expanded in releases newer than dedupe will have the extra scratch space required for conversion and the following procedure is unnecessary.

Before you start the dedupe conversion, use the following procedure to determine if a file system has sufficient scratch and free space available. It is not an offline procedure.

Procedure

  1. Call customer support to obtain the instructions and dev password to execute the fs-capacity-info dev command.

    This command can be run on a mounted file system.

  2. Run the command with the name of the file system that is to be checked (for example, fs-capacity-info f filesystem1).

    This command generates the following sample output: 
    fs-capacity-info/fs3: Underlying length : 0x017666400000 bytes, 0x00bb332000 sectors, 0x017666400 blocks 
fs-capacity-info/fs3: Formatted length : 0x017636000000 bytes, 0x00bb1b0000 sectors, 0x017636000 blocks 
fs-capacity-info/fs3: Scratch-space offset : 0x017636000000 bytes, 0x00bb1b0000 sectors, 0x017636000 blocks 
fs-capacity-info/fs3: Scratch-space length : 0x000030400000 bytes, 0x0000182000 sectors, 0x000030400 blocks 
fs-capacity-info/fs3: Reservable available space : 0x01748e51f000 bytes, 0x00ba4728f8 sectors, 0x01748e51f blocks 
fs-capacity-info/fs3: Physical available space : 0x0175ff4bb000 bytes, 0x00baffa5d8 sectors, 0x0175ff4bb blocks 
fs-capacity-info/fs3: Space required for RCB conversion : 0x0000e9e1c000 bytes, 0x000074f0e0 sectors, 0x0000e9e1c blocks 
fs-capacity-info/fs3: Scratch-space required for RCB conversion: 0x00002eccc800 bytes, 0x0000176664 sectors, 0x00002eccc blocks

  3. Locate the "Scratch-space length" value. In this example, 0x000030400000 bytes.

  4. Locate the "Scratch-space required for RCB conversion" value. In this example, 0x00002eccc800 bytes.

  5. If the Scratch-space length is greater than Scratch-space required for RCB conversion, then there is enough scratch space. In this example, 0x000030400000 is greater than 0x00002eccc800, indicating that there is enough scratch space for the conversion.

  6. Locate the "Reservable available space" value. In this example, 0x01748e51f000 bytes.

  7. Multiply the Space required for RCB conversion value by 1.6. In this example, the value is 0x000176360000 bytes.

  8. If Reservable available space is greater than Space required for RCB conversion * 1.6, then there is enough free space. 0x01748e51f000 is greater than 0x000176360000, indicating there is enough free space for the conversion.

Parent Topic

Preparing for dedupe conversion

In addition to the space required for the ondisk component (for example, 45 GB) of the dedupe index, a conversion process may also require additional scratch space that is normally carved out of the underlying partition when the file system is first formatted or expanded. It will not utilize storage from the free space available on the file system (for example, free space reported by df).

If you receive the following failure message during the dedupe conversion process, this indicates that additional scratch space needs to be setup. 

2013-03-11 09:15:49.744-04:00 1 MMB1 fs-convert-to-support-dedupe request for evs12-fs-d01-tgt: Checking if sufficient space is free on target file system... 2013-03-11 09:15:49.744-04:00 1 MMB1 fs-convert-to-support-dedupe request for evs12-fs-d01-tgt: Scratch space requires 4.21 GB. 2.31 GB is available. 2013-03-11 09:15:49.744-04:00 1 MMB1 fs-convert-to-support-dedupe request for evs12-fs-d01-tgt: Expand the file system size before re-submitting the conversion

You can resolve this issue by expanding the file system. Usually one chunk is all that is required. Any storage beyond what is required for the scratch space from the chunk will be made available to the file system. Once this has been done, you can retry the conversion process.

Parent Topic

Viewing the deduplication file system page
To view the Deduplication file system page:

Procedure

  1. Navigate to Home Storage Management File Systems Dedupe File Systems. The Deduplication page appears.

    GUID-D7466663-E977-4E17-A9D0-F1AFC53FD99E-low.png

    The following table describes the fields on this page:

    Field/Item Description
    File System Name Use to search by file system name.
    EVS Use to search by a particular EVS.
    Show

    Use to search by type of file system version:

    • Converted file systems
    • Not converted file systems
    • All file systems
    Filter Use to initiate your search criteria.
    File Systems Displays a list of file systems that match the search criteria.
    Used Capacity Displays the total amount of file system space in use, in GiB or TiB and as a percentage of the total.
    Reclaimed Capacity Displays the space savings based on deduplication, presented in bytes and in a percentage.

    The percentage is the ratio of the Deduped column to the sum of the Deduped and Use columns. Note that the dedupe process begins by taking a temporary snapshot of the file system and then dedupes it.

    Status Displays the status:
    • Disabled: dedupe is disabled on the file system.
    • Enabled: dedupe is enabled on the file system.
    • Not converted: The file system needs to be converted in order to use the dedupe capability on the file system.
    • Not assigned to an EVS: File system is not assigned to an EVS and cannot be converted until the file system is assigned to an EVS.
    • EVS name (disabled): File system cannot be converted because the EVS is disabled. Enable the EVS first.
    • Converting: Starting to convert a file system to support deduplication.
    • Full run: Performing a full deduplication on the file system.
    • Incremental run: Performing an incremental deduplication on the file system.
    EVS EVS to which the file system is assigned.
    Last Run The date and time of the last dedupe run.
    enable Dedupe-enables an existing file system.
    disable Disables dedupe on an dedupe-enabled file system.
    convert Starts the conversion process on a file system that needs conversion to support and enable dedupe. See the fs-convert-to-support-dedupe man page for important information and considerations before starting the conversion process. For example:

    • The file system should be in an unmounted state.
    • The file system must have sufficient space.
    File System Returns to the File System page.
    Active Tasks Displays the current status of the conversion process.
Parent Topic

Enabling dedupe for a file system
To enable dedupe for a file system:

Procedure

  1. Navigate to Home Storage Management File Systems Dedupe File Systems. The Deduplication page appears.

  2. Enter the search criteria in the Filter section of the page and then click Filter.

  3. Under the File System label, fill the check box next to the file system to be enabled. You can enable one or more file systems at a time.

  4. Click enable.

    The system immediately starts the dedupe-enable process. The Status column displays Enabled to reflect this action.
Parent Topic

Converting a file system to enable dedupe
You can convert an existing file system to support and enable dedupe. After the conversion process takes place, the file system is dedupe-enabled. Converting a file system can take some time and it is recommended that you plan accordingly for the conversion time. It is recommended that you convert one file system at a time. See the fs-convert-to-support-dedupe man page for important information and considerations before starting the conversion process. For example:

  • The file system should be in an unmounted state.
  • The file system must have sufficient space.

Procedure

  1. Navigate to Home Storage Management File Systems Dedupe File Systems. The Deduplication page appears.

  2. Select Not converted file systems from the Show list to display file systems do not have dedupe support enabled.

  3. Click Filter.

  4. The system displays the file systems that need conversion in order to be dedupe-enabled

  5. Fill in the check box next to the file system to convert. It is recommended that you one file system at a time.

  6. Click Convert and read the messages in the dialog that appears.

  7. After you have read the message and ensure that you want to proceed with conversion, click OK.

  8. Click Active Tasks to view the current conversion status.

    After the conversion is done, the file system is dedupe capable and the file system is now queued for a full dedupe run. The dedupe process will start when the file system is queued for dedupe. The Status column displays Enabled.

    If the status remains in the Needs Conversion status, check the Events page. Navigate to Home Status & Monitoring Event Log. This log reports any conversion errors. For example, an error may occur if there is not sufficient space for the file system to be converted or if the user-data-write-policy of the file system is set to anything other than never-overwrite. See the following CLI man page for more information:

    wfs-user-data-write-policy

Parent Topic

 

  1. Back to top
  • Was this article helpful?

Recommended articles

  1. Article type
    Reference
    Page Type
    Documentation Article
  2. Tags
    1. administrator
    2. HNAS 13.9