Configuration considerations when using HDP with NAS
When your system must work with HDP-based storage, there are things to remember that affect both the storage and the NAS server. For detailed information on using HDP with your particular storage, consult the HDP software documentation.
When using HDP pools with NAS servers, consider the following:
- All current Hitachi Vantara storage supports HDP as a licensed option.
- All NAS servers support HDP without requiring a server-side license.
- The HDP software has no effect on protocol clients, such as NFS, and SMB.
- Each DP-Vol draws space from multiple pool volumes, which helps to relieve the storage bottleneck by distributing I/O to more disks.
- If you attempt to create or expand a file system and either there are not enough free chunks on the span or there is not enough free disk space on the DP pool, the server will make space by recycling one or more deleted file systems, if the span contains any.
NoteWhen you recycle or delete a file system, the amount of free space shown in a storage configurator such as Hitachi Ops Center Administrator or Hitachi Storage Navigator does not reflect the new space until after you have run the span-unmap-vacated-chunks command. Do not run this command unnecessarily because performance may be reduced.
- Recycling a file system causes the chunks that stored the file system data to be moved into the vacated-chunks list, which contains records of which freed chunks were used by which file system.
- Creating or expanding a file system draws space from the vacated chunks list, if any is available, without using new space from the HDP pool. Any further space is pre-allocated at once.
- Writing to a file system costs no space because that space was pre-allocated.
- When creating or expanding a storage pool on HDP pools, you must use the DP-Vols from a single DP pool. This rule applies whether you are using the CLI or the NAS Manager. Later storage pool expansions can be done using storage from different DP pools.
Using HDP to expand a span
Using HDP to add space provides additional benefits.
These benefits include:
- You can add disks in small increments, even just a single pool volume.
- Data gets restriped.
- Span gets faster and performance remains almost even.
Consider the following use case for using HDP to expand a span:
If you originally created a pool containing 10 TiB of real storage and eight DP-Vols of 2.5 TiB each, totalling 20 TiB, the pool is over-committed by a ratio of 2:1. As always, a storage pool (span on the CLI) resides on the DP-Vols. As time goes by, you make a series of expansions of 4 TiB each by adding new parity groups or pool volumes. The first expansion increases the amount of real storage in the pool to 14 TiB and the second expansion takes it to 18 TiB.
After each of these expansions, no further action is necessary. However, after a third 4 TiB expansion, the pool contains 22 TiB of real storage, but its DP-Vols total only 20 TiB. As a result, 2 TiB of the disk space that you have installed are inaccessible to the server.
More DP-Vols are needed, but any expansion should always add at least as many DP-Vols as were provided when the span was created. You must therefore create a further eight DP-Vols, preferably of the same 2.5 TiB as the original ones, and add them to the storage pool by using the span-expand command or NAS Manager equivalent. This addition brings the total DP-Vol capacity to 40 TiB. No further DP-Vols will be necessary unless the real disk space in the pool is expanded beyond 40 TiB.
Configuration guidelines for HNAS with HDP
Follow these configuration guidelines for best results.
On the storage:
- Make every new HDP pool thinly provisioned.
- Create enough DP-Vols to meet the expected size of the storage pool and to provide enough queue depth (minimum of four).
- Limit each HDP pool to hosting only a single NAS server storage pool, particularly with low performing HDD’s; when using flash media using multiple spans per HDP pool is more acceptable. With the exception of tiered file systems, if you need fifty NAS server storage pools, create fifty HDP pools.
- Do not share an HDP pool between two or more clusters, as disparate clusters are unaware of the other’s storage needs; additionally, troubleshooting performance concerns is exacerbated due to unknown workloads.
- Do not share an HDP pool between an HNAS system and a foreign server, as disparate clusters are unaware of the other’s storage needs; additionally, troubleshooting performance concerns is exacerbated due to unknown workloads.
On the NAS server:
- To avoid server timeouts when creating a new NAS server storage pool/span, wait for the HDP pool to finish formatting before creating the NAS server storage pool/span using NAS Manager or the span-create command. NoteIf the HDP pool has finished formatting, but the NAS server does not detect the new DP-Vols, run the scsi-refresh CLI command so the NAS server will detect the new DP-Vols, , though this should not be necessary when HMO 7 is enabled on the target port host group.
- For tiered file systems, you may either use two HDP pools when creating the NAS server storage pool that will host the tiered file systems or one can use a single HDP pool and pin DP Vols to the highest performing disks and use those as HNAS Tier 0.
- Do not mix HDP DP-Vols and plain parity groups in a single NAS server storage pool/span. However, it is acceptable for some NAS server storage pools/spans to use HDP DP-Vols while other NAS server storage pools use parity groups.
- For best performance, when creating a new NAS server storage pool based on HDP DP-Vols, specify all the DP-Vols in a single span-create command or NAS Manager equivalent. Do not create the storage pool on just a few of the DP-Vols and then make a series of small expansions.
- Create as many file systems as needed and expand them as often as necessary (or allow them to auto-expand). For maximum flexibility and responsiveness, create small file systems and allow them to auto-expand as data is written to them.
Note The maximum size of a newly created file system on an HDP pool is 1 TiB, but file system expansions can increase the file system size to the maximum limit supported by your NAS server and underlying storage.
Disable zero page reclaim
HDP offers the ability to unmap individual pages within a file system. This capability is called zero page reclaim.
Consult the Hitachi Dynamic Provisioning software documentation for more information about zero page reclaim.
Storage pool chunks
Storage pools are made up of multiple small allocations of storage called “chunks.”
The size of the chunks in a storage pool is defined when the storage pool is created, and a guideline chunk size is set. The guideline chunk size is between 500 MiB and 18 GiB. It determines the maximum size that the storage pool (span) can ever reach, because a storage pool can contain up to a maximum of 60,000 chunks.
Chunk size is an important consideration when creating storage pools. The size of the chunks in a storage pool is defined when the storage pool is created, and a guideline chunk size is set. The guideline chunk size is between 500 MiB and 18 GiB. It determines the maximum size that the storage pool (span) or file system can ever reach, because a storage pool or file system can contain up to a maximum of 60,000 chunks.
Larger chunks maximize scalability, but smaller chunks provide more granular file system expansion, because a file system always expands by a whole number of chunks.
If you create a storage pool using NAS Manager, the guideline chunk size is 18 GiB (the maximum allowable size). The default chunk size set by NAS Manager can be larger than the guideline chunk size calculated and suggested by the server if you created the storage pool using the CLI.
Storage pool pre-allocation
In normal operation, on HDP, the NAS Server prevents you from running out of real disk space. If creating or expanding a file system would use more space than is available on the parity groups or pool volumes in the DP pool, the operation fails safely. Without this check, if the DP pool ran out of space, write operations would fail and file systems would be forcibly unmounted.
An important part of this strategy is to pre-allocate real disk space for all the HDP pages in a chunk when that chunk is allocated to a file system. The server achieves this pre-allocation by writing one non-zero block to each HDP page. Without this pre-allocation, the free space on the DP pool would fall some time later, when data was first written to the HDP pages in the chunk; meanwhile, the server would overestimate the amount of free space on the DP pool, and would be in danger of running out of space.
Although pre-allocation protects your system from running out of disk space on a DP pool, it has a number of disadvantages:
- It slows down filesystem creation and expansion.
- Owing to the amount of time that it takes, the system limits the size of a new filesystem or a single manual expansion.
- Pre-allocation writes places stress on the storage, and flushes data out of the cache.
- Pre-allocation is not a sufficient safeguard if your storage system uses FMDs that perform data compression. On such systems, deleting compressible data from a file system and writing incompressible data in its place can cause the system to run out of space. The Administrator is therefore responsible for monitoring free space, expanding file systems and adding physical media as required.
It is possible to disable pre-allocation and enable the Administrator to manage free space by using the span-hdp-preallocation command. For further information, see the span-hdp-preallocation man page in the Command Line Reference.
You can re-enable pre-allocation at any time using the same command. However, because some chunks may recently have been allocated without the pre-allocations taking place, some HDP pages in the file systems may not yet be mapped to real storage, and the server may overestimate the amount of free space available on one or more DP pools. It is therefore necessary to prevent filesystem auto-expansion until the server has written to all of the HDP pages in allocated chunks and obtained an accurate assessment of the amount of free space on DP pools.
Using HDP pools with a NAS server
You must configure the NAS server so that the HNAS software and HDP software can work together.
The high-level process for using HDP pools with a NAS server is as follows:
Procedure
Configure the HDP pools and DP-Vols on the storage:
Use your storage configurator to create an HDP pool containing sufficient pool volumes to meet your immediate requirements for capacity and performance.
Using your storage configurator, create DP-Vols on the HDP pool. The total capacity of the DP-Vols should significantly exceed that of the pool volumes in order to fulfil future storage requirements. There should be enough DP-Vols to provide enough queue depth for good performance. As a guideline, DP-Vols usually have a capacity of 8 TiB.
Place the new DP-Vols into host groups and assign host LUNs to them. Before assigning host LUNs, enable Host Mode Option 7 and 68 on every host group so that the server can detect the new DP-Vols automatically. Alternatively, assign host LUNs to the DP-Vols and then run the server's scsi-refresh command.
Use the HDP-based storage on the NAS server:
Allow access to the DP-Vols (SDs) on the NAS server. You can allow access using NAS Manager or the command line interface (see the sd-allow-access command).
Create the NAS server storage pool from the HDP DP-Vols.
NoteIf you plan to create tiered file systems, you must create a tiered storage pool using DP-Vols from two HDP pools.To display a table that relates server device identifiers to DP-Vol internal LUNs, use the sd-list --hdp command. This information is useful when you create the storage pool. See the span-create man page.Create the file system on the storage pool.
Format the file system.
Mount the file system.
Performing large file system expansions
Large file system expansions should not be performed when the system is heavily loaded.
The system is heavily loaded when:
- experiencing heavy I/O load, especially write load
- rebuilding a parity group or pool volume after a disk failure
- formatting new disk space
- zero-initializing space after recent use of the span-unmap-vacated-chunks command.
Therefore, if a large expansion is requested, the filesystem-expand command prompts you to supply the --storage-is-lightly-loaded switch and will not proceed without it.
Upgrading from older HNAS systems
Any pre-existing storage pool (span in the CLI) should be left thickly provisioned after a recent upgrade.
When upgrading from an older version of an HNAS system, be aware that certain conditions produce the results and restrictions described in this section.
The conditions:
- A storage pool (span in the CLI) that was created in an earlier release violates the restriction that SDs in each stripeset must come from one HDP pool.
- A storage pool (span) contains a mixture of HDP DP-Vols and plain parity groups. This configuration is unsupported.
The following results are produced:
- Events are logged at boot time
- The span-list and trouble span will issue warnings
- Some operations fail cleanly, for example:
- You cannot create a file system
- You cannot expand a file system
- You cannot delete a file system
- You can still load Cod
- You can still mount file systems