Managing filesystems and object stores
The details on how to view and manage filesystems, object stores, and filesystem groups using the GUI are provided.
Overview
The management of object stores, filesystem groups and filesystems is an integral part of the successful running and performance of the Content Software for File system and overall data lifecycle management.
The pages in this section cover the following subjects:
Managing object stores
Using the GUI, you can perform the following actions:
Editing default object stores using the GUI
Object store buckets can reside in different physical object stores. To achieve good QoS between the buckets, Weka requires to map the buckets to the physical object store.
You can edit the default local and remote object stores to meet your connection demands. When you add an object store bucket, you apply the relevant object store on the bucket.
Editing the default object store provides you with the following additional advantages:
- Set restrictions on downloads from a remote object store. For on-premises systems where the remote bucket is in the cloud, to reduce the cost, you set a very low bandwidth for downloading from a remote bucket.
- Ease of adding new buckets. You can set the connection parameters on the object store level and, if not specified differently, automatically use the default settings for the buckets you add.
Procedure
From the menu, select Manage > Object Stores.
On the left, select the pencil icon near the default object store you want to edit
On the Edit Object Store dialog, set the following:
- Type: Select the type of object store.
- Buckets Default Parameters: Set the protocol, hostname, port, bucket folder, authentication method, region name, access key, and secret key.
NoteIf using the AWS object store type and access from the Weka EC2 instances to the object store is granted by the IAM roles, it is not mandatory to set the access and secret keys in the Edit Object Store dialog.
Viewing object stores using the GUI
The object store buckets are displayed on the Object Stores page. Each object store indicates the status, bucket name, protocol (HTTP/HTTPS), port, region, object store location (local or remote), authentication method, and error information (if exists).
From the menu, select Manage > Object Stores.
The following example shows two object store buckets.
Adding an object store using the GUI
Add object store buckets to be used for tiering or snapshots.
Procedure
From the menu, select Manage > Object Stores.
Select the +Create button.
In the Create Object Store Bucket dialog, set the following:
- Name: Enter a meaningful name for the bucket.
- Object Store: Select the location of the object store. For tiering and snapshots, select the local object store. For snapshots only, select the remote object store.
- Type: Select the type of object store.
- Buckets Default Parameters: Set the protocol, hostname, port, bucket folder, authentication method, region name, access key, and secret key.
To validate the connection to the object store bucket, select Validate.
Optional: If your deployment requires a specific upload and download configuration, select Advanced, and set the parameters:
- Download Bandwidth: Object store download bandwidth limitation per core (Mbps).
- Upload Bandwidth: Object store upload bandwidth limitation per core (Mbps).
- Max concurrent Downloads: Maximum number of downloads concurrently performed on this object store in a single IO node.
- Max concurrent Uploads: Maximum number of uploads concurrently performed on this object store in a single IO node.
- Max concurrent Removals: Maximum number of removals concurrently performed on this object store in a single IO node,
- Enable Upload Tags: Whether to enable object-tagging or not.
Select Create.
NoteIf an error message about the object store bucket configuration appears, to save the configuration, select Create Anyway.
Editing an object store using the GUI
You can modify the object store bucket parameters according to your demand changes over time.
Procedure
From the menu, select Manage > Object Stores.
Select the three dots on the right of the object store you want to modify, and select Edit.
In the Edit Object Store Bucket dialog, modify the details, and select Update.
Deleting an object store using the GUI
You can delete an object store bucket if it is no longer required. The data in the object store remains intact.
Procedure
From the menu, select Manage > Object Stores.
Select the three dots on the right of the object store bucket you want to delete, and select Remove.
To confirm the object store bucket deletion, select Yes.
Managing filesystem groups
A filesystem group defines the policy of the drive retention period and the tiering cue time. The Content Software for File system can include up to eight filesystem groups.
Viewing filesystem groups using the GUI
The filesystem groups are displayed on the Filesystems page. Each filesystem group indicates the number of filesystems that use it.
From the menu, select Manage > Filesystems.
The following example shows two filesystem groups defined in the system.
Adding a filesystem group using the GUI
Adding a filesystem group is required when adding a filesystem. If you want to apply a different tiering policy on specific filesystems, you can create more file system groups.
Procedure
From the menu, select Manage > Filesystems.
Select the + sign right to the Filesystem Groups title.
In the Create Filesystem Group dialog, set the following:
- Name: Enter a meaningful name for the filesystem group.
- Drive Retention Period: Set the number of days to keep the data on the SSD before it is copied to the object store. After this period, the copy of the data is deleted from the SSD.
- Tiering Cue: Set the time to wait after the last update, before the data is copied from the SSD and sent to the object store.
Select Create.
Editing a filesystem group using the GUI
You can edit the filesystem group policy according to your system requirements.
Procedure
From the menu, select Manage > Filesystems.
Select the filesystem group you want to edit.
Select the pencil sign right to the filesystem group name.
In the Edit Filesystem Group dialog, update the settings as you need. (See the parameter descriptions in the Add a Filesystem Group topic.
im
Select Update.
Deleting a filesystem group using the GUI
You can delete a filesystem group that is no longer used by any filesystem.
Procedure
From the menu, select Manage > Filesystems.
Select the filesystem group you want to delete.
Verify that the filesystem group is not used by any filesystems (indicates 0 filesystems).
Select the Remove icon. In the pop-up message, select Yes to delete the filesystem group.
Managing filesystems
How to view and manage filesystem groups using the GUI.
Viewing filesystems using the GUI
The filesystems are displayed on the Filesystems page. Each filesystem indicates the status, tiering status, backup status, encryption status, SDD capacity, total capacity, and the filesystem group used.
From the menu, select Manage > Filesystems.
Adding a filesystem using the GUI
When creating a Weka system on-premises, it does not contain any filesystem. You need to add it and set its properties, such as capacity, group, tiering, thin provisioning, and encryption.
When creating a Weka system in AWS using the cloud formation, the Weka system contains a default filesystem, which is provisioned with the maximum capacity. If your deployment requires more filesystems with different settings, first reduce the provisioned capacity of the default filesystem, and then add a filesystem with the properties that meet your specific needs.
Before you begin
- Verify that the system has free capacity.
- Verify that a filesystem group is already set.
- If tiering is required, verify that an object store bucket is set.
- If encryption is required, verify that a KMS is configured.
Procedure
From the menu, select Manage > Filesystems.
Select the +Create button.
In the Create Filesystem dialog, set the following:
- Name: Enter a meaningful name for the filesystem.
- Group: Select the filesystem group that fits your filesystem.
- Capacity: Enter the storage size to provision, or select Use All to provision all the free capacity.
Optional: If Tiering is required and an object store bucket is already defined, select the toggle button, and set the details of the object store bucket:
- Object Store Bucket: Select a predefined object store bucket from the list.
- Drive Capacity: Enter the capacity to provision on the SSD, or select Use All to use all free capacity.
- Total Capacity: Enter the total capacity of the object store bucket, including the drive capacity.
Optional: If Thin Provision is required, select the toggle button, and set the minimum and the maximum capacity of the SSD to use for thin provisioning.
Optional: If Encryption is required and your Weka system is deployed with a KMS, select the toggle button.
Select Save.
Editing a filesystem using the GUI
You can modify the filesystem parameters according to your demand changes over time. The parameters that you can modify include, filesystem name, capacity, tiering, and thin provisioning (but not encryption).
Procedure
From the menu, select Manage > Filesystems.
Select the three dots on the right of the filesystem you want to modify, and select Edit.
In the Edit Filesystem dialog, modify the parameters according to your requirements. (See the parameter descriptions in the Add a filesystem topic.
Select Save.
Deleting a filesystem using the GUI
You can delete a filesystem if its data is no longer required. Deleting a filesystem does not delete the data in the tiered object store bucket.
Procedure
From the menu, select Manage > Filesystems.
Select the three dots on the right of the filesystem you want to delete, and select Remove.
To confirm the filesystem deletion, enter the filesystem name and select Confirm.
Attaching or detaching object stores to or from filesystems
Two local object store buckets can be attached to a filesystem, but only one of the buckets is writable. A local object store bucket is used for both tiering and snapshots. When attaching a new object store bucket to an already tiered filesystem, the existing object-store bucket becomes read-only, and the new object store bucket is read/write. Multiple object stores allow a range of use cases, including migration to different object stores, scaling of object store capacity, and increasing the total tiering capacity of filesystems.
When attaching an object store bucket to a non-tiered filesystem, the filesystem becomes tiered.
Detaching a local object-store bucket from a filesystem migrates the filesystem data residing in the object store bucket either to the writable object store bucket (if one exists) or to the SSD.
When detaching, the background task of detaching the object-store bucket begins. Detaching can be a long process, depending on the amount of data and the load on the object stores.
Attachment of a local object store bucket to a filesystem
Two local object store buckets can be attached to a filesystem, but only one of the buckets is writable. A local object store bucket is used for both tiering and snapshots. When attaching a new object store bucket to an already tiered filesystem, the existing object-store bucket becomes read-only, and the new object store bucket is read/write. Multiple object stores allow a range of use cases, including migration to different object stores, scaling of object store capacity, and increasing the total tiering capacity of filesystems.
When attaching an object store bucket to a non-tiered filesystem, the filesystem becomes tiered.
Detachment of a local object store bucket from a filesystem
Detaching a local object-store bucket from a filesystem migrates the filesystem data residing in the object store bucket either to the writable object store bucket (if one exists) or to the SSD.
When detaching, the background task of detaching the object-store bucket begins. Detaching can be a long process, depending on the amount of data and the load on the object stores.
Migration to a different object store
When detaching from a filesystem tiered to two object store buckets, only the read-only object-store bucket can be detached. In such cases, the background task copies the relevant data to the writable object store.
Un-tiering a filesystem
Detaching from a filesystem tiered to one object store bucket un-tiers the filesystem and copies the data back to the SSD.
On completion of detaching, the object-store bucket does not appear under the filesystem when using the weka fs command. However, it still appears under the object store and can be removed if it is not being used by any other filesystem. The data in the read-only object-store bucket remains in the object-store bucket for backup purposes. If this is unnecessary or the reclamation of object store space is required, it is possible to delete the object-store bucket.
Migration considerations
When migrating data (using the detach operation) you would like to copy only the necessary data (to reduce migration time and capacity), however, you may want to keep snapshots in the old object-store bucket.
Migration workflow
The order of the following steps is important.
Procedure
Attach a new object store bucket (the old object store bucket becomes read-only).
Delete any snapshot that does not need to be migrated. This action keeps the snapshot on the old bucket but does not migrate its data to the new bucket.
Detach the old object store bucket.
NoteIf you perform the workflow steps in a different order, the snapshots can be completely deleted from any of the object store buckets. It is also possible that the snapshots are already in a migration process, and cannot be deleted until the migration is completed.
Attaching a remote object store bucket
One remote object-store bucket can be attached to a filesystem. A remote object store bucket is used for backup only, and only snapshots are uploaded to it using Snap-To-Object. The snapshot uploads are incremental to the previous one.
Detaching a remote object store bucket
Detaching a remote object-store bucket from a filesystem keeps the backup data within the bucket intact. It is still possible to use these snapshots for recovery.
Attaching an object store bucket to a filesystem using the GUI
Before you begin
Verify that an object store bucket is available.
One remote object-store bucket can be attached to a filesystem. A remote object store bucket is used for backup only, and only snapshots are uploaded to it using Snap-To-Object. The snapshot uploads are incremental to the previous one.
Procedure
From the menu, select Manage > Filesystems.
On the Filesystem page, select the three dots on the right of the filesystem that you want to attach to the object store bucket. Then, from the menu, select Attach Object Store Bucket.
On the Attach Object Store Bucket dialog, select the relevant object store bucket.
Detaching an object store bucket from a filesystem using the GUI
Detaching a local object store bucket from a filesystem migrates the filesystem data residing in the object store bucket either to the writable object store bucket (if one exists) or to the SSD.
Procedure
From the menu, select Manage > Filesystems.
On the Filesystem page, select the filesystem from which you want to detach the object store bucket.
From the Detach Object Store Bucket dialog, select Detach. If the filesystem is attached to two object store buckets (one is read-only and the other is writable), you can detach only the one that is read-only. The data of the detached object store bucket is migrated to the writable object store bucket.
In the message that appears, to confirm the detachment, select Yes.
If the filesystem is tiered and only one object store is attached, detaching the object store bucket opens the following message:
Object store buckets usually expand the filesystem capacity. Un-tiering of a filesystem requires adjustment of its total capacity. Select one of the following options:
- Increase the SSD capacity to match the current total capacity.
- Reduce the total filesystem capacity to match the SSD capacity or the used capacity (the decrease option depends on the used capacity).
- Configure a different capacity.
NoteUsed capacity must be taken into account. Un-tiering takes time to propagate the data from the object store to the SSD. When un-tiering an active filesystem, to accommodate the additional writes during the detaching process, it is recommended to adjust to a higher value than the used capacity.Select the option that best meets your needs, and select Continue.
In the message that appears, select Detach to confirm the action.