Technology preview
This chapter describes a Technology Preview feature that is included with the software code.
Use of this feature is only recommended in a test environment as it is not covered under any Hitachi Vantara LLC support plans, may not be functionally complete, and is not intended for production use.
For full information about Technology Preview, review the release notes.
Stretched PVC
The provisioning of synchronous replication for conventional systems requires the storage system administrator, cluster administrator, and user to cooperate closely. By using the Stretched PVC feature, the cluster administrator and user can perform the provisioning of synchronous replication by themselves by using the command line tool for Kubernetes or OpenShift.
The following figure gives an overview of a cluster environment in which Stretched PVC is installed.
Requirements for using the Stretched PVC feature
The following table shows the system-wide requirements for using Stretched PVC.
Item | Description |
Cluster configuration | Configure a single Kubernetes or OpenShift cluster (stretched cluster) that spans two sites. Then, allocate one storage system to each site. |
Communication between sites (REST API) | Specify settings so that Storage Plug-in for Containers can communicate with the storage systems at the primary site and at the secondary site by using the REST API. |
Communication between sites (data path) | Specify settings so that each node can perform data communication with the storage systems at the primary site and at the secondary site. Only FC is supported for communication between nodes and storage systems. |
The following is an overview of connecting the server and storage systems in a stretched cluster.
Check the following requirements and settings for both the primary site and secondary site. For details, see the Global-Active Device User Guide.
Item | Description |
Storage system models | The following storage systems are supported:
|
License | Apply the required license by referring to the Global-Active Device User Guide. |
Virtual storage machines | For each storage system, create one virtual storage machine. The virtual storage machine must meet the following requirements:
Note the following when using the same combination of storage systems from multiple clusters:
|
Users and user groups | Create and set a user group that can view only the created virtual storage machines. In addition, create users to which only the created user group is assigned. |
Pool | Create a pool from the pool volumes in the created virtual storage machines. |
|
LDEV | Assign the required number of unused LDEV IDs to the virtual storage machines. For the assigned LDEVs, change the virtual LDEV ID to unassigned (65534). |
Host groups | For each port that you want to use, assign the required number of unused host group IDs for the virtual storage machines. The number of host group IDs must be at least the total number of hosts + 1. |
Remote paths | Create remote paths. The same path group ID must be used for both the primary site and the secondary site. |
Quorum disks | Create quorum disks. The same quorum disk ID must be used for both the primary site and the secondary site. |
To use a regular PVC instead of Stretched PVC in a stretched cluster, specify settings in advance by referring to Resource partitioning. Note the items listed in the following table when specifying settings in advance:
Item | Description |
Resource groups | Create regular resource groups instead of virtual storage machines. |
Host groups | Use ports that are not used by Stretched PVCs. |
When connecting to a Pod, if you want to control the site to which the Pod will be deployed, use the Node Affinity or Node Selector feature of Kubernetes and OpenShift.
The following table shows whether Stretched PVC can be used together with other features:
Feature | Can be used with Stretched PVC? |
Volume snapshot | No |
Volume cloning | No |
Volume expansion | No |
Raw block volume | Yes |
Creating a Stretched PVC
When Storage Plug-in for Containers receives the Secret and StorageClass specifically for the Stretched PVC, the Storage Plug-in for Containers creates volumes at the specified primary site and secondary site and generates a global-active device pair. The volumes created at the primary site and secondary site and the global-active device pair are to be deleted when the Stretched PVC is deleted. Use Storage Navigator to manage the status of the global-active device pair in the period between the creation and deletion of the Stretched PVC.
To create a Stretched PVC:
Procedure
Create a Secret specifically for the Stretched PVC. A sample (secret-sample-stretched.yaml) can be found in the yaml directory.
apiVersion: v1 kind: Secret metadata: name: secret-sample-stretched type: Opaque stringData: primarySerial: "11111" #(1) primaryURL : http://172.16.0.1 #(2) primaryUser : primary-user #(3) primaryPassword : primary-password #(4) secondarySerial: "22222" #(5) secondaryURL : http://172.16.0.2 #(6) secondaryUser : secondary-user #(7) secondaryPassword : secondary-password #(8)
Legend:
(1) Serial number of the storage system at the primary site
(2) URL of the REST API of the storage system at the primary site
(3) User of the storage system at the primary site
(4) Password for the storage system at the primary site
(5) Serial number of the storage system at the secondary site
(6) URL of the REST API of the storage system at the secondary site
(7) User of the storage system at the secondary site
(8) Password for the storage system at the secondary site
Create a StorageClass specifically for the Stretched PVC. A sample (sc-sample-stretched.yaml) can be found in the yaml directory.
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: sc-sample-stretched annotations: kubernetes.io/description: Hitachi Storage Plug-in for Containers provisioner: hspc.csi.hitachi.com reclaimPolicy: Delete volumeBindingMode: Immediate allowVolumeExpansion: false parameters: connectionType: fc replicationType: stretched #(1) quorumID: "30" #(2) primaryPoolID: "10" #(3) primaryPortID: CL1-A,CL2-A #(4) secondaryPoolID: "20" #(5) secondaryPortID: CL1-F #(6) csi.storage.k8s.io/node-publish-secret-name: "secret-sample-stretched" csi.storage.k8s.io/node-publish-secret-namespace: "default" csi.storage.k8s.io/provisioner-secret-name: "secret-sample-stretched" csi.storage.k8s.io/provisioner-secret-namespace: "default" csi.storage.k8s.io/controller-publish-secret-name: "secret-sample-stretched" csi.storage.k8s.io/controller-publish-secret-namespace: "default" csi.storage.k8s.io/node-stage-secret-name: "secret-sample-stretched" csi.storage.k8s.io/node-stage-secret-namespace: "default"Legend:
(1) Parameter indicating that the replication type is Stretched PVC
(2) ID of the Quorum disk
(3) Pool ID of the storage system at the primary site
(4) Port ID of the storage system at the primary site
(5) Pool ID of the storage system at the secondary site
(6) Port ID of the storage system at the secondary site
Create a PVC by specifying the created StorageClass.
Display the created PVC to make sure that STATUS is Bound.
NoteCreating a Stretched PVC takes more time than creating a regular PVC.
Checking information on Stretched PVCs
A PV is created after a Stretched PVC is created. Therefore, by running kubectl describe pv <pv-name> and checking the spec.csi.volumeAttributes field, you can check the resources on the storage systems created by Storage Plug-in for Containers. You can also use this information for troubleshooting.
The following table lists the items displayed in spec.csi.volumeAttributes.
Item | Description |
|
nickname | Nickname of the created LDEV (the same for both the primary site and the secondary site) |
|
primarySerial | Serial number of the primary site |
|
primaryVolumeID | ID of the LDEV created at the primary site |
|
secondarySerial | Serial number of the secondary site |
|
secondaryVolumeID | ID of the LDEV created at the secondary site |
Deleting a Stretched PVC
You can delete a Stretched PVC in the same way as deleting a regular PVC. When a Stretched PVC is deleted, the global-active device pair and the volumes at the primary and secondary sites are also deleted.
Forcibly deleting a Stretched PVC
To delete a Stretched PVC, the storage systems at the primary site and secondary site must be running normally. If a failure occurs on the system at either site, deletion of the Stretched PVC will fail. To delete a Stretched PVC in this situation, perform the following steps:
Procedure
Check information about the resources created at the primary and secondary sites by referring to Checking information on Stretched PVCs.
Delete the PVC and PV.
Based on the information you checked in step 1, use Storage Navigator to delete the global-active device pair, volumes, and the associated LUNs. In addition, change the virtual LDEV IDs of the deleted volumes to unassigned (65534).
Connecting a Stretched PVC to a Pod
Similar to a regular PVC, a Stretched PVC can be used from a Pod.
When you create a Pod, the following message is output to the log of Storage Plug-in for Containers. The log displays the serial number of the storage system that successfully connected to the Pod. Generally, both the storage system at the primary site and the one at the secondary site will be used. However, depending on the status of the storage systems, only one of these storage systems (either the one at the primary site or the one at the secondary site) might be used.
[INFO]storages used for publish : [11111 22222]
Failover
If a failure occurs on a storage system or at a site, allocate the Pod to a different site at your discretion. A Stretched PVC can continue to operate by using the volumes of the storage system that is running normally.
Troubleshooting
If processing to create a Stretched PVC does not finish (the status does not change
to Bound), run kubectl describe pvc <pvc-name> to check the error details. In addition, if the following
error message appears, this indicates that volumes have already been created on the storage
system. If you delete the PVC before the status of the PVC changes to Bound, the volumes
will remain on the storage system without being deleted. After the PVC is deleted, use
Storage Navigator to delete
the volumes and the associated LUNs. In addition, change the virtual LDEV IDs of the deleted
volumes to unassigned (65534).
failed to provision volume with StorageClass ... : volumes created : [{id:100 serial:11111} {id:200 serial:22222}]