Skip to main content
Hitachi Vantara Knowledge

Technology preview

This chapter describes a Technology Preview feature that is included with the software code.

Use of this feature is only recommended in a test environment as it is not covered under any Hitachi Vantara LLC support plans, may not be functionally complete, and is not intended for production use.

For full information about Technology Preview, review the release notes.

Stretched PVC

The Stretched PersistentVolumeClaim (PVC) feature automates the provisioning of synchronous replication between the storage system at each site in a single Kubernetes or OpenShift cluster that spans two sites. By using this feature, you can build a high-availability cluster that includes storage systems at two sites.

The provisioning of synchronous replication for conventional systems requires the storage system administrator, cluster administrator, and user to cooperate closely. By using the Stretched PVC feature, the cluster administrator and user can perform the provisioning of synchronous replication by themselves by using the command line tool for Kubernetes or OpenShift.

The following figure gives an overview of a cluster environment in which Stretched PVC is installed.

GUID-064AD7CA-53A2-46E6-A7FC-DBA8911E3CA5-low.png

Requirements for using the Stretched PVC feature

The Stretched PVC feature automates replication provisioning. Note that this feature includes neither the management and monitoring of the replication status, nor the management and monitoring of the status of the primary and secondary sites.
Requirements for all systems

The following table shows the system-wide requirements for using Stretched PVC.

Item

Description

Cluster configuration

Configure a single Kubernetes or OpenShift cluster (stretched cluster) that spans two sites. Then, allocate one storage system to each site.

Communication between sites (REST API)

Specify settings so that Storage Plug-in for Containers can communicate with the storage systems at the primary site and at the secondary site by using the REST API.

Communication between sites (data path)

Specify settings so that each node can perform data communication with the storage systems at the primary site and at the secondary site. Only FC is supported for communication between nodes and storage systems.

The following is an overview of connecting the server and storage systems in a stretched cluster.

GUID-B05D4975-2675-434F-BEF6-2EE95CD13E47-low.png

Requirements and settings for storage systems

Check the following requirements and settings for both the primary site and secondary site. For details, see the Global-Active Device User Guide.

Item

Description

Storage system models

The following storage systems are supported:

  • VSP E series
  • VSP 5000 series
  • VSP F350, F370, F700, F900
  • VSP G350, G370, G700, G900

License

Apply the required license by referring to the Global-Active Device User Guide.

Virtual storage machines

For each storage system, create one virtual storage machine. The virtual storage machine must meet the following requirements:

  • At the primary site and the secondary site, specify the same serial number and model for the virtual storage machines. Note that a virtual storage machine is also required at the primary site.
  • For the model of the virtual storage machines, specify any storage system model supported by the Stretched PVC feature.
  • Use the virtual storage machines only for Storage Plug-in for Containers. Do not use the virtual storage machines for other purposes.

Note the following when using the same combination of storage systems from multiple clusters:

  • For each cluster, use the same storage system users and virtual storage machines.
  • Make sure that the combination of the storage system set for the primary site and the storage system set for the secondary site for each cluster matches each other.

Users and user groups

Create a user group that has access permissions only for the resource group of the created virtual storage machine. In addition, create users to which only the created user group is assigned.

Pool

Create a pool from the pool volumes in the created virtual storage machines.

LDEV

Assign the required number of unused LDEV IDs to the virtual storage machines. For the assigned LDEVs, change the virtual LDEV ID to unassigned (65534).

Host groups

For each port that you want to use, assign the required number of unused host group IDs for the virtual storage machines. The number of host group IDs must be at least the total number of hosts + 1.

Remote paths

Create remote paths. The same path group ID must be used for both the primary site and the secondary site.

Quorum disks

Create quorum disks. The same quorum disk ID must be used for both the primary site and the secondary site.

Using a regular PVC in a stretched cluster

To use a regular PVC instead of Stretched PVC in a stretched cluster, specify settings in advance by referring to Resource partitioning. Note the items listed in the following table when specifying settings in advance:

Item

Description

Resource groups

Create regular resource groups instead of virtual storage machines.

Host groups

Use ports that are not used by Stretched PVCs.

When connecting to a Pod, if you want to control the site to which the Pod will be deployed, use the Node Affinity or Node Selector feature of Kubernetes and OpenShift.

When using Stretched PVC together with other features

The following table shows whether Stretched PVC can be used together with other features:

Feature

Can be used with Stretched PVC?

Volume snapshot

No

Volume cloning

No

Volume expansion

No

Raw block volume

Yes

ReadOnlyMany

No

Adaptive data reduction

No

NoteDo not use the Stretched PVC feature and Hitachi Replication Plug-in for Containers together in the same Kubernetes cluster.

Creating a Stretched PVC

You can create a Secret and StorageClass specifically for a Stretched PVC, and then use the created Secret and StorageClass to create a Stretched PVC. You can also use the created Stretched PVC in the same way as a regular PVC.

When Storage Plug-in for Containers receives the Secret and StorageClass specifically for the Stretched PVC, the Storage Plug-in for Containers creates volumes at the specified primary site and secondary site and generates a global-active device pair. The volumes created at the primary site and secondary site and the global-active device pair are to be deleted when the Stretched PVC is deleted. Use Storage Navigator to manage the status of the global-active device pair in the period between the creation and deletion of the Stretched PVC.

To create a Stretched PVC:

Procedure

  1. Create a Secret specifically for the Stretched PVC. A sample (secret-sample-stretched.yaml) can be found in the yaml directory.

    apiVersion: v1
    kind: Secret
    metadata:
      name: secret-sample-stretched
    type: Opaque
    stringData:
      primarySerial: "11111"                  #(1)
      primaryURL : http://172.16.0.1          #(2)
      primaryUser : primary-user              #(3)
      primaryPassword : primary-password      #(4)
      secondarySerial: "22222"                #(5)
      secondaryURL : http://172.16.0.2        #(6)
      secondaryUser : secondary-user          #(7)
      secondaryPassword : secondary-password  #(8)
    
    

    Legend:

    (1) Serial number of the storage system at the primary site

    (2) URL of the REST API of the storage system at the primary site

    (3) User of the storage system at the primary site

    (4) Password for the storage system at the primary site

    (5) Serial number of the storage system at the secondary site

    (6) URL of the REST API of the storage system at the secondary site

    (7) User of the storage system at the secondary site

    (8) Password for the storage system at the secondary site

  2. Create a StorageClass specifically for the Stretched PVC. A sample (sc-sample-stretched.yaml) can be found in the yaml directory.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: sc-sample-stretched
      annotations:
        kubernetes.io/description: Hitachi Storage Plug-in for Containers
    provisioner: hspc.csi.hitachi.com
    reclaimPolicy: Delete
    volumeBindingMode: Immediate
    allowVolumeExpansion: false
    parameters:
      connectionType: fc
      replicationType: stretched               #(1)
      quorumID: "30"                           #(2)
      primaryPoolID: "10"                      #(3)
      primaryPortID: CL1-A,CL2-A               #(4)
      secondaryPoolID: "20"                    #(5)
      secondaryPortID: CL1-F                   #(6)
      csi.storage.k8s.io/node-publish-secret-name: "secret-sample-stretched"
      csi.storage.k8s.io/node-publish-secret-namespace: "default"
      csi.storage.k8s.io/provisioner-secret-name: "secret-sample-stretched"
      csi.storage.k8s.io/provisioner-secret-namespace: "default"
      csi.storage.k8s.io/controller-publish-secret-name: "secret-sample-stretched"
      csi.storage.k8s.io/controller-publish-secret-namespace: "default"
      csi.storage.k8s.io/node-stage-secret-name: "secret-sample-stretched"
      csi.storage.k8s.io/node-stage-secret-namespace: "default"
    

    Legend:

    (1) Parameter indicating that the replication type is Stretched PVC

    (2) ID of the Quorum disk

    (3) Pool ID of the storage system at the primary site

    (4) Port ID of the storage system at the primary site

    (5) Pool ID of the storage system at the secondary site

    (6) Port ID of the storage system at the secondary site

  3. Create a PVC by specifying the created StorageClass.

  4. Display the created PVC to make sure that STATUS is Bound.

    NoteCreating a Stretched PVC takes more time than creating a regular PVC.

Checking information on Stretched PVCs

A PV is created after a Stretched PVC is created. Therefore, by running kubectl describe pv <pv-name> and checking the spec.csi.volumeAttributes field, you can check the resources on the storage systems created by Storage Plug-in for Containers. You can also use this information for troubleshooting.

The following table lists the items displayed in spec.csi.volumeAttributes.

Item

Description

nickname

Nickname of the created LDEV (the same for both the primary site and the secondary site)

primarySerial

Serial number of the primary site

primaryVolumeID

ID of the LDEV created at the primary site

secondarySerial

Serial number of the secondary site

secondaryVolumeID

ID of the LDEV created at the secondary site

Deleting a Stretched PVC

You can delete a Stretched PVC in the same way as deleting a regular PVC. When a Stretched PVC is deleted, the global-active device pair and the volumes at the primary and secondary sites are also deleted.

Note
  • Deleting a Stretched PVC takes more time than deleting a regular PVC.
  • If the Stretched PVC cannot be deleted, take action in accordance with the error message. If the problem persists, see Forcibly deleting a Stretched PVC.

Forcibly deleting a Stretched PVC

To delete a Stretched PVC, the storage systems at the primary site and secondary site must be running normally. If a failure occurs on the storage system at either site or on the global-active device pair, deletion of the Stretched PVC will fail. To delete a Stretched PVC in this situation, perform the following steps:

Procedure

  1. Check information about the resources created at the primary and secondary sites by referring to Checking information on Stretched PVCs.

  2. Delete the PVC and PV.

  3. Based on the information you checked in step 1, use Storage Navigator to delete the global-active device pair, volumes, and the associated LUNs. In addition, change the virtual LDEV IDs of the deleted volumes to unassigned (65534).

Connecting a Stretched PVC to a Pod

Similar to a regular PVC, a Stretched PVC can be used from a Pod.

When you create a Pod, the following message is output to the log of Storage Plug-in for Containers. The log displays the serial number of the storage system that successfully connected to the Pod. Generally, both the storage system at the primary site and the one at the secondary site will be used. However, depending on the status of the storage systems, only one of these storage systems (either the one at the primary site or the one at the secondary site) might be used.

[INFO]storages used for publish : [11111 22222]

Failover

If a failure occurs on a storage system or at a site, allocate the Pod to a different site at your discretion. A Stretched PVC can continue to operate by using the volumes of the storage system that is running normally.

Troubleshooting

If processing to create a Stretched PVC does not finish (the status does not change to Bound), run kubectl describe pvc <pvc-name> to check the error details. In addition, if the following error message appears, this indicates that volumes have already been created on the storage system. If you delete the PVC before the status of the PVC changes to Bound, the volumes will remain on the storage system without being deleted. After the PVC is deleted, use Storage Navigator to delete the volumes and the associated LUNs. In addition, change the virtual LDEV IDs of the deleted volumes to unassigned (65534).

failed to provision volume with StorageClass  ... : volumes created : [{id:100 serial:11111} {id:200 serial:22222}]

 

  • Was this article helpful?