Skip to main content

We've Moved!

Product Documentation has moved to docs.hitachivantara.com
Hitachi Vantara Knowledge

Troubleshooting Hitachi Block

This section provides guidelines for how to troubleshoot issues that might occur when using Hitachi Block storage.

Cannot create any more snapshots

Problem:

When trying to create a snapshot, the following error message is logged:

Handler call failed: [...] failed to create snapshot pair [...]

Block storage has a limit of 1024 snapshots per LDEV. When this limit is reached, no more snapshots can be created.

Solution:

Review the RPO and Retention you have specified. If you set an overly aggressive RPO combined with long retention times then the snapshot limit can easily be breached.

Cannot create snapshots in the specified pool

Problem:

The following error message is logged when creating a snapshot:

Handler call failed: [] snapshots that are in a different pool

This indicates that the primary volume (P-VOL) in question already has a pool assigned for snapshots, and this is not the same pool as defined in the policy. Block storage will only allow one pool to be used for snapshots (S-VOLs) per P-VOL.

Solution:

Review the policy to ensure that the correct snapshot pool is specified.

Cannot make a mounted snapshot multipathed

Problem:

When mounting a snapshot there is always only one path to the disk.

Solution:

To make the mounted snapshot multipathed requires additional steps outside of Protector, dependent on the type of OS on mount host.

Cannot revert snapshot if source machine is unavailable

Problem:

If the source machine where the primary volume is mounted is unavailable, it is not possible to revert a snapshot using Protector. This is because Protector needs to unmount the primary volume prior to reverting it using the snapshot.

Solution:

In the case of a Hitachi Block hardware path classification, the source machine does not need to be available but it is important to unmount the volume before performing the revert.

Cannot tear down adopted replication having other pairings

Problem:

If a replication is adopted in Protector but other replications or snapshots are associated with the replication pair it will not be possible for Protector to remove the replication.

Solution:

Before removing the replication from Protector, first remove the associated snapshots and replications.

Clean-up actions not performed if replication setup fails

Problem:

If the set-up phase of a replication fails, Protector will not clean-up the changes made on the hardware.

Solution:

Detailed steps to perform the clean-up by hand can be found on the Hitachi Vantara knowledge base.

Error 10360 activating UR data flow with grouped source nodes

Problem:

When attempting to activate a data flow in which a node group is specified as the source of a UR replication, the rules compiler generates the following error, indicating that the journal is already assigned to one of the sources within the group:

The Hitachi Block Storage node <Destination Name> has a target journal selected for data from <SourceName2 'OperationName'> operation 'UR' but that journal is already assigned to <SourceName1 'OperationName'> operation 'UR'

Solution:

For UR, the selected journals must be unique to each operation and policy in the data flow. This means that a node group cannot be specified as a source when drawing a UR data flow.

Identify each source node explicitly on the data flow and assign unique journals for each replication operation.

Error when removing replications from Protector

Problem:

When removing a replication from Protector the following error message maybe logged:

Replication pair status change *** Attachment count: 1 ***

Solution:

In this circumstance (i.e. when removing a replication), this error can be safely ignored.

In all other circumstances this error should not be ignored and should be investigated.

ISM crashed, shutdown or rebooted whilst activating a replication or snapshot data flow

Problem:

If an ISM node crashes, shuts down or reboots whilst a replication or snapshot data flow is being activated, Protector will indicate that the job(s) associated with the activation has failed.

Solution:

Protector is resilient to this type of failure. Try retriggering the operations on the associated data flow(s) once the ISM has finished rebooting.

Policy returns 'Could not start HORCM instance [...]'

Problem:

This error message occurs when a policy is triggered and there is a problem with the command device (CMD) used by the HORCM instance to communicate with the storage device:

  • The command device is being shared with another server.
  • The command device has been initialized (and possibly put online) as a disk at the OS level.

Solution:

The array maintains a reference count for the command device instances. There can be a maximum of 4096 instances per storage system. If HORCM is not shutdown cleanly using horcmshutdown then the reference count will not be decremeted and the instance will be leaked. Over time this can cause an inability to start HORCM instances for that array.

Try detaching the command device from the Protector proxy node, clearing its command device attribute then setting it up again (this subtracts the number of instances for that command device from the total count). Alternatively, try unmapping the existing command device and mapping a new one.

Proxy node cannot access the LDEV's resource group

Problem:

When the user account does not have permission to access the LDEV the following error message is logged:

Handler call failed: refreshLogicalDevices Failed [...]

This is because the LDEV is not in a resource group that the user account can access.

This can be confirmed by navigating to C:\HORCM\etc on the proxy node and running the following command:

raidcom.exe get ldev -ldev_id LdevId -I0

The following message will be returned:

raidcom: [EX_EGPERM] Permission denied with the Resource Group

Solution:

Ensure that the user account specified for the block device proxy node has access to the resource group being used.

Renaming an LDEV fails due to unresolved variables

Problem:

When trying to name or rename an LDEV using a variable (e.g. %ORIGIN_SERIAL%), the following error message is logged:

Unable to generate custom secondary logical device name because one or more variables could not be resolved.

If the referenced LDEV has no name then %XXXX_LDEV_NAME% variables will evaluate to an empty string and an error will be logged.

If upgrading from a version earlier than v6.6, the information required to resolve substitution variables may not be present in the metadata for exising LDEVs created prior to upgrade. For example the %ORIGIN_SERIAL% (serial number of the origin storage array for the LDEV) is not included in previous Protector versions.

  • For replications, this metadata is added when the replication is next evaluated, following an upgrade to Protector v6.6.
  • For snapshot, this metadata is never added since snapshots are not re-evaluated, and will eventually be retired.

Solution:

Review the naming string in the Secondary Logical Device Name field of the Replication/Snapshot Operation Properties or Rename Secondary LDEVs dialog, and for:

  • %XXXX_LDEV_NAME% variables - Ensure the existing LDEV has a name, then retry the naming operation.
  • All other variables - For replications, ensure they have been re-evaluated, then retry the naming operation. For snapshots, remove the offending substitution variable(s).

Windows 2008 Server doesn't show changes to reverted disk

Problem:

On Windows 2008 Server, after successfully reverting a disk it is possible that changes are not reflected on the OS.

Solution:

Use the disk management console on the server to offline and then online the disk.

 

  • Was this article helpful?