Cluster switch configuration
When configuring switches for use with NAS server clusters, there are several guidelines to follow in order to optimize the transfer of data.
Network infrastructure
Each cluster must have its own isolated/private cluster interconnect (ICC) layer-2 network as system performance depends upon the cluster interconnect network delay and bandwidth. The cluster interconnect network must be designed to limit network congestion and therefore it requires dedicated bandwidth rather than sharing bandwidth with other networks. Consider using a redundant network infrastructure for cluster interconnect.
A cluster with 2 nodes ( HNAS server models) can have direct connections. Port C1 in one node must connect to port C1 in the other node, and port C2 in one node must connect to port C2 in the other node. NAS modules are inherently 2-node clusters and use their own internal connections, so no cabling or switch infrastructure is required.
A cluster of 3 nodes or more requires at least two switches for redundancy. Connections should be as follows:
-
NAS Platform Series 4000 models: All C1 ports must connect to the same switch infrastructure, and all C2 ports must connect to the same switch infrastructure. The switch infrastructure for C1 and C2 can be separate and isolated from each other.
-
NAS modules and NAS Platform Series 5000 models: C1 ports, and optionally C2, should connect to each other in the same switch infrastructure. C3, and optionally C4 ports, should connect to each other in the same infrastructure. Note that the use of C2 and C4 ports is optional and only necessary when additional NVRAM mirroring bandwidth is required.
Jumbo frames
Jumbo frames must be enabled. Specifically, the switch must be able to pass frames of at least 9216 bytes.
Spanning Tree
This feature must be disabled on the switch ports connected to the NAS servers, so that the ports will not attempt negotiation and therefore boot faster. This is sometimes known as portfast.
Interswitch link
Each physical connection must have a minimum of 10Gbps bandwidth, to avoid potential congestion and packet loss. Techniques to accommodate for asymmetric bandwidth (such as 10Gbps to multiple 1Gbps) might not be sufficient in this context.
For redundancy, the ISL in a cluster with more than 3 nodes needs to be an aggregation; there is also a small benefit for aggregating 3-node clusters, but it is not required. The recommended minimum number of (10 Gbps) physical links is half the number of cluster nodes, rounding down for odd-sized clusters.
You can also use ISLs for the connections between sites with cluster members on separate networks. ISLs must provide adequate bandwidth, as cluster protocol performance is not optimised in the event of packet loss.
Where more than one link's bandwidth is needed, close attention must be paid to the switches' frame distribution algorithm(s). The small number of source and destination addresses in a cluster creates a significant risk of very uneven traffic distribution within a link aggregation. Do not use non-compliant algorithms that could allow frame reordering, such as round-robin. Consider using switches with faster "uplink" ports instead.
Egress tagging must be enabled on ISL ports.
VLANs
ICC traffic must be isolated from any other traffic that may run on the switch. This is usually done by the use of VLANs.
For HNAS server models, the C1 and C2 networks must be isolated from each other. Therefore, if they share the same switch infrastructure each should have its own dedicated VLAN.
For NAS modules, there is no requirement to isolate the two ICC networks (C1C2 and C3C4) from each other.
The server's cluster interface is not VLAN-aware and does not support VLAN configuration, so the switch ports must be untagged members.
Priority
The server uses 802.1p priority tagging to manage congestion. Management traffic is carried in priority tagged frames with Priority Code Point 2; mirroring traffic is tagged with Priority Code Point 0; CNS traffic is tagged with Priority Code Point 1. Under the recommended default traffic class mappings (see IEEE 801.1G-2014, Annex I.4), the management traffic has a higher priority than the mirroring traffic and the CNS traffic has the lowest priority.