How to Change FEC Settings When 25Gbps iSCSI Ports on Array and Cisco Switch Won't Connect

How to identify and change FEC settings on Cisco switch when SFPs have been verified.

Connectivity port effected:

• 25-Gbps iSCSI ports

Storage Array:

• Hitachi Virtual Storage Platform G/Fx00 models (VSP G130, G/F350, G/F370, G/F700, G/F900)
• Hitachi Virtual Storage Platform E Series (VSP E590, VSP E790, VSP E990, VSP E1090)
• VSP 5000 Series (DKC910I, R900, R10K)
• Virtual Storage Platform 5100 (VSP 5100)
• Virtual Storage Platform 5100H (VSP 5100H)
• Virtual Storage Platform 5200 (VSP 5200) 
• Virtual Storage Platform 5500 (VSP 5500)
• Virtual Storage Platform 5500H (VSP 5500H)
• Virtual Storage Platform 5600 (VSP 5600) 
• VSP One 20 Series (B24, B26, B28)

Cisco switch type:

• Cisco Nexus 93180YC-FX
• Cisco Nexus 93180YC-FX2
• Cisco Nexus 93180YC-FX3

Please review the Cisco documentation on the FEC settings with the following link: Reference in the Cisco ACI Auto-Negotiation and Forwarding Error Correction

Procedure

1. Verify the SFP health on the array via one of the following methods.
   1. Confirm shipped SFPs via Asset shipment
   2. Confirm with Customer and Customer Engineer SFP Speed installed
   3. Verify on HRO Website under asset > Port > Port Speed Column
2. Verify Switch speed with customer by reviewing the configuration by using the following command:
   1. Command: show interface brief
   2. Example Output:
      `show interface brief`
      
      -----------------------------------------------------------------------------------------
      Interface  Vsan   Admin  Admin   Status       SFP    Oper  Oper   Port     Logical
                        Mode   Trunk                       Mode  Speed  Channel   Type
                               Mode                              (Gbps)
      -----------------------------------------------------------------------------------------
      fc1/1       ---    auto   off     up           swl   F      25     --       edge
      fc1/2       ---    auto   off     up           swl   F      25     --       edge
      fc1/3       ---    auto   off     up           swl   F      25     --       edge
      fc1/4       ---    auto   off     notConnected swl   --     --     --       --
      fc1/5       ---    auto   off     down         swl   --     --     --       --
      fc1/6       ---    auto   off     notConnected swl   --     --     --       --
      fc1/7       ---    auto   off     notConnected swl   --     --     --       --
      fc1/8       ---    auto   off     up           swl   F      25     --       edge
      fc1/9       ---    auto   off     up           swl   F      25     --       edge
      fc1/10      ---    auto   off     up           swl   F      25     --       edge
      fc1/11      ---    auto   off     up           swl   F      25     --       edge
3. 1. Please verify the FEC settings by reviewing the confirmation using the following command:
   2. Command: show interface Ethernet<port>
   3. Example Output: FEC is enabled by Default
      
      switch1# show interface Ethernet1/2
      Ethernet1/2 is up
       admin state is up, Dedicated Interface
       Hardware: 40000/100000 Ethernet, address: 00eb.d56e.9fd0 (bia 00eb.d56e.9fd0)
       MTU 9216 bytes, BW 40000000 Kbit, DLY 10 usec
       reliability 255/255, txload 1/255, rxload 1/255
       Encapsulation ARPA, medium is broadcast
       Port mode is access
       full-duplex, 40 Gb/s, media type is 40G
       Beacon is turned off
       Auto-Negotiation is turned on
       Input flow-control is off, output flow-control is off
       Auto-mdix is turned off
       Rate mode is dedicated
       Switchport monitor is off
       EtherType is 0x8100
       EEE (efficient-ethernet) : n/a
         admin fec state is auto, oper fec state is off
4. Configuring FEC to ieee-rs-fec
   1. Enter Configuration mode: 
      switch1# configure terminal
   2. Enter Interface configuration mode
      switch1 (config)# interface Ethernet<port>
   3. Enable Auto-Negotiate 
      switch1 (config-if)# negotiate auto-negotiation
   4. Enable FEC mode ieee-rs-fec
      switch1 (config-if)# fec ieee-rs-fec
      • Options
      • auto-fec
      • cl74-fc-fec
      • cl91-rs-fec
      • cons16-rs-fec
      • disable-fec
      • ieee-rs-fec
      • inherit
      • kp-fec
   5. switch1 (config-if): exit
   6. Save configuration across reboots: switch(config)# copy running-config startup-config
5. Each Array type has reference to these settings under section "25-Gbps iSCSI channel board ports". The following links are provided in the Documentation page
   1. Hitachi Virtual Storage Platform G/Fx00 models (VSP G130, G/F350, G/F370, G/F700, G/F900) "Modifying the storage system port settings"
   2. Hitachi Virtual Storage Platform E Series (VSP E590, VSP E790, VSP E990, VSP E1090) "Restrictions for using 25-Gbps iSCSI channel boards"
   3. VSP 5000 Series (DKC910I, R900, R10K) "Creating iSCSI targets and registering hosts in an iSCSI target"
   4. VSP One 20 Series (B24, B26, B28) "25-Gbps iSCSI Channel Board Ports"
6.  CAUTION: Auto negotiation cannot be set as the transfer speed. Set the transfer speed of the CHB to 10Gbps or 25 Gbps to mat the port transfer speed of the connection-target NIC or switch. 
7. If you use Router Advertisement (RA) to automatically set the default gateway, do NOT manually set the IP address of the default gateway. If you set it manually, performance delays might occur when multiple IPv6 routers exist in the same network. You can avoid manual setting in the following ways:
   1. Device Manager - Storage Navigator: In the window for setting iSCSI ports, specify a blank for the IP address.
   2. CCI: When you run the raidcom modify port command, do not use the option for which an IP address must be specified.
      • Command Control Interface User and Reference Guide "Creating external volumes (iSCSI)"