I've been around and around with attempting to establish a connection between two Chelsio T580-SO-CR cards and a Gnodal GS7200 switch. Because this is my first attempt with 10G/40G/fiber, I'm sure that I'm overlooking a minor detail. Unfortunately, however, to date I've been unable to identify exactly what that detail is.
Hardware:
GS7200
2x Chelsio T580-SO-CR
WS2016 DC Server
FreeNAS Server
2x Fiberstore.com 4x SFP:1x QSFP cables (generic)
On the switch I've:
Updated the firmware to version 2.6.4 per Gnodal's instructions
Configured the default IP and subnet
Configured the mgmt port IP and subnet
Enabled the ports to be used
Set the system clock
Created a port-channel/link aggregation for the two groups of 4 ports
On the cards I've:
Installed the latest version of Unified Wire from the Chelsio Website
Configured the IP address to be on the same subnet as the switch
Set the VLAN Identifier to match that of the switch
Observations:
I get absolutely nothing out of the FreeNAS machine really haven't even attacked that yet. Everything below describes interaction with the Windows machine.
Prior to installation of Unified Wire, I get green led indicators on the switch for each connected port and "show interfaces status" reports:
"Status= Connected; Duplex= Full; Speed= 10 Gbps; Negotiation= No-Negotiation"
Post installation of UW, I have no lit port indicators on the back of the switch and status= not connected.
"show interfaces description": Status= Up; Protocol= down
"show interfaces transceiver": Status= present; Name= Fiberstore; Part Number= QSDP-4SFP10G-DAC
"show etherchannel summary":
Port-channel Module Admin Status is enabled
Port-channel Module Oper Status is enabled
Port-channel System Identifier is **:**:**:**:**:**
Flags:
D - down P - in port-channel
I - stand-alone H - Hot-standby (LACP only)
U - in-use
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
-------------------------------------------------------------------------
1 Po1(D) LACP Eth0/1(D),Eth0/2(D),Eth0/3(D),Eth0/4(D)
"GNOS# show interfaces port-channel 1":
po1 up, line protocol is down (not connect)
Bridge Port Type: Customer Bridge Port
Hardware Address is **:**:**:**:**:**
MTU 1500 bytes
"GNOS# show interfaces etherchannel":
Port : Eth0/1
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 1
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
Port : Eth0/2
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 2
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
Port : Eth0/3
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 3
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
Port : Eth0/4
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 4
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
LACP Port Admin Oper Port Port
Port State Priority Key Key Number State
---------------------------------------------------------------------
Eth0/1 Down 128 1 1 0x1 0xa2
Eth0/2 Down 128 1 1 0x2 0xa2
Eth0/3 Down 128 1 1 0x3 0xa2
Eth0/4 Down 128 1 1 0x4 0xa2
Port-channel : Po1
-------------------
Number of Ports = 4
HotStandBy port = null
Port state = Port-channel Ag-Not-Inuse
Protocol = LACP
Aggregator-MAC **:**:**:**:**:**
Maximum number of Ports = 8
Hardware:
GS7200
2x Chelsio T580-SO-CR
WS2016 DC Server
FreeNAS Server
2x Fiberstore.com 4x SFP:1x QSFP cables (generic)
On the switch I've:
Updated the firmware to version 2.6.4 per Gnodal's instructions
Configured the default IP and subnet
Configured the mgmt port IP and subnet
Enabled the ports to be used
Set the system clock
Created a port-channel/link aggregation for the two groups of 4 ports
On the cards I've:
Installed the latest version of Unified Wire from the Chelsio Website
Configured the IP address to be on the same subnet as the switch
Set the VLAN Identifier to match that of the switch
Observations:
I get absolutely nothing out of the FreeNAS machine really haven't even attacked that yet. Everything below describes interaction with the Windows machine.
Prior to installation of Unified Wire, I get green led indicators on the switch for each connected port and "show interfaces status" reports:
"Status= Connected; Duplex= Full; Speed= 10 Gbps; Negotiation= No-Negotiation"
Post installation of UW, I have no lit port indicators on the back of the switch and status= not connected.
"show interfaces description": Status= Up; Protocol= down
"show interfaces transceiver": Status= present; Name= Fiberstore; Part Number= QSDP-4SFP10G-DAC
"show etherchannel summary":
Port-channel Module Admin Status is enabled
Port-channel Module Oper Status is enabled
Port-channel System Identifier is **:**:**:**:**:**
Flags:
D - down P - in port-channel
I - stand-alone H - Hot-standby (LACP only)
U - in-use
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
-------------------------------------------------------------------------
1 Po1(D) LACP Eth0/1(D),Eth0/2(D),Eth0/3(D),Eth0/4(D)
"GNOS# show interfaces port-channel 1":
po1 up, line protocol is down (not connect)
Bridge Port Type: Customer Bridge Port
Hardware Address is **:**:**:**:**:**
MTU 1500 bytes
"GNOS# show interfaces etherchannel":
Port : Eth0/1
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 1
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
Port : Eth0/2
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 2
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
Port : Eth0/3
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 3
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
Port : Eth0/4
-------------
Port State = Down, Not in Bundle
Channel Group : 1
Mode : Active
Port-channel = Null
Pseudo port-channel = Po1
LACP port-priority = 128
LACP Wait-time = 2 secs
LACP Port Identifier = 4
LACP Activity : Active
LACP Timeout : Long
Aggregation State : Aggregation, Defaulted
LACP Port Admin Oper Port Port
Port State Priority Key Key Number State
---------------------------------------------------------------------
Eth0/1 Down 128 1 1 0x1 0xa2
Eth0/2 Down 128 1 1 0x2 0xa2
Eth0/3 Down 128 1 1 0x3 0xa2
Eth0/4 Down 128 1 1 0x4 0xa2
Port-channel : Po1
-------------------
Number of Ports = 4
HotStandBy port = null
Port state = Port-channel Ag-Not-Inuse
Protocol = LACP
Aggregator-MAC **:**:**:**:**:**
Maximum number of Ports = 8