Chelsio T580-CR connected to a Gnodal GS4008

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

nephri

Active Member
Sep 23, 2015
541
106
43
45
Paris, France
Hi,

I'm planning to connect a Chelsio T580-CR to a Gnodal GS4008

From T580-CR | Chelsio Communications , the NIC should use SM40G-SR optics.

I would determine which compatible one i can get from Fiberstore(FS.COM) - Run To Win The Future in order to save some costs ?

I would to know if Buy 40GBASE-SR4 QSFP+ Transceiver Module for MMF | Fiberstore is compatible with the NIC and also with the gnodal switch ?

The gnodal switch is "open-sourced", that mean it don't need trademarked optics but i want to be sure that is "connect type" compatible !

At last (but not the least), i search which cable i should use from fiberstore, but i'm loosing myself in the variety of cables. which one i should get (for very short distance < 20m) ?
I saw 2M(6.6ft) Passive Copper AWG30 40GBASE QSFP+ Direct Attach Cable -Fiberstore but it's twinax copper. It will be compatible ?

I'm noob in 40gb et 10b fiber connectivity and i'm afraid to make build mistakes.
Thanks for any advices.

Best regards,
Sébastien.
 
Last edited:

Aluminum

Active Member
Sep 7, 2012
431
46
28
DACs do not require transceivers, they are all-in-one and definitely the cheapest option for 40G home use if you can keep things 5M or closer. The main caveats are very stiff cables and making sure both ends will support it.
 

nephri

Active Member
Sep 23, 2015
541
106
43
45
Paris, France
I undertand.
It would be good i ask this question before buying 10gb/s stuffs (still in shipping) from fiberstore.
I bought DAC cables + transceivers, .... i must contact them now :-/

So, how determine if a DAC cables will be compatible with the NIC and the Switch ?
 

nephri

Active Member
Sep 23, 2015
541
106
43
45
Paris, France

PithyChats

Active Member
Feb 3, 2015
173
85
28
I think to use
- 5M(16.4ft) 10GBASE SFP+ Active Optical Cable -Fiberstore for 10Gb/s with Chelsio T420
- 10M(32.8ft) 40GBASE QSFP+ to QSFP+ Active Optical Cable -Fiberstore for 40Gb/s with Chelsio T580

do you think i can go with copper cable or it's better to go with theses optical ones ?

The Quanta LB4M will accept 5M(16.4ft) 10GBASE SFP+ Active Optical Cable -Fiberstore in their 2 SFP+ uplink ?

Best regards,
Sébastien.

The only thing I cannot help you with is which QSFP+ adapters that the chelsio card likes. For the gondal, it will take just about anything.

Is there a reason you are going Chelsio (Driver support in BSD for example)? Otherwise it might be cheaper to just go with a Mellanox 40Gbe cards which are going for around $150, and QSFP+ DACs, which can be found under $50. What length do you need from switch to nic?

If you really want to go the optical route, before you buy the active cable consider the optics and cables mentioned in this thread:

https://forums.servethehome.com/index.php?threads/40gb-qsfp-optics.7023/#post-63008
 

nephri

Active Member
Sep 23, 2015
541
106
43
45
Paris, France
The Chelsio is for FreeBSD support (version 9.3 or 10.x)
If i'm right, mellanox is supported in 10.x version but i will probably start with a 9.3 version (FreeNas distribution)

I will receive a mellanox MHQH29-XTC 40gb.s : http://www.mellanox.com/related-docs/prod_adapter_cards/PB_ConnectX_VPI.pdf

Interconnections:

- Switch "gs4008" <-----> switch "lb4m"
> distance : 1 meter is **largely** enough
> type : 2x SFP+ (10gb/s)

- Switch "gs4008" <-----> "server storage"
> distance : at least 2 meters, 5 meters is ** largely** enough
> type : 1x QSFP+ (40gb/s) to a Chelsio T580-CR (maybe later i will use the second port but it's clearly over sized)

Now i understand if i buy DAC cables, i just have to plug, transceivers comes with the cable.
If i go on dedicated transceivers + fibers. How the junction is done. it's requires special stuff for do it ?

Sébastien.
 

PithyChats

Active Member
Feb 3, 2015
173
85
28
The Chelsio is for FreeBSD support (version 9.3 or 10.x)
If i'm right, mellanox is supported in 10.x version but i will probably start with a 9.3 version (FreeNas distribution)

I will receive a mellanox MHQH29-XTC 40gb.s : http://www.mellanox.com/related-docs/prod_adapter_cards/PB_ConnectX_VPI.pdf

Interconnections:

- Switch "gs4008" <-----> switch "lb4m"
> distance : 1 meter is **largely** enough
> type : 2x SFP+ (10gb/s)

- Switch "gs4008" <-----> "server storage"
> distance : at least 2 meters, 5 meters is ** largely** enough
> type : 1x QSFP+ (40gb/s) to a Chelsio T580-CR (maybe later i will use the second port but it's clearly over sized)

Now i understand if i buy DAC cables, i just have to plug, transceivers comes with the cable.
If i go on dedicated transceivers + fibers. How the junction is done. it's requires special stuff for do it ?

Sébastien.

While the MHQH29-XTC 40gb.s is a great card, unfortunately it will not work at 40Gb/s with the Gnodal, but rather at 10Gb/s since it only support 10Gbe. Mellanox is natively supported in 10.x but not 9.3 and therefore FreeNas.

If you go DAC, you are correct, one cable end plugs into each device. If you go fiber, you will need two transceivers and a fiber patch cable. For 40Gbe he fiber patch cable will have MPO ends which plug into the transceivers, which in turn plug into the devices. Make sure you get a female to female type B MPO cable if you want to patch between two 40Gbe transceivers. For 10Gbe the fiber will have LC connectors in general, as will the transceivers.

Also note that while the chelsio is dual port, the PCIE bus can really only saturate one port. The second port is for redundancy only.
 
  • Like
Reactions: neo

nephri

Active Member
Sep 23, 2015
541
106
43
45
Paris, France
The MHQH29-XTC was for test purpose (buy a 125$ with shipping + duty)
For the limitation to 10Gb/s, it comes from the NIC or from the GS4008 ? i didn't understood why it's limited !

For the mellanox support, i saw it was not done in 9.3... That's why i would go with Chelsio that is well supported in FreeBSD.

For the PCIE, i saw than PCIe 3.0 8x can go up to 64Gb/s that mean dual 40gb/s connectivity for increasing speed is not a good solution. Maybe it exists network cards with PCIe 3.0 16x that could handle 2x 40Gb/s. In all ways, i don't need it.

Mellanox seems "open sourced" like the gnodal and accepts a wide range of transceivers.
Is it the same with Chelsio ?

I'm playing for home.... but i don't want to waste my money...
 

PithyChats

Active Member
Feb 3, 2015
173
85
28
The MHQH29-XTC was for test purpose (buy a 125$ with shipping + duty)
For the limitation to 10Gb/s, it comes from the NIC or from the GS4008 ? i didn't understood why it's limited !

For the mellanox support, i saw it was not done in 9.3... That's why i would go with Chelsio that is well supported in FreeBSD.

For the PCIE, i saw than PCIe 3.0 8x can go up to 64Gb/s that mean dual 40gb/s connectivity for increasing speed is not a good solution. Maybe it exists network cards with PCIe 3.0 16x that could handle 2x 40Gb/s. In all ways, i don't need it.

Mellanox seems "open sourced" like the gnodal and accepts a wide range of transceivers.
Is it the same with Chelsio ?

I'm playing for home.... but i don't want to waste my money...
The MHQH29-XTC is a VPI adapter so it supports both Infiniband and Ethernet. While it supports 40Gb/s it does so only through Infiniband. The Gnodal switch however is an Ethernet switch so it does not "speak" Infiniband. The MHQH29-XTC on the other hand "speaks" both Ethernet and Infiniband, but on the ethernet side can only speak at 10Gb/s.

Unfortunately I don't know how Chelsio cards are with transceivers. Can anyone else chime in?
 
  • Like
Reactions: neo