Connecting 10gb - choosing cable and modules question

ChasW

New Member
Nov 3, 2019
9
0
1
My question is regarding connecting a few ConnectX-3 devices (SFP+).

For my home lab, I am looking to upgrade the connections on 3 systems, all linux.
They are already connected at 1gb ethernet and I would like to get them all to 5gb or 10gb if possible. I mention 5gb first because I am not sure what to expect from these older Asus motherboards, but I plan to find out.
Each system has at least 1 free PCIe 2.0 16x physical slot capable of linking at 8x.

system 1 is 30m (about 100 feet) from system 2 and has an existing cat 6 cable connecting them.
system 3 is about 8m from system 1 and has an existing cat 6 cable connecting them.

If possible, I would like to reuse the 100 foot run, but that is not a critical factor, but overall cost, of course, is.

Since I am connecting 3 systems and not 2, and am trying to avoid the costly expense of a 10gb switch, I am looking at, for example, 1x MCX312A-XCBT (dual SFP+) for system 1 and perhaps 2x MCX311A-XCAT (single SFP+) for each of system 2 and 3.

If I am correct, the use of the dual port MCX312A-XCBT would avoid having to use a 10gb switch, likely at the cost of some performance, but still better than 1gb.

The spec sheet (PB_ConnectX3_EN_Card.pdf) shows the card's connectivity to include:
Passive copper cable with ESD protection
Powered connectors for optical and active cable support

so for the 10gb over an 8m run, would a good choice be SFP+ DAC Twinax Cable, Active?

Are there reasonable options for using the existing 100 foot cat6 run while preserving 10gb capability?

Would a couple of these or similar work?
https://www.amazon.com/dp/B06XQBFHNL/ref=psdc_537316_t1_B01KFBFL16

Or should I be looking at an optical run and if so, which modules or transceivers, if any, would be required?

Thank you and best regards.
 

Smbaker

New Member
Oct 9, 2019
16
14
3
My understanding is that the copper modules run hot and consume additional power. I went with a new fiber run rather than trying to use my copper. I found preterminated fiber to be pretty easy to work with. It's quite flexible and the connectors are reasonably small.

There are some affordable switch options, see the Brocade ICX thread in this forum (4 x 10Gbe SFP+ and 24 1Gbe copper ports for around $200 on eBay). There's also new 10 Gbe Mikrotiks for around $130. I went with a Brocade ICX6450-24P.
 
  • Like
Reactions: fohdeesha

ChasW

New Member
Nov 3, 2019
9
0
1
My understanding is that the copper modules run hot and consume additional power....
Thank you for the reply. I had some time to dig into this today and read about experiences different people have had with different modules. Some report that theirs seem to work fine, other comment that they can be touchy. I think a new fiber run is a good idea. Thank you again.