10Gb SFP+ single port = cheaper than dirt

PnoT

Active Member
Mar 1, 2015
606
135
43
Texas
I have SFP+ at the house and those T520's are $$$. Was that the card you recommend me looking at?
 

neo

Well-Known Member
Mar 18, 2015
672
361
63
I have SFP+ at the house and those T520's are $$$. Was that the card you recommend me looking at?
Well you don't have to go for the top. The T520 is roughly equivalent to a ConnectX-3 - both are PCIe 3.0
For a Chelsio equivalent of a ConnectX-2 I would recommend a T420 - both are PCIe 2.0.
 

xbliss

Member
Sep 26, 2015
68
0
6
43
I have an HP ProCurve 24-port GigE Managed Switch and CAT6 cabling in the home.

Would these cards make any sense/ value for me?
 

Keljian

Active Member
Sep 9, 2015
429
71
28
Melbourne Australia
I have an HP ProCurve 24-port GigE Managed Switch and CAT6 cabling in the home.

Would these cards make any sense/ value for me?
Unless you get a switch with a 10gb port, essentially the answer is no.

I am moving from a similar switch to a mikrotik switch with two 10gb ports and 24x1gb ports - I can justify it by using 10gb on my desktop and 10gb on my server, (the rest of the household is on 100mbit, 1gb or wireless)

I probably won't saturate the 10gbit (be pretty hard to in my use) but the lower latency for some of the things I do will be welcome.
 
Last edited:
  • Like
Reactions: xbliss

xbliss

Member
Sep 26, 2015
68
0
6
43
Unless you get a switch with a 10gb port, essentially the answer is no.

I am moving from a similar switch to a mikrotik switch with two 10gb ports and 24x1gb ports - I can justify it by using 10gb on my desktop and 10gb on my server, (the rest of the household is on 100mbit, 1gb or wireless)

I probably won't saturate the 10gbit (be pretty hard to in my use) but the lower latency for some of the things I do will be welcome.
I might consider getting a similar switch, but will this work with the CAT 6 cabling I have or will need Fibre Cable?

Hmm.. Are you / do you ever saturate your existing GigE/ 1G ?
How much would the latency change from now to then? And what are some of the things you do where it will be beneficial?
 

Keljian

Active Member
Sep 9, 2015
429
71
28
Melbourne Australia
I saturate the 1 gig link when I am transferring files to/from the server generally. It has a 7 disk raid6 config.

My desktop has an assortment of SSDs and 2.5 inch hdds.

The other thing is that I Remote Desktop into some of the VMs on the server, so latency improvements there would be beneficial, especially when other machines are hitting the server at the same time.

In terms of cables, you would need optical cable + transceivers (sfp+) or alternatively passive twinax.

Optical cable isn't expensive, you would want OM3 or OM4 duplex LC-LC

Beyond all this, for me, the key thing is that the existing nic in the server is a Chinese clone of the i350-T4, which has me a little concerned about it potentially failing. Also there is the fact that changing it up for a cheap Mellanox and adding better network infrastructure(which I had planned to do as I had run out of ports) is just a little more than buying a genuine Intel.

This is the switch that is on its way to me:
http://www.servethehome.com/mikrotik-crs226-24g-2s-in-review-you-want-one/
 
Last edited:

xbliss

Member
Sep 26, 2015
68
0
6
43
Hmm.. My CAT6 Cabling is internal inside a Concrete brick walled, 2 level-Condo. So, I am wondering if/how I could connect the NAS {in closet} to a machine at all to enjoy any of the said benefits you are getting.

Maybe in the future, when I next revamp cabling - Maybe such a cable can be pulled using and existing CAT6 cable - joined and pulled?

Would the prices of these stay at this level, or go down in the future? As these are tempting at the price even though I have/ would not be able to use them for a while.

I guess I wont be able to saturate until I have a Fast RAID or major SSDs that surpass SATA bandwidths.

I saturate the 1 gig link when I am transferring files to/from the server generally. It has a 7 disk raid6 config.

My desktop has an assortment of SSDs and 2.5 inch hdds.

The other thing is that I Remote Desktop into some of the VMs on the server, so latency improvements there would be beneficial, especially when other machines are hitting the server at the same time.

In terms of cables, you would need optical cable + transceivers (sfp+) or alternatively passive twinax.

Optical cable isn't expensive, you would want OM3 or OM4 duplex LC-LC

Beyond all this, for me, the key thing is that the existing nic in the server is a Chinese clone of the i350-T4, which has me a little concerned about it potentially failing. Also there is the fact that changing it up for a cheap Mellanox and adding better network infrastructure(which I had planned to do as I had run out of ports) is just a little more than buying a genuine Intel.

This is the switch that is on its way to me:
http://www.servethehome.com/mikrotik-crs226-24g-2s-in-review-you-want-one/
 

Peanuthead

Active Member
Jun 12, 2015
819
141
43
41
For the any new readers that are reading this thread and want a simplified read. Someone correct me if I am incorrect.

Connectors/Connections needed if using this card (or a similar SFP+ card):
So in a nutshell is someone an this from server to server - Twinax SFP+ DAC is needed
From server to switch (or vice versa) you need 10Gb SFP+ modules (which are LC type)

Mellanox seems to be SFP+ module agnostic but the switch you use may not be. If using a Mikrotik switch here is a link that may help you out.
Supported Hardware - MikroTik Wiki
 
  • Like
Reactions: xbliss

neo

Well-Known Member
Mar 18, 2015
672
361
63
Hmm.. My CAT6 Cabling is internal inside a Concrete brick walled, 2 level-Condo.
This is not a 10GBASE-T NIC. It uses the SFP+ form factor. For data cables I always recommend using a conduit so it will be easier to pull new additional cabling if a need arrises.
 
  • Like
Reactions: xbliss

neo

Well-Known Member
Mar 18, 2015
672
361
63
Connectors/Connections needed if using this card (or a similar SFP+ card):
So in a nutshell is someone an this from server to server - Twinax SFP+ DAC is needed
From server to switch (or vice versa) you need 10Gb SFP+ modules (which are LC type)
LC is a optical fiber connector used in SFP+ transceivers. TwinAx DAC is a copper cable with built in SFP+ transceivers (DAC = direct attached cable). The main difference is TwinAX can not be run for very long lengths. I prefer to use TwinAx when I can for simplicity.

Mellanox seems to be SFP+ module agnostic but the switch you use may not be. If using a Mikrotik switch here is a link that may help you out.
Supported Hardware - MikroTik Wiki
My MikroTik switch is agnostic.
 

Peanuthead

Active Member
Jun 12, 2015
819
141
43
41
LC is a optical fiber connector used in SFP+ transceivers. TwinAx DAC is a copper cable with built in SFP+ transceivers (DAC = direct attached cable). The main difference is TwinAX can not be run for very long lengths. I prefer to use TwinAx when I can for simplicity.
Correct. I don't think you can disconnect the TwinAx cable like you can the LC/Fiber cable can you? (Fiber newbie.) I personally like the idea of fiber just because if I need a longer run I just get a longer cable and keep my receivers.
 

canta

Well-Known Member
Nov 26, 2014
1,029
198
63
39
just a simple question :
can I do direct connection between two cards? the goal is to connect my primary Zol to my backup server since gaining more than 1Gb connection is awesome to shorten transfer time.

waiting patiently to buy fiber switch from ebay...
 

NetWise

Active Member
Jun 29, 2012
599
130
43
Edmonton, AB, Canada
just a simple question :
can I do direct connection between two cards? the goal is to connect my primary Zol to my backup server since gaining more than 1Gb connection is awesome to shorten transfer time.

waiting patiently to buy fiber switch from ebay...
Yes, you can. Note that if your backup isn't multithreaded the cable/bandwidth will likely not solve all your problems. But your use case would certainly fit...
 
  • Like
Reactions: canta

Peanuthead

Active Member
Jun 12, 2015
819
141
43
41
FYI - if anyone is planning to order from Fiberstore they have a coming holiday of Nation Day for the factory. They will be on holiday from Oct 1st to 7th.
 
  • Like
Reactions: Chuckleb and canta

canta

Well-Known Member
Nov 26, 2014
1,029
198
63
39
Yes, you can. Note that if your backup isn't multithreaded the cable/bandwidth will likely not solve all your problems. But your use case would certainly fit...
thanks
I plan to run mulitple rsync :D. yeah rsync is old but this the most trusted backup that I rely on as now :D
 

JayG30

Active Member
Feb 23, 2015
226
46
28
34
From what I have always gathered, if you only need a short connection Copper Twinax is the cheapest route. The fiber transceivers aren't going to be as cheap if you want to go fiber. The fiber is usually "better" I believe do to less interference, but for 99.99% of people I don't see this mattering and if you can connect with copper twinax, it is simpler and cheaper.

For longer runs you really don't have a choice.
 
  • Like
Reactions: Keljian

Fritz

Well-Known Member
Apr 6, 2015
2,120
483
83
66
I just ordered 2 Chelsio NICs that came with 2 TwinAx cables. Knew nothing about them until I read this thread. Thanks. :)
 
  • Like
Reactions: Keljian