Home Lab: 10G SFP+ vs 10G Base-T in 2017 Q3
- Thread starter fmatthew5876
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
The switch logged warnings for my "mellanox compatible" finisar transceivers, but they work fine.
In a homelab context SFP+ and fiber is more popular only because of cost and the mass availability of second hand ConnectX-2 adapters.
Fiber optics is still the only choice if you want more than 10GbE.
If you are looking at new controllers the difference isn't so big. Also there is not much of a price difference between high port count switches with 10Gb SFP+ vs 10GBASE-T.
I already invested in SFP+ so I will stick with it for a while. Neither is a bad choice at home.
I guess I will migrate to 10GBASE-T when the price is right and adapters come down further. Preferably at the same level good quality 1gigabit NICs cost now.
Fiber optics is still the only choice if you want more than 10GbE.
If you are looking at new controllers the difference isn't so big. Also there is not much of a price difference between high port count switches with 10Gb SFP+ vs 10GBASE-T.
I already invested in SFP+ so I will stick with it for a while. Neither is a bad choice at home.
I guess I will migrate to 10GBASE-T when the price is right and adapters come down further. Preferably at the same level good quality 1gigabit NICs cost now.
I have some Java applications that executes tens of thousands queries every minute. The reduced latency with SFP+ / Fiber dramatically reduced the resource usage / threads needed as "blocking" time was significantly reduced. For me, choosing 10G-BaseT is like throwing money out of your window.
IMO SFP+ or DAC for homelab...even for work. The only play I really see unless gear starts shipping with copper ports is handful of desktops scenario that CAT6 is already installed and running glass would get expensive or can't be done easily.
And yes, that TPLink T1700G-28TQ is sexy!
And yes, that TPLink T1700G-28TQ is sexy!
This might be the right place to ask it, but will CAT wiring fade away? It cannot support the faster link speeds, and we already starting to see 100GbE adopted/needed.
A bit of a threadjack but years from now IMO it will eventually. Short distance will be wireless with long distance and high bandwidth scenarios will likely be glass. Corporate will take a full shift from traditional desktops/laptops to thin/cloud-based solutions. Current architectures rely upon machine imaging in which the network booting needs a wired connection. This could shift to glass but given the general costs associated with pulling fiber to the desktop outside of some high bandwidth/research scenarios, will favor copper due to cheaper costs.
Higher speeds over copper is possible via 802.3bq which allows for 25GB or 40GB links of up to 30M over a 4-pair Cat8 solution (IEEE ratifies 802.3bq standard for 25GBase-T and 40GBase-T). Practically though, if you need this kind of speed just go glass.
We will only see 40G on copper, CAT8 cables, its limited to 30m or so. For real high bandwidth Fiber it will be ! I Guess QSFP28 will be the next standard, expect to see much greater usage of the form factor.
If you need the number of ports good option...Netgear M4300 10G SFP+ and 10G Base-T , 8+8 or 12+12 or 24+24 port
50w idle power consumption according to spec sheet if a bit ugly for a home lab though.
It all comes down to:
At some point the rest will be wireless, NBase and MBase, although not any time soon I'm afraid. So the question might be: while clients will be on these "lower" speed ports, should the few "uplink" ports from servers be using SFP+ or Base-T? For a few ports it shouldn't make much of a difference, apart from cost. But generally speaking Xeon D or Atom boards from SuperMicro mainly target small and medium businesses where being able to plug these servers into an RJ45 plug is a must have.
If you can have your desktops/laptops and servers close in the same room (and stand the noise), then SFP+ is viable. But by and large Base-T is too, especially if you do not need a switch - BTW the A2SDV series might help).
- your definition of HomeLab
- whether servers and clients can be co-located
- your pre-existing cabling
At some point the rest will be wireless, NBase and MBase, although not any time soon I'm afraid. So the question might be: while clients will be on these "lower" speed ports, should the few "uplink" ports from servers be using SFP+ or Base-T? For a few ports it shouldn't make much of a difference, apart from cost. But generally speaking Xeon D or Atom boards from SuperMicro mainly target small and medium businesses where being able to plug these servers into an RJ45 plug is a must have.
If you can have your desktops/laptops and servers close in the same room (and stand the noise), then SFP+ is viable. But by and large Base-T is too, especially if you do not need a switch - BTW the A2SDV series might help).
I recently bought a Netgear S3300 with 2x SFP+ 10g and 2x 10GBase-T ports. The SFP+ is used for servers in the office and the 10GBase-T is there if I want to take copper through the wall to other parts of the house (future expansion).
My house was wired with Cat5e (yeah, wish it had been 6), and I've been pleasantly surprised with how well short distance cat5e will handle 10GBase-T under testing scenarios (maybe 1 bad frame out of several TB tested). Testing used snake topology from server1->swSFP+ ->10gbase-Tport1 ->10gbase-Tport2 -> swSFP+ -> server2. Used iperf running either one direction or both directions. The switch does get a little warm (46C).
My house was wired with Cat5e (yeah, wish it had been 6), and I've been pleasantly surprised with how well short distance cat5e will handle 10GBase-T under testing scenarios (maybe 1 bad frame out of several TB tested). Testing used snake topology from server1->swSFP+ ->10gbase-Tport1 ->10gbase-Tport2 -> swSFP+ -> server2. Used iperf running either one direction or both directions. The switch does get a little warm (46C).
Blocking time is idle time while having many threads isn't a problem either: creating them might rather be, but it has nothing to do with IO waiting or SFP+. The application should probably redesigned anyway.
Wrong assumptions, wrong conclusions.
Oh it has everything to do with that. The reduced wait time from 10us to 1-2us is very noticeable in the benchmarks we have done. Having thousands of threads that consume valuable memory that could be used better for other things is not always the right choice. An event loop would probably help slightly, but it also has it drawbacks. Event loops requires slightly more cpu power and it's harder to write good asynchronous code than it is to write good synchronous code.
So there is no silver bullet, but choosing 10GBase-SR over 10GBase-T is an easy and cheap way to speed up things. Refactoring working production code is a very risky business move.
So there is no silver bullet, but choosing 10GBase-SR over 10GBase-T is an easy and cheap way to speed up things. Refactoring working production code is a very risky business move.
A thread itself consumes negligible amount of memory both in kernel and user space. I've never claimed that SFP+ doesn't provide lower latency nor that is not there measurable or beneficial, but that's all it does. It doesn't change software design, it doesn't solve the hunger in the world and the likes.
It depends, often it's part of the software business, but definitely it's not an excuse for "fixing" problems elsewhere.
I'm talking about keeping most stuff inside the L2 and L3 cache. Many threads will just cause slower execution because you will have a significant memory management overhead.
No you're constantly talking about random stuff, not surprisingly without elaborating (and I challenge you to do so...), that you barely understand and is not at all related to SFP. Contention has nothing to do with cache size, cache thrashing is not caused by threading and depends on the workload type and/or how the code is written. Not a single thing you mentioned is related to the network stack or SFP+ in particular. I'm not going to spend any more time on this, Google can offer you (and the other four people who liked your post) a good starting point for a better understanding.
So, again, whether "choosing 10G-BaseT is like throwing money out of your window" or not depends entirely on a different set of factors I already mentioned.
Many idle threads does not cause trashing, too many working threads can. The reason why we prefer a fewer threads, rather than tens of thousands is that they do sequential work over some time. Your assumptions are correct, and our code may not be optimal. But we have a lot of other stuff to prioritize than spending expensive engineering hours on something that give us maybe 5% better performance.
Last edited: