10Gbit networking on Supermicro X10SDV-4C+-TLN4F

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

DRSpalding

New Member
Jul 14, 2020
17
3
3
Hi,

I recently rebuilt my Unraid 6.9.1 server onto a machine based on the above Supermicro mb. I added a Mikrotik 10Gb CRS305-1G-4S+ switch and an Intel x510-1 10Gb NIC into my main computer. I used Ipoplex SFP+ 10Gbase-t transceivers to flesh out the connections with Cat 7 patch cables. The main cable run for the PC to the switch is a ~15m run of Cat5e cabling (yeah, I know, I'll get to that in a sec). With this set up, I am able to get ~4 Gb/s throughput using iperf3. Not bad, but not 10Gb/s. I figure the cabling is limiting the throughput, so no biggie.

I received my 20m of OM3 MM fiber today to pair with the SFP+ fiber transceivers and testing the cabling directly to the switch before I start fishing cable through the wall. It's better to know the cable works before stuffing it in a hole, right? So, I am *still* only getting about ~4 Gb/s! Hmmm. I dump the switch and wire directly with the Cat7 and 10Gbase-t SFP+ transceiver to my PC and still only getting about the same. It's not the switch, evidently.

I also moved all the MTU to 9018 bytes. Is there any other tuning to do? What is the most likely culprit in this setup? The x510 board in my PC with EVGA FTW mb with a I9-9900K running Win10 Pro 20H2? Are 8 PCIEx lanes enough for it to sustain 10Gb? Can the X10SDV with the Xeon D-1518 SOC actually sustain 10Gb? Can Unraid?

Looking for answers before I bother running the fiber now, since the only thing it appears to be able to do for me is to reduce the power consumption of the link between my PC and the switch.

Thanks for any insight,

Dan
 

DRSpalding

New Member
Jul 14, 2020
17
3
3
How are you measuring 4G/s?

x8 is more than enough for 10G
Measuring with iperf3 on each end.

And I thought x8 was plenty as well, since I think that the NIC is x8 anyway. I put it into the second x16 slot so my 16 lanes are now 2 x8 instead of 1 x16.
 

DRSpalding

New Member
Jul 14, 2020
17
3
3
Ok, I got to spend a bit more time on this and poked about with iperf3's options. If I run it with a 1MB window size, I get 10Gbit for all intents and purposes, and this is on the Cat5e cable run through the Mikrotik switch:

C:\tmp> iperf3 -c samwise -w 1024k
Connecting to host samwise, port 5201
[ 4] local 192.168.1.24 port 56755 connected to 192.168.1.32 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.11 GBytes 9.51 Gbits/sec
[ 4] 1.00-2.00 sec 1.15 GBytes 9.87 Gbits/sec
[ 4] 2.00-3.00 sec 1.15 GBytes 9.89 Gbits/sec
[ 4] 3.00-4.00 sec 1.15 GBytes 9.87 Gbits/sec
[ 4] 4.00-5.00 sec 1.14 GBytes 9.82 Gbits/sec
[ 4] 5.00-6.00 sec 1.15 GBytes 9.88 Gbits/sec
[ 4] 6.00-7.00 sec 1.15 GBytes 9.89 Gbits/sec
[ 4] 7.00-8.00 sec 1.15 GBytes 9.87 Gbits/sec
[ 4] 8.00-9.00 sec 1.15 GBytes 9.89 Gbits/sec
[ 4] 9.00-10.00 sec 1.15 GBytes 9.89 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 11.5 GBytes 9.84 Gbits/sec sender
[ 4] 0.00-10.00 sec 11.5 GBytes 9.84 Gbits/sec receiver

iperf Done.



So, there doesn't seem to be any need for the fiber run at all. <shrug>