1U Supermicro Server 6x 10GBE RJ45 X10SLH-LN6TF LGA 1150 H3 X10SLH-N6-ST031

Churchill

Admiral
Jan 6, 2016
835
208
43
no I didnt, can you post the link?
Btw I am using Mikrotik SFP+ 10GBase-T modules and the Mikrotik 16 port sfp+ switch. Those modules and the supermicro boards are the only 10GBase-T hardware I have. I have other SFP+ hardware though which I could test.
Not sure if it's related or not:
 

Churchill

Admiral
Jan 6, 2016
835
208
43
I don't think it's related. It is totally different hardware.

I tested with Arch Linux (kernel 5.17) and was able to get >9Gbps in both directions. So it's definitely something with OpnSense and not the hardware
Wasn't sure if it was the driver or not. I'm disappointed that you aren't getting 10gbps bandwidth on that hardware.

Can you try pfsense and see if you get the same problem?
 

joellinn

New Member
Apr 5, 2022
7
1
3
I'm disappointed that you aren't getting 10gbps bandwidth on that hardware.
Unfortunately I have yet to see a pfsense/opnsense system without NIC performance or compatibility issues. Xen VMs needed tweaks and can't do VLANS (need to make one interface per vm as a workaround). APU2 not only suffers from BSD single thread limitation but I also couldn't replicate the performance claims after tweaking. Plus I had the weirdest issues with PPoE WAN using pfsense/opnsense on an APU2.

Feature wise opnsense is really great but to me it looks like the praise for the BSD networking stack doesn't live up to the "it just works" state of Linux :confused:.

Can you try pfsense and see if you get the same problem?
Can as a test but wouldn't run that myself. I am trying to migrate away from it (currently running in VM) as development has really stalled.
 

Churchill

Admiral
Jan 6, 2016
835
208
43
Unfortunately I have yet to see a pfsense/opnsense system without NIC performance or compatibility issues. Xen VMs needed tweaks and can't do VLANS (need to make one interface per vm as a workaround). APU2 not only suffers from BSD single thread limitation but I also couldn't replicate the performance claims after tweaking. Plus I had the weirdest issues with PPoE WAN using pfsense/opnsense on an APU2.

Feature wise opnsense is really great but to me it looks like the praise for the BSD networking stack doesn't live up to the "it just works" state of Linux :confused:.


Can as a test but wouldn't run that myself. I am trying to migrate away from it (currently running in VM) as development has really stalled.
I want to move away from pfsense myself but if we can say that pfsense gives you 10gbps and opnsense does not then we can go back to the opnsense folks with some solid things to fix.
 

joellinn

New Member
Apr 5, 2022
7
1
3
I want to move away from pfsense myself but if we can say that pfsense gives you 10gbps and opnsense does not then we can go back to the opnsense folks with some solid things to fix.
I checked with pfSense and I am getting 5.5Gbps (iperf3 -s on pfsense, with -P2 it reaches >9Gbps) and > 9Gbps in reverse direction.

Goto the networking forum and ask that question. Enjoy the answers.
I personally wish for OpenWrt kernel and drivers with OPNsense Ui and features.

Edit:
Just tested with OpenWrt. It can basically do full speed 10Gbps across the firewall (default rules) using vlans on wan and lan
 
Last edited:

joellinn

New Member
Apr 5, 2022
7
1
3
Could have told you ;). There is a sticker on the case that says "Softlayer, IBM". It's also written in the thread someplace.
 

Churchill

Admiral
Jan 6, 2016
835
208
43
Could have told you ;). There is a sticker on the case that says "Softlayer, IBM". It's also written in the thread someplace.
I was scanning through the thread and saw the "Well the vendor was going to make us pay for the unlock but we figured it out.." part and then skipped to the end.

Reading comprehension is not 100% my thing.

Also, did you hear back from Opnsense about the speed issue? They released 22.1.6 maybe there's some tweaks in it? I saw the thread and Franco hasn't commented sadly :(
 

joellinn

New Member
Apr 5, 2022
7
1
3
Not really thread there is pretty much dead. Like this one :p. Might actually switch to OpenWrt. Bit less features but it usually just works with no tuning
 

EasyRhino

Active Member
Aug 6, 2019
250
144
43
I dimly recall trying to run openwrt on it, and failing to get it to install. (it might have been attempting in a VM, or maybe bare metal, can't remember any more)
 

petersnows

New Member
Jan 22, 2021
4
0
1
View attachment 22282
Registered just off this thread, great community here, have a build in progress used a lot of the information here to get up and running. It’s functional, gotta make a top for it though, it’s part of a bigger themed project so you’ll have to excuse some oddities for looks

planning a right up once it’s a bit more complete, quite the learning experience
Great Build.
Can you share a bit more ?
Case ?
Fans ?

Thanks
 

DAMatson

New Member
Apr 28, 2022
2
1
3
Good evening folks,

I too have found this SuperMicro system to be a good deal, most notably for the numerous 10G ports. I have a 5GBe connection from AT&T, but when I connect the 10G port on the motherboard to the 5G of the AT&T Router (Humax BGW320-500), the most I see on a connection basis (as reported by pfSense) is 1000baseT.

I've tried the pciconf -lv command and have this output, which I believe indicates that the driver for the Intel NIC is loaded properly at least (but I could be wrong about this):

ix0@pci0:4:0:0: class=0x020000 card=0x152815d9 chip=0x15288086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10-Gigabit X540-AT2'
class = network
subclass = ethernet
ix1@pci0:4:0:1: class=0x020000 card=0x152815d9 chip=0x15288086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10-Gigabit X540-AT2'
class = network
subclass = ethernet
ix2@pci0:5:0:0: class=0x020000 card=0x152815d9 chip=0x15288086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10-Gigabit X540-AT2'
class = network
subclass = ethernet
ix3@pci0:5:0:1: class=0x020000 card=0x152815d9 chip=0x15288086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10-Gigabit X540-AT2'
class = network
subclass = ethernet
ix4@pci0:6:0:0: class=0x020000 card=0x152815d9 chip=0x15288086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10-Gigabit X540-AT2'
class = network
subclass = ethernet
ix5@pci0:6:0:1: class=0x020000 card=0x152815d9 chip=0x15288086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Controller 10-Gigabit X540-AT2'
class = network
subclass = ethernet

I have selected ix0 for the WAN, and ix1 for the LAN. However, I have tried other port combinations and I'm having the same problem on each of them.


Speed and Duplex
In the "Speed and Duplex" setting under each interface, it does reflect that 10Gbase-T media "is Supported by this interface", which again supports the theory that it is correctly configured. However, when I do select to hardcode the 10Gbase-T setting, it still does not change the status page:

Status > Dashboard
WAN 1000baseT <full-duplex,rxpause,txpause> xxx.xx.xxx.xx
LAN 1000baseT <full-duplex,rxpause,txpause> 192.168.1.1

Speedtest (AT&T Internal)
The AT&T Router has been testing the connection on a recurring basis, so I will take this at face value until we can prove otherwise:
Upstream 4945.743
Downstream 5432.466

Speedtest
I took the liberty of installing the CLI version of Speedtest on the pfSense box as nothing more than an indicator of whether or not I was truly connecting at a speed greater than 2+ or 3+ GBps, it unfortunately reflects that I am still sitting at sub-gigabit speeds:

Retrieving speedtest.net configuration...
Testing from AT&T U-verse (xxx.xx.xxx.xx)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Open10G (Emeryville, CA) [114.99 km]: 17.479 ms
Testing download speed................................................................................
Download: 866.67 Mbit/s
Testing upload speed......................................................................................................
Upload: 784.22 Mbit/s

I'm relatively new to pfSense and the various *nix flavors, so I don't really know the best way to troubleshoot this further, so hoping someone here can assist my newb self...

Lastly, I have attempted to access the BIOS on my system via DEL button during POST, but it never gains access. I am hoping there is an easy fix, as this doesn't feel like a defective board.
 

EasyRhino

Active Member
Aug 6, 2019
250
144
43
yep. the x540 ports are not compatible with multi-gig. They can't do 2.5gb or 5gb. Only 1gb and 10gb.
 

DAMatson

New Member
Apr 28, 2022
2
1
3
Wow. Thank you for that brutal but honest solution. My bad for not asking the right question of the seller.

Ok, so then does anyone have a system that has two "multi-gigabit" compatible ports rated up to 5GB each that they can recommend?
 
  • Like
Reactions: cb0r