1U Supermicro Server 6x 10GBE RJ45 X10SLH-LN6TF LGA 1150 H3 X10SLH-N6-ST031

B

Bert

Active Member
Mar 31, 2018
470
177
43
43
I originally bought this motherboard to use it as a 6 port 10Gb unmanaged switch and also it can be used as a server to host a file server/VMs/firewall etc. After spending several weeks, I just found that easiest way to get that is simply installing linux (mine is debian) and enabling a bridge:

www.cyberciti.biz

Debian Linux: Configure Network Interfaces As A Bridge / Network Switch

My server has five Ethernet ports and one ADSL port. How do I setup IPv4 software bridge using Debian Linux operating systems so that the rest of five ports act as a network switch?
www.cyberciti.biz www.cyberciti.biz

then everything else works out of the box with perfect line speed of 9.41Gb/sec as measure from iperf. I was not able to measure total agg speed but this is already more than good enough for me. It is amazing to see how performant the bridge implementation in linux, quite amazing I must say.

Just wanted to share what I tried so far:
- PfSense on Hyper-V: I spent 2 weeks on online guides to configure a bridge on pfsense but it is too complicated and couldn't figure out. Pfsense, being a firewall is not designed to run a switch.
- Windows Bridge on Windows Server 2016: This is a built-in bridge functionality in windows and has been there for decades but it looks MS didn't update the implementation and its performance is horrible.
- VyoS pn Hyper-V: This also took me several weeks to make it work and got lots of help from @Marsh. VyOS was an overkill for switch purposes as it designed to be as router. I was missing some crucial steps in hyper-v configuration which prevented me to make VyOS running on Hyper-V. After all that, I must say VyOS didn't work good as I was limited to ~5 - 6 Gbit/s. My guess is that this is due to the inefficiency of running VyOS on Hyper-V.

I think a better set up would be using Proxmox instead of Debian but I am fine with using VirtualBox on Debian.
 
  • Like
Reactions: techtoys and Marsh
eduncan911

eduncan911

The New James Dean
Jul 27, 2015
635
482
63
eduncan911.com
+100 for Proxmox. Once you install it, figure out the odd/different networking (it makes a lot of sense once you "get it"), you'll never ever go back to anything. Doubly so when you learn about Proxmox's native OVS built-right in as a first-class citizen (Open Virtual Switch, or Open vSwitch).

Especially if you setup a NAS with shared storage on that 10G network. Then you can "move" VMs around from machine to machine all through a simple interface (given, changes to IP DNS has to be resolved first).

Also, if you have more than one server: they all can be joined to a cluster, managed by a single interface (like vSphere).

I have Proxmox installed on SoC devices as small as 4 GB of ram running lightweight containers (and a VM or two). You basically disable the heavy Proxmox services to bring it down to around 1GB of usage.

It really irks me that I don't have the same control over all of my RPi devices like I can a Proxmox cluster.
 
Last edited:
E

EasyRhino

Active Member
Aug 6, 2019
250
144
43
I tried setting up both opnsense and pfsense with both proxmox and esxi.

I couldn't get the *sense to successfully bridge the Ethernet ports on either.

With proxmox I was able to set up a Linux bridge there and pass it to *sense, so that worked, but I couldn't get GPU pass thru to work which was kind of the point of other VMS.

So it's just bare metal opnsense now. And works fine
 
  • Like
Reactions: Bert
D

discoeels

Member
May 8, 2013
42
8
8
I recently got this board off EBay. Flashed the new Bios from Ez and then reset IPMI with IPMItools. It's a cool board. I'm a huge fan of oddballs but I decided to go with something that I can put in my CSE-731I-300B chassis (X10-SLF). The bios fan control completely freaked out when I mixed an 80mm Noctua with 120mm fans so YMMV depending on your setup. A fun project but I need "just works" for labs/self-study.
 
  • Like
Reactions: Samir
M

Marsh

Moderator
May 12, 2013
2,547
1,398
113
I transplant the system board from the 1u chassis to a iStar 2u matx case

Fan profile set to Optimal speed, no need to run ipmitool to set threshold.
2 Front fan running at 1100rpm, CPU fan is at 1600 rpm.
Nice and quiet , just low hum.
 
  • Like
Reactions: Samir, cesmith9999 and Chuntzu
E

EasyRhino

Active Member
Aug 6, 2019
250
144
43
yeah in my home router setup I just screwed a 120mm case fan on top of the built in heatsink. it spins at minimal speed and the Xeon 1225v3 stays cool under the modest load. There's actually a very small 'click' sound in the power supply fan, but it's only noticeable if you stick your head up to the side.
 
  • Like
Reactions: Samir
C

chris heyworth

Member
Jan 24, 2020
64
23
8
if you put a good passive heatsink on the CPU and a fan over the X-540 heatsinks then it's near silent (even with the side off my case)

I am not using mine much anymore so may look to sell soon (UK only)
 
  • Like
Reactions: Samir and ullbeking
U

ullbeking

Active Member
Jul 28, 2017
506
69
28
43
London
chris heyworth said:
if you put a good passive heatsink on the CPU and a fan over the X-540 heatsinks then it's near silent (even with the side off my case)

I am not using mine much anymore so may look to sell soon (UK only)
Click to expand...
I am in the UK (London) and would be very happy to consider purchasing this if the time is right, etc.
 
  • Like
Reactions: Samir
E

EasyRhino

Active Member
Aug 6, 2019
250
144
43
TXAG26 said:
They pull a lot of power for a lightly used home server.
Click to expand...
Yeah, my old netgear home router probably used about 10W. This uses around 65W (and I still have the netgear as a wireless AP).

But, on the upside, it's got opnsense installed, and I....
a) have never had a bittorrent like download slow down due to router getting overloaded
b) haven't had anything like a whiff of a quality of service complain (streaming, Zoom etc)
c) have been confident that any time I lose internet connection, the solution is always to restart the cable modem to my dumb ISP, never the router.

and eventually I'll get some other device that supports 10GB RJ45 so I can actually use all those great ports :)

so it's probably worth the power wastage.
 
  • Like
Reactions: Samir
D

discoeels

Member
May 8, 2013
42
8
8
I ultimately decided that the RJ-45 ports limited my options. SFP+ add-in cards, direct attach copper, cheap modules, etc are out there. 10Gbe wasn't the main reason I was putting this together anyway. More Wireshark and troubleshooting l fun with LACP and MPIO too.
 
S

Samir

Post Liker and Deal Hunter Extraordinaire!
Jul 21, 2017
2,439
982
113
47
HSV and SFO
EasyRhino said:
Yeah, my old netgear home router probably used about 10W. This uses around 65W (and I still have the netgear as a wireless AP).

But, on the upside, it's got opnsense installed, and I....
a) have never had a bittorrent like download slow down due to router getting overloaded
b) haven't had anything like a whiff of a quality of service complain (streaming, Zoom etc)
c) have been confident that any time I lose internet connection, the solution is always to restart the cable modem to my dumb ISP, never the router.

and eventually I'll get some other device that supports 10GB RJ45 so I can actually use all those great ports :)

so it's probably worth the power wastage.
Click to expand...
Yep considering the power difference probably amounts to $10 USD for the whole year (if even that much).

Power isn't as big a factor in home labs as it is in the datacenter--unless your homelab looks like a datacenter. :D
 
Bjorn Smith

Bjorn Smith

Well-Known Member
Sep 3, 2019
538
281
63
48
r00t.dk
Samir said:
Power isn't as big a factor in home labs as it is in the datacenter--unless your homelab looks like a datacenter. :D
Click to expand...
Power is expensive in Denmark, so even a small homelab runs up into more than $1000 USD per year in power.

The "few" servers I run probably uses around 4-500w/h, which quickly amounts to something, when every kw/h cost around 0.38 USD :-(
 
eduncan911

eduncan911

The New James Dean
Jul 27, 2015
635
482
63
eduncan911.com
discoeels said:
I recently got this board off EBay. Flashed the new Bios from Ez and then reset IPMI with IPMItools. It's a cool board. I'm a huge fan of oddballs but I decided to go with something that I can put in my CSE-731I-300B chassis (X10-SLF). The bios fan control completely freaked out when I mixed an 80mm Noctua with 120mm fans so YMMV depending on your setup. A fun project but I need "just works" for labs/self-study.
Click to expand...
Since you modified the chassis from non-stock with the Noctua, no Supermicro board will "just work" any longer.

Either install the stock fan, or modify the IPMI fan settings like the rest of us.
 
D

discoeels

Member
May 8, 2013
42
8
8
eduncan911 said:
Since you modified the chassis from non-stock with the Noctua, no Supermicro board will "just work" any longer.

Either install the stock fan, or modify the IPMI fan settings like the rest of us.
Click to expand...
I guess I wasn't very clear. I didn't do anything to that chassis. I want to reuse it and repurpose the board that's in there to storage. I don't think this board would fit in there anyway.
 
  • Like
Reactions: eduncan911
Z

zanechua

Member
May 6, 2016
76
12
8
29
Upgraded the ipmi on the board from 3.84 to 3.90.

Used the ipmi firmware for the X10SLH-F for the upgrade.

Did a backup in case anything went wrong but seems to be working fine.
 
E

EasyRhino

Active Member
Aug 6, 2019
250
144
43
zanechua said:
Upgraded the ipmi on the board from 3.84 to 3.90.

Used the ipmi firmware for the X10SLH-F for the upgrade.

Did a backup in case anything went wrong but seems to be working fine.
Click to expand...
Interesting. Does the new ipmi do anything useful?
 
You must log in or register to reply here.
Top Bottom