1U Supermicro Server 6x 10GBE RJ45 X10SLH-LN6TF LGA 1150 H3 X10SLH-N6-ST031

[Bert]

I originally bought this motherboard to use it as a 6 port 10Gb unmanaged switch and also it can be used as a server to host a file server/VMs/firewall etc. After spending several weeks, I just found that easiest way to get that is simply installing linux (mine is debian) and enabling a bridge:

Debian Linux: Configure Network Interfaces As A Bridge / Network Switch

My server has five Ethernet ports and one ADSL port. How do I setup IPv4 software bridge using Debian Linux operating systems so that the rest of five ports act as a network switch?

www.cyberciti.biz

then everything else works out of the box with perfect line speed of 9.41Gb/sec as measure from iperf. I was not able to measure total agg speed but this is already more than good enough for me. It is amazing to see how performant the bridge implementation in linux, quite amazing I must say.

Just wanted to share what I tried so far:
- PfSense on Hyper-V: I spent 2 weeks on online guides to configure a bridge on pfsense but it is too complicated and couldn't figure out. Pfsense, being a firewall is not designed to run a switch.
- Windows Bridge on Windows Server 2016: This is a built-in bridge functionality in windows and has been there for decades but it looks MS didn't update the implementation and its performance is horrible.
- VyoS pn Hyper-V: This also took me several weeks to make it work and got lots of help from @Marsh. VyOS was an overkill for switch purposes as it designed to be as router. I was missing some crucial steps in hyper-v configuration which prevented me to make VyOS running on Hyper-V. After all that, I must say VyOS didn't work good as I was limited to ~5 - 6 Gbit/s. My guess is that this is due to the inefficiency of running VyOS on Hyper-V.

I think a better set up would be using Proxmox instead of Debian but I am fine with using VirtualBox on Debian.

 

[eduncan911]

+100 for Proxmox. Once you install it, figure out the odd/different networking (it makes a lot of sense once you "get it"), you'll never ever go back to anything. Doubly so when you learn about Proxmox's native OVS built-right in as a first-class citizen (Open Virtual Switch, or Open vSwitch).

Especially if you setup a NAS with shared storage on that 10G network. Then you can "move" VMs around from machine to machine all through a simple interface (given, changes to IP DNS has to be resolved first).

Also, if you have more than one server: they all can be joined to a cluster, managed by a single interface (like vSphere).

I have Proxmox installed on SoC devices as small as 4 GB of ram running lightweight containers (and a VM or two). You basically disable the heavy Proxmox services to bring it down to around 1GB of usage.

It really irks me that I don't have the same control over all of my RPi devices like I can a Proxmox cluster.

 

[EasyRhino]

I tried setting up both opnsense and pfsense with both proxmox and esxi.

I couldn't get the *sense to successfully bridge the Ethernet ports on either.

With proxmox I was able to set up a Linux bridge there and pass it to *sense, so that worked, but I couldn't get GPU pass thru to work which was kind of the point of other VMS.

So it's just bare metal opnsense now. And works fine

 

[discoeels]

I recently got this board off EBay. Flashed the new Bios from Ez and then reset IPMI with IPMItools. It's a cool board. I'm a huge fan of oddballs but I decided to go with something that I can put in my CSE-731I-300B chassis (X10-SLF). The bios fan control completely freaked out when I mixed an 80mm Noctua with 120mm fans so YMMV depending on your setup. A fun project but I need "just works" for labs/self-study.

 

[Marsh]

I transplant the system board from the 1u chassis to a iStar 2u matx case

Fan profile set to Optimal speed, no need to run ipmitool to set threshold.
2 Front fan running at 1100rpm, CPU fan is at 1600 rpm.
Nice and quiet , just low hum.

 

[EasyRhino]

yeah in my home router setup I just screwed a 120mm case fan on top of the built in heatsink. it spins at minimal speed and the Xeon 1225v3 stays cool under the modest load. There's actually a very small 'click' sound in the power supply fan, but it's only noticeable if you stick your head up to the side.

 

[ullbeking]

Can you actually get one of these servers stay near-silent when run at home (not pushing it too hard)?

 

[chris heyworth]

if you put a good passive heatsink on the CPU and a fan over the X-540 heatsinks then it's near silent (even with the side off my case)

I am not using mine much anymore so may look to sell soon (UK only)

 

[TXAG26]

They pull a lot of power for a lightly used home server.

 

[ullbeking]

if you put a good passive heatsink on the CPU and a fan over the X-540 heatsinks then it's near silent (even with the side off my case)

I am not using mine much anymore so may look to sell soon (UK only)

I am in the UK (London) and would be very happy to consider purchasing this if the time is right, etc.

 

[Erlipton]

They pull a lot of power for a lightly used home server.

agreed, can you think of any budget friendly alternatives? RJ-45 specifically...?

 

[EasyRhino]

They pull a lot of power for a lightly used home server.

Yeah, my old netgear home router probably used about 10W. This uses around 65W (and I still have the netgear as a wireless AP).

But, on the upside, it's got opnsense installed, and I....
a) have never had a bittorrent like download slow down due to router getting overloaded
b) haven't had anything like a whiff of a quality of service complain (streaming, Zoom etc)
c) have been confident that any time I lose internet connection, the solution is always to restart the cable modem to my dumb ISP, never the router.

and eventually I'll get some other device that supports 10GB RJ45 so I can actually use all those great ports

so it's probably worth the power wastage.

 

[discoeels]

I ultimately decided that the RJ-45 ports limited my options. SFP+ add-in cards, direct attach copper, cheap modules, etc are out there. 10Gbe wasn't the main reason I was putting this together anyway. More Wireshark and troubleshooting l fun with LACP and MPIO too.

 

[Samir]

Yeah, my old netgear home router probably used about 10W. This uses around 65W (and I still have the netgear as a wireless AP).

But, on the upside, it's got opnsense installed, and I....
a) have never had a bittorrent like download slow down due to router getting overloaded
b) haven't had anything like a whiff of a quality of service complain (streaming, Zoom etc)
c) have been confident that any time I lose internet connection, the solution is always to restart the cable modem to my dumb ISP, never the router.

and eventually I'll get some other device that supports 10GB RJ45 so I can actually use all those great ports

so it's probably worth the power wastage.

Yep considering the power difference probably amounts to $10 USD for the whole year (if even that much).

Power isn't as big a factor in home labs as it is in the datacenter--unless your homelab looks like a datacenter.

 

[Bjorn Smith]

Power isn't as big a factor in home labs as it is in the datacenter--unless your homelab looks like a datacenter.

Power is expensive in Denmark, so even a small homelab runs up into more than $1000 USD per year in power.

The "few" servers I run probably uses around 4-500w/h, which quickly amounts to something, when every kw/h cost around 0.38 USD :-(

 

[eduncan911]

I recently got this board off EBay. Flashed the new Bios from Ez and then reset IPMI with IPMItools. It's a cool board. I'm a huge fan of oddballs but I decided to go with something that I can put in my CSE-731I-300B chassis (X10-SLF). The bios fan control completely freaked out when I mixed an 80mm Noctua with 120mm fans so YMMV depending on your setup. A fun project but I need "just works" for labs/self-study.

Since you modified the chassis from non-stock with the Noctua, no Supermicro board will "just work" any longer.

Either install the stock fan, or modify the IPMI fan settings like the rest of us.

 

[discoeels]

Since you modified the chassis from non-stock with the Noctua, no Supermicro board will "just work" any longer.

Either install the stock fan, or modify the IPMI fan settings like the rest of us.

I guess I wasn't very clear. I didn't do anything to that chassis. I want to reuse it and repurpose the board that's in there to storage. I don't think this board would fit in there anyway.

 

[zanechua]

Upgraded the ipmi on the board from 3.84 to 3.90.

Used the ipmi firmware for the X10SLH-F for the upgrade.

Did a backup in case anything went wrong but seems to be working fine.

 

[EasyRhino]

Upgraded the ipmi on the board from 3.84 to 3.90.

Used the ipmi firmware for the X10SLH-F for the upgrade.

Did a backup in case anything went wrong but seems to be working fine.

Interesting. Does the new ipmi do anything useful?

 

[chris heyworth]

i am having a bit of a clear out so if anyone in the UK is looking for this motherboard let me know (currently on ebay)