Time to upgrade my Supermicro Server

[DrTebi]

Hello there, I need some motherboard suggestions, and whether I can salvage parts of my old server:

It has been 13 years since I built a file server for my home:

• SUPERMICRO SYS-5027R-WRF 2U Server Barebone LGA 2011 DDR3 1600
• Intel Xeon E5-2609 2.4GHz LGA 2011 80W Quad-Core Server Processor
• Kingston 8GB 240-Pin DDR3 SDRAM Server Memory DR x4 Hynix C Model KVR16R11D4/8HC
• 6 x Western Digital Caviar Black 1TB 3.5" SATA 6.0Gb/s Internal Hard Drive -Bare Drive
• Adaptec RAID 6805 2271200-R 6Gb/s SATA/SAS 8 internal ports w/ 512MB cache memory Controller Card, Kit

OS is FreeBSD 7.
This was not a cheap home server, but considering how long it lasted (actually, it's still working fine), it was definitely worth it.

Nevertheless, here are a couple of reasons why I want to upgrade:

• It uses too much power
• It's loud (I moved, no more basement space)
• It's a bit slow
• OS urgently needs an upgrade

I have already done a bunch of research, but have a hard time finding a good compromise for these requirements:

• Less than 50W idle, possibly a sleep mode feature
• Supports at least 4 SATA SSD drives
• 2.5Gbe, preferably 10Gbe ethernet
• Decent performance
• Reliable manufacturer (Supermicro, ASRock...)

My use case is really just for personal files. These can be very large, e.g. movies (e.g. GoPro footage), Music, large Blender files, and things like 1 GB files of stitched together medium format film scans
However, there won't be much multi-user requirements, in rare cases two users at the same time.

Also: If I could salvage some parts of my old server, that would be great. Especially the case, I suppose it shouldn't be too difficult to fit it with new components.

My real question is: What motherboard? I have a hard time finding one that fits my requirements (and doesn't cost over $1000).

Any comments would be greatly appreciated.

 

[nexox]

Under 50W with a proper server board is going to be tricky, but still possible. I don't have personal experience upgrading a Supermicro W chassis from an X9 board to and X10 or X11, you'll need to check or ask someone else on here, but since those boards are proprietary they can be had very cheaply, at least in the US, you can get an LGA-1151 board with a small Xeon for quite a low price (eg: X11SCW-F - SUPERMICRO - SERVER MOTHERBOARD Single Socket LGA-1151 (Socket H4) US | eBay). If the chassis and risers are in the same physical configuration as later boards then you should be able to go to at least X11 because they're all PCIe 3.0 up to that point. Note the small Xeons (as opposed to the LGA3647 or LGA2011) require ECC UDIMMS, which are more expensive than RDIMMS, You may be able to get a single socket Xeon E5 v4 under 50W but that's only a couple years newer than your current setup, a Skylake first gen Scalable Xeon is probably going to come in a little over your desired idle power and that will take a bit of effort.

You can also quiet that chassis down at least a bit by going to an active CPU cooler and replacing the mid-wall fans with some which can spin down a little better (I'm using Arctic P8 Max fans in a similar 2U, temperatures are mostly fine, though I did add an extra fan to cool my hot PCIe devices.) If you select a CPU with a higher temperature rating the fans will generally spin slower.

That chassis should take more modern PSUs too, which are also cheap, so you can go Platinum rated with a 500W supply to cut idle power further.

 

[DrTebi]

That's a lot of great info already, thank you so much.

Another thought I had about saving energy is some kind of sleep mode... I only heard about it recently. I suppose it's similar to what you do on a desktop OS, and the server would wake up on LAN? Do server boards support this? And I wonder what power usage it would have "sleeping".

I also thought about replacing the fans, thanks for the Arctic fans tip. CPU cooling depends a bit on the board I can fit in there I guess.

 

[nexox]

Assuming you go for a system with IPMI you're looking at 7W to 10W usage when the system is off, I've never really bothered with Wake on LAN, but have seen several compatibility issues come up on this forum, though you still have the IPMI to potentially use to power on. Server boards tend to have little or no support for suspend or hibernate modes, and the documentation generally doesn't mention them at all, so you would have to either buy one and try it or find someone with a very similar model to check.

Aside from that you want to select hardware with the best ASPM support you can, which, if you get everything right, will allow the CPU to drop down to lower power modes when idle, without suspending or similar. Fortunately you don't want a SAS controller and there's at least one 10G NIC with full ASPM support (one of the newish Intels, I forget which, but the details are on this forum,) so if the motherboard isn't too poorly configured you can probably make it work.

 

[DrTebi]

Thanks again. I will try to find that card; I was already thinking about using an add-on 10G card for my upgrade, in case I cannot find a motherboard that has it onboard.

If I may ask another question: Would it make any sense at all to try to just upgrade what I have to 10G and replace the HDDs with SDDs? I am sure there is some kind of adapter for the hotbays so that I could use SDDs...? I am afraid the answer will be "why risk it"... it's an old server, who knows how long before it dies...

 

[nexox]

I was already thinking about using an add-on 10G card for my upgrade, in case I cannot find a motherboard that has it onboard.

Onboard 10G is very unlikely to be good on power, and SFP+ ports are hard to find if you're looking for that.

I am sure there is some kind of adapter for the hotbays so that I could use SDDs...?

You can get 3.5" to 2.5" bay adapters which have a short bit of wiring to put the 2.5" drive in a good spot, or you can find newer Supermicro trays that have a couple extra screw holes on one side to fit the 2.5" drive, which then only needs a cheap plastic (I 3D print mine) adapter to just support the other edge of the drive, the SATA/SAS connector is placed in the same spot as the 3.5" drive.

SSDs are definitely good for boot drives or specific uses with a lot of random IO, but maybe not to replace bulk storage on spinning drives, which are really pretty fast for sequential reads on large files. I don't think that you can find a backplane for that chassis to add U.2 NVMe support, but those drives will bring more power consumption and ASPM issues anyway, enterprise SATA drives are way better for a lower power system.

 

[DrTebi]

I have the feeling that, if you were in my case, you would say "never change a running system" ?
I am considering other options now instead of rebuilding the entire server. After all, it's all good quality components and still working flawlessly.

Some approaches I am considering:
- Upgrade to the latest FreeBSD version
- Replace fans with quieter ones
- Upgrade the server with a PCI 3.0 8x NIC that supports 10Gb/s
- Add a bridge router to my network that supports 10Gb/s
- Upgrade my desktop also with the approriate NIC

I have the feeling that my speed issues would be minor after doing that.

I could additionally get onto a schedule to make monthly backups to an external SSD.

In terms of power use, since it's 99% of the time just me using the server, I could try to do some automatic shut-offs over night, or maybe look into the possibility to implement ASPM.

Maybe a better strategy?

 

[kapone]

Nevertheless, here are a couple of reasons why I want to upgrade:

• It uses too much power
• It's loud (I moved, no more basement space)
• It's a bit slow
• OS urgently needs an upgrade

All of those are debatable reasons.

Your system uses an X9SRW-F motherboard. Unless it is really ancient (i.e. not v1.2 if I recall correctly), it can support Ivy Bridge CPUs. You're using an E5-2609 Sandy Bridge era CPU. That's a quad core CPU with no hyper-threading. That's seriously ancient and under powered.

What's too much power? - An 8C/16T Ivy bridge in a similar board with 4-8 RDIMMS will idle at ~40w. Is that "too much"?
It's loud - Of course it is. It's a server...in a server chassis...with server fans...and server fan profiles. Read up on how to make them quieter.
It's a bit slow - Doing what?
OS urgently needs an upgrade - That's a software problem, not a hardware problem.

 

[DrTebi]

It uses about 120W of power. 50W would be a good compromise, unless I keep it as is and change my habits and shut it down over night.
The "slow" part is solved... it was a cable issue.
Yes, I will look for quieter fans.
Upgrading the OS may be a bit of a task, but certainly doable.

I think my biggest concern should be longevity?

 

[kapone]

I think my biggest concern should be longevity?

Why?

 

[nexox]

The most likely parts to fail are the HDDs, should be pretty easy to upgrade since you only have 6TB raw storage, then maybe the PSU(s), also easy to replace, beyond that it's thermal cycling fatiguing soldered connections on the motherboard, which is what I think is slowly killing my Supermicro H8 system. Probably replace your CMOS battery too if it's more than about 5 years old.

 

[DrTebi]

Replacing the drives is probably a good idea. And I suppose I can replace them with SSDs, which should theoretically also cut down in power consumption just a bit.

Longevity is probably then not really that big of a deal if I would replace the drives, because even some of the other parts should break, I wouldn't loose my data.

I am still toying with the idea to add a faster networking card. From what I understand, the board can take 2 PCIe 8x boards, but I am not 100% sure. The manual is a bit confusing on this.

The battery definitely needs to be replaced... totally slipped my mind.

 

[nexox]

Replacing the drives is probably a good idea. And I suppose I can replace them with SSDs, which should theoretically also cut down in power consumption just a bit.

Even just swapping to two 10TB spinning drives in a mirror would save you ~15 watts, plus if you could go to a software RAID you can get rid of the hardware RAID controller and save some more.

If you go for SSDs, that board probably only has two SATA 6G ports so you would want some kind of controller to connect them fast enough, plain SATA controllers on PCIe are pretty low power and I'm pretty sure they usually support ASPM. Either way stick to enterprise grade SSDs with power loss protection, consumer drives are not great in RAID and lack of PLP causes all sorts of data corruption and performance issues.

I am still toying with the idea to add a faster networking card. From what I understand, the board can take 2 PCIe 8x boards, but I am not 100% sure. The manual is a bit confusing on this.

This depends on the chassis and the risers you have, the motherboard manual doesn't know about that, which is why it's vague. You should be able to find part numbers on the risers themselves to determine how many lanes each slot gets, but the board should provide access to most of the 40 lanes provided by the CPU via some slot configuration, I would imagine you could get at least three x8 slots unless you're missing risers.

 

[WANg]

Hello there, I need some motherboard suggestions, and whether I can salvage parts of my old server:

It has been 13 years since I built a file server for my home:

• SUPERMICRO SYS-5027R-WRF 2U Server Barebone LGA 2011 DDR3 1600
• Intel Xeon E5-2609 2.4GHz LGA 2011 80W Quad-Core Server Processor
• Kingston 8GB 240-Pin DDR3 SDRAM Server Memory DR x4 Hynix C Model KVR16R11D4/8HC
• 6 x Western Digital Caviar Black 1TB 3.5" SATA 6.0Gb/s Internal Hard Drive -Bare Drive
• Adaptec RAID 6805 2271200-R 6Gb/s SATA/SAS 8 internal ports w/ 512MB cache memory Controller Card, Kit

OS is FreeBSD 7.
This was not a cheap home server, but considering how long it lasted (actually, it's still working fine), it was definitely worth it.

Nevertheless, here are a couple of reasons why I want to upgrade:

• It uses too much power
• It's loud (I moved, no more basement space)
• It's a bit slow
• OS urgently needs an upgrade

I have already done a bunch of research, but have a hard time finding a good compromise for these requirements:

• Less than 50W idle, possibly a sleep mode feature
• Supports at least 4 SATA SSD drives
• 2.5Gbe, preferably 10Gbe ethernet
• Decent performance
• Reliable manufacturer (Supermicro, ASRock...)

My use case is really just for personal files. These can be very large, e.g. movies (e.g. GoPro footage), Music, large Blender files, and things like 1 GB files of stitched together medium format film scans
However, there won't be much multi-user requirements, in rare cases two users at the same time.

Also: If I could salvage some parts of my old server, that would be great. Especially the case, I suppose it shouldn't be too difficult to fit it with new components.

My real question is: What motherboard? I have a hard time finding one that fits my requirements (and doesn't cost over $1000).

Any comments would be greatly appreciated.

Brand-new or cascading hand-me-downs? If it's brand new, maybe an HPE Microserver Gen 11 (although I don't like the quad 1GbE Broadcom NIC versus the i350-AM4 in the Gen 10s)...but it's fundementally overpriced for what it is.

If you want something new from a decent (not great, decent) brand, eeeeh, Terramaster F6-424 Max - Newegg quotes it for 799 USD until August 31st. Takes laptop SODIMMs up to 2 x 64GB (no ECC though), 6 SATA bays, 2 PCIe 4x4 NVMe slots and a pair of Intel i226 nbaseT NICs. No PCIe slot for further expandability though - Terramaster quotes 56w with 6 WD Red 22TB drives in the bay and running, they tend to do well with TrueNAS Scale/Proxmox 9 and all that other fun stuff. I don't remember if they can do VPro for out of band access.

Both are probably more of a reasonable setup if you are running off a home office.

 

[DrTebi]

Even just swapping to two 10TB spinning drives in a mirror would save you ~15 watts, plus if you could go to a software RAID you can get rid of the hardware RAID controller and save some more.

If you go for SSDs, that board probably only has two SATA 6G ports so you would want some kind of controller to connect them fast enough, plain SATA controllers on PCIe are pretty low power and I'm pretty sure they usually support ASPM. Either way stick to enterprise grade SSDs with power loss protection, consumer drives are not great in RAID and lack of PLP causes all sorts of data corruption and performance issues.



This depends on the chassis and the risers you have, the motherboard manual doesn't know about that, which is why it's vague. You should be able to find part numbers on the risers themselves to determine how many lanes each slot gets, but the board should provide access to most of the 40 lanes provided by the CPU via some slot configuration, I would imagine you could get at least three x8 slots unless you're missing risers.

I am a little confused. Don't I have a riser already for the Adaptec RAID controller? I opened the case and see one vertical card with four slots, one of which is occupied with the Adaptec. The Adaptec manual says it supports SATA II only, which should be fine for newer SSDs, but not as fast I suppose?
I have to look up the specs and report back on the details.

 

[DrTebi]

Brand-new or cascading hand-me-downs? If it's brand new, maybe an HPE Microserver Gen 11 (although I don't like the quad 1GbE Broadcom NIC versus the i350-AM4 in the Gen 10s)...but it's fundementally overpriced for what it is.

If you want something new from a decent (not great, decent) brand, eeeeh, Terramaster F6-424 Max - Newegg quotes it for 799 USD until August 31st. Takes laptop SODIMMs up to 2 x 64GB (no ECC though), 6 SATA bays, 2 PCIe 4x4 NVMe slots and a pair of Intel i226 nbaseT NICs. No PCIe slot for further expandability though - Terramaster quotes 56w with 6 WD Red 22TB drives in the bay and running, they tend to do well with TrueNAS Scale/Proxmox 9 and all that other fun stuff. I don't remember if they can do VPro for out of band access.

Both are probably more of a reasonable setup if you are running off a home office.

Thank you for these suggestions. I am now considering to keep the server though, and just upgrade it, still trying to figure out how to as you can see in the posts following my initial post.

 

[nexox]

I am a little confused. Don't I have a riser already for the Adaptec RAID controller? I opened the case and see one vertical card with four slots, one of which is occupied with the Adaptec. The Adaptec manual says it supports SATA II only, which should be fine for newer SSDs, but not as fast I suppose?
I have to look up the specs and report back on the details.

You definitely have at least one riser, but I think they come in different slot configurations, plus that chassis can fit two risers, one for each stack of slots. Your RAID controller is fine for SATA SSDs, but if you can get rid of it and go to software RAID (or ZFS or whatever,) either with onboard SATA ports or a non-RAID SATA adapter you'll save some power.

 

[DrTebi]

Some specs on the Adaptec RAID 6805:
- PCIe Gen2 x8
- 6 Gb/s per port
- 8 direct-attached disks
And, one confusing spec:
- maxCache SSD support: 6805Q (I have 6805): Up to eight solid state drives, 1TB capacity max.
--> Does this mean I cannot install SSDs with more than 1TB? Or is that related only to that maxCache feature?

Some specs on the SUPERMICRO SYS-5027R-WRF:
- A SATA controller is integrated into the chipset to provide a six-port, SATA subsys-
tem, which is RAID 0, 1, 5 and 10 supported. The SATA drives are hot-swappable
units. Two of the ports support SATA 3.0 (6 Gb/s) while the other four are SATA
2.0 (3 Gb/s) ports.
- An additional SCU SATA controller integrated into the chipset provides four SATA 2
ports. RAID 0, 1 and 10 are supported. The SATA drives are hot-swappable units.

- PCI Expansion Slots
Two riser cards are used to support PCI expansion cards in the system. The RSC-
R2UW-E8R riser card can support a PCI-E 3.0 x8 card and the RSC-R2UW-4E8
can support four PCI-E 3.0 x8 cards.

So from what I understand, I have another PCIe 8x slot free, which I could potentially use for a 2.5Gb/s card.

 

[kapone]

Lemme see if I can say this plainly.

- Your Adaptec RAID card is e-waste at this point. Junk it. You can get 3 generations newer (like 81650) for $30.
- Your HDDs are e-waste at this point. Junk them and get newer ones. You have six 1TB drives, you could get two 6TB (or bigger) drives and simply mirror them (for redundancy) and cut down on your power consumption. I don't know how much storage you need.
- That SSD spec above is for maxcache. If you don't know what it is, don't think about it.
- You likely have one or both risers for the X9SRW in your chassis. You have more than enough x8 slots. You can use them for whatever you want.

That all said, you don't seem too knowledgable about these things. Get a friend to help you. Sorry, no other way to say this.

 

[DrTebi]

I appreciate your comments. I am not an expert, nevertheless I managed to build a NAS server that ran flawlessy for 13 years, so maybe not a total beginner

The idea to junk everything because it's old is a bit odd. The expenses for a complete new server will some take some time to offset the cost of energy usage. The system is also still running perfectly fine, so I am not sure why I should "junk" everything. A new system would give me piece of mind that it wouldn't start to fail, but I would have to probably put together a server that is of the same quality of what the current one was at the time. If I junk everything and build a cheap server, I may get faster transfer rates, lower power consumption, but what guarantee that it will last 13 years like this system did?