recommendation for quiet, power efficient, rackmount server?

[BLinux]

I have a pair of Dell PE2900 that I have rack mounted in my rack. It's sort of my main server, runs as storage server w/ about 16TB of ZFS, KVM hypervisor w/ 10~20 VMs, also my backup server, etc. The VMs run my small consulting business, testing lab, and some personal VMs. It's done well over the years, but over the holidays, the primary one had what looks like a main board failure so I swapped over to the secondary right now.

I can replace the failed motherboard for under $100, but these machines are starting to show their age. So, I'm wondering if I should just replace them with something more modern. Quiet and power efficient are important factors for me. I picked these PE2900 tower servers because they have 5U of height which allows me to use quieter fans. As for power efficiency, they have the L54xx Xeons (8 cores total) and at idle w/ RAID controller and 8x 3.5" HDD consume about 250W as seen by the managed PDUs. I'd like something quieter, more power efficient, and at least the same core count and memory (64GB). I definitely want a lot of 3.5" HDD slots, at least 8 - maybe more since I'm upgrading. Nothing crazy like a 45-bay or 60-bay, but wouldn't mind 16-bays to allow for expansion later.

I was considering the Dell T620 or T630. Seems to be the descendant of the PE2900. I think they are even quiet right out of the box without fan mods. Significantly more cost than $100 motherboard replacement; but if I can save on power it might be worth it over the next 3 yrs. Any opinions about the T620 or T630?

Any suggestions for other options I should consider?

Thanks.

 

[manxam]

Many of the supermicro 3u+ servers will fit your bill as they use regular 80mm fans which are easy to swap and have varying power supply options from normal to SQ (super quiet). You can get anywhere from 8 bay to 24 bay configurations. The 24 bay are all over eBay right now for a few hundred bucks USD with X8 (5500/5600) motherboards. I have a few of the 836 16-bay chassis' and love em.

If you want something more run-of-the-mill, buy a used dell R510. They have a 12 bay 3.5" option, decent amount of RAM slots, and fit dual 5500/5600 processors. They're quite cheap used and very quiet compared to your 2900...

 

[BLinux]

Many of the supermicro 3u+ servers will fit your bill as they use regular 80mm fans which are easy to swap and have varying power supply options from normal to SQ (super quiet). You can get anywhere from 8 bay to 24 bay configurations. The 24 bay are all over eBay right now for a few hundred bucks USD with X8 (5500/5600) motherboards. I have a few of the 836 16-bay chassis' and love em.

If you want something more run-of-the-mill, buy a used dell R510. They have a 12 bay 3.5" option, decent amount of RAM slots, and fit dual 5500/5600 processors. They're quite cheap used and very quiet compared to your 2900...

thanks for the suggestions. i'll have to look into those Supermicros a bit more; i'm not as familiar with them. Do you have specific models you suggestion?

As for the R510, the specs looks like it would meet might needs, but I'm unsure about the sound level with those smaller fans in the R510. I've gotten my PE2900 noise down quite a bit - there's a slight whine from the tiny fans of the PSU while the other 6x 92mm fans have been swapped out and running quietly. What type of noise level are we talking about with the R510? Do you have any numbers?

 

[KC0ISG]

I don't have any sound level measurements, but I am running a R510, R710, and a R310 in the same room and they are all fairly quiet. I can have a conversation on the phone without any issues and you can't really hear them in the next room. Here is a comparison I found of the R610 and R710 noise levels Dell and HP Server Noise Level and Comparison. A 510 is a bit quieter than a 710 in my experience.

 

[Terry Kennedy]

I was considering the Dell T620 or T630. Seems to be the descendant of the PE2900. I think they are even quiet right out of the box without fan mods. Significantly more cost than $100 motherboard replacement; but if I can save on power it might be worth it over the next 3 yrs. Any opinions about the T620 or T630?

Any suggestions for other options I should consider?

The Rx10 and Tx10 servers were the last models where features are enabled by (inexpensive) parts from eBay. x20 and x30 have everything on the mainboard, but you need a license to enable the advanced remote access, for example.

I have a fair number of R510/NX3100 servers (same hardware) and R710 servers. The R710's are my "go-to" servers for just about anything. They come with either 8 x 2.5" or 6 x 3.5" drive slots, have built-in DVD drive, lots of expansion slots. The R510/NX3100 are the same 2RU chassis as the R710, but stuff 12 x 3.5" drives in there, don't have an internal optical drive, and have controls and connectors on the rack ears. They also have fewer expansion slots (only 1 riser compare to the 2 in the R710). Changing a motherboard in an R510/NX3100 is substantially more complicated than doing it in the R710 (I've converted a bunch of R710's from 3.5" to 2.5" by ordering "barebones" 2.5" chassis and moving all the parts over).

 

[BLinux]

thanks guys for the suggestions. based on your input, i've spent some time looking at Supermicro options and have become very interested in the 846 chassis, but also found out there are many variants and options for backplanes, PSUs, etc. if possible, i could use some guidance on my endeavor here...

i know the 846 chassis has several variants, but I think i'm particularly interested in the 24-bay with the PSUs on the side, instead of across the bottom. my thought here is that it gives me more vertical height between the top cover and the motherboard allowing me better options for heatsinks for the CPUs - which I'm hoping will aid my quest for a quieter system.

as for backplane, it appears there are some old ones I need to avoid, such as those that only support up to 2TB drives. there are also the ones with expander chipsets vs direct attached. I'm okay with direct attach (just need break out cables) so long as I can avoid the 2TB limit. I don't mind the expander chip either. what are my best options here? for either approach, what are some good HBAs to pair with these backplanes? I'm planning to use ZFS, so probably some LSI card in "IT mode" ?

I also understand that the 846 chassis has some "SQ" PSU options that appear to be very quiet. This is of course very interesting to me, but from reading various forums about this, it appears that the PSU may be particular to a particular PDB, If so, then I have to be careful about what "SQ" PSU I get so it will be compatible with the PDB in the chassis I eventually get. Any help here with particular part numbers?

If possible, I'd like to pickup a refurbished/used unit that is close to what I need to minimize the effort required to get to my end goal. I'm hoping to find a unit that has L56xx Xeon CPUs.

As for quieting the system down, here are my thoughts so far:

1) the fan wall in the middle of the chassis are often hooked up to the SAS backplane which runs them at full speed. controlling these fans via the motherboard PWM controllers appears to help a lot. this should be fine as long as I don't use very hot hard drives, which I don't.. and I'm starting out with 8 drives, not the full 24 drives.

2) replacing the fans on the fan wall and rear of the chassis appears to help too. I'm considering modifying the fan wall to accommodate either 140mm or 120mm PWM fans. I think I can re-use the "frame" and cut out the middle section and replace it with a custom piece of sheet metal that I can rivet to the frame. the custom cut sheet metal will have holes to fit 140mm or 120mm fans x 3.

3) i'm considering getting rid of the air shroud that goes over the CPU+RAM section of the motherboard. this in order to do a similar modification on the rear of the chassis to accommodate a single 140mm or 120mm fan instead of the 2x 80mm fans. this to also be controlled via PWM from the motherboard. I want to start by trying to use the tallest/widest CPU heatsink I can find that fits and see if the fan wall and rear fan will be enough to cool the L56xx CPUs with a passive heatsink. If not, then I may bolt on a fan to the heatsink.

after being suggested to look at the Supermicro options, and actually doing a little research about them, what I find most appealing is that I'm hoping when this new system starts getting old, I can re-use the chassis and simply replace the mobo+cpu+backplane internals for future upgrades. this is very appealing to me vs the Dell PowerEdge route where it seems not worth the hassle to modify in order to accommodate newer internals into a PowerEdge chassis.

Any suggestions? Any pitfalls I need to be aware of and avoid?

thanks..

 

[i386]

You might get problems with add on cards (or the io hub on 1366 mainboards that gets very hot) with quiet (read low rpm) 120/140mm fans: I have a workstation with 3x lian li 140mm 3pin fans that run at 1.5k rpm in the front and 1x noctua 120mm rear fan (usually 350 rpm) and my mellanox cards get very hot (mget_temp reports 66°c for the connectx2 and 82°c for the connectx3). With pwm fans in the fron that would be even worse.

 

[BLinux]

hmmmm... starting to look at Supermicro 847 chassis options as well. I can get a 847 for about $1300 w/ 2x E5-2660 and already has "SQ" PSUs. If I go the 846 route, and after sourcing "SQ" PSUs, and various components for a L5640 system, it comes out over $1600+. Seems like a better deal to go 847, but that has a 2U layout for the motherboard and I'm wondering if that will limit my options to make a quiet server? Anyone have experience making 847 quiet?

 

[cheezehead]

Another option is buy 2 and resell the other....did that last time I couldn't find the exact config I wanted for the price I wanted to pay.

 

[FlashEngineer]

You might get problems with add on cards (or the io hub on 1366 mainboards that gets very hot) with quiet (read low rpm) 120/140mm fans: I have a workstation with 3x lian li 140mm 3pin fans that run at 1.5k rpm in the front and 1x noctua 120mm rear fan (usually 350 rpm) and my mellanox cards get very hot (mget_temp reports 66°c for the connectx2 and 82°c for the connectx3). With pwm fans in the fron that would be even worse.

How do you check temps of AOC? I have a M1015 in my sc836 but no idea how to check temps. I'm running Nas4free

 

[i386]

Cli tools that come with the driver packages, for the mellanox cards these tools are in the mft package.

I think lsi has a tool for their raid controllers, but I don't know if it works with the hbas.