Server upgrade (swapping components around)

[lpallard]

Hello there,

so I am planning to upgrade my freenas server to better hardware to save electricity and improve performance. I currently use a Supermicro 836TQ-R710B chassis with 16 hard drive bays and redundant 710W PSU.

Supermicro | Products | Chassis | 3U | SC836TQ-R710B

Motherboard is X7DBE+ and CPU's are dual Xeon L5420. HDD controller is M1015 flashed to IT.

New hardware will be Supermicro X9SCL motherboard with single Xeon E3-1220v2 and 32GB DDR3-1600 ECC UDIMM. I intend to reuse the chassis, backplane, redundant PSU's and M1015 controller.

Do you guys see a major show-stopper in swapping the internals of the existing server (mobo, CPU and RAM) for the new hardware? I think everything will be fine since the PSU have a 24 pin power connector and a 8pin CPU power connector compatible to 4 pin motherboards. The X9SCL has a 4 pin CPU power connector and the usual ATX 24 pin power connector.

I'd like also to reduce noise. These chassis are notorious for noise (fans). I intend to change the existing chassis 80mm fans (3 at the front between the backplane and CPU, and 2 at the back of the chassis) for Noctua 92mm REDUX fans. The X9SCL supports PWM fans but doesn't have IPMI (crap) so I am not sure how I will setup the fans speed from within the BIOS (full speed) or will they be controlled at the OS level? Should I even bother at all with upgrading these fans or can I make them quieter? Perhaps since the new CPU cooler will be active (with fan) the fans will mostly be to cool off the hard drives and motherboard and M1015 so maybe they can be quieter?

Finally, I intend to install a Noctua NH-D9L which is 110mm high... Do you think this will fit in a 3U chassis???

NH-D9L

Thanks!

 

[j_h_o]

I found the PSUs in my SC846 to be the major source of noise, not the case fans. Can you swap in PWS-920P-SQ into your chassis?

 

[lpallard]

Thats strange, when the server is powered off (but PSU's are still powered on) the tiny 24mm fans (or so) are barely noticeable compared to when its running (case fans ON).

I am not sure if I can swap PWS-920P-SQ in my chassis but what would be the benefit? These are 920W power modules. Are they more efficient or have NO fans?

 

[j_h_o]

Fire up the chassis, then unplug all the case fans (temporarily). Are the case fans really the source of the noise?

 

[CyberSkulls]

As to the PWS-920P-SQ, yes they would be more efficient and the fans don't spin when the server is powered off. Mine don't spin when the unit is powered on as my power draw is so low. So I drastically reduced the noise coming from my 846 chassis. I also left the stock fans in and run them off the motherboard fan controller. Much quieter now and drive temps still stay below 30 C.


Sent from my iPhone using Tapatalk

 

[cheezehead]

If your current fans are not PWM, replacing them with PWM fans would help. They've used a couple different model 80mm fans over the years, I've swapped out the stock ones for newer models which helped with the noise some but cut the power consumption per fan almost in half.

 

[lpallard]

As to the PWS-920P-SQ, yes they would be more efficient and the fans don't spin when the server is powered off. Mine don't spin when the unit is powered on as my power draw is so low

I assume going to much more powerful power modules than the actual server draws allows them to run away from their design point which in turns translates into lesser stress, heat and cooling requirements? Is that the rationale behind 920W? I am not sure what you guys use these servers for, but for FreeNAS, 920W is probably 6-7 times the actual power consumption I am targeting through this upgrade...

The other choice would be to completely remove the redundant PSU setup and power distribution board, and install a supermicro platinum PSU (not swappable and not redundant). Is that even feasible in their chassis?? Anyways whats the benefit for redundant PSU at home?

Another point is the chassis fans. The Supermicro page states "3x 80mm Hot-swap PWM Fans" so I assume they are PWM capable but I will confirm this once I pull the server out of the rack. Otherwise, I agree, I wil get 3 new Supermicro 80mm PWM fans to replace the existing ones...

Finally, nobody has commented on the Noctua CPU cooler in a 3U chassis, so I assume this will work clearance wise... ?!

 

[CyberSkulls]

I went for the power supply simply because it is quiet. I honestly didn't need the wattage. I only run one power supply in each of mine. That other hot swap bay is still housing the old screamer power supply, but it's pulled out enough to not be plugged into the PDU but still in far enough to block the air flow.

Yes the fans are PWM. I run mine off a Asus motherboard.

I just needed to quiet mine down till the next rebuild when I'll go back to tower chassis.


Sent from my iPhone using Tapatalk

 

[lpallard]

Ok, so far so good upgrading the 3U server with newer hardware:

• Except for the Noctua cooler than barely clears the top cover (about 1mm clearance left!)

• The EPS 8 pin power cable from the PDU that is WAYYYY too short (about 4 inches) (=>what was supermicro thinking?). I will get an extension cable from newegg and call it a day.

• I was mistaken: all case fans are PWM capable and have proper 4 wire connectors. The X9SCL-F supports PWM fans to this is gonna help a lot I think. I just need to get a PWM Y cable (splitter) from newegg since I am lacking one PWN header (6 fans overall and 5 headers on the mobo).

• The PSU's are NOT 710W as I initially expected. They are PWS-801-1R modules (800W). I owe an apology to you guys, the PSU fans are indeed louder than the chassis fans! I may replace the modules when finances allow. Can I do a replacement "in place" or do I need to shutdown the server? Do I need to replace the PDU as well? I assume if I replace the modules by lower wattage then the PDU will be OK?

• Finally, I noticed there are 2 more cables from the PDU that I am not sure what they do exactly. One is a 3wire/5pin cable named "PF" and the other is a 4 pin/4wire cable named "PD" (identical to a CPU 4 pin connector). I guess the "PF" cable is for power module failure alarm signal? Supermicro's documentation is non-existent for PDU's... But what about the PD ??.. Where do I connect them on the X9SCL-F? I can post pics if need be.

Looking forward to restoring the ZFS pool on this machine so I can get going once again!
Thanks

 

[cheezehead]

I was mistaken: all case fans are PWM capable and have proper 4 wire connectors. The X9SCL-F supports PWM fans to this is gonna help a lot I think. I just need to get a PWM Y cable (splitter) from newegg since I am lacking one PWN header (6 fans overall and 5 headers on the mobo).

• The PSU's are NOT 710W as I initially expected. They are PWS-801-1R modules (800W). I owe an apology to you guys, the PSU fans are indeed louder than the chassis fans! I may replace the modules when finances allow. Can I do a replacement "in place" or do I need to shutdown the server? Do I need to replace the PDU as well? I assume if I replace the modules by lower wattage then the PDU will be OK?

I'm one short on the PWM mobo headers as well and so far have just been leaving the left most fan unplugged without any issues, I've thought about the Y-splitters but hasn't been an issue for me yet.

The 801's work but for at home are very loud and use more power than what they need to. The PWS-920P-SQ are your best bet if money is no issue. I opted instead going for the PWS-1K21P-1R, overkill on the PSU size, still gold rated and a lot better on the noise (once fan control kicks in). The reason I went for those vs the 920's was price alone as these can be picked up for around $25/each vs for the 920's which cause an arm and leg in comparison. Be sure to verify the PDU revision for compatibility with the newer PSU's, they did change the hotswap plug a bit over the years.

As an aside, if anyone has moved from the PWS-1K21P-1R to the PWS-920P-SQ how much of an actual noise difference do you notice if the equipment is in another room? For me, my dehumidifier is louder than the 1200w's so the only time I hear it is during a reboot.

 

[lpallard]

Thanks cheezehead for replying.

I am too interested in the power modules from supermicro but before I pull the trigger on buying a 1200 Watt power module I would need to know if my PDU is compatible.

I am quite shocked to see that the 920 is a lot more expensive than the PWS-1K21P-1R..... the price difference is like 8x at least and it's less powerful. Perhaps highest efficiency is the culprit?

With all of that I am still wondering what are the PD and PF cables from my PDU???????

Do they connect somewhere on the x9scl????

If I don't connect them will my PSU still work?

I don't want to burn either the PSU or motherboard!!!!

 

[cheezehead]

Gold vs Platinum and also the SQ's are extremely quiet (ie you sleep a few feet from the box).

I would contact SM support to verify the PSU and PDU compatibility before pulling the trigger, they are usually very helpful.

Not sure on the PDU cables for 100%, I've got a slightly newer rev. The 5pin-3wire cable I suspect is the Power I2C connector...monitoring the power supplies, fan, and system temp. The 4-pin one I suspect has a long cable compared to the other one and is for the IPMB I2C header...for the optional IPMI card on supermicro servers of that day.

 

[lpallard]

OK server components successfully swapped around!

I have contacted Supermicro about the PD & PF connectors, and according to their reply, the PD connector is no longer used with modern motherboards and PDU's. Basically, the PD is only able to detect the power fail event of the redundant power supply.

The PF connector is able to read more information of the power supply module via the PMBus if the power supply module supports PMBus, but according to them, my PWS-801-1R does not support PMBus...

Other thing, I confirm the PSU module fans are LOUD, at least mine... Much noisier than the case fans.. I will upgrade them to a pair of PWS-1K21P-1R's once Supermicro confirms my current PDU supports these more modern PSU modules. All that but I am debating if I shouldn't go instead for a standard Supermicro single PSU unit. After all, I run only one of the PSU modules at a time, the other one being in standby. I recognize that I dont have redundancy, but having an extra 30-40w being drawn only for "in case something goes wrong", plus the noise of having 2 modules working simultaneously..

Do you have both of your PWS-1K21P-1R connected and powered on at the same time? Did you notice an significant power draw when using 2 modules instead of only one? I would imagine being 1200w, they must run "off curve" significantly and generate lots of heat??

If my PDU is compatible, running 2x PWS-1K21P-1R simultaneously does not cause a significant power draw, and they are quiet, then Id run them both and connect them to 2 separate UPS's.