UPS to server connections

[NB2020]

Hello. I am planning out a home server room build. As I try to figure out the physical wiring set up that people normally use, I am hoping some of you are able to assist.

So there are going to be 1-2 racks in the room. The servers have dual power supplies, and the room will be wired with at least 2 electrical circuits. There will also be AV equipment but I don't think they come with redundant PSUs.

For UPS setup, I plan to have a UPS for each circuit and every server would have its PSUs plugged into a different UPS. This would mean that the two PSUs in every server would be supplied by different circuits. Is that considered to be best practice?

In regards to battery replacement in the UPS, is it possible to swap the batteries without having to power down any equipment? I'm guessing the servers with redundant PSU would not have any issues, but what about those with only 1 power input? This may be dependent on the type and model of UPS, or maybe its ok as long as there is no power outage during the replacement process. I plan on getting online UPS's if that makes a difference.

Another thing I am looking into is whole-house surge protection. Would something like this have an impact on the equipment? I know we're not supposed to daisy chain surge protectors. Not sure if adding whole-house surge protection is an equivalent scenario.

 

[PigLover]

Having the redundant PSUs served by separate UPS would work and does give some level of redundancy. Not quite the same as having A/B power distribution the way you'd find in a data center, but workable. Its also MASSIVE overkill for a home lab!

Battery replacement procedure varies by UPS, but most of the common ones do allow you to replace the batteries "live" (as long as your AC power is live too). On most of them you just unplug the battery tray, slide it out, do your replacements and then slide it back in/plug it back in and all done. Tiny risk if you have a power failure while the batteries are disconnected, of course.

For the surge protector question I think you may need advice from a professional electrician! Its not something you want to get wrong.

 

[NB2020]

Haha I agree it’s overkill. Maybe I’m better served by getting a bigger UPS instead of 2 redundant ones. The way I justified it in addition to the redundancy is that it also allows me to use both 120v poles from the electrical power supply and evenly distribute the load. It’s supposed to be more efficient than having everything on 1 pole. I also haven’t come across any UPS that has multiple power inlets.

For servers with redundant PSUs, it seems the majority out there have a full-load/standby setup as opposed to half/half load. I will probably need to figure out which one is default active for each server to set everything up correctly.

I believe the surge daisy chain issue is not inherently bad, but it is due the setup allowing someone to plug in many more devices and making it a lot easier to overload the circuit. I will double check that with the electrician and contact the manufacturer when I decide on the hardware.

 

[mb300sd]

Haha I agree it’s overkill. Maybe I’m better served by getting a bigger UPS instead of 2 redundant ones. The way I justified it in addition to the redundancy is that it also allows me to use both 120v poles from the electrical power supply and evenly distribute the load. It’s supposed to be more efficient than having everything on 1 pole. I also haven’t come across any UPS that has multiple power inlets.

For servers with redundant PSUs, it seems the majority out there have a full-load/standby setup as opposed to half/half load. I will probably need to figure out which one is default active for each server to set everything up correctly.

I believe the surge daisy chain issue is not inherently bad, but it is due the setup allowing someone to plug in many more devices and making it a lot easier to overload the circuit. I will double check that with the electrician and contact the manufacturer when I decide on the hardware.

If you have both 120v poles, why not go 240v? More efficient and you can get 240v UPS's for cheap.

 

[NB2020]

If you have both 120v poles, why not go 240v? More efficient and you can get 240v UPS's for cheap.

I wanted to do that initially, but another forum member pointed out that the 240V UPSs out there are meant for 240V from a single pole. I can combine 2 120V poles to get 240V split phase, but I would need a transformer to convert that into 240V single phase. From how it was explained, it seems the efficiency losses from using a transformer would cancel out the gains from running 240V in the first place.

I've been looking at various 240V UPS models and their inlet/outlet type, but can't find anything concrete. There are a few that accept 240V split phase power, but the output is also split phase and I'm not sure if the PSUs from the servers would be able to safety draw power from that. I haven't come across any UPS with 240V split phase input and 240V single phase output. I'm guessing it is because it would need a built-in transformer? From what I understand, the prongs on a split phase plug would be hot-hot-ground whereas a single phase plug would be hot-neutral-ground. I'm not an electrician, so that is the part I am stuck at (i.e., can a psu plug into an outlet wired as hot-hot-ground?)

I have read other threads where people have wired their server room with 240V using 2 120V poles, but they do not explain how everything is connected together. Please let me know if there is something I am missing or misunderstanding.

 

[Rand__]

For devices with single power supply you need to get an Automatic Transfer switch to draw power from either USV (or the wall) if you dont want a half ar*ed solution...


Kidding of course, overkill as previously mentioned, but an ATS would indeed cover the other parts if you want more hardware

 

[mb300sd]

I wanted to do that initially, but another forum member pointed out that the 240V UPSs out there are meant for 240V from a single pole. I can combine 2 120V poles to get 240V split phase, but I would need a transformer to convert that into 240V single phase. From how it was explained, it seems the efficiency losses from using a transformer would cancel out the gains from running 240V in the first place.

I've been looking at various 240V UPS models and their inlet/outlet type, but can't find anything concrete. There are a few that accept 240V split phase power, but the output is also split phase and I'm not sure if the PSUs from the servers would be able to safety draw power from that. I haven't come across any UPS with 240V split phase input and 240V single phase output. I'm guessing it is because it would need a built-in transformer? From what I understand, the prongs on a split phase plug would be hot-hot-ground whereas a single phase plug would be hot-neutral-ground. I'm not an electrician, so that is the part I am stuck at (i.e., can a psu plug into an outlet wired as hot-hot-ground?)

I have read other threads where people have wired their server room with 240V using 2 120V poles, but they do not explain how everything is connected together. Please let me know if there is something I am missing or misunderstanding.

I have 240 split phase running to all my equipment. The PSUs are built for it, if you open any isolated SMPS (all server PSUs) and look at the PCBs the Line/Neutral are not differentiated at all - same circuit on both sides. The ground wire just goes to the case. Same for all the 240 UPSs I've encountered.

 

[NB2020]

Ah ok. That is good to know. I'm thinking one 240V 20A circuit per rack should be plenty. To sum up the basic wiring: Wall outlet to UPS would be NEMA L6-20 or similar. Then from UPS to PDU using whatever outlet type depending on the model. And from PDU to server PSU using C14-C13 plugs. Is this similar to your setup, which carries 240V split phase entirely from wall to server?

I looked at on-line UPS's from the APC, eaton, and cyber power. I wasn't able to find any from eaton or APC that lists split phase in their specs. All of the input voltages were either 120V or 208V. Would you be able to tell me which one you use?

 

[mb300sd]

Yep, it's split phase all the way down.

One setup has a Tripp-Lite SU6000RT4U, I wired a L6-30 on the input cord and it goes straight out to a C13 power strip.

The other setup has 2 GE VH2000s that I cut a the plug off a C13/C14 splitter cord and replaced it with a L6-30. C13 outputs go directly into the equipment with some C14/5-15R adapters thrown in for stuff with power bricks (check bricks are rated for 240, most are).

Check the spec sheets for the ones that list 208v input, they're usually good for 180-260. My GEs even have a 277v output setting.

 

[Tom5051]

What is so important in your home lab to require redundant anything?

 

[NB2020]

Yep, it's split phase all the way down.

One setup has a Tripp-Lite SU6000RT4U, I wired a L6-30 on the input cord and it goes straight out to a C13 power strip.

The other setup has 2 GE VH2000s that I cut a the plug off a C13/C14 splitter cord and replaced it with a L6-30. C13 outputs go directly into the equipment with some C14/5-15R adapters thrown in for stuff with power bricks (check bricks are rated for 240, most are).

Check the spec sheets for the ones that list 208v input, they're usually good for 180-260. My GEs even have a 277v output setting.

Thanks for the info! I think this is the route I will go with. In the other thread I started, someone mentioned that 240V split phase wouldn't be possible and got me second guessing everything.

What is so important in your home lab to require redundant anything?

Nothing is important really. This is something I am choosing to do. I will start small and build my way up, so there will be very little redundancy at the beginning. It's just that I don't want to end up needing to re-wire the room if the time to add equipment comes.

I plan to run the servers for file storage, media streaming, surveillance/security, and home automation. The only thing I care about is the family photos/videos. Everything else is just a mild inconvenience if the system goes down and data gets lost. Any equipment that come later would be for experimenting with VMs, testing out new software/hardware configs, etc. In any case, I acknowledge that a lot of my design choices are probably overkill.

 

[Tom5051]

fair enough. Well I've had no issues with the APC 1500 UPS I bought 10yrs ago, I modified mine to run on 100A external batteries which gives nearly 2 hours of run time. You can change the batteries without having to power down, the risk is if you lose power during this time your servers will ungracefully turn off.
I put all the important stuff on a RAID 6 array and also back it up to another array.
Everything is VM, even the router. So much better than the old days of physical servers for each
Remember you gotta power this stuff which also costs money.

 

[Stephan]

This would mean that the two PSUs in every server would be supplied by different circuits. Is that considered to be best practice?

For a home lab absolutely, also for small business entry solution. Make sure nevertheless that your filesystems are crash-tolerant and do not have any RAID write-hole should power go out in inopportune moments. SSDs should have capacitors and implement some power-loss-protection (PLP) scheme.

In regards to battery replacement in the UPS, is it possible to swap the batteries without having to power down any equipment?

No problem usually. The beeper and warning LEDs will go on. Use safety gloves like Hase Padua Pro against cuts because of sharp edges and in case you touch something juicy. If the UPS appears servicable without disconnecting anything on the back, batteries are hot-swap. If you are comfortable to do this and the UPS is older, after 2-3 days of charging check the voltage of the entire connected idle pack with a voltmeter. For 48V sealed lead acid pack it should be ~54.54V i.e. spot-on right at lower bound of battery's floating voltage spec. Otherwise if the UPS has drifted it will cook new batteries dry within 2-4 years with too high of a floating voltage. Or undercharge them when voltage is too low. Well-specced UPS will make batteries last more like 10 years.

I'm guessing the servers with redundant PSU would not have any issues, but what about those with only 1 power input?

Not much you can do. If you run a VM cluster, migrate services away to that. If its an RPi to drive the TV, tough. Make a backup of the SD card and that is that. For some things like PoE injectors for VOIP phones I like to keep a spare quality PSU in the drawer e.g. by MeanWell.

This may be dependent on the type and model of UPS, or maybe its ok as long as there is no power outage during the replacement process. I plan on getting online UPS's if that makes a difference.

Make sure you have well-tested automatic server shutdown in case batteries run low. Also a sheet for wife/girlfriend/kids in case automatic restart of servers after some charge level is reached is not working and you are not at home. Here in central Europe power is either out for way less than 30 minutes or much much longer. Online UPS is less efficient than line-interactive because inverters are constantly running but will produce a clean sine wave on output even with dirty power input.

Another thing I am looking into is whole-house surge protection. Would something like this have an impact on the equipment? I know we're not supposed to daisy chain surge protectors. Not sure if adding whole-house surge protection is an equivalent scenario.

You will daisy chain anyway, if you get a whole-house surge arrestor and use a UPS, because the UPS has MOVs (metal oxide varistor) on input to filter voltage spikes. But I do not see a problem. What you usually use in Europe is a spark gap arrestor like a DEHN DEHNshield. Once voltage goes above a threshold a spark will form and short everything to ground. Anything that is left will be shorted by the MOVs. Those MOVs age though, with every spike they have to absorb.

Make sure to review other ways of surges into the house, e.g. DSL lines. Best would be a fiber connection because that doesn't carry voltage inducted by nearby lightning. Again here in Europe, there are devices like "VDÜ-DSL" by l-tek dot net. This one uses a spark gap arrestor (again) for coarse protection like the DEHN and right behind that voltage suppressor diodes for fine protection. It will save your DSL router or modem but will lower the line rate by 10 MBit/s or more due to signal attenuation. If that is not workable (it wasn't for me) get a modem that acts as a victim in case of lightning. Anything over 400 volts will fry the DSL frontend of the device but the rest will live. Replace with cold standby from the drawer...

No affiliation to any of the products mentioned.

 

[edge]

Where are you planning to use the surge protectors? With the UPS you should use power distribution units (PDU) and not surge protectors.

 

[NB2020]

Where are you planning to use the surge protectors? With the UPS you should use power distribution units (PDU) and not surge protectors.

The whole house surge protector is wired in at the breaker panel. I certainly wouldn't connect one after a UPS in the circuit.

 

[ramicio]

I have 240 split phase running to all my equipment. The PSUs are built for it, if you open any isolated SMPS (all server PSUs) and look at the PCBs the Line/Neutral are not differentiated at all - same circuit on both sides. The ground wire just goes to the case. Same for all the 240 UPSs I've encountered.

You have answered the question I was seeking an answer to. I am going to be building a shed and want to run 240 volts (with 2 hot legs) to it. I want to do so only because I only have 50 amps of each leg as a budget between my home-shed and the server-shed (I live at a campground, so everything is wired for campers/RVs). Thank you for that information!