Stopping The Cardinal Sin of Colocation Rack Power Sizing

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

NablaSquaredG

Bringing 100G switches to homelabs
Aug 17, 2020
1,857
1,234
113
One thing to remember when sizing power supplies is that, in case you are using two power supplies with the goal of redundancy in mind, a single power supply must be able to handle the load of the entire server. So usually it will appear that power supplies are twice as large as they have to be, which might not be the case.
 
  • Like
Reactions: zunder1990

Terry Kennedy

Well-Known Member
Jun 25, 2015
1,169
618
113
New York City
www.glaver.org
I've been involved in this for years (my first colo was in 1993 at 60 Hudson).

Some datacenters will offer you both "A" and "B" feeds to the cabinet, where A and B are claimed to run to independent UPS / generator / utility entrance facilities. Sounds nice in theory. You need to thoroughly investigate these claims, even to the point of requiring a tour of the power system rooms at the colo. You might even be offered the A/B option for a small upcharge, or even at no extra cost, as long as you keep your maximum steady-state load to something that could be satisfied by A or B only.

Another trap is sizing for the "normal case" where all of your power supplies are operating normally, split between A and B and both the A and B feeds are available. If one of those feeds drops out, there will be a transient where all of the no-longer-redundant power supplies ramp up to providing all of the power the devices require. This can cause you to momentarily exceed the breaker capacity on the remaining feeder, and all of a sudden your cabinet goes dark.

Having outlet strips that can monitor current in real-time on each individual outlet is handy. You can also graoh this using something like MRTG, and even sum up various pieces of equipment (or entire racks) the same way.

You might think that rack automatic transfer switches are the way to go. That way lies madness. I've used them from several different manufacturers, and my experience has been that they're all horrible in various ways (dropping the load during a firmware upgrade or random management interface is a specialty of one brand, for example). You want a transfer switch that's as close to instantaneous as possible. But it HAS to be break-before-make, because even when the datacenter promises you that A and B are both on the same phase, that isn't always the case. And even if it is, if there really are 2 separate power distribution systems there will be minor voltage variations between them. If your transfer switch decided to change sources even though A and B are both available, you'll damage the relays. If A and B are on different phases, make-before-break will produce a short between the phases. Your transfer switch will be toast, and you'll have tripped one or both of your A and B feeder breakers. If either A or B drops completely, make-before-break is going to briefly try to backfeed power to the entire datacenter, and you trip the breaker that is providing your sole remaining power. Break-before-make has a period of "dead air" (no power) as it is switching sources. Most equipment power supplies can handle a loss of a cycle or two of AC power, but some can't. Some of those supplies will reboot the host system when this happens, some will go catatonic and wait for someone to either disconnect and reconnect them, or (worst case) need to be replaced.

Some colo facilities offer measurement-based billing, where they measure your current consumption at unannounced times. With that, you can oversize (back in the day, a 30A breaker for both A and B was overkill for a cabinet, but if you were only paying for what you used it could make sense to oversize).

Some facilities offer both the measured (see previous paragraph) and "all you can eat" (up to whatever your feeder breakers are sized at) configurations. Sometimes you can get both in the same cabinet - it all depends on how flexible the colo facility is and how big a (potential) customer you are. I've done this in the past, where I had a fixed current contract for things like routers, switches, firewalls, management interfaces, etc. and a measured current contract for devices that have quite extreme variations in current draw (servers, disk arrays, etc.).

I miss the old days - I used to run everything on telco -48V power. 200A fusing on both the A and B legs, nobody measured and nobody cared...
 
  • Like
Reactions: Patrick

eece_ret

New Member
Jul 31, 2024
5
0
1
Let's also not commit the cardinal sin of using single phase math on a three phase feed. For a three phase feed the correct calculation is V*A*1.73=W. So 208@30A is actually slightly over 10kw with 80% safety factor clears 8kw. Also don't forget power factor which with switching pws is usually in the mid to high 90%. Yielding a safe wattage capacity of around 7.5 kw. A full 50% more capacity than single phase feeds at the same amperage.
 

eece_ret

New Member
Jul 31, 2024
5
0
1
Single phase 208 is an odd way to put that but what you reference is 3 phase off a delta wye transformer with a neutral tap. So voltage to neutral. As I wrote above 3 phase with 208v line to line voltage, not line to neutral.
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,530
242
63
Totally, but it's also what APC is using even in the product title. They're pretty big in DC power too.
 

eece_ret

New Member
Jul 31, 2024
5
0
1
Also note that a 3 phase delta wye with neutral will give you 240v not 208 (off a 480v 3 phase line feed). Hence when you see 208 the normal assumption is 3 phase sans neutral
 

eece_ret

New Member
Jul 31, 2024
5
0
1
Specifically the diagram for balanced delta (lower left). Key item here is neutral or no neutral. 208v to neutral would be an odd duck at least in my experiences