Advise on cooling 22,000 watts

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

ELit3

Banned
Aug 10, 2017
309
24
18
41
Texas
Anyone have a GPU mining setup that is around 44,000 watts if so how much space is it taking up and what sort of cooling are you utilizing to cool the required space?
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,514
5,805
113
Datacenter.

Being serious here, 44kW is enough that electric delivery is going to require custom work on a residence/ office so it will be cheaper to colo.
 
  • Like
Reactions: ELit3

ELit3

Banned
Aug 10, 2017
309
24
18
41
Texas
Datacenter.

Being serious here, 44kW is enough that electric delivery is going to require custom work on a residence/ office so it will be cheaper to colo.
My setup is not residential I have the power capacity for the setup I'm just not exactly sure how to cool it in 150 sq ft. A data center wouldn't be able to get me the power cost I have at $0.039 per kwh. I used a 14,000 BTU portable unit for my 6,600 watt setup with a ceiling exhaust fan out to the roof and that keeps the units at 60 degrees +/- 5.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,641
2,058
113
I think it would depend on what office/space you have/get.

I know businesses that have more power to 5k/sqft warehouses for industrial purposes... it's really not that much power in that regard... it's what ~200A @ 240v single phase so you'd need 400AMP single phase to accommodate 200A utilization with minimal room to grow (you can't use all 400).

Then you'd have 3 phase for cooling systems.

It's not unheard of for warehouse space to have single and three phase power at those capacities, especially if you're subletting 5-10k sq/ft warehouse that has many more buildings onsite.

Obviously this doesn't take into account the huge cost and PITA to do the configuration/setup/management/monitoring/testing/etc yourself vs. datacenter ;) as Patrick mentioned, but it's not that much power it couldn't be done by any means.
 
  • Like
Reactions: ELit3

ELit3

Banned
Aug 10, 2017
309
24
18
41
Texas
I think it would depend on what office/space you have/get.

I know businesses that have more power to 5k/sqft warehouses for industrial purposes... it's really not that much power in that regard... it's what ~200A @ 240v single phase so you'd need 400AMP single phase to accommodate 200A utilization with minimal room to grow (you can't use all 400).

Then you'd have 3 phase for cooling systems.

It's not unheard of for warehouse space to have single and three phase power at those capacities, especially if you're subletting 5-10k sq/ft warehouse that has many more buildings onsite.

Obviously this doesn't take into account the huge cost and PITA to do the configuration/setup/management/monitoring/testing/etc yourself vs. datacenter ;) as Patrick mentioned, but it's not that much power it couldn't be done by any means.
I was thinking to use a Liebert DS CRAC Unit, 30 Ton and expand the space to about 300 sq ft. I believe most of the problems would come it exhausting all of that heat.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,641
2,058
113
Remember you don't need (and becoming more common) to keep them super cool.

I can't comment on AC unit, or required capacity out of my experience but I have a feeling it's more than just slapping a big AC and being good to go :) There probably are regulations for not only how much exhaust can flow but the quality of it too, in the USA that is.
 

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
Yeah by the time you are talking about spending enough on hardware to consume that much power you are probably better off looking for a physical location far enough north that ambient temps are cool enough that you can skip the A/C and just use massive fans to the outdoors. Then you can add even more gear cause you don't have to pay/power an A/C unit.
 
  • Like
Reactions: ELit3

ELit3

Banned
Aug 10, 2017
309
24
18
41
Texas
Remember you don't need (and becoming more common) to keep them super cool.

I can't comment on AC unit, or required capacity out of my experience but I have a feeling it's more than just slapping a big AC and being good to go :) There probably are regulations for not only how much exhaust can flow but the quality of it too, in the USA that is.
Ill do my due diligence on that.

Yeah by the time you are talking about spending enough on hardware to consume that much power you are probably better off looking for a physical location far enough north that ambient temps are cool enough that you can skip the A/C and just use massive fans to the outdoors. Then you can add even more gear cause you don't have to pay/power an A/C unit.
The reason I am doing this project now is because winter is coming so to speak. I have all the equipment already I'm not paying for it alone me and a couple of buds have invested highly into the equipment and property/space it will be in but its our responsibility to cool the placement.

So you think if temperatures outside are cold/cool it would be worthwhile utilizing an incoming airflow while its cool?
 

StammesOpfer

Active Member
Mar 15, 2016
383
136
43
So you think if temperatures outside are cold/cool it would be worthwhile utilizing an incoming airflow while its cool?
Mother Nature is for sure cheaper than doing it artificially. Just have to have a plan for the hot months too.
 

RBE

Member
Sep 5, 2017
60
34
18
At that scale it would be worth employing an HVAC engineer to scope the system for you. I would also strongly suggest you pay careful attention to your fire and flood management strategies.
 
  • Like
Reactions: mackle

_alex

Active Member
Jan 28, 2016
866
97
28
Bavaria / Germany
Even better would be if you could use the heat for something meaningful, i.e. growing strawbery, heat an office or whatever.
if it's mining i would also consider running a lower load on extremely hot days over investing huge into cooling. doing some math could tell what is more preferable.
 

Rahvin9999

Active Member
Jan 14, 2016
135
86
28
Rotterdam, The Netherlands
44000 watt is about 63A three phase fully utilized.
Not that exotic here. Not many homes would have it but you can get to about 80A 3 phase residential and 250A 3 phase on an LV line.
Everything above that and you will need multiple lines if you want to stay LV or go MV with your own transformer.

As for cooling it. If the climate allows, use outside air.
Make hot and cold aisles. Suck the hot air out of the hot aisles. Make sure there is enoug return air coming in. Its surprising how little delta T is needed to keep the cards happy. The rest is volume and that is not as much as you might think.
 

chilipepperz

Active Member
Mar 17, 2016
212
64
28
54
44000 watt is about 63A three phase fully utilized
Is it 15% or 20% safety margin? Yea that's good load on 3 phase.

You'll want to do outside air. Need to have good fans with temp regs. Make sure everything can handle constant load like that. One fire and that's a big operation to go to flames.
 

mackle

Active Member
Nov 13, 2013
221
40
28
Your outside climate is an important factor - keeping the inside air at 70degs (or 60degs) on a 100 deg day is quite a bit different to just getting the 44kW out. Or is your space within an otherwise air conditioned space?

How many racks of equipment do you have within that space? Is it traditional 44u racks or a more ghetto setup? Can you hot/cold it (you should) and duct accordingly? A colder cool isle costs more but gives a longer grace period if something goes wrong with the AC. Speaking of which, what sort of uptime do you need?

For how long are you expecting to run the operation? A one year pump and dump vs. 5 year vs. indefinitely?

The required circuit for the crac will easily be 25+kVA - required for the compressor inrush I believe. Our 20 ton AC requires a 240v/100a circuit. (As a building AC it also has an airside economizer that switches between recirculated air & ‘free cooling*’.)

For the amount of investment you must be dropping I’ll echo the others that say you should really get a consult from an HVAC pro.

*free cooling, but the longer runtime of the 5kw fan offsets it more than I expected.
 

vrod

Active Member
Jan 18, 2015
241
43
28
31
If you get air directly from the outside, make sure to “de-humid” with filters or so. You want as low of a humudity inside as possible. Also make sure to have a real airflow system. Separate cold from hot areas. That way you can control where heat goes much easier.
 

Aestr

Well-Known Member
Oct 22, 2014
967
386
63
Seattle
If you get air directly from the outside, make sure to “de-humid” with filters or so. You want as low of a humudity inside as possible. Also make sure to have a real airflow system. Separate cold from hot areas. That way you can control where heat goes much easier.
I think you're correct that you want to control humidity, but you definitely don't want it as low as possible. Too low and you'll have ESD issues.
 

Blinky 42

Active Member
Aug 6, 2015
615
232
43
48
PA, USA
The question of timeline to run the whole setup is going to be helpful to suggest a best course of action. If you are running it just for the winter then selling off the gear ad doing something else then that is different than running it for 5+ years. And if you are going to have it for > 1yr, is it going to grow?
44kw isn't much at all for a small data center. It sounded like you found some space already but if not you can find some office space with and old data room already decked out with a generator and cooling.

Having built out data rooms in the past, I would consult a HVAC engineer to make sure you end up with the most bang for the buck, and get hooked up with people who can do maintenance on the unit(s) - regardless if you do new or used units. Also get fire control squared away, you don't want to run afoul of the local codes.
 

vrod

Active Member
Jan 18, 2015
241
43
28
31
I think you're correct that you want to control humidity, but you definitely don't want it as low as possible. Too low and you'll have ESD issues.
You are right, I formulated that wrong. :D however low enough so there won’t be moisture in the room. And I also forgot to add that an HVAC expert would be the right one to ask about this.
 

ELit3

Banned
Aug 10, 2017
309
24
18
41
Texas
I just decided to go with a 3 ton ductless mini split ill let everyone know how it performs.