Let's watercool a 42U Rack and some servers
- Thread starter s0lid
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Arise old one, your time has come yet again!
Got custom waterblock tops from co-worker ^^
Next I need to get quick release fittings... expensive as hell tbh :I
Got custom waterblock tops from co-worker ^^
Next I need to get quick release fittings... expensive as hell tbh :I
Last edited:
G'day fellas, for many years, I have had the pain of noisy hardware that has lived indoors with me and the family driving us nuts. This topic has been well researched by myself and a few other locals who share my pain.
A tip for those stuck in the PC world, stop wasting money on expensive PC-based WC options !!!!! If you are stuck, then maybe some 3-5RU solutions maybe in order:
https://www.google.com.au/search?q=...m_3k4u_water_cooled_server_chassis%2F;500;272
http://www.nordichardware.com/Archive/Koolance-ERM-3K4U-water-cooled-server-chassis.html
I see from the images, a lot of wasted rack space is about to follow for no good reason along with some serious potential issues. The placement of WC gear above the fronts of expensive servers is not a prudent idea either.
Options to look at, bigger rads in the form of those designed for vehicles and other bigger situations. Place the rads in the top of the rack where the normal top vent fans would be, if not venting up, place rads at rear of rack or situate them somewhere else to get rid of the heat rather than keeping it in the area.
Always use multiple pumps with check-valves and flow monitors, a failure can be messy or costly.
Always monitor the level of the coolant in the res to make sure that there are no leaks. Just for the record, Evaporation is not possible either if the loops are sealed.
P.S Solid, the CPU block looks good, just need to get the hose barbs facing away from the RAM and VRM's. May I suggest maybe copper pipe to route the lines along the internals to the rear then attach barbs or fittings at rear?
Water-cooled processors and blades - The future of computing at IBM Zurich research labs
IBM's Hot-Water Supercomputer Goes Live | Data Center Knowledge
A tip for those stuck in the PC world, stop wasting money on expensive PC-based WC options !!!!! If you are stuck, then maybe some 3-5RU solutions maybe in order:
https://www.google.com.au/search?q=...m_3k4u_water_cooled_server_chassis%2F;500;272
http://www.nordichardware.com/Archive/Koolance-ERM-3K4U-water-cooled-server-chassis.html
I see from the images, a lot of wasted rack space is about to follow for no good reason along with some serious potential issues. The placement of WC gear above the fronts of expensive servers is not a prudent idea either.
Options to look at, bigger rads in the form of those designed for vehicles and other bigger situations. Place the rads in the top of the rack where the normal top vent fans would be, if not venting up, place rads at rear of rack or situate them somewhere else to get rid of the heat rather than keeping it in the area.
Always use multiple pumps with check-valves and flow monitors, a failure can be messy or costly.
Always monitor the level of the coolant in the res to make sure that there are no leaks. Just for the record, Evaporation is not possible either if the loops are sealed.
P.S Solid, the CPU block looks good, just need to get the hose barbs facing away from the RAM and VRM's. May I suggest maybe copper pipe to route the lines along the internals to the rear then attach barbs or fittings at rear?
Water-cooled processors and blades - The future of computing at IBM Zurich research labs
IBM's Hot-Water Supercomputer Goes Live | Data Center Knowledge
Last edited:
Well the plan is to use one cpu per server in cpu0 socket so it doesn't matter if the cpu1's rams get blocked, atleast in my case.
Copper pipes, now that's an interesting idea. I was kinda wondering how to do the tubing so it doesn't go outside the nodes height. So copper pipes would be just perfect for the task. Now I need to ask around what kind of fittings work with copper pipe.
Copper pipes, now that's an interesting idea. I was kinda wondering how to do the tubing so it doesn't go outside the nodes height. So copper pipes would be just perfect for the task. Now I need to ask around what kind of fittings work with copper pipe.
The Koolance is a nice idea but is still not cheap at US$1,700. I would imagine a PC watercooling setup with a couple of video cards could come in at less than that quite easily and that solution still does not include water blocks for the CPU / GPUs or pipe / connectors outside of the unit. I would also imagine that a lot of people, like myself, who do have a rack at home dont actually have it filled so using a chunk of space at the bottom for an external pump and res may not be so bad. If the solution were to be moved in to a data center then the Koolance may be a better solution although I don't see any redundancy / leak monitoring on it. It does look good though
. I may look to go that way for my phase 2 on my own watercooling rack project.Yep, car or motorcycle rads are an interesting option, leak detection and protection for the watercooled servers and the rack are also preferable but it really depends on the scope of the solution as to the importance. S0lid is doing for his own hoem rack with a few servers and lots of spare space. It has different requirements to an enterprise solution and is really just a proof of concept for him as the one I am doing is for me. Hopefully once the base build is up then he will look at enhancements and protection.
Good ideas and links there, thanks.
RB
Last edited:
The Muc Supercomputers copper pipework in the blades looks great but the pipes are of small bore (easier to block with corrosion etc). It would also be interesting to see what sort of wattage pipes of that size can cope with and an idea of what sort of pressures they are using. Going along these lines, how about a feed into a manafold in the server splitting a 1/4" external connection to two 1/8" internal loops, one for each CPU., possibly one also taking in the chipset as well.
I am also experimenting with acrylic tubing for my watercooled C6100 node based workstation. It is strong, can be heat bent, non-conductive, heat resistant to over 100Deg C, transparent so you can see any blockages or buildup. Not sure how small bore piping will cope though. Connectors to the G1/4" fittings are expensive though and you cannot just solder the joints.
Of course you could just submerge the server in mineral oil and then have no CPU blocks or piping to worry about.
RB
A lot of pump is going to be needed for the rise and to get through the rads. Maybe look into a larger pond pump for the return and another on the cold side if doing a single loop. I would go with larger supply and return lines, like 3/4in. I think a single loop with vales on the hot side to control flow to each system would work out.
Howdy again, the choice of copper is fairly easy. It allows precise bends than can be done with cheap tools and you can cover the tube in coloured heat-shrink tubing to ID the line but also insulate it electrically.
The bore of the line would be at least 1/4" and blockages should be minimized as these WC solutions are for the Pro WC guys who know how to keep systems clean.
Pumps: 2 good pumps are the min but adding more with check-valves will help. Flow indication is very wise and add pumps as you add systems.
I will put more in the other thread including a vital tip not covered in here.
The bore of the line would be at least 1/4" and blockages should be minimized as these WC solutions are for the Pro WC guys who know how to keep systems clean.
Pumps: 2 good pumps are the min but adding more with check-valves will help. Flow indication is very wise and add pumps as you add systems.
I will put more in the other thread including a vital tip not covered in here.