If you actually want it quiet, put a 120mm AIO water block on the cpu : PQuieter Deltas? Is it April fools? I jest. Yeah probably best to not risk components overheating. Can you put the server in another room or even a garage?
It'll be about 4u at that point, though.