Which cases offer the best air circulation for 9 or more internal 3.5" hard drives?

[EffrafaxOfWug]

There were about 150 different models, they were cooked online for about three hours and cooked offline for another six or so all told. Includes big fat SAS enterprisey drives and cheap off-the-shelf SATA gubbins. Once we put the piping in place we added HDD temperature monitoring to our hardware data, we kept it turned on as it's just another 100GB of WORN data. Since then there's been no correlation between temperature and failure rate for us.

Our Top Tips for HDD procurement generally goes, in list of most to least important:
Avoid known bad batches/families
Avoid physical damage
Buy Hitachi, avoid Seagate. Go WD for best perf per pound if you need to go SATA,
Makre sure the enclosures are firmly bolted and there's no vibration
...
...
...
Don't look at it funny or make eyes at the hard drive's bird
...
...
...
...
...
...
...
Don't open the drive and place a doughnut on the platter no matter how cool it seems at the time
Similarly avoid immersing hard drive in custard
...
...
...
...
...
Watch HDD temperatures like an OCD hawk on methamphetamines

 

[cptbjorn]

OP why not just toss the drives you have in the case you bought and see if the temps are dangerous?

 

[NeverDie]

OP why not just toss the drives you have in the case you bought and see if the temps are dangerous?

Great question. Thanks for asking. I've answered it (or thought I did) three different times now: first in post #10, then again in post #14, and once more in post #19. I hope at least one of those answers will do, as at the moment I can't think of any other ways to say it. If not, please try to ask about the part of the answer you didn't understand, or that you disagree with, and I'll focus on that. I don't think I'm willing to start over from zero and answer yet again without at least some clue of where the communication is getting lost.

 

[cptbjorn]

Great question. I've answered it (or thought I did) three different times now: first in post #10, then again in post #14, and once more in post #19. I hope at least one of those answers will do, as I can't think of any other ways to say it.

I read the whole thread and you have not yet managed to measure/post any actual temperature numbers. That's all the drives care about and that's what you should base your decision on so I suggest starting there.

 

[chinesestunna]

I read the whole thread and you have not yet managed to measure/post any actual temperature numbers. That's all the drives care about and that's what you should base your decision on so I suggest starting there.

Thank you. That was my whole point in multiple posts but OP doesn't seem to trust the in drive temperature module, at least not without knowing where it is. All drive health studies or user stories I've seen have been based on that reading, wherever it is and I have not seen any evidence whereby the sensor's temperature readings are called into question, enterprise conditions with dedicated HVAC or home desktop use.

 

[NeverDie]

Hmmm.... Let's see now. Why might sensor location matter? Also, if the operational temperature is up to 60C, does it mean all parts must be below 60C, or just what the temperature sensor (wherever that is) reports as the temperature?

And by the way, that's a 20C temperature difference across the Red drive. Do you think that temperature spread will increase, decrease, or remain the same if the drives a packed tightly together? Now suppose the temperature sensor happens to be near the edge of the drive (maybe for no reason other than manufacturing covenience), and that edge happens to be the edge facing the fans in the case. In that event, do you really believe the reported temperature will accurately reflect the highest temperature inside the drive?

 

[chinesestunna]

I don't think that proves anything... The same comparison is looking at difference in core temps of CPUs, in most 4+ core setups temperatures between coolest and hottest core can be up to 10C. In this case you have onboard ICs vs. enclosure vs PCB so of course temperature would be significantly different. But this doesn't illustrate that the sensor temp (wherever it's located) isn't a good "overall" indicator

 

[Entz]

At least as far as WD is concerned, the sensors used to be located in the top left of those images (source http://tinyurl.com/pzw8zhw). Closer to point #4. May still be. Makes sense as the first part of the drive is the more temperature sensitive portion vs the controllers on the PCB which can handle far more heat (and would be dissipated into the aluminum body).

 

[NeverDie]

Thank you. That was my whole point in multiple posts but OP doesn't seem to trust the in drive temperature module, at least not without knowing where it is. All drive health studies or user stories I've seen have been based on that reading, wherever it is and I have not seen any evidence whereby the sensor's temperature readings are called into question, enterprise conditions with dedicated HVAC or home desktop use.

OK, I checked the drive temps with the case open and 4 fans (including two aftermarket fans I added to make a push-pull) attempting to blow through the drives. According to smartctl, the idle temperature is just 25C, so there's actually good reason for hope. The remaining 3 drives will arrive tomorrow or the next day, at which point I'll check how the temperature plays while fully loaded and while running badblocks. FWIW, Tom's Hardware recommends drive temps be kept below 30C, though they didn't elaborate as to exactly how they picked that number.

 

[T_Minus]

Why are you picking a fight with a a temperate that EVERYONE ELSE uses except you?

I don't get what the struggle is to do it right by simply measuring the temp like EVERYONE ELSE keeps an eye on. Do you think 24 or 48 drive enclosures don't rely on that temp???? How do you think they work?

What you've done is gone and created a problem for yourself that might not exist.

EVeryone here is tellinig you to measure temps and then make decisions... you should listen to their advice.




EDIT: Looky there you took the temps and they're "OK". as most suspected.

 

[NeverDie]

Why are you picking a fight with a a temperate that EVERYONE ELSE uses except you?

I don't get what the struggle is to do it right by simply measuring the temp like EVERYONE ELSE keeps an eye on. Do you think 24 or 48 drive enclosures don't rely on that temp???? How do you think they work?

What you've done is gone and created a problem for yourself that might not exist.

EVeryone here is tellinig you to measure temps and then make decisions... you should listen to their advice.




EDIT: Looky there you took the temps and they're "OK". as most suspected.

Well, I still think very little is blowing through the drives, but maybe it turns out that blowing on just the small sides of the drive is enough to keep it cool enough. It's counter-intuitive (well, at least to me anyway), because the surface area is so much less than if the top and bottom also had good airflow contact (as would be true in the twelve hundred case mentioned earlier).

 

[NeverDie]

I'm currently running SMART's extended self-test (about 6 hours long on a WD Green 3TB drive), and I'm noticing I can dial-in pretty much any hard drive temperature I want by adjusting fan speeds. Naturally, lower fan speeds are quieter and consume less energy. Therefore, what should the target HDD temperature be? Relying only on the HDD's built-in temperature sensor, as you all do, should I just let the temperature rise to the top end of the manufacturer's stated operating temperature range (60C), or should I try to keep it under 30C, as recommended by Tom's Hardware? What targets have you all picked for yourselves?

 

[chinesestunna]

Glad to hear you're up and running. I have my drives under 40C, 45C I get notified and 55C the system spins array down.
These are just my settings and the reason I have 45C as notification temp is my drives sit around 30C idle and <35C full activity. 45C would almost indicate something else going really wrong in the overall server cooling and environment

 

[EffrafaxOfWug]

I picked a number that turned the case fans at about ~1000rpm which keeps the noise down and HDD speeds hovering at about 10 over ambient:

root@wug:~# hddtemp /dev/sd?
/dev/sda: WDC WD30EZRS-00J99B0: 31°C
/dev/sdb: WDC WD30EFRX-68AX9N0: 24°C
/dev/sdc: WDC WD30EZRS-00J99B0: 28°C
/dev/sdd: WDC WD30EFRX-68AX9N0: 25°C
/dev/sde: WDC WD30EZRS-00J99B0: 27°C
/dev/sdf: WDC WD30EFRX-68AX9N0: 24°C
/dev/sdg: Crucial_CT120M500SSD1: 23°C
/dev/sdh: Crucial_CT120M500SSD1: 23°C

Temps during a sync on a hot day might get as high as 40.

 

[NeverDie]

I picked a number that turned the case fans at about ~1000rpm which keeps the noise down and HDD speeds hovering at about 10 over ambient:

root@wug:~# hddtemp /dev/sd?
/dev/sda: WDC WD30EZRS-00J99B0: 31°C
/dev/sdb: WDC WD30EFRX-68AX9N0: 24°C
/dev/sdc: WDC WD30EZRS-00J99B0: 28°C
/dev/sdd: WDC WD30EFRX-68AX9N0: 25°C
/dev/sde: WDC WD30EZRS-00J99B0: 27°C
/dev/sdf: WDC WD30EFRX-68AX9N0: 24°C
/dev/sdg: Crucial_CT120M500SSD1: 23°C
/dev/sdh: Crucial_CT120M500SSD1: 23°C

Temps during a sync on a hot day might get as high as 40.

I think you're right: roughly 1000RPM seems to be a very good tradeoff between fan noise and cooling. I tried the 2000RPM Noctua, and it was without question too loud, so it's going back. Even the 1200-1500RPM Noctua seem louder than desirable, so back they go as well. It turns out the 140mm Nanoxia fans (5 come with the case) are fairly quiet, and they're around 1000RPM. It's neither here nor there, but I believe Nanoxia was making fans before they ventured into making enclosures, so maybe that's why their fans are actually decent.

 

[5teve]

You do not need very much air to make a difference. I was one of 2 engineers involved in the concept, design and testing of the Nexsan SATAbeast which had 48 drives 3.5" in a 4u case plus raid controllers and redundant PSU's I believe it was the first of its kind. We used 3x 120mm fans at the front (admittedly not quiet, but that wasnt a requirement) and just the centrifugal blowers in the controllers and PSU fans at the rear. We worked within the operating parameters set by the manufacturer, so we had to operate in a +35degC environment and keep the drives under 60degC (25deg C rise over ambient) and as such tested the units by putting them in a testing oven and operating them while cooking them... we easily managed within the specs (and pushed them a lot further in testing). obviously the drives at the rear were warmer than the front drives... but that was the tradeoff of the design.

a small amount of airflow makes a huge difference, you would be surprised at how little airflow you need to significantly impact the temperature.

Steve

 

[NeverDie]

You do not need very much air to make a difference. I was one of 2 engineers involved in the concept, design and testing of the Nexsan SATAbeast which had 48 drives 3.5" in a 4u case plus raid controllers and redundant PSU's I believe it was the first of its kind. We used 3x 120mm fans at the front (admittedly not quiet, but that wasnt a requirement) and just the centrifugal blowers in the controllers and PSU fans at the rear. We worked within the operating parameters set by the manufacturer, so we had to operate in a +35degC environment and keep the drives under 60degC (25deg C rise over ambient) and as such tested the units by putting them in a testing oven and operating them while cooking them... we easily managed within the specs (and pushed them a lot further in testing). obviously the drives at the rear were warmer than the front drives... but that was the tradeoff of the design.

a small amount of airflow makes a huge difference, you would be surprised at how little airflow you need to significantly impact the temperature.

Steve

Thanks, that's good info to know.

Because of your background, which I imagine gives you good insight, I'm wondering what target HDD temperature you personally target when setting your fan speeds? Or, if the answer is, "It depends," how do you go about deciding what the target should be?

 

[badskater]

The Nexsan is a good exemple, another one would be the NetApp DS4486 which has 48 drives in tandem. It only uses the fans from the PSUs to cool it, and it stays quite cool in a normal environment. I have 2 of those at work outside the server room (In the lab room, where AC isn't really working fine, temps of 30C about all the time) and they run at 45C of the hard drives, which is quite fine according to specs. Hard drives doesn't need that much cooling compared to say a 150W CPU.

 

[5teve]

Thanks, that's good info to know.

Because of your background, which I imagine gives you good insight, I'm wondering what target HDD temperature you personally target when setting your fan speeds? Or, if the answer is, "It depends," how do you go about deciding what the target should be?

Bear in mind I was at Nexsan a number of years ago when 500gb hard drives were the biggest SATA available and they ran hot. Probably very different now.

If you look at my new system in the process of being built (8x 2tb 7k3000 hitachi drives connected to a P420 possibly) the environmental operating specs on the drive for < 3tb drives are 0-60degC AMBIENT - This is the maximum temperature the air around the hard disk can be, not the temp of the drive. The drive temperature will be a set amount above ambient (in still air) when operating. These specs are what Hitachi will warrant, and I doubt with the low power usage of modern spinners that any of the drives will see close to their max operating temperatures in a normal case environment. http://www.hgst.com/tech/techlib.nsf/techdocs/155901D3B251D9A9862577D50023A20A/$file/ds7K3000_ds.pdf

The netapp example above, should show a small increase on the rearmost drive as the air will warm slightly.. but a 15degC increase over ambient is kind of indicative of how cool these can run with minimal airflow.

My case is a fractal design R2xl and has 2x 140mm fans moving air slowly over the hard drives (very disrupted path) 1x 140mm fan pushing in from the bottom and 3 expelling from the top and rear. The hard drives get the fresh air from the front and the ambient in the house is unlikely to pass 45degC (Australia) once built I will have a look to see what temps they are reaching, but as long as I am under the ambient spec set by hitachi, i'll be happy that the warranty is safe, and the reliability should be as per Hitachi specs.

On a completely different note when designing LED lighting systems recently we were getting large amounts of light output drop off due to heat in the housing (we were testing in a lab) with the addition of a small desk fan to simulate a very light (barely perceptible) breeze from a distance of 3m from the light, the temps dropped dramatically and the light fall off reduced dramatically too. You dont need fast powerful air to remove heat any more with hard drives.

Steve

 

[NeverDie]

Bear in mind I was at Nexsan a number of years ago when 500gb hard drives were the biggest SATA available and they ran hot. Probably very different now.

If you look at my new system in the process of being built (8x 2tb 7k3000 hitachi drives connected to a P420 possibly) the environmental operating specs on the drive for < 3tb drives are 0-60degC AMBIENT - This is the maximum temperature the air around the hard disk can be, not the temp of the drive. The drive temperature will be a set amount above ambient (in still air) when operating. These specs are what Hitachi will warrant, and I doubt with the low power usage of modern spinners that any of the drives will see close to their max operating temperatures in a normal case environment. http://www.hgst.com/tech/techlib.nsf/techdocs/155901D3B251D9A9862577D50023A20A/$file/ds7K3000_ds.pdf

The netapp example above, should show a small increase on the rearmost drive as the air will warm slightly.. but a 15degC increase over ambient is kind of indicative of how cool these can run with minimal airflow.

My case is a fractal design R2xl and has 2x 140mm fans moving air slowly over the hard drives (very disrupted path) 1x 140mm fan pushing in from the bottom and 3 expelling from the top and rear. The hard drives get the fresh air from the front and the ambient in the house is unlikely to pass 45degC (Australia) once built I will have a look to see what temps they are reaching, but as long as I am under the ambient spec set by hitachi, i'll be happy that the warranty is safe, and the reliability should be as per Hitachi specs.

On a completely different note when designing LED lighting systems recently we were getting large amounts of light output drop off due to heat in the housing (we were testing in a lab) with the addition of a small desk fan to simulate a very light (barely perceptible) breeze from a distance of 3m from the light, the temps dropped dramatically and the light fall off reduced dramatically too. You dont need fast powerful air to remove heat any more with hard drives.

Steve

Thanks for your post. Has this always been true, or has something changed (better components? less heat generated? both?) so that hard drives no longer need much cooling? i.e. it's strange that backblaze (and others) engineered their "pods" to produce good airflow over their harddrives if there was never a need in the first place.