Uncontained fan failure

[jabuzzard]

Something really unusual happened last week. We had two different 4 nodes in 2U chassis in our HPC system have what is best described as an uncontained fan blade failure. See attached pictures. The systems have just come up on their third birthday. The fans have literally disintegrated. I though in quarter of a century of IT support I had seen everything when it comes failures but this is a new one on me. I mean the normal failure mode of a fan is for the bearings to fail, not that they fly apart.
What are peoples thoughts? It is so out of the ordinary we are not sure what to make of it and the fact that two fans in different enclosures have failed makes us a little concerned for the rest.

.

 

[TXAG26]

Wow! That's wild!

 

[EffrafaxOfWug]

What are peoples thoughts? It is so out of the ordinary we are not sure what to make of it and the fact that two fans in different enclosures have failed makes us a little concerned for the rest.

Smacks of a bad batch of fans. If possible I'd take a few more out of the systems and inspect for hairline fractures in the plastic and see if any others are showing signs of falling to bits.

What's the brand of fans and/or the chassis they came with?

 

[Evan]

That’s unusual for sure, never seen it before in an enterprise server as far as I can remember. Doesn’t surprise me when you say it how it could happen with those little fans spinning at 14k rpm or more but the plastic always seemed very strong and unlikely to fail like that.

 

[jabuzzard]

Smacks of a bad batch of fans. If possible I'd take a few more out of the systems and inspect for hairline fractures in the plastic and see if any others are showing signs of falling to bits.

That's a decent idea.

What's the brand of fans and/or the chassis they came with?

I am reluctant to name the vendor of the chassis as I don't think it's their fault. The vendor of the fans I am not sure, will have to check. They where left in the data centre as the server vendor didn't want them. Except that has now changed and they want to them back having seen the photos. Grrr means another trip to the data centre to retrieve them, which is right now a right pain in the backside what with a global pandemic and the whole WFH thing.

 

[EffrafaxOfWug]

I am reluctant to name the vendor of the chassis as I don't think it's their fault.

Sorry, I should have explained better - merely that it's quite a coincidence to have this sort of thing happen multiple times, so it'd be worthwhile seeing if anyone else using the same servers or fans (presumably server fans from a well known vendor but I assumed the stickers had come off the fans when they disintegrated). There might be a publicised flaw or a product recall in action. Assuming they're either under warranty or service contract, have you already informed your server vendor about this?

 

[Patrick]

Yikes. Better a server fan than your airplane's engine.

 

[Blinky 42]

Were you able to tell if it was just the fans that failed directly, or if there was some other stuff that was loose in the chassis and got sucked into the fans and cause them to crack and self-destuct? I have had the latter happen before when a loose stand-off or screw got sucked into a fan.

 

[Evan]

I am reasonably sure based on the images of the service info that I know the vendor and I don’t think they would be at fault directly.

Anyway you will see the fans are normally made by one of a few companies and supplied to the large server vendors.

I have to find the photo by I remember some fans in a 4u IBM power system were rated at ~200w each ! Can you imagine the power in that !

 

[tinfoil3d]

totally read this as uncontained engine failure lmao

 

[jabuzzard]

The fan vendor is AVC

 

[gb00s]

A minimal failure in production (material and/or assembly) and 22.1bln noisy rounds later ....