Hardware Failures in 2020 - Post yours!

sboesch

Active Member
Aug 3, 2012
467
91
28
Columbus, OH
I lost 3 Hitachi H3U2000'S, 2 Toshiba DT01AC300's and one Western Digital Red WD10EFRX, and a defective new Intel 520 10Gbe NIC within the last 30 days. I've been in the process of converting my rust to flash, so the disk failures were a good motivator to finish the job.
 
  • Like
Reactions: Patrick

stamasd

Member
May 30, 2020
38
10
8
So far this year I've had 2 hardware failures:

1. IBM M5015 card, blew 2 out of the 3 PCIe fuses. Already repaired, repair thread: https://forums.servethehome.com/index.php?threads/psa-repairing-dead-ibm-m5015-sas-card.28905/

2. Mushkin Reactor 1TB SSD, this one is a bit puzzling. It was in a laptop onto which my daughter spilled soda. The laptop's keyboard was ruined, and curiously the SSD appeared dead as well although I have inspected it very carefully and there was no sign of the liquid having made it to the drive. It was completely dry when I took the laptop apart. But the drive was invisible to a SATA controller (tested it in 2 different motherboards). I let it sit in a drawer for a couple of months, and just found it again last week. For the hell of it I plugged it to see if there's anything I can do to fix it, but it showed right up and it now works perfectly fine. I did an extended test on it, no faults whatsoever. Anyway, I'll keep it as a spare. :)

I'm not counting the parts that I received DOA from ebay and other places and which were returned ASAP. There were a few of those (Seagate 2.5" 2TB HDD, power supply etc) - those didn't fail on my watch.
 
Last edited:
  • Like
Reactions: sboesch and Patrick

edge

Active Member
Apr 22, 2013
129
38
28
I have been using netgear r7000p/r7000 as router and access points for years (netgear stuff for decades). Friday right around 5:00pm, I had an r7000 in bridge mode lose connectivity to the main router. I spend a stupid amount of time debugging (while noticing my neighbor has switched his router to DD-WRT and boosted his signal - I have considered retribution) but eventually the easy fix was rebooting the EX8000 range extender (hard wired backhaul) that is next to the neighbor. Along about 11:00, everything is hunky dory, sort of.

The R7000p as an internet facing device has bugged me for quite some time. The 0 day vuln reported Thursday made that it become an outright rash. So, I start reconfiguring my base servers - I have an SM X9DA7 as my main hyper-v host with an X9SRA as the failover back up. So today, I decided I wanted to put a pfsense vm on the X9SRA NOW (not wait for nic cards to arrive), so that meant dismantling my live migration network so I could use that onboard nic for the wan. I also took the opportunity to make sure all the OS's were patched to current. Last step is to reboot the x9da7 after patching...

Of course I fail order dependency 10. Live migration dismantled with both DCs on x9da7 after removing live migration, I reboot it. Of course, when I attempt to RDP back into the x9da7 after the reboot, I get the NLA error... I go down to the server room in the basement and try to switch the kvm from the x9sra to the x9da7, it goes beep (instead of beep beep) and its' lights go dark. Another hour getting the KVM out of the loop while switching in my old office monitors to the server room. Finally, I am where I should have been 6 hours earlier.

I come up out of the basement grumbling and my wife says: "mercury is in retrograde, you shouldn't be changing tech".

I am surprised my children still have a mother. I, somehow, have not gotten a Darwin award.
 
  • Like
Reactions: Fritz

ViciousXUSMC

Active Member
Nov 27, 2016
221
113
43
39
Ubiquiti US‑8‑60W died on me. That thing always did seem to run really hot.

It was replaced with the switch it replaced originally because I wanted more Ubiquiti stuff a ZyXEL GS1200-8HP and its still going strong supporting the kids computers and a Ruckus R710 in the back of the house.

I also had a HDD failure in my computer, but not sure what kind it was and its been around for a LONG time so it deserved to finally die.
99% It was a WD HDD I think it was a 2TB capacity.
 

Spartacus

Well-Known Member
May 27, 2019
781
323
63
Austin, TX
Ive found the ubiquiti stuff seems to have specific positions they like to sit in.
Like the USG prefers to be vertical mounted otherwise it turns into a griddle.

Just had my old APC BX1300 finally die with the F02 error, replaced with a little 425va version since i'm just running my modem router and 8 port green switch in the closet now.
 

Fritz

Well-Known Member
Apr 6, 2015
2,446
657
113
67
A couple of weeks ago I had a Supermicro BPN-SAS2-846EL1 die on me. No drives detected. Today i had a Supermicro u1 backplane with one dead slot. Luckily I had a spare chassis so just swapped the MB and disks and all is well again.

If anyone has any ideas about the SAS2 backplane it would be mucho appreciated. Nothing obvious to my eyeballs so I suspect it's deceased. :(

EDIT - I replaced it with a SAS backplane I had lying around and it works so the problem is in the backplane.
 

Zoltan

New Member
Dec 18, 2019
6
5
3
Bought a Ruckus R700 on ebay, from a seller with a 99.5% feedback score.

Here's what I got:
IMG_20200812_133331090_.jpg IMG_20200812_133506026_.jpg IMG_20200812_133500051_.jpg

All the major IC's decapitated with acid.
 

b-rex

Member
Aug 14, 2020
59
13
8
Bought two R610 on eBay to act as monitoring hosts in each of my server cabinets. Both failed within two weeks of each other shortly after purchasing. Both failed of the same thing...high fan speed, won't power on. No firmware updates...mobo just died on both. I think that's just what I get for purchasing 11G Dell servers for $85/each in 2020.
 

bmorepanic

New Member
Oct 24, 2020
13
1
3
Baltimore, Hon
Volunteered to help someone "fix" their camera security system. They had stuffed eight 10TB Iron wolf sata nas drives into the hot swap bays of a 12 year old Dell server with an elderly SAS controller, "upgraded" to run Windows Server 2012R2, with an old Xeon processor and 8 whole gigs of ram. The software had created a 60TB database that ended up with the camera recordings of fairly high resolution images. Whoever had set it up just said stripe all the disks, but one, together. So, a drive failed - because of course it would. So this database started to merrily eat itself to death until there was nothing left. There was no backup and at this point, really, who cared anymore. Their original installer got out of the business - and no wonder.

It took three months to get the software re-licensed through a new dealer, because it was only sold through dealer. Forced them to get some new hardware with maintenance. They can't believe how much better it runs with 80 gigs of ram.

The real kicker? Upgrading their software to a level that actually worked was only $145.
 
  • Like
Reactions: Patrick

penrhos

New Member
Nov 23, 2020
17
9
3
This is why you have N+1 on your chiller units on a datacentre... Got woken up by multiple SMS messages saying a chiller had gone offline....

This is the picture I got from our facilities maintenance manager when he got to site to investigate.

Crispy!

Contactor.jpg
 

penrhos

New Member
Nov 23, 2020
17
9
3
Couldn't get onto the net a couple of days ago - popped down to the garage where my home-lab is to be met by silence,,,

UPS shutdown so the Cisco 3750G switch and all the other kit was off as well - unplugged everything from the UPS and powered it back up, Amber & red lights, reset the trip-switch on the rear and Amber & Green light....

Checked the output with my multi-meter - all OK. Assumed something plugged into the UPS had gone pop and tripped the trip-switch.

Plugged the switch back in - OK booted up, so worked through the rest of the kit, plugging it into a non-UPS backed socket first so if it tripped the mains the stuff I'd powered back up would stay on because of the UPS.

Got to the last item - my QNAP TS-659 and plugged it back in - big blue flash and the power went off.

FFS - that's the second QNAP PSU to die on me in 6 months. Hopefully my data is OK as it contained all my Anime and Pron (and isn't backed up).
 
  • Like
Reactions: Fritz

Fritz

Well-Known Member
Apr 6, 2015
2,446
657
113
67
Couldn't get onto the net a couple of days ago - popped down to the garage where my home-lab is to be met by silence,,,

UPS shutdown so the Cisco 3750G switch and all the other kit was off as well - unplugged everything from the UPS and powered it back up, Amber & red lights, reset the trip-switch on the rear and Amber & Green light....

Checked the output with my multi-meter - all OK. Assumed something plugged into the UPS had gone pop and tripped the trip-switch.

Plugged the switch back in - OK booted up, so worked through the rest of the kit, plugging it into a non-UPS backed socket first so if it tripped the mains the stuff I'd powered back up would stay on because of the UPS.

Got to the last item - my QNAP TS-659 and plugged it back in - big blue flash and the power went off.

FFS - that's the second QNAP PSU to die on me in 6 months. Hopefully my data is OK as it contained all my Anime and Pron (and isn't backed up).
I'd rather get nothing for Christmas.
 

edge

Active Member
Apr 22, 2013
129
38
28
Xmas morning, my 7 yo gets $100 worth of robux as a gift - buys upgrades on his tablet. Just as he starts to play, samsung updates his android and then his tablet goes into a boot loop due to missing files in the update.

Give me an irate CEO any day
 
  • Like
  • Wow
Reactions: nasi and Patrick