9341-8i Unexpected Sense + CRC Errors

[coolrunnings82]

Setup:
Chassis: CSE-216 with SAS216A backplane (non-expander)
Motherboard: Supermicro X9DR7-LN4F with very latest BIOS
RAM: 64gb Samsung ECC Registered on the HCL
CPU: E5-2620V2
Storage Controllers: onboard LSI-2308 - works perfect
2x Intel RS3WC080 - LSI 9341-8i
Drives: 9x Samsung PM853T

So here's the scoop:
I'm using a group of 6 of the Samsung drives in a StorageSpaces RAID0 for testing. No critical data on them. I'm running the controllers in JBOD mode and they have the latest Intel firmware on them. Here's the rub: write performance is HORRIBLE. I loaded up MegaRaid Storage Manager to look for errors and the system was spamming dozens of "Unexpected Sense: Power on, reset, or bus device reset occurred" errors. I checked the controller's temp and they're 55C give or take 5 degrees. I checked the cabling and I'm using brand new LSI-branded MiniSAS 8643-8087 cables. There is no strain on the cables and they're properly attached to both the cards and the backplane.

I tried hard-setting the link speed to 6G and that removed most of the unexpected sense errors. I started backing up the data to an off-system backup drive I have and got a few Unexpected Sense Information CRC error detected messages. I haven't tried any more writing because I don't want any dataloss but this is getting kinda nuts. The backplane including those exact slots works perfectly with the onboard LSI-2308 controller so I'm pretty sure it's not defective. Any hints here? Should I try flashing the RAID cards to an LSI factory 9341-8i ROM? Will doing so remove the RAID5 and Cachecade keys that these cards came with? Any way to back those up so I could recover them if so?

Sorry, I'm at a bit of a loss here!

 

[coolrunnings82]

Update: Still spamming TONS of CRC errors and performance is plummeting to 16Mb/s while trying to back up the array to an external NAS. Ugh.

 

[coolrunnings82]

Well, I think I found my culprit. After much swearing, it does appear that I have a pair of faulty ports on the backplane on this chassis. Bought it a month and a half ago so I'm doubting the seller (DougDeals.com) has any left in stock but we'll see. That cost me an entire day of troubleshooting but I did learn some stuff. Anyone know if Supermicro fixes stuff like that or am I better off buying a new one off eBay and replacing it? I'm quite regretting buying this chassis at this point...

 

[mrkrad]

Use megacli/storcli to show # of PHY errors if you think there is a problem, perhaps force to 3GBPS mode to avoid a faulty backplane that can't handle 6gbps!

 

[canta]

Well, I think I found my culprit. After much swearing, it does appear that I have a pair of faulty ports on the backplane on this chassis. Bought it a month and a half ago so I'm doubting the seller (DougDeals.com) has any left in stock but we'll see. That cost me an entire day of troubleshooting but I did learn some stuff. Anyone know if Supermicro fixes stuff like that or am I better off buying a new one off eBay and replacing it? I'm quite regretting buying this chassis at this point...

email them and explain you situation,
if they can give you a good deal on backplane.... or almost fee (with or without shipping fee).

I bought 2U SM server, they shipped free caddies later after sending via ebay PM 4 times for negotiation.

Now... one fan make "ticking: noise, I just bought one with my ebay bucks.

 

[neo]

A few months ago I had a similar issue - constant CRC errors. The cables were name brand, looked in perfect condition and etc.. But I tried swapping them out and it fixed the issue. Might be worth giving it a shot.

 

[coolrunnings82]

Neo: yeah I thought of that too. I tried switching up which cables went where but the issue remained the same unfortunately. Seems it is just those 2 ports on the backplane.

Canta: Yeah I just did and got a nice response. We're working out a deal where they'll help cover the cost of the replacement backplane since they don't have one in stock.

Mrkrad: Not quite sure how to interpret the readings. Got any suggestions or know where I might find a guide that would explain how to interpret the numbers? Will try the lower speed for kicks since I was testing with SSD's.

Got a deal on a brand new never used one for $80 shipped from an eBay seller. Hope USPS doesn't do their ususal and run a freaking truck over it! Todd, keep your bad luck with eBay & shipping companies away from me on this one! ;-) LOL!

 

[coolrunnings82]

Well neo, you were partially correct. Not only did I have a bad backplane but I had a single bad cable too. What are the odds? I got a brand new backplane for the case - remarkably easy to change I might add! - and the seller of the case (DougDeals) made good on the backplane cost plus gave me a 10% off coupon for a future purchase. I'm thoroughly impressed. I finally have the server up and going after a month of trials! Waiting for my 4x 800gb S3700s and then we go into Hyper-V testing mode.