9305-16e can’t see SATA disks

[Ezekial66]

Hello everyone,

I’m starting to exhaust available sources of information as none of us can seem to determine the issue with this HBA so far. Been on the Unraid forums, discord, Reddit, etc and so far no luck.

Current system:
-Dedicated server chassis with i9-12900k, 64GB DDR5
-2x SM JBOD’s connected to server HBA via 8644 SAS cables
-15 disks per JBOD, 30 total
-10 SATA disks
-20 SAS disks

I purchased this card second hand and immediately flashed to IT mode, flashed latest Firmware (no BIOS) and ran it through lsiutil in Unraid terminal to reconfigure the ports and confirm they are open (annoying Dell card thing).

Now upon installation and replacement of my old 9206-16e card, the new 9305-16e is only seeing the SAS drives for some reason.

If I switch the cables back to the 9206-16e (both cards currently installed for troubleshooting and same 8644 ports on both hba’s) it sees everything no problem, but not on the 9305-16e. The SATA disks do not show up in the boot up, system devices, or the array gui.

Current troubleshooting attempts:
-triple check documentation
=card is confirmed to support SAS & SATA both
-removed SAS drives so only SATA drives in JBODs
=HBA then sees nothing at all
-swap PCIe slots
=No change, confirmed working
-swapped SAS cables
=No change, confirmed working
-triple checked FW, BIOS, & Ports
=All check out good
-checked all disks individually
=All check out good

We’re kind of lost at this point of what it could be, if it’s a bad card somehow only effecting SATA, or some weird setting within the lsiutil that is blocking the SATA.

Any help is much appreciated

 

[CyklonDX]

(*this controller does support sata 3 & 6gbps options)
Are you running in efi mode or legacy?
Are you using mpio?

Next, are you mixing those sata disks with sas on same connector? (Have you tried not mixing)
i.e. connectors
0 sata
1 sata
2 sas
3 sas

and so on

I presume you have some sas3 disks mixed in?
The sata tunneling protocol may be misbehaving when they are mixed with sas3 - depending on backplane obviously.
Have you tried sata disks alone?

 

[Ezekial66]

What are you referring too regarding efi or legacy?

My understanding was the SAS3 cards do not use any form of legacy.

For mpio, the SAS3 backplane does have the secondary expander for SAS failover/redundancy but it isn’t used, 2x cables are simply using the primary ports on each JBOD out to the HBA.

SAS and SATA disks were mixed on the same 4x lane connector in some instances. I know it’s not recommended but it hasn’t been an issue in the past. I did pull all the SAS disks while shutdown and then booted with only the SATA disks to test but that didn’t change anything, the HBA just saw nothing instead of only the SAS disks.

Please let me know if there’s any other information I can provide to help!

 

[CyklonDX]

the card supports both modes (even 9400); in efi you should potentially be able to access controller from bios - in legacy during post. (since your system isn't on server mobo i assume menu for your sas controller isn't added under bios.) There might be settings in there or info that will tell you why its not seeing those disks. (changing to legacy mode might give you access to the sas controller management - and maybe its been set in sas3 mode only or something...

Do also check if you connect just 1 cable.


(also y did you get 9305-16e, and not 9300-16e? There's potential your bios flash was dodgy too - price wise 9300-16e should be 80 usd)
*I do have 9300-8i on one of my systems and i do have sas3 and sata ssd's mixed in ~ but its a supermicro server and supermicro sas3 backplane.

 

[Ezekial66]

in efi you should potentially be able to access controller from bios - in legacy during post. (since your system isn't on server mobo i assume menu for your sas controller isn't added under bios

My understanding was you will only get the boot up prompt when there is an actual BIOS installed on the HBA, which in this case there is no BIOS installed. May be worth installing one to see what i can find in there though...

Do also check if you connect just 1 cable

Checked all cables are seated properly and working properly. They were seated and removed a multitude of times over half a dozen different troubleshooting tests

also y did you get 9305-16e, and not 9300-16e?

9305-16e is the superior card using only a single SAS controller chip vs the dual 8e chip on the 9300. uses about half the power and far less heat than the 9300.

 

[CyklonDX]

I'd lean you flashed wrong bios that supports only sas, or this card only did physically support sas in first place.
*(yes there are some cards even in past that were produced to only support sas - or look for signatures of oem.)

 

[Ezekial66]

Again, there is no BIOS. And I did confirm with another forum member who also has this card and he flashed with the same IT FW that I did.

And I don't believe there is a variant of this card that only supports SAS...

 

[CyklonDX]

can you post a photo of the card front and back?
Thanks

(just saying i do recall someone flashing ?dell? oem cards of 9200, with it stopping to recognize non-oem disks, and one person complained not recognizing sata disks just like you describe now.)

But it could be just faulty card too...

 

[Ezekial66]

Sure thing! Please find them attached.

The title listing for the card was this:

“LSI SAS Logic 9305-16e IT Mode 12Gbs 4xSFP PCIe3 x8 16 port SAS HBA 00VYM4”

 

[CyklonDX]

This seem to be dell branded card.

Try doing full erase, and flashing again. (sometimes there are leftovers bits from oem's)

• Sas3flsh -o -e 6 (erase everything excl. Manufacturing area)
• Sas3flsh -f firmwarefile
• Sas3flsh -b biosfile

 

[Ezekial66]

Yes it is a Dell card (I don't know how I keep finding the damn things lol, such a pain in the @ss).

Yes you may be right though I can try just erasing it completely instead of flashing over the existing, I'll give it a go thank you!

 

[i386]

What jbod exactly? Dual expanders? (sata devices don't have a second port)
Do the sata discs show up when directly connected to the hba?

 

[Ezekial66]

The JBOD’s are the:

SuperChassis 836BE2C-R1K03JBOD

They are dual expanders but none of the secondary ports are being used.

As I mentioned in the original post, without changing the configuration at all except for switching the 4 SAS cables over from one HBA to the other in the exact same layout (cables were not removed from the JBOD’s at all), we connected to the old HBA (9206-16e) and everything worked perfectly. So it tells me the disks, backplanes, and cables are almost definitely not the issues. The system also ran in that configuration for 2+ years before this HBA upgrade and the issues regarding it.

 

[itronin]

They are dual expanders but none of the secondary ports are being used.

I know you said nothing changed and you probably already know this - but where I think @i386 was going is that with dual expanders (dual-port), if SAS works and SATA does not -> that can point to SATA drives plugged into backplane connections that are ONLY active on the second/dual port or for whatever reason the HBA is expecting to only see them on the second port which with SATA won't ever happen SATA not being dual ported. this occurs quite a bit with people not using interposers on SATA drives in NETAPP chassis and not connecting to the correct IOM module for the primary connectors on the backplane.

I don't recall seeing the correct drive mounting holes to use an interposer on the SM trays, but if they are there you could try that...

If it were me and to protect what little sanity I have left ... I'd pull the 836 out, pop the cover and trace internal SFF8643s and make sure I have them labeled on the chassis back as to which are expander 1 and which are expander 2 - just to be sure things really are cabled up as I thought.

expander 2 chip and connectors are on the left hand side of the back of the backplane and
expander 1 chip and connectors are on the right hand side of the back of the backplane.

elves are sneaky little buggers and sometimes change things without us realizing it. just saying.

 

[movax]

This seem to be dell branded card.

Try doing full erase, and flashing again. (sometimes there are leftovers bits from oem's)

• Sas3flsh -o -e 6 (erase everything excl. Manufacturing area)
• Sas3flsh -f firmwarefile
• Sas3flsh -b biosfile

Agreed with this -- use the sas3flash command that dumps all the info prior and save it a text file (if you care about preserving the SAS address, etc.) just in case. I did this (but accidentally did -o -e 7) to erase a latched temperature fault on my ex-Dell 9305-16e that was resulting in all the PHYs being down.

What does the utility report for PHY status? The user guide lists the command to print that.

 

[sdail]

This seem to be dell branded card.

Try doing full erase, and flashing again. (sometimes there are leftovers bits from oem's)

• Sas3flsh -o -e 6 (erase everything excl. Manufacturing area)
• Sas3flsh -f firmwarefile
• Sas3flsh -b biosfile

Warning!
Using the "-o -e 6" erase function on the 9305 16e - the e(xternal) model only - has a defect per Broadcom's site that causes it to also delete the SAS address, board tracer number, and board assembly number.

Flashing SAS9305-16e with ‘–o –e 6’ parameters erases some controller values

.
ID: 1211171734131
Flashing SAS9305-16e with ‘–o –e 6’ parameters erases some controller values
Solution
Q. When flashing to an older version of firmware I need to use the ‘-o –e 6’ option in sas3flash. However, when running the command and then flashing the firmware and BIOS, the SAS address, board tracer number, and board assembly number are missing. Why did this happen and what can I do. Only the DOS and EFI versions of sas3flash support the ‘-o –e 6’ option.

A. This is a defect in the SAS9305-16e only. When running the ‘-o –e 6’ (which is only needed if flashing older firmware), the controller erases the , the SAS address, board tracer number, and board assembly number.

The work around for this is to capture the , the SAS address, board tracer number, and board assembly number before flashing, then after downgrading the firmware, reflash the SAS address, board tracer number, and board assembly number using the steps below. This example is for the EFI version of sas3flash. Use ‘sas3flsh’ for the DOS version.
sas3flash -list [to see the controller details. Write down SAS address, Board Assembly and Board Tracer Number.]
sas3flash -o -e 6
sas3flash -o -f <firmware file name> -b <bios>
sas3flash -o -sasadd <SAS address>
sas3flash -o -tracer <tracer number>
sas3flash -o -assem <assembly number>
The controller is now fully functional.