Problem booting proxmox with 9400-16i installed

adgenet

Member
Apr 12, 2016
39
14
8
I've got a Proxmox box at home that's been running fine for years now (Gigabyte consumer X58 board, X5690), with my mirrored zfs boot disks hanging off of the onboard Intel SATA ports, and a bunch of other storage connected to a pair of 9310-8i HBAs (SC836 w/ TQ backplane).

I just got a 9400-16i with the intention of consolidating my controllers into one slot but am having trouble booting when I install the new HBA.
With a temporary GPU installed, I can see that the bios will post, then the HBA option ROM will list all of my storage disks, then proceed to GRUB, then after doing loading the initramfs, get stuck at a black screen with blinking cursor. I tried waiting it out but it never proceeded past this even after an hour.
I tried moving the card around to various slots, as well as disconnecting all of my storage drives, but nothing seems to work.

Returning the configuration to my 9310-8i pair allows the machine to boot again without issue.
I was running a pair of flashed H310s in the past before this as well and never encountered any issues. The only other PCIe card installed is a ConnectX-3 for 40GbE.
My desktop Ryzen machine boots arch and windows 10 with the 9400-16i installed, and I can interact with it via storcli so I don't think the card has a hardware fault either.

I am suspecting some sort of issue with my X58 board's bios but I have no real way of knowing since I can't even get it to boot to anything useful to diagnose.

Anybody have any suggestions?
 

pricklypunter

Well-Known Member
Nov 10, 2015
1,671
497
83
Canada
Have you tried taping of the SMBUS pins on the card? I doubt that is the issue, but stranger things. Much more likely a driver getting stuck or something I think :)
 

adgenet

Member
Apr 12, 2016
39
14
8
Have you tried taping of the SMBUS pins on the card?
You're referring to the taping of pins B5 and B6 correct?
I never had to do it with my H310s in the past but I know it's an issue on some boards. I guess it doesn't hurt to try on the 9400 either.
Will give it a shot and report back.
 

pricklypunter

Well-Known Member
Nov 10, 2015
1,671
497
83
Canada
Yea, I agree it's a bit of a long shot and hardware dependant, but it's a quick check and won't do any harm either way. My money is still on a driver problem. I guess, thinking about this, it's also possible that the card is not being properly enumerated for some reason by the BIOS and hence driver hanging :)
 

BLinux

cat lover server enthusiast
Jul 7, 2016
2,600
1,052
113
artofserver.com
Have you tried booting to single user mode? Point is to get to a state where you can interact with the system to further diagnose the problem.

I've seen SAS 3408 cards behave strangely on PCIe 2.0 system, but work fine on newer PCIe 3.0 systems. Might be a factor here... Can you perhaps update the firmware on a machine that will boot with the card and then try again in the system with the issues?

Just a few thoughts for consideration...
 

adgenet

Member
Apr 12, 2016
39
14
8
I went ahead and tried booting debian from a usb stick, same behavior. I tried FreeBSD as well, and athough it gets further along in the boot process, it eventually just hangs for a bit and reboots.
I also tried taping the smbus pins, but that didn’t do anything either.
The card has been updated to the latest firmware using my Ryzen machine and I know the card itself works fine and can see and access attached drives no problem in that machine.

Conclusion is that there must be a compatibility issue with my specific motherboard. Unfortunate, but I suppose it’s a sign that it’s time to upgrade the rest of the system.

I guess there is a PSA here for this unlikely combo: If you are using a Gigabyte X58A-UD3R 1.0, you will not be able to boot with a 9400-16i installed (or likely any of the 9400 HBAs). Stick to the 9300 series.
 

adgenet

Member
Apr 12, 2016
39
14
8
This affects probably zero other people in the universe, but just for the sake of completeness, I'd like to report that after updating to P17, everything works fine with no other changes.
I had written the whole thing off as a lost cause and left it on my parts shelf, but I'm glad I had some free time to try and update it again.
I guess something got fixed on the firmware side of things between my last post and now.
 
  • Like
Reactions: Tyrant82