ROMED8-2T Finally Replaced (RMA), Won't Boot with New GPUs

[shpitz461]

Hi,

After waiting ~1.5 months for a replacement board (original one had one network port non-functional), I've reinstalled all previous hardware but having issues booting up.

If I do not install any GPUs the board boots up fine.

The replacement came with BIOS v3.20, I tried to update to v3.40 but it still won't boot with any GPU installed.

What works:
LSI 9305-16i SATA SAS 12Gbs RAID - running in SLOT7 @ x16 in GEN3 mode
ASUS Hyper M.2 X16 PCIe 3.0 X4 Expansion Card V2 - has four NVMe's running in SLOT6 @ x4x4x4x4 in GEN3 mode

What doesn't work:
MSI RTX 3080 VENTUS 3X PLUS 10G OC LHR
Sapphire Radeon 5770 HD 1GB

It doesn't matter which slot I put the GPUs, I tried each one on its own, PCIE power adapter(s) connected to each card, but the ROMED8 refuses to boot.

It gets stuck at "DXE -- PCI Bus Enumeration 93" showing on the screen, and BIOS 7-segment on the board displays b2.

Searching the code suggests something to do with legacy devices with their own BIOS (i.e. GPUs).



Nothing has changed in terms of hardware, does anyone have any idea what I am missing?

Thanks

 

[gsrcrxsi]

When you’re using the GPU, do you have the monitor hooked up to the GPU? Because you have it set to primary graphics on External. Meaning it switches display output to the GPU.
If you want the GPU attached but display through the onboard VGA, then switch the setting from External to Onboard or Internal or whatever the verbiage is the onboard VGA

 

[RolloZ170]

It gets stuck at "DXE -- PCI Bus Enumeration 93" showing on the screen, and BIOS 7-segment on the board displays b2.

Searching the code suggests something to do with legacy devices with their own BIOS (i.e. GPUs).

boot mode UEFI or legacy ? CSM en. or disabled ? oproms legacy or UEFI ?
edit:
set BIOS to default: and set UEFI only CSM disabled and try.

 

[shpitz461]

When you’re using the GPU, do you have the monitor hooked up to the GPU? Because you have it set to primary graphics on External. Meaning it switches display output to the GPU.
If you want the GPU attached but display through the onboard VGA, then switch the setting from External to Onboard or Internal or whatever the verbiage is the onboard VGA

I have a monitor connected to the onboard vga port, which works. I was actually playing with that primary graphics setting, it has no effect on the issue.

boot mode UEFI or legacy ? CSM en. or disabled ? oproms legacy or UEFI ?
edit:
set BIOS to default: and set UEFI only CSM disabled and try.

Good question, I'll have a look and see what it is set to, I don't recall changing any of the boot mode settings.

Thanks guys for the suggestions!

 

[shpitz461]

boot mode UEFI or legacy ? CSM en. or disabled ? oproms legacy or UEFI ?
edit:
set BIOS to default: and set UEFI only CSM disabled and try.

Bingo, that was it!
I enabled CSM and set Video OpRom to Legacy Only.
Both cards are working great.
I also fixed IPMI access, had to reset its config in the BIOS, no idea what broke it.

 

[shpitz461]

Just an FYI, now, with the latest bios (v3.40) it appears there is no need to specify PCIE lane speed, AUTO works now where it didn't with BIOS v3.20. There is also an additional GEN4 entry which didn't exist in v3.20.

 

[shpitz461]

Updated BIOS to vP3.50 and BMC to v1.27.00.

Do you guys know of the settings that will speed up the boot process? It takes forever to boot with the screen going blank several times.

 

[RolloZ170]

Do you guys know of the settings that will speed up the boot process? It takes forever to boot with the screen going blank several times.

its mem.check and ddr4 training. remove allmemory except one stick

 

[shpitz461]

Haha, well, I thought 128GB of ram is enough, now I regret not buying 256GB... But 256GB is almost 3x the cost.
The reason I'm complaining is that with the original motherboard I didn't experience such long wait-time until it actually boots, but with the replacement board the POST duration is very long...
No big deal since I don't reboot much...

 

[shpitz461]

I have to tell you guys, the ASRock support service is amazing, the other day I went to install a GPU and accidently bumped the door where the BMC chip is, the board was vertical and the chip just fell down and pin #1 got broken
The board will not boot without the BMC chip the way I have the BIOS configured, it just counts down on the 7-segment from 99 to 00, then after a minute or so the board reboots and it gets to do another countdown, and another, and then I gave up. The num-lock on the keyboard turns on, but there no video output from any video out (GPUs or onboard).
I emailed William from support and he will send me a replacement chip.
Kudos to William and ASRock for the awesome service!!!

 

[RolloZ170]

the other day I went to install a GPU and accidently bumped the door where the BMC chip is, the board was vertical and the chip just fell down and pin #1 got broken

impossible, you should play lotto.

 

[shpitz461]

Haha, maybe I should...

Although, I'm not sure if you're referring to my accident or to ASRock's awesome service...

 

[shpitz461]

Hi guys, I'm @ my wits' end...

I cannot get the motherboard to boot with any newer GPU.

I got BIOS v3.50 flashed, reset to defaults, configured the following config (attached pics).

It boots with an ATI HD 5770 no problem, with on-board GPU as default or not, doesn't matter, it boots fine.

As soon as I add/replace any of my other GPUs, the motherboard gets stuck in enumerating PCI bus (showing code 93 on-screen, code B2 in 7-segment on the board).

All GPUs work fine in other PC's, an RTX 3080, RTX 3090, AMD RX 550 Low-profile, Nvidia Titan Xp.

Any combination I try, in any port (1 through 5), will not boot.

If I remove the newer GPU's it boots fine.

Anything you can think of I'm missing?

I've spent 10 hours today trying different options, flashing different BIOS versions, clearing CMOS, to no avail.

 

[juma]

Try changing "Launch Video OpROM Policy" to something other than "Legacy".

 

[shpitz461]

Thanks @juma for the inspiration, I really needed it!

Your suggestion actually made the POST freeze with BIOS code B2 once I change from 'Legacy Only' to 'UEFI Only'

What I ended up doing was just disable CSM and remove the old GPU (ATI HD 5770 1GB from 2010).

Turns out I'm retarded, I had it working but didn't know.

What happened is that part of the boot process was on the on-board VGA display, and then it switched to one of the GPU's, which I never noticed until today when all of a sudden the TV was showing the boot process.

When posting is complete the BIOS diagnostic 7-segment display turns off, which I mistaken for a fault since I had no output on the on-board VGA out.

All is well with three GPU's installed, two running in VM's.

Thanks all for the persistence, I sure learned a lot this time around!

 

[shpitz461]

On another note, I'm struggling with IPMI, when I edit BIOS settings they don't stick, I change a setting in the webpage:



From 'PCI Express' to 'Onboard'.

I then go to to Exit BIOS menu and click Save Changes:


What am I doing wrong? How do I make BIOS configuration changes via IPMI webpage and make them stick during next boot?

https://192.168.1.253/bios/Index.html is where I go to make the changes. I already enabled the 'Sync' toggle in the main IPMI webpage (https://192.168.1.253/#dashboard).

Thanks!

 

[juma]

Glad to help. I've run into the display issue before as well, it's disappointing that there isn't an option to set a specific GPU as the primary display.

Try reverting to the 3.40 BIOS. I had multiple problems with the 3.50 BIOS. With 3.40 I was able to edit the BIOS settings via the IPMI with no issue.

 

[shpitz461]

Thanks Juma!
I actually want the primary display to be the on-board VGA, this way all the other GPU's are available in unraid to be used for other things.
And is my BIOS setting update process proper? Is that how you update the settings via web interface? Just like I show in the pictures?
I'll go back to 3.40 in a few days, still doing some testing with the VM's/GPU's.

 

[juma]

No problem. The first ROMED8-2T system I built was very unpredictable in its display output, but that one had 5 GPUs, even with the onboard VGA set to primary. The second ROMED8-2T system I built was much more consistent when turning setting primary VGA to PCIE.

Yes, I think you're updating the BIOS correctly. I used the same interface and was able to have it working without any configuration. I just made the changes and rebooted manually through the IPMI right after.

 

[shpitz461]

Cool, I'll give it a shot with BIOS 3.40 and see if it works properly.