First ASRock Rack / EPYC Milan build! but... no POST?

[RolloZ170]

CPUs properly installed ? (torque)

 

[bardlam]

it does with BIOS 3.10

BIOS is updated. you have to resolve the critical state.

Okay. You're more confident than I am that 3.10 adds support for Milan-X. I don't see anything in the BIOS version notes mentioning Milan-X or 3D V-Cache support being added in 3.10, and there are no Milan-X CPUs listed in the supported CPU list.

CPUs properly installed ? (torque)

Yeah, that seems fine. I've reseated the CPU, as well as swapped it out for the second 7773X CPU. With the carrier frame, it's difficult to mess it up.

 

[RolloZ170]

I don't see anything in the BIOS version notes mentioning Milan-X or 3D V-Cache support being added in 3.10,

AGESA

and there are no Milan-X CPUs listed in the supported CPU list.

many other SKUs missing. asrock style.

 

[RolloZ170]

Okay. You're more confident than I am that 3.10 adds support for Milan-X

but it is highly possible, ASROck hides a required BIOS update on they're site. we know that.

 

[RolloZ170]

remove battery for a while (AC unplugged)

 

[bardlam]

remove battery for a while (AC unplugged)

Good idea. I removed the battery to test voltage, but I didn't let the board rest for a while without it. I'll do that now.

 

[bardlam]

Oh heck. I picked up a server that uses 200v-240v and didn't realize it. My home lab is normal U.S. 120v. That's what was causing those voltage sensors to report 0V.
It's a wonder that the BMC was turning on at all.

 

[bardlam]

ASRock is checking to see if they can swap out the PSUs for me.
Can't believe I missed that.

I would have realized it much sooner if the PSUs hadn't been clicking on with the BMC running the whole time
Thanks for all your help and suggestions, @RolloZ170 !

 

[wings]

Oh heck. I picked up a server that uses 200v-240v and didn't realize it. My home lab is normal U.S. 120v. That's what was causing those voltage sensors to report 0V.
It's a wonder that the BMC was turning on at all.

Oooooof. You're extremely lucky it wasn't the other way around - plugging a 110V server into 220-240V...

 

[bardlam]

Oooooof. You're extremely lucky it wasn't the other way around - plugging a 110V server into 220-240V...

I know, right?

tbf, I did check the voltage when buying it... it's just that many US power supplies will handle 220/240v as well as 110/120v (like when we travel with our laptops to EU and can use a simple adapter to plug them in without down-stepping voltage).
So I read it and it didn't sink in that it wouldn't handle normal US voltage and higher voltages.

 

[RolloZ170]

it's just that many US power supplies will handle 220/240v as well as 110/120v

server PSUs with this also exists
but the negative is, they have 1100W @230V and only 850W @120V i.e.
from the efficiency view its better to design for a small voltage range.
also the wide range PSU will have high AMP fuses for 120V operation, they blow hard @230V, sometimes after fire...

 

[bardlam]

server PSUs with this also exists
but the negative is, they have 1100W @230V and only 850W @120V i.e.
from the efficiency view its better to design for a small voltage range.
also the wide range PSU will have high AMP fuses for 120V operation, they blow hard @230V, sometimes after fire...

Yep, I understand the reasoning. If I had a higher voltage option at home, I'd keep them for better efficiency.

 

[bardlam]

Depending on how ASRock responds, I may end up swapping the 2U2E-F/ROME2 server for a Gigabyte R282-Z90 (rev. A00)
Does anyone have any opinions on ASRock vs Gigabyte?

It'd be a $200 price increase, but I'd also get a few extra PCIE x8 slots, which would be nice.

Edit: It looks like Gigabyte has released new BIOS versions more recently, supporting AGESA 1.0.0.A for Milan.

 

[RolloZ170]

if you want always up to date with BIOS, stay away from ASRock Rack

 

[bardlam]

I haven't found a barebones Supermico server for sale that fits what I need, so I ended up with that Gigabyte R282-Z90 I mentioned previously.
It cost a bit more than the ASRock 2U2E-F/ROME2, but it has a few advantages:

• PSUs that handle either 100-120V or 200-240V.
  • essential for my home lab that uses standard U.S. residential power.
  • ASRock was kindly searching for PSUs that would work for me, but I was under a time constraint to get this server up and running.
• 4 more PCIE x8 slots.
  • with GPUs installed, it's actually 2 more.
  • I didn't think 2 extra x8 slots would be important at first, but for both servers I needed to use an expansion card for PCIE 4.0 m.2 nvme drives, so the asrock server was getting cramped quickly.
• 2DPC "overclocking" is available using plan-of-record (POR)
  • Filling all 32 DIMM slots (2 dimms per memory channel) normally downclocks the ram from 3200 to 2933. This Gigabyte motherboard can still clock 2DPC at 3200 MHz.
  • I don't understand how this works under the hood yet, but BIOS reports ram running at 3200 MHz.
• OCP 2.0 or OCP 3.0
  • asrock was only OCP 3.0
  • not really an issue, but might be useful. my home lab is still using 40 GbE copper, and an OCP 2.0 card might be cheaper.
• Onboard 1GbE
  • not essential, but useful.
  • the asrock server didn't have an onboard NIC aside from the management port, and my home lab uses a mix of 1GbE and 40GbE.

 

[RolloZ170]

2DPC "overclocking" is available using plan-of-record (POR)

• Filling all 32 DIMM slots (2 dimms per memory channel) normally downclocks the ram from 3200 to 2933. This Gigabyte motherboard can still clock 2DPC at 3200 MHz.
• I don't understand how this works under the hood yet, but BIOS reports ram running at 3200 MHz.

it is just a recommendation for stability:
2DPC 3200, POR enabled = 2933
2DPC 3200, POR disabled =3200 - but you can run into memory errors if you don't have latency headroom.

 

[bardlam]

Just as a fun little benchmark, I was able to compile LLVM's current main branch in 2 minutes 9 seconds

git clone https://github.com/llvm/llvm-project
cd llvm-project
mkdir build
cd build
cmake -G Ninja ../llvm -DCMAKE_BUILD_TYPE=Release
time ninja