EU ASRock Rack B450D4U-V1LQ5 mATX AM4

[dbram]

They are e-Waste

not sure this is e-waste : good enough for me

 

[dbram]

try to get rid of these 5700X

another long shot:

• did you use the latest microcode updates ?
• are you using bios 2.12 ?

it's a step i took when installing proxmox nodes, apply microcode updates, one of the proxmox community scripts, you never know

 

[luckylinux]

Its this 8U case from innovision, its a "blade style" approach with 10 vertical rack cases in the outer 8U shell.
Still direct access to all from the rear like a normal case.

While its a bit steep of a pricetag its cheaper than 10x 1U cases.

I got a few Supermicro CSE-512 for around 80 EUR each, with a X11SSL-F or X11SSM-F Motherboard, 200W PSU and Intel Xeon E3-1230 v5 ...

I tend to blow my budget every month, but i try to keep selling off toys as i get new ones.

I feel I don't have enough Time for that.

I could sell off some AM3 Motherboard+CPU+RAM Combos that I didn't even turn on for the past 4 years or so. Or also the Supermicro X10SLL-F and similar of that Generation. Or heck even the X9DRi and similar Xeon E5 v2 which now I plan to use as Desktop otherwise.

Problem is Time ... As well as Dealing with unreliable People not showing up etc.

I either need to go compact like those or just dedicate a rack to the ryzen nodes.
Then it would be drawing up a custom 3U case for that mobo to 3d print, to get full card height and more strength on the front ears.

I'm curious as to how you can use both PCIe Slots in such a Scenario (or a 1U Chassis for that Matter).
The x16 Slot there is probably just a 90° Adapter, but the x8 Slot you would need to "Displace" it by say 12 cm and do a 90° there as well, wouldn't you ?

So far, despite its Issues, and NOT being rack-mounted (need a Shelf), the Silverstone ML03 has been my favourite platform for this Type of (relatively) small Factor Builds. It's slightly more than 2U though.

 

[dbram]

actually the command we're supposed to use is this below, since the other also show the ECC of L1, L2, L3 cache of the cpu

dmidecode -t memory | grep -i 'error correction'

without ECC: Error Correction Type: None

dmidecode -t memory | grep -i 'error correction'

with ECC: Error Correction Type: Multi-bit ECC

 

[dbram]

I seem to recall Proxmox blocks microcode Updates by default as well.

well I haven't checked if it actually does update but it thinks it can, some 202502xx update i believe on Debian 12.1 (underlying the proxmox)

Proxmox VE Helper-Scripts

 

[luckylinux]

This one positive side of living in the middle of nowhere, nobody wants to pick up stuff and its always shipping it.

Yeah positive ... I'm also kind of in the middle of nowhere but then you also need to Pack it and almost daily trips to the Parcel Store to ship them out .

I used to do that a few Years ago but between the Time to take Pictures, write a Listing for it, put it on whatever platform is available locally to you (I used DBA, but could also be Facebook Marketplace), Test Stuff, Package Stuff, deal with Customers that either don't show up or change their Mind, plus all the Negotiations etc, it's usually very Tiring. Especially if you are dealing with just relatively cheap Motherboards/NICs you can easily lose more than a Couple Hours for each Item and *maybe* get 20 EUR or so back ...

 

[dbram]

Isn't it just to install the amd64-microcode Package and possibly removing the Blacklisting in e.g. /etc/modprobe.d/amd64-microcode-blacklist.conf ?

Why do we need a custom Script just for that ?

it does more than that I think...

msg_info "Downloading AMD processor microcode package $microcode"
curl -fsSL "https://ftp.debian.org/debian/pool/non-free-firmware/a/amd64-microcode/$microcode" -o $(basename "https://ftp.debian.org/debian/pool/non-free-firmware/a/amd64-microcode/$microcode")
msg_ok "Downloaded AMD processor microcode package $microcode"

msg_info "Installing $microcode (this might take a while)"
dpkg -i $microcode &>/dev/null
msg_ok "Installed $microcode"

 

[dbram]

You are on BIOS L2.12 with the AMD Ryzen 3700X ? just to make sure it will still work with AMD Ryzen 3700X if I upgrade the BIOS now

yes, boards are upgraded 2.12 (instant flash using usb stick)

 

[dbram]

FYI: the same memory module on the Gigabyte MC12LE0's using a 5600X

sudo dmidecode -t memory | grep -i 'error correction'
    Error Correction Type: Multi-bit ECC

 

[azee]

I received my 4 sticks of "ATP X4C16QE8BNRCSE-7-LI1" in the morning. I plugged the 2 stick and wanted to verify ECC, but I can't find any ECC menu at all.
I don't even see Advanced -> AMD CBS -> UMC Common Options

I am running on Ryzen 5600x at L2.11.

I don't have any good cooler at the moment for long testing, currently using an old AM3 based cooler for verification, temp jumped to mid 60C.

imgur link

 

[luckylinux]

why can someone who quotes another post change the link and use this 'feature' to link someone to another targeted malicious address? Is this a feature, a bug or ....

What do you mean exactly ? Just curious. And yeah, everybody can modify any Part of any Quote of Anything/Anyone (not only limited to URLs).

EDIT 1: and @gb00s, who/when/where changed the Link to a Malicious Address ? I don't see this having happened here, or ?

 

[RolloZ170]

microcode of BIOS can be updated with a Hex editor, but will not be accepted by AFU flash tools after mod. i guess.
the microcode are located in a padding after AmiNvramMainAreaGuid

Base: 57000h, Offset: 57000h
Full size: 8F8000h (9404416)
the microcodes have a size of 0xC80 or 0x15C0
the first microcode ( cpu00800F82_ver0800820D_2019-04-16_B0DE605C ) is located at 24D000h

 

[luckylinux]

It's really the Damn REAR USB Ports.

Got a bit of Time to test with a USB3 Header -> 2 x USB3 Type A Ports Adapter where I could connect my Keyboard + USB Flashdrive (with LiveUbuntu). Worked on the first attempt. After losing an entire Weekend 1 Week ago trying to make the rear Ports work ....

EDIT 1: if only this Forum would fix their Usual "Oops! We ran into some problems." I would also be able to upload some Pictures . This is really getting on my Nerves now. First I need to RESIZE Pictures because they are not accepted as it is. On top of that, even when resized, they are not accepted anyways. HTTP Error 500 - Internal Server Error.

EDIT 2: Embedding from imgur, since it won't work at all here

http://imgur.com/CNMQgIi

http://imgur.com/ysuc25q

http://imgur.com/1BZtWoX

http://imgur.com/2nflEZ7

 

[RolloZ170]

and installed a NIC and now ECC seems to be recognized !!

unpowered the CPU the first time ? ( standby power to PCH / SoC )

 

[gb00s]

In my humble opinion and being retarded for life, why a motherboard shall support ECC w/3700x but won't with 5700X. If I'm not mistaken these were HETZNER boards for AX41 environments that were offered with Ryzen 5 3600 and 5600's with ECC. They are still in business .... with ECC.

 

[luckylinux]

It seems there are three ROM-s available for ASPEED:

• VBIOS
• UEFI
• VBIOS+UEFI

Maybe we just need to flash the combined VBIOS+UEFI ROM to ASPEED and then VGA will also work pre-os in legacy BIOS mode.
You can flash in EFI console.

Download link:

https://www.aspeedtech.com/tw/support_driver/

Is there a Way to flash from Linux or can it only be done from EFI Shell ?

EDIT 1: And which File do you think we should flash ? Is it uefi_2500.rom or uefi_2500_r4.rom ?

The README Mentions

;===============================================================================
; ASPEED Graphics VBIOS + UEFI Driver ReadMe
;===============================================================================
1.Overview:
- ASPEED Graphics VBIOS + UEFI GOP Driver (Expansion ROM format)

2.UEFI Driver Release Architecture:
- X64 : X64 Merge Binary with signed

3.UEFI Driver Release Image Description:
- uefi_2500.rom: AST2500/2510/2530, MCLK=800Mhz
- uefi_2500_r4.rom: AST2500/2510/2530 + Reserve 4MB Buffer, MCLK=800Mhz
- uefi_2600.rom: AST2600 + ASTDP + D-sub, MCLK=800Mhz
- uefi_2600_r4.rom: AST2600 + ASTDP + D-sub + Reserve 4MB Buffer, MCLK=800Mhz
- uefi_2600_vga.rom: AST2600 + D-sub , MCLK=800Mhz

4.Contact Window:
- aspeed_vga_support@aspeedtech.com

What the Hell is "+ Reserve 4MB Buffer" ? Do we need this extra Buffer (thus use uefi_2500_r4.rom) or should we use the other one (uefi_2500.rom) ?

 

[azee]

OK, so it is like @RolloZ170 said.

You need a COMPLETE Poweroff (don't know for how long exactly, give it at least 30 Seconds with the Power Cord DETACHED).

I rebooted with the Graphics Card installed and it still works with ECC. And I still can't see these Menus that should exist in the BIOS ...

Note that I am on BIOS L2.12 now. But BIOS update alone did NOT fix this !

root@LiveUbuntu02:~# sudo dmidecode -t memory | grep -i 'error correction'
    Error Correction Type: Multi-bit ECC
root@LiveUbuntu02:~# dmesg | grep -i edac
[    0.650354] EDAC MC: Ver: 3.0.0
[    6.543072] EDAC MC0: Giving out device to module amd64_edac controller F19h_M20h: DEV 0000:00:18.3 (INTERRUPT)
[    6.543076] EDAC amd64: F19h_M20h detected (node 0).
[    6.543080] EDAC MC: UMC0 chip selects:
[    6.543082] EDAC amd64: MC: 0:     0MB 1:     0MB
[    6.543084] EDAC amd64: MC: 2:     0MB 3:     0MB
[    6.543088] EDAC MC: UMC1 chip selects:
[    6.543089] EDAC amd64: MC: 0:     0MB 1:     0MB
[    6.543091] EDAC amd64: MC: 2:  8192MB 3:  8192MB

I used the Scripts inside the SSP Subfolder. Boot into UEFI Shell. Run patch.nsh. Reboot with reset. Boot into UEFI Shell. Run read.nsh. Shutdown. Disconnect Power Cord for at least 30 Seconds. Boot into Debian/Ubuntu and check dmesg / dmidecode.

Note: feel free to download setup_var.efi from the Official Github Page, you don't need to use the one that I provided if you don't trust it. I just don't want starting to have to edit Subfolders and whatnot or Upload the 8 Patch + Read Scripts individually depending on what CPU we want to test against.

It still sucks that nothing shows up in the BIOS Menu. But I guess this is better than nothing, right ?

EDIT 1: maybe also worth trying with JUST the BIOS Update + Poweroff. But from what I recall from @ALinden, that didn't seem to work. Not sure if he did a complete Power Off though ...

EDIT 2: @ALinden, @azee: can you try the Procedure with the Poweroff as Indicated in this Post ? I'm NOT getting the Menu showing up in BIOS but ECC seems to have started working now !

will this file works with 5600X, or do i need to change any thing? i am quite new with bios mod, so i just want to be sure.
and i attached no graphic or lan card in either of pcie slots. with stock L2.11 (complete power off)

 sudo dmidecode -t memory | grep -i 'error correction'

output was

Error Correction Type: None

 

[gb00s]

AX41's with the 3000 & 5000 CPU's at Hetzner is available from the 'Auctions' section only as these systems are kind of 'EOL' in their terms. From an operation and financial risk perspective, I can't imagine Hetzner would offer ECC environments not functioning as advertised .... because of this 100 Million EUR database that may .... Finish it by yourself.

 

[luckylinux]

No video from onboard VGA pre-OS, but system POSTs and tries to boot. So if you boot linux it will initialize the onboard AST graphics and you get picture.

That's not my Experience though .

No Video AT ALL from the onboard VGA using Default BIOS (2.09 as it arrived here, still need to upgrade to 2.12). No matter if PCIe GPU installed or not.

I tried the ASPEED VROM flash utility for EFI shell too. It says it can't find VGA and won't do anything.

Ah, sad to hear that . Did you try the different Files or just one of them ?

 

[gb00s]

No conclusions made at all.

 

[luckylinux]

Im doing this for multiple stacks of 4-8 boards each.

Pico psus like these ones and they just use pci-e 6pin leads to a breakout board like this.
(Both are random example listings, might not be the cheapest etc)

Cheap and power efficient.

How were you planning on "Mechanically" laying out the 6-pin from one Chassis to the other though ?

I tried to see if I could find some 6-pin Female to 6-pin Female Pass-through Adapters but couldn't find any.

Otherwise just get a standard Cable, some heatshrink Sleeve and a Cable gland and let it hang out of the Chassis ?

What I really do NOT like about these PicoPSUs, besides the questionable Quality overall, is the SATA Power Connectors. It's those soldered (NON-crimped) à la MOLEX-to-SATA-lose-all-your-Data kind of Fire Risk. Of course I could chop it off and try to re-crimp a Molex Connector out of it, but it would have been nicer if these SATA Connectors were not there to begin with .

 

[ALinden]

Hey, so is this summary correct at the moment:

@luckylinux ECC started working with 5700X but it is not clear what was the final thing that made it work. One possibility is microcode update?

@dbram ECC just works with 3700X and bios 2.12

@azee ECC not working with 5600X, also tried setup_var.efi to change ECC bios setting to "Enable"

As for me:
I did not use any GPU while testing.
I think I did complete poweroff, but not 100% sure.
Will test today.

 

[ALinden]

I found out that there is a utility by Ami to manage UEFI settings form linux command line. Its name is SCELNX. Googling that I found that someone uploaded a version to github: tools/SCELNX64 at master · doublechiang/tools
Running it like this: ./SCELNX_64 /o /s nvram.txt /a produces a text file with all settings and options. Interestingly variable (varstore_id) is not listed. Looking at the output I see that with 5700X "ECC Enable" has offset 0x11B. So this confirms that with Vermeer CPU CbsSetupDxeSSP is used.

First test today was the total power off for couple if minutes. No change.

Then I tried to change the installed 1x8Gb to 2x16Gb. Just to see that maybe the memory training or something the bios does upon detecting memory change will trigger ECC activation - no change.

Then I reviewed the offsets that @luckylinux used.
I could not see how the "Overclock" setting with 0xD4 offset could be "parent" or dependency for "ECC enable". However I tought I will try it.
So I did change it from 0xFF to 0x01 (manually with setup_var.efi and only that setting). Rebooted with uefi shell command reset (so no power off) and surprise surprise ECC is active dmidecode shows it and also there are many EDAC lines in dmesg log.

I made couple of reboots and ECC was working. Very nice I thought.

Then I thought I need to verify the procedure to get ECC working.
Rebooted to BIOS and selected "Load UEFI defaults".
Reboot to Linux, verified that ECC was not active.
Reboot to uefi shell, set AmdSetup:0x11B=0x01 and AmdSetup:0xD4=0x01
Reboot to Linux and . . . nothing . . . ECC not active
Tried total poweroff, change memory modules - still nothing.

Some setting or event will trigger activation of ECC but it is not simply changing these two settings.

 

[luckylinux]

Its not like crimped vs non-crimped alone say anything regarding if its a risk or not tho.

Low quality of either can be a fire risk, neither is a fire risk as a general rule.
(The old rule of thumb is not about non-crimped plugs in general either, its about some specific low quality adapters that sellers used to include.)

I also had a CRIMPED Molex to SATA Adapter went up in Smoke as soon as I started a PC, so I agree, that is unfortunately NOT true all the Time.

I'm still not sure about what happened there (somehow I got a Short-Circuit, as I don't think the Inrush Current should have caused that) and of course the PSU did NOT trip in Overcurrent ...

I can only imagine your disappointment unpacking a brand new server and first task is to start chopping off connectors.

???? Well first would be to Select a Chassis and PSU ... For the Price I was tempted to actually get the Fujitsu TX1320 M3 Barebones and just get the Motherboard and PSU Out. Still cheaper than the Silverstome ML03 and has 4 x SSD Hot Swap Bays.

The Inter-Tech IT-503, Inter-Tech IT-502 and Inter-Tech IT-607 (all Desktop Cases) are slighly smaller than the Silverstone ML03B are all significantly cheaper than the Silverstone ML03. Not a lot of Options for installing extra Fans seems to be the Limitation though .

On the Silverstone ML03 I typically just fill up the Side with 80mm Fans, that keeps the RAM as well as the PCIe Slots sufficiently cool (the Static Pressure of the Fans is enough for some Air to go out through the PCIe Slots, thus cooling the NIC and HBA).

How were you planning on "Mechanically" laying out the 6-pin from one Chassis to the other though ?

I tried to see if I could find some 6-pin Female to 6-pin Female Pass-through Adapters but couldn't find any.

Otherwise just get a standard Cable, some heatshrink Sleeve and a Cable gland and let it hang out of the Chassis ?

You didn't answer the Question .

 

[ALinden]

When I got ECC active I did not set 0x20 to 0xFF -- just trying to set 0xD4 activated ECC

After "load defaults" and not getting ECC active I have tried setting 0x20 to 0xFF too but ECC still not activating.

If you say you can repeat this, do you mean:
1. load defaults
2. verify ECC not active
3. change the three settings
4. verify ECC active
?

 

[luckylinux]

I'm actually thinking if the Fujitsu TX1320 M3 Barebone could indeed be a viable Solution.

MicroATX Board should Fit. I think there isn't much space for the CPU Cooler though. Plus not much Space at all for additional Fans. But there is a nice hotswap Bay and with some SFF-8643 to 4xSATA REVERSE Breakout Connector it could be used .

The Proprietary PSU has a weird 16-Pin Proprietary Connector, supplying only +12V/GND AND +11V Standby (instead of +5V Standby).

So maybe that can be permanently be switched on and supply the 12V to the PicoPSU ?

Some People here discussed the possibility of using those 12V Rails:
- 16-pol Stromeingang Mainboard + 20-pol Anschluss NT? (Fujitsu-PC)
- stupid fujitsu 12v only psu 16pin connector

 

[ALinden]

Just for clarity: I did the reset to default only to verify that our thinking what is needed for ECC activation is correct.

And now I can't get ECC active again. And yes I have set 0xD4

 

[ALinden]

That SCELNX kinda does that.
Download it from here: tools/SCELNX64 at master · doublechiang/tools
Then run it like this:
./SCELNX_64 /o /s nvram.txt /a /v
Then upload nvram.txt somewhere.

I will try to compare the settings you currently have with active ECC to mine without ECC active

 

[luckylinux]

I received Ryzen 5600X and Ryzen 2200G on weekend

I started with Ryzen 2200g

VGA display no output, No Input signal detected
connect the keyboard to internal USB3.0 header, keyboard respond to NUM, SCROLL and CAPS lock.
Dr. Debug complete the initialization with 62 and 02 and then turn OFF
connect external GPU to both X16 and X8 PCI-E, still no display.
Clear CMOS pad did not help. did multiple time with removing the battery, Power cord , RAM and etc.

Next day i swap the process with Ryzen 5600X

and VGA TURN ON at first boot.
The first thing i notice was the BIOS version, it was L2.11
Rear USB ports are working as well.

http://imgur.com/a/rASZZpo

I have few more boards to test during weekend.

Did you use ECC Memory by any Chance ?

The Ryzen 2200g (non-PRO APU) does NOT support ECC RAM ...

In my Case with Ryzen 2600X, Rear USB3 Ports would NOT work no matter what and VGA would also NOT work. ONLY installing an external GPU in the x16 Slot worked to get a Display Output.

 

[ALinden]

Linux