EU ASRock Rack B450D4U-V1LQ5 mATX AM4

[Cruzader]

I was looking for a 4u 3x140mm Panel but I cannot find anything unfortunately, neither on printables nor thingiverse .

For a 42U Rack that would need 10 x 4u (plus 2 Blanks) otherwise 14 x 3u. Saying 3 fans on each Panel, that would be 30 x 140mm Fans or 42 x 120 mm Fans respectively. But on the good Note, they should be able to be tuned down a lot either by PWM or by using an Adjustable Power Supply .

For rack panels/blanks i usualy order from thomann, they got 3u 3x120mm at 119dkk.
Standard stuff that is used for stage/audio also tends to be cheaper there than in the IT space for consumers.
(They also got a solid selection of PDUs if you need some)

Arctic cooling has 5packs of their f12 and p12 fans at a decent price.

 

[luckylinux]

What exactly are you afraid of when running it on a system without network and without harddrives only with a usb stick with a bootable linux and another usb stick with a r/w partition with SCELNX_64 ?

Why do I need 2 USB Drives now ?

And "No Network" might be your Case. I have LAN connected and SSH working ...

 

[Cruzader]

Anyways, 119 dkk * 14 blocks + say 5 eur / fan * 42 fans is still quite the money...

Its not something you need either way, so id guess you have a bit of a budget for the setup when buying it to begin with.

 

[Cruzader]

Well yes and no.

Unless you want the loud 1u blower fan screaming and rattling at you

I was just thinking plastic is cheaper than metal.

But I might also do with wood maybe.

You will need some airflow in the case regardless of your rear pull config tho.

Personally id just replace the blower with a lower rpm fairly silent one possibly paired with a 40mm for nic/nvme.

 

[luckylinux]

1. disconnect network and any nvme/ssd/hdd
2. boot up any live linux usb stick
3. if that live linux usb has a r/w partition put SCELNX_64 there (no need for second usb stick), if it does not have r/w partition put SCELNX_64 on another usb stick

I have my usual LiveUSB in Debian and Ubuntu Flavour. R/W of course otherwise it would be like an ISO ...

That counts as a Hard Drive so if you are paranoid about HDD/SSD why not the LiveUSB, I don't get it ...

 

[Cruzader]

Well, 3 x 40mm each Server (1U) isn't much different than 3 x 120mm each 3U in Terms of Cost. But it will likely be a HUGE Difference in Terms of Noise.

The noise depends on what fans you go with for either option, neither of them has to be loud.

But its not like you can just remove the blower and get good airflow with only pulling air in rear of rack either.
You are looking at placing atleast 1 fan in the case regardless.

 

[luckylinux]

@ALinden

Apparently many of the Variables (approx. 50%) don't exist in every CPU Configuration so there are quite a few Errors.

script.txt to be renamed to script.nsh and executed in UEFI Shell.

You will need to convert current.txt to UTF-8. I did it from Geany: Document -> Set Encoding -> Unicode -> UTF-8.

By default UEFI Shell generates UTF-16LE .

Probably best to filter with:

grep -a -Eiv "Error reading variable|No variable with specified name found|Specified offset" current.utf8.txt

Remember: I'm on BIOS L2.12

 

[gb00s]

I'll just do an Order using CraftCloud for ~30 Pieces of that IO Shield (I'll see if I can get it printed in Metal Sheet, might be way too expensive though), as well as some other Stuff I could 3D Print as well.

Last Time i went with a Belgian Company that was the Cheapest and the Results were "OK" I would Say. Since it was some SSD Caddies and it wasn't a huge premium, I went with 40% Fill instead of the "Standard" 20%, hopefully to make them a bit more sturdy.

EDIT 1: I don't have a 3D Printer, that's why I go with CraftCloud

EDIT 2: I ended up buying 26 of the Chassis CSE-512

EDIT 3: For Cooling I'm honestly considering (hence why I want to mount the Chassis Rack Ears in Reverse, so I can have all Connections and Ears on one side and I shouldn't even need to use the Front Panel Anymore) to put a bunch of external 120mm or 140mm (better) Fans all on one Side of the Chassis, by using something like this to mount them

19＂ 3U Server Rack Fan Panel by jackknrsh7 | Download free STL model | Printables.com

A 3u 19＂ server fan panel, which can be used to either add cool air intake or hot air exhaust to a full-size rack. | Download free 3D printable STL models

www.printables.com

And 1u/2u/3u/4u Blank Plates where Servers are NOT installed, on the other Side:

19＂ Server Rack Blank Plates (1U, 2U, 4U) (EIA-310) by btadeus | Download free STL model | Printables.com

Server rack blank plates sized at 1U, 2U, and 4U. Used for controlling airflow\temperature in a server rack. | Download free 3D printable STL models

www.printables.com

I was looking for a 4u 3x140mm Panel but I cannot find anything unfortunately, neither on printables nor thingiverse .

For a 42U Rack that would need 10 x 4u (plus 2 Blanks) otherwise 14 x 3u. Saying 3 fans on each Panel, that would be 30 x 140mm Fans or 42 x 120 mm Fans respectively. But on the good Note, they should be able to be tuned down a lot either by PWM or by using an Adjustable Power Supply .

FS.com offers 10x blank plates for ~80US$ in a package. They are tool-less. Super easy to use. Not sure if you can print 10 for 80 incl fast shipping.

 

[luckylinux]

@ALinden

Please provide yours. Both using my Script and also the SCELNX_64 if you want me to do that as well (although I'd have to find another USB Flashdrive as I don't really want to compromise the one I currently have).

 

[dbram]

Another short update. I'll be writing it up on a github project or so if time allows it.

• tried the usb hdmi capture stick (no name thing)

  • Bus 003 Device 002: ID 534d:2109 MacroSilicon USB Video
    Device Descriptor:
    bLength 18
    bDescriptorType 1
    bcdUSB 2.00
    bDeviceClass 239 Miscellaneous Device
    bDeviceSubClass 2
    bDeviceProtocol 1 Interface Association
    bMaxPacketSize0 64
    idVendor 0x534d MacroSilicon
    idProduct 0x2109
    bcdDevice 21.00
    iManufacturer 1 MACROSILICON
    iProduct 2 USB Video
    iSerial 0
      bNumConfigurations      1

• installed the benfei vga to hdmi
  • note: this model has a vga input and uses the usb power cable as an audio source (so it contains an usb audio input too)
• created a lxc container in one of the proxmox hosts
• passthrough of usb device to the lxc container and some extra things for permissions in an unprivileged container
• the lxc container can be skipped of course when doing it on a dedicated kvm device
• cloned the ustreamer project from pikvm and is 'streaming' the usb capture to a html5 browser
  • 
    
  • another attempt using ffmpeg or aiortc was working too but much more resource intensive

So what's left,

• continue on the esp32-s3 to act as USB HID for keyboard and mouse
• capture the mouse and keyboard when hovering over the displayed vga/hdmi output and send it to the esp32-s3 that feeds it to the corresponding host
• add a VGA 4-1 switch with (another esp thing) to remotely switch
• add a simple server deamon (similar to pikvm kvmd), that organises everything a bit, allows for vga-node and esp32-s3 selection
• add the a decent webserver e.g. nginx with all required security, (mfa) authentication, lets encrypt certificates, waf, ...

total cost for a 4 node setup would be : around 50,- eur (excluding a dedicated pc or raspberry pi)

• 1x pc with decent usb ports or raspberry pi (any lying around, fill in your estimate)
• 1x usb capture stick 7,- eur or CSI2, 35,- eur capture device
• 4x esp32-s3 : around 2-3,- eur each
• 4x svga cables : 1-2 eur each
• 1x svga switch : 6 - 10,- eur (to be confirmed)
  • 4 -> 1 svga : one that allows to hijack the input button, preferably with a direct assignment (and not rolling through the inputs)
  • if the vga switch supports usb ports too, only one esp32-s3 per 4 nodes would be possible too
• benfei SVGA to HDMI, 9 - 12,- eur was available through amazon, sometimes the price is reduced.
• an additional esp device (can even be an esp8266) for the gpio button or infrared press to control the vga (or hdmi switch)
• optional:
  • a couple of pcb's that allow atx controls power, reset, led detection of the hosts

 

[dbram]

@dbram: what's the Status on the IO Shield, does it work as expected with your latest Changes ?

What were the Issues (before the Changes you made ?), something about the Screw Holes not working properly ?

latest version is on printables.com

The asrockrack board b450d4u-v1lq5 is beig sold at a very good price some cases were needed. had some supermicro CSE512 left over, combine them, started from the very identical board x470d4u (that has more connectors).
todo's:

• no holes yet for screws. : fixed in v1.5
• the connectors at the bottom need to be slimmer, fixed in v1.5
• the margin around the vga connector needs to be increased fixed in v1.5
• add opening for the top case cover clip : fixed in v1.5

the thickness of the shield is ok.

 

[dbram]

Do you think the extra complexity of the VGA/HDMI Switch is worth it though ?

good question, a usb-hub with 4 hdmi captures is feasable, but adding 4x benfei is not cheap, 9-12 eur each. it almost doubles my total cost (per 4 nodes).
I had a lot of trouble to get my low-cost vga-hdmi adapter to work (also in the range of 2-3,- eur), very few low-resolutions worked
the benfei ( or hdmi capture) is not the fastest but it works.

EDIT 1: why LXC and not Docker/Podman Container though ?

that's also possible of course.
i'm trying everything using material that is 'lying around'

edit:
1. it is possible to run the kvm thing 'self contained' on one of the proxmox host nodes in this way passing it again through a docker,podman would be a bit overkill.
2. the esp32-s3 is connected over the network ethernet or wifi to the kvm controller and usb is plugged to each node that needs to be controlled,
or
3. ethernet can be done using additional ethernet connector, adding a low cost ethernet is 3-4,- eur for each esp32 (POE increases the cost quite a bit again)

 

[luckylinux]

I ran your script.
Basically most that differs from your result is boot entries that depend what drives are connected.

Seems your script does not read anything from Setup or PBS.

If we want to compare settings, I think you have to send me SCELNX_64 output.

So I provide Data, you provide Anecdotal Evidence and no Data.

In that Case no, I won't provide anything anymore.

 

[VirMach]

I've contacted ASRock and reported the USB issue thoroughly (as well as how many units of them we have plus how many units of their other boards.) They did mention it was built to order but so far they've been receptive enough to contact the BIOS engineer for this board. So best case scenario maybe we'll get a 2.13 version.

@luckylinux I ended up giving up trying to fix the USB if you can't tell but I found a bunch of 9 pin to dual USB and I'm 3D printing a bracket to mount them in the rear as backup if ASRock doesn't respond.

Why ?

As long as the external Fan has enough Static Pressure (at the required Airflow Operating Point) and there are no Bypass paths (i.e. if a Server is not installed, then a 1u Plug / Blank is installed instead), all Air should be able to flow through the Chassis ...

But I'm all ears if you have suggestions for cheap, good and quiet 40mm Fans

If I'm understanding you correctly, you want to mount a bunch of 120mm fans on the outside of the chassis, 3 of them per 3 servers?

You should go with these monsters:

9CR1212P0G03 | Counter Rotating Fan | San Ace | Product Site | SANYO DENKI

Arctic P12 - 56CFM, 1800 RPM, 2.2 mmH2O static pressure
This one - 300 CFM, 6200 RPM, 1.93 inH2O static pressure

But you might think "wait that one is only 1.93 static pressure and the Arctic is 2.2" no it's inH2O as in 49 mmH2O. Okay don't look up the price or weight though. Or power usage. Or decibels. The cool part is though if you did theoretically go with these, at 0% duty cycle which is 1800-2700 rpm, you still get 134CFM and 9.5 mmH2O.

They do have a bunch of models, look around and see if you can find any for cheap and then look at the spec sheet, they almost always perform way better even at the same decibel range. Oh no why'd I do this now I'm tempted to buy 20 of the older models and use it to improve my HVAC, someone's selling them for $5 a piece.

 

[dbram]

I've contacted ASRock and reported the USB issue thoroughly (as well as how many units of them we have plus how many units of their other boards.) They did mention it was built to order but so far they've been receptive enough to contact the BIOS engineer for this board. So best case scenario maybe we'll get a 2.13 version.

That would be very nice ! Thanks Virmach and Asrockrack already !

 

[luckylinux]

Here are my files.

Also lets compare our CPUs. Here is beginning of /proc/cpuinfo of mine:

vendor_id    : AuthenticAMD
cpu family    : 25
model        : 33
model name    : AMD Ryzen 7 5700X 8-Core Processor
stepping    : 2
microcode    : 0xa201210

I actually had a look and am re-running my Script to extract the VarOffset Values.

Actually you are probably right, the Setup was missing. I actually only extracted one Section from uefitool (the only one that seemed to work ?) and it had most of the Stuff in it (IMHO). Like it generated 400+ Files when extracted.

Now I extracted 6 Volume Sections but only 2 could be extracted by ifrextractor (1 of which is what we already have).

I decided to bite the bullet and just create a Github Repository for it, at least it's in some Version Control now ....

GitHub - luckylinux/ASRock-B450D4U-V1LQ5: Tools to make the 2nd Hand ASRock B450D4U-V1LQ5 work correctly

Tools to make the 2nd Hand ASRock B450D4U-V1LQ5 work correctly - luckylinux/ASRock-B450D4U-V1LQ5

github.com

 

[dbram]

Some Updates from my side.

I received my super cheap VGA to HDMI Adapters (2.7 USD / each) x 23 Pieces:

https://www.aliexpress.com/item/1005006371360750.html

As said, based on a Review on Aliexpress, this seems to use the zy5621 Chip.

I also received a HDMI to USB Capture Card (5.5 USD / each) x 1 Piece - USB3 Version:

https://www.aliexpress.com/item/1005006825608006.html

The Boot Sequence is very fast, but if I keep pressing DEL (or possibly F2, didn't try that one though), I can successfully enter the BIOS and see everything fine there.

Most likely I messed up the Framerate in the below Commands, I think 60 fps is more than this Adapter supports, it seemed to work correctly though.

That's very good news for your svga and hdmi capture adapters. I might have to retry my cheap adapter, granted I only tried it on 2 real monitors with SVGA inputs, not the capture card...

 

[luckylinux]

Here are my files.

Also lets compare our CPUs. Here is beginning of /proc/cpuinfo of mine:

vendor_id    : AuthenticAMD
cpu family    : 25
model        : 33
model name    : AMD Ryzen 7 5700X 8-Core Processor
stepping    : 2
microcode    : 0xa201210

Are you sure you uploaded YOUR File generated from my Script ? It looks EXACTLY the same as Mine ... Diff Outputs exactly ZERO differences ...

Which is weird since you said there were some Differences

 

[dbram]

in the mean time I've advanced on the esp32-s3 combined with a lxc container running nginx and ustreamer (low resource < 1% cpu). currently still going through lightweight mqtt messages for the keystrokes using a homeassistant embedded mosquito. meaning you could even automate the bios setup and upgrades

a very rough version just got me into the bios of a node installed in the rack
the hardest part: getting the correct keys for azerty be keyboard. quite some examples and libraries but difficult to find the right working combination. also avoiding a html5 browser to intercept F1, F11 keys didn't work on the first attempts.



edit:
1. will probably be better to go for a docker compose for easy setup
2. ctrl-c would be nice too
3. mqtt still is good enough, but probably not embedded into home assistant but inside the compose setup or on the lxc container.

 

[luckylinux]

I think I just noticed (one of ?) the Issues: the VarStoreId is NOT unique. Neither is the VarStore Guid for that Matter ...

For instance VarStoreId: 0x1 can correspond both to PcieSataSetupVar and Setup (and probably many other):

Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.16.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.0.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.13.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.5.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.11.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.7.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.14.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.2.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.27.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.22.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.10.8.en-US.ifr.txt:    VarStore Guid: FAE2664D-C098-4DB5-9CD8-9B429CF93351, VarStoreId: 0x1, Size: 0x12, Name: "PcieSataSetupVar"

Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.24.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.2.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.16.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.18.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.7.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.13.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.8.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.21.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.20.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.12.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.8.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.7.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.17.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.25.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.19.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.2.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.15.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.27.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.0.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.22.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.10.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.5.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.5.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.11.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.23.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.26.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.0.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.6.32.en-US.ifr.txt:    VarStore Guid: EC87D643-EBA4-4BB5-A1E5-3F3E36B20DA9, VarStoreId: 0x1, Size: 0x1A7, Name: "Setup"
Section_Volume_image_9E21FD93-9C72-4C15-8C4B-E77F1DB2D792_body.vol.9.14.en-US.ifr.txt:    VarStore Guid: 80E1202E-2697-4264-9CC9-80762C3E5863, VarStoreId: 0x1, Size: 0x7, Name: "Setup"

 

[VirMach]

So... one thing led to another. Providing an update for anyone interested for what's going on my end. I'm pursuing heatsinks and rails. No update from ASRock on BIOS. I did the math on the rails (doing it how I recommended to @luckylinux then realized I'm going to have to turn into a full-time metalworker so I created a CAD design and sent it off to China for bulk quotes.

If we do go that route with the rails, and anyone wants rails, I don't know how economical it will be to ship heavy rails across to Europe but I'd sell them at cost as I'm going to have to buy way more than we need to make it work. Same with heatsinks, I'm probably going to have to buy several hundred units more than I need, I'd still rather do that than buy $50 heatsinks and $40 rails.

 

[VirMach]

Not sure how many People would be interested in Europe. I might be (I already ordered those Supermicro Ear Brackets though) .

Yeah I have multiple designs, one is static one size, shouldn't be hard to also make it work with those ears and "adjust" a little optionally. If it ends up being expensive, I'll cut it up and go fully adjustable/modular and I already have those ears so it'll be made for that. Third version is just an inner rail/"shelf" combo so it'll (a) work with the Supermicro inner slider rails (these are the cheap/available part for the longer chassis) or you can just use it with those ears specifically because those are for the sets I already have like hundreds of inner rails for anyway.

You can be our European hub for them if you want to go halfsies on the order and also have way too many rails, maybe start a server rail business. They should be dirt cheap, most of the price on what you can get for universal rails is markups, I just need to find a good supplier.

Basically here in the US, the rails that end up being $35-40 are actually closer to maybe $9 plus $3 shipping ($12 a set.) That's not even with extreme volumes, with extreme volumes they go down to like... $3-4 plus $1-2 shipping. And the versions I'm making should theoretically be cheaper (not cheaper than the extreme volume, hopefully cheaper than the $12 sets, so I'm hoping for around $8-9 delivered per set.) Who knows though maybe they'll come back and say $80 a set unless I order 6 tons of them.

 

[luckylinux]

Arghh wrong file indeed.

BTW did you take a look at nvram-al.txt ?

VERY briefly. It looks like it contains also Comments etc. I'll have to do this Evening . I still need to start my work Day .

 

[VirMach]

Do you mean $9 plus Shipping for ONE Rail or ONE PAIR (SET) or Rails ?

My target is max $9, total, per pair (set.) For 200 to 500 sets~

It's technically not the lowest they could be, but at the same time the number of units I'm saying might not be worth it to most these factories so sometimes they'll just quote really high as they don't want to deal with it.

1 ton of cold rolled steel is like $1200. Which is like 4.5 cubic feet, which is the material for about 1260 rails (or 630 sets.) They have to cut it, so let's say laser cut out 1260 templates. It probably takes a few minutes each? More realistically with the type of machines they have I'm guessing closer to 15 seconds probably. Some super rough math and with their profit margins for the manufacturing, it should be about $800 for the work, or $2000 total. Except the steel I said earlier is the US prices, China is closer to $600 so $1400 total, or $2.22 per set.

Shipping is probably going to be around the same price per set, so it'll be $4.44 and then whatever they add on that is their sales profit margin. If they just double it then it should be plausible for it to cost that much. Or maybe I could be dead wrong on all my math.

If we had the $50,000+ in machinery we could definitely get it done ourselves in half a week for $2 a set. I guess by the time everyone adds in their margins and expenses they end up being 20x the cost.

 

[luckylinux]

Arghh wrong file indeed.

BTW did you take a look at nvram-al.txt ?

Now I see what you mean with "Boot" Order and USB ...

The only one that differs and Stands out would be (which you do NOT have) SystemAccess(4):0x0=0x00 but unsure what that does ...

 

[VirMach]

If I think about the original Idea you illustrated to me, that will add around 1.5mm-2mm below each Chassis, thus the 1U Alignment will be completely messed up (if not immediately, after 3-4 Servers on top of each other for Sure).

That doesn't happen because 1U chassis are 1.7 inches but "1U" is 1.75 inches (or something like that, let me look up more specifically...)

EDIT -- Yeah I don't know if they get that specific with measurements and there's also +/-% tolerances but pretty much I know for a fact that these work out fine in the real world, and if you're off by like a fraction of a millimeter or something like that, that would only end up causing a problem maybe after maybe like 20+ units since you have some play on where exactly you end up mounting the ears. You usually also don't end up just straight up filling an entire 42U/48U cabinet to the brim with just servers, I assume you might have at least on blank, one switch mounted the other direction, one patch panel, one "something" and that's where you can make up for any potential minor discrepancy.

EDIT 1: but your design is with "real" Rails (one "Inner" Steel Bar sliding alongside another "outer" Steel Bar) ? I guess to keep it cheap you'd do without Bearings, but then of course you rely on your Mechanical Tolerances and Friction Coefficient to be "good" (otherwise it would seize and you wouldn't be able to pull it out). Well nothing that a bit of WD40 or similar Lubricant can't fix I guess (although better that it does NOT drip inside the Chassis with the Motherboard !)

Yeah the inner rails for these specifically has a "fabric" on it that's meant to absorb/keep lubricant on it.

 

[luckylinux]

Here are my files.

Also lets compare our CPUs. Here is beginning of /proc/cpuinfo of mine:

vendor_id    : AuthenticAMD
cpu family    : 25
model        : 33
model name    : AMD Ryzen 7 5700X 8-Core Processor
stepping    : 2
microcode    : 0xa201210

But at least I can give you this

beginning of /proc/cpuinfo:

processor    : 0
vendor_id    : AuthenticAMD
cpu family    : 25
model        : 33
model name    : AMD Ryzen 7 5700X 8-Core Processor
stepping    : 2
microcode    : 0xa201210
cpu MHz        : 2200.000
cache size    : 512 KB
physical id    : 0
siblings    : 16
core id        : 0
cpu cores    : 8
apicid        : 0
initial apicid    : 0
fpu        : yes
fpu_exception    : yes
cpuid level    : 16
wp        : yes
flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local user_shstk clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca debug_swap
bugs        : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass srso ibpb_no_ret
bogomips    : 6787.29
TLB size    : 2560 4K pages
clflush size    : 64
cache_alignment    : 64
address sizes    : 48 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

Or lscpu if you prefer:

Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          48 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   16
  On-line CPU(s) list:    0-15
Vendor ID:                AuthenticAMD
  BIOS Vendor ID:         Advanced Micro Devices, Inc.
  Model name:             AMD Ryzen 7 5700X 8-Core Processor
    BIOS Model name:      AMD Ryzen 7 5700X 8-Core Processor              Unknown CPU @ 3.4GHz
    BIOS CPU family:      107
    CPU family:           25
    Model:                33
    Thread(s) per core:   2
    Core(s) per socket:   8
    Socket(s):            1
    Stepping:             2
    Frequency boost:      enabled
    CPU(s) scaling MHz:   56%
    CPU max MHz:          4661.7178
    CPU min MHz:          2200.0000
    BogoMIPS:             6787.29
    Flags:                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmu
                          lqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext
                          perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_t
                          otal cqm_mbm_local user_shstk clzero irperf xsaveerptr rdpru wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip pku ospke v
                          aes vpclmulqdq rdpid overflow_recov succor smca debug_swap
Virtualization features: 
  Virtualization:         AMD-V
Caches (sum of all):     
  L1d:                    256 KiB (8 instances)
  L1i:                    256 KiB (8 instances)
  L2:                     4 MiB (8 instances)
  L3:                     32 MiB (1 instance)
NUMA:                     
  NUMA node(s):           1
  NUMA node0 CPU(s):      0-15
Vulnerabilities:         
  Gather data sampling:   Not affected
  Itlb multihit:          Not affected
  L1tf:                   Not affected
  Mds:                    Not affected
  Meltdown:               Not affected
  Mmio stale data:        Not affected
  Reg file data sampling: Not affected
  Retbleed:               Not affected
  Spec rstack overflow:   Mitigation; Safe RET
  Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:             Mitigation; Retpolines; IBPB conditional; IBRS_FW; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
  Srbds:                  Not affected
  Tsx async abort:        Not affected

 

[VirMach]

So your Solution has also inner/outer Rails with Lubricant ?

Doesn't matter anymore, all the quotes came back pretty much double where they needed to be with shipping, the custom design higher.

 

[luckylinux]

And yes, found the Issue with the missing Setup Entries. I was doing a Unique entry for VarStoreId: X & VarOffset: Y but of course since VarStoreId is NOT unique then that is wrong ...

EDIT 1: 309 Entries added for Setup -> ASRock-B450D4U-V1LQ5/extract-bios/extract.nsh at main · luckylinux/ASRock-B450D4U-V1LQ5

 

[VirMach]

Damn it. So just mounting the Servers on top of each other and screw in via the Rack Ears basically ?

I learned a little bit more about how they do everything specifically, and I have some ideas on how to modify my design to simplify it in a way where I can potentially bring the cost down for the custom design. It ends up leaning toward what we talked about, so this is where I'm at right now:

If it ends up being expensive, I'll cut it up and go fully adjustable/modular and I already have those ears so it'll be made for that

The shipping for the custom one ended up being much lower than the other quote I received. It did look "too high" but I don't really have a frame of reference so I just assumed the other would be similar. So that's what makes it potentially viable. I believe it's because the premade ones are also prepackaged individually for resale, and the custom ones aren't so maybe it saves a lot of volume (still heavy.)

I really do still want to avoid cutting it up too much, or trying to be "smart" and introducing some structural design flaw into it that makes them all useless.

@VirMach: a simple Solution I thought of, but can turn out ugly as ****, would be to get ideally these

https://www.penn-elcom.com/dk/42u-rack-strip-with-square-holes-2mm-thick-r0863-2mm-42

Otherwise on the Cheap some say 75mm x 45mm Pieces of Wood and place these in the space between the 2 Choices for the Rack Ear Holes Pattern (M6 or some Imperial Screw Size) on the SAMSON SRK21 (or just after the Hole Pattern, in other Cases where you don't have such Obstacle):
View attachment 43730

Pre-drill 3 Holes per 1U as usual in the Wood and Optionally add a Wood Insert, so you can attach the L-profile "Rails bottom" using a Metal Threaded Screw (instead of a Wood self-threading Screw).

Otherwise take a cheap Rack Strip such as one of these:
View attachment 43732

Thomann - Hjertelig velkommen!

Søgeresultater - side 1 - "Rack Strip"

www.thomann.dk

e.g.

Adam Hall 61535 BLKRack Strip 45U

Rack Rail Punched steel rack rail with three holes per U, 9.5 mm hole size (according to IEC 60297), Powder-coated steel, Material thickness 2 mm, Weight 1.45 kg, Length: 45 U, Colour: Black

www.thomann.dk

And mount that at 90° with a spacer (e.g. Wood) in the same Place.

Then put a L-Profile "Rails Bottom" like this (challenge is to get it thin enough, 3mm is standard, 2mm is a bit more difficult to source and get it shipped to Denmark for a reasonable Price):

Winkelstahl Feuerverzinkt 30x30x3mm 1000mm

Material:Stahl feuerverzinkt Form:Winkelstahl gleichschenklig Werkstoff:S235JR Oberfläche:innen und außen verzinkt Herstellungsverfahren:gewalzt Norm:verzinkt nach EN ISO 1461 Außenabmessung:30 mm x 30 mm x 3 mm Länge:1000 mm Ge…

www.stahl-shop24.de

View attachment 43731

But if you go full-custom and don't have the Rack already, maybe it's just easier to get the Rack Strip and the Steel Profiles, then Screw everything in some IKEA-like Furniture Cabinet or Cupboard. You might at that point not even care about the Rack Strip Holes Pattern for the Rack Ears mounting as you can oversize slightly anyway (e.g. do a "Shelf" every 50mm instead of 1U).

Okay that took a while but I finally pictured all this in my head. Just one thing for our case specifically though, these are ending up in a datacenter facility, so I think they would be pretty upset once they see me bringing in Ikea furniture and wood into their multi-million dollar facility.

And once I'm done explaining to them it's to save a few dollars per server, I don't think I'll ever be allowed back in.

I've done (or tried) what you said in the past, several years ago. I really hated it. A nightmare in many ways once you start using it but maybe I could have done it better (not with wood, etc, by the way, the posts you mentioned, 90 degree, etc.) This was for my personal cabinet at home. Then in the end you still have to basically buy a bunch of stuff so it doesn't really save money. It seems like every solution ends up adding up to the same thing.

By the way, I just checked the price of those steel slotted things I told you to purchase as part of the cheap solution, maybe it is in the US, but I did not realize how much more expensive they have gotten. Crazy. So I think at least for me, that solution no longer makes sense either.


EDIT -- I will say though perhaps one solution for you specifically, if you have all these in the same cabinet, is to essentially do what you said but skip any actual rails, just bring one set of posts closer, use your rack ears, mount it back and front, all 4 sides each server, they won't go anywhere. As in, attach the ears directly to the server at that point and every time you have to take a server out and have to undo 8 screws, just remember of all the money you saved. Plus you wanted to screw them in anyway, correct (so it doesn't slide around.) You're going to be making a lot of holes anyway, just make a couple in the chassis before you put everything in those CSE512's as needed.

EDIT2 -- Oh right you don't even need 4 of them just two, in the back only. It already has front ears.

EDIT3 -- You don't even need to make a hole I don't think, it lines up, but it doesn't "clasp" it, I just had a look. Depending on tolerances should work.

 

[luckylinux]

I am still getting quite a bit of Errors but apparently a lot of Setup entries (299) have been exported now ...

Maybe you can see something more now @ALinden

 

[nilfisk_urd]

But if you go full-custom and don't have the Rack already, maybe it's just easier to get the Rack Strip and the Steel Profiles, then Screw everything in some IKEA-like Furniture Cabinet or Cupboard. You might at that point not even care about the Rack Strip Holes Pattern for the Rack Ears mounting as you can oversize slightly anyway (e.g. do a "Shelf" every 50mm instead of 1U).

The IKEA "LACK" table fits 19" equiipment and is quite cheap

LackRack - Eth0Wiki

wiki.eth0.nl

 

[b3rrytech]

Haven't catched up with reading the whole thread, neither did I find anything while searching it.

Have anyone managed to get fan pwm to work in any OS? It registers RPM and seems to work ok with "Smart Control" in BIOS. Haven't found a driver that works for fan readout in the OS though.

System is running Ryzen 7 Pro 3700.


Ran sensors-detect and the recommended driver is NCT6775.

Now follows a summary of the probes I have just done.
Just press ENTER to continue:

Driver `k10temp' (autoloaded):
  * Chip `AMD Family 17h thermal sensors' (confidence: 9)

Driver `nct6775':
  * ISA bus, address 0x290
    Chip `Nuvoton NCT5532D/NCT6779D Super IO Sensors' (confidence: 9)

Ran modprobe nct6775 and at least I got some more entities when running sensors

sensors
k10temp-pci-00c3
Adapter: PCI adapter
Tctl:         +28.0°C 
Tccd1:        +25.8°C 

nct6779-isa-0290
Adapter: ISA adapter
Vcore:                  96.00 mV (min =  +0.00 V, max =  +1.74 V)
in1:                   224.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
AVCC:                    3.33 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
+3.3V:                   3.33 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
in4:                   384.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
in5:                   216.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
in6:                   240.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
3VSB:                    3.39 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
Vbat:                    3.20 V  (min =  +0.00 V, max =  +0.00 V)  ALARM
in9:                     0.00 V  (min =  +0.00 V, max =  +0.00 V)
in10:                  944.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
in11:                  304.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
in12:                  224.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
in13:                  200.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
in14:                  248.00 mV (min =  +0.00 V, max =  +0.00 V)  ALARM
fan1:                     0 RPM  (min =    0 RPM)
fan2:                     0 RPM  (min =    0 RPM)
fan3:                     0 RPM  (min =    0 RPM)
fan5:                     0 RPM  (min =    0 RPM)
SYSTIN:                 +27.0°C  (high =  +0.0°C, hyst =  +0.0°C)  ALARM  sensor = thermistor
CPUTIN:                 +27.5°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor
AUXTIN0:                +92.0°C    sensor = thermistor
AUXTIN1:                +29.0°C    sensor = thermistor
AUXTIN2:                +98.0°C    sensor = thermistor
AUXTIN3:               +102.0°C    sensor = thermistor
PCH_CHIP_CPU_MAX_TEMP:   +0.0°C 
PCH_CHIP_TEMP:           +0.0°C 
PCH_CPU_TEMP:            +0.0°C 
PCH_MCH_TEMP:            +0.0°C 
PCH_DIM0_TEMP:           +0.0°C 
PCH_DIM1_TEMP:           +0.0°C 
TSI0_TEMP:              +28.0°C 
intrusion0:            ALARM
intrusion1:            ALARM
beep_enable:           disabled

Ran pwmconfig again but it stops since it can't find any working fan sensors.

Found the following devices:
   hwmon0 is k10temp
   hwmon1 is nct6779

Found the following PWM controls:
   hwmon1/pwm1           current value: 255
   hwmon1/pwm2           current value: 255
   hwmon1/pwm3           current value: 255
   hwmon1/pwm4           current value: 255
   hwmon1/pwm5           current value: 255

Giving the fans some time to reach full speed...
Found the following fan sensors:
   hwmon1/fan1_input     current speed: 0 ... skipping!
   hwmon1/fan2_input     current speed: 0 ... skipping!
   hwmon1/fan3_input     current speed: 0 ... skipping!
   hwmon1/fan5_input     current speed: 0 ... skipping!

There are no working fan sensors, all readings are 0.

I know some of these drivers are more or less a clusterf*ck due to manufacturers not caring, but I'll keep looking. I'll gladly let BIOS control the CPU fan but the case fans I'd rather control myself.