EU ASRock Rack B450D4U-V1LQ5 mATX AM4

[luckylinux]

@Cruzader: any Progress on your Side ? This lack of ECC is quite a big Issue IMHO ...

 

[dbram]

Riser / 1xPCIe only

it depends on how and what you mount, I'm trying to squeeze 2 nvme's and a 10Gb nic on a x4x4x8 riser and an nvme ssd to boot in the second pcie slot...

Centrifugal Fan on those CSE-512

that's why I removed them

here is a first draft io shield for a 1U supermicro case
b450d4u-v1lq5 supermicro CSE-512 1u

 

[Cruzader]

Curious how you do that. The Riser that has been mentioned before (2x NVMe + x8 Slot) turned a Low-Profile PCIe Adapter from Aliexpress into a Solution where you plugged the NIC "on top". But then you would need a FULL High PCIe Bracket Case (4u, possibly 3u), definitively NOT 1U !

So I don't think you did that on the CSE-512.

a 90degree angle riser from slot and install it like you would any other card in a 1u.

Since i happend to be near my workbench atm, something like this.

 

[luckylinux]

most options do. but if ECC is not wanted or supported by code its checked at every POST and set according to rules.

But I would NOT expect the BIOS to reset a setting on its own just like that.

To Ignore it I can understand. But to reset it automatically. Well. Then there is no way out :/.

I have ZERO experience in Building custom BIOS ROMs, that's why setup_var.efi seemed a very good Option.

I'm also NOT sure that when disassemblying with uefitool and ifrextractor it's 100% like it should be (i.e. you can disassemble, modify a bit, then reassemble and it would just work). I fear it's MUCH more complicated than that ...

 

[nilfisk_urd]

1U cases have a full height pcie slot that is rotated 90 degrees

 

[Cruzader]

And are you doing 2x100gbps there ? That looks more of a Mellanox ConnectX-4 QSFP28 than a Mellanox ConnectX-4 LX SFP28 ...

Should be a 2x100g connectx4 or connectx5, got a bit of each on the bench and borrowed from closest stack.
(Dont really have a plan for them, impulse buys at 20-25$/ea pretty much.)

But for the ryzens its 2x25gbe connectx4 lx im normally using.

 

[dbram]

Wasn't there one already available you said ? Did you have to customize it ?

Not required to customize, the x470 is similar but would leave some open space. Serial port extra nic, ipmi nic. Our b450 has far less connectors populated. It’s also cleaner like this.
I wanted a design to work on anyhow to add a holder or something to fit an esp32 on the free space, with the c3 wifi antenna sticking out

 

[luckylinux]

look at the set of numbers. you can find them with a hex editor in the sct file. you could move a ref. out of the suppress If e.g.

0x1C593         Suppress If: {0A 82}
0x1C595             Variable 0x9 equals 0x2 {12 86 09 00 02 00}
0x1C59B                 Not {17 02}
0x1C59D                 Variable 0x9 equals 0x3 {12 06 09 00 03 00}
0x1C5A3                 Not {17 02}
0x1C5A5                 And {15 02}
0x1C5A7                 Variable 0x9 equals 0x4 {12 06 09 00 04 00}
0x1C5AD                 Not {17 02}
0x1C5AF                 And {15 02}
0x1C5B1             End {29 02}
0x1C5B3             Ref: NTB Common Options, Variable: 0xFFFF {0F 0F 0A 00 0A 00 06 00 00 00 FF FF 00 06 70}
0x1C5C2         End If {29 02}

Uhm but again how would that help with the hardcoded Hardware based Checks ? isn't that just another way of solving the QuestionId Issue like I just tried with setup_var.efi, but still potentially bypassed by the Hardware Check ?

And how would you rebuild the BIOS after modification ?

And Flashing must be done via a SOP/SOIP Clip, I don't think the UEFI Easy Flash tool would work with a custom Image (I guess their Images are signed) ....

 

[dbram]

all steam ahead with the ESP32 for PiKVM

yes, exactly, the esp32 does usb hid for keyboard and mouse and the gpio's do the atx power/reset control.

granted i manage quite well with wake on lan and autoboot on power-on for the moment

 

[luckylinux]

Got the first set of compute nodes moved onto these boards, they are working as expected and have not really looked much more at them beyond that.

So is ECC working for you for whatever Reason ? Or you did NOT check (aka "it just works") ?

 

[VirMach]

As in buying a whole AMD Ryzen AM4 Server ? That wasn't an Option for these Motherboards unfortunately.

Or did you mean buy an Entire Server (1U/2U), sell ALL components except Chassis and PSU, then reuse those Components (Chassis and PSU) that I keep ? In this case there aren't many Short Depth 2U Chassis and 1U is loud as Hell ...

Once in a while, like with these boards, a company will get rid of entire servers and the work to part it out isn't worth it them. When that happens, they end up selling for about the same cost as acquiring just the chassis or slightly above it, and you end up usually also getting the power supply, fans, brackets, motherboard, and some other parts left inside.

Then you can usually make it work out for whatever you're trying to do with a drill and a 3D printer.

1U is ***** loud though .

Heck even 2U with SATA/SAS Drives is quite loud in Summer.

It's theoretically possible to get 1U to be quiet, but not financially feasible. I did it once with 1U + Ryzen... 3700X? 3800X? Don't remember the exact one but somewhere around there. Will cost around $200-300 to do. But yeah can achieve the same results doing 2U. Nothing crazy just specific expensive fans and such and works better with a longer depth chassis.

Indeed a cheap one, but only 1 for every 4 b450 boards with a hdmi 4-1 switch or a vga 4-1, and put the vga to hdmi adapter before or after the 4-1 switch. Only drawback is that you can only view one of them at a time…

I think I finally resolved my KVM dilemma. I guess for this to be helpful, there's a lot of use case questions that need to be asked, such as whether or not you need to view more than one, how many you have, and what compromises are acceptable.

I'll test it out once I receive it all but I found the right combination of model numbers to get it all possibly working as low as $3.80 per server not included taxes/shipping (a little bit higher for lower quantities.) The tradeoffs are that you lose (1) HTML5, (2) precise mouse syncing which shouldn't matter, (3) any 2FA/smart card features, (4) virtual media support, this might be a big one, (5) audio support, (6) any chaining, which technically shouldn't be required unless you go past... 64 of them. You do get (A) mouse and keyboard input, (B) over IP although kind of janky.

All of this assumes I'm not wrong and it actually works. If anyone's interested, then this would work best for 16, 32, or 64 systems -- and I don't know how much it would be in Europe.

The pricing is honestly not bad considering it probably costs the same as just a VGA and USB cable.

 

[dbram]

. Will cost around $200-300 to do.

prices do fluctuate. i got these cse512 cases including 1u power supply for 17.99,- eur /each back in 2017, some of them had 1U angle risers (pcie 3). in fact I only bought the cases with a X9SCL-F to avoid damage of the mainboard in shipping...
so if I sumup a 1u case 3700x compute 32gb ddr4 node small nvme boot ssd, i'm between 100,- eur and 150,- nothing fancy for networking, no big storage... I admit had most of the 'parts' lying around unused.

 

[luckylinux]

I'm kinda tempted to buy 3700X instead going forward and see if those work better. Sucks but no ECC working sucks too . This is turning into a huge side Project.

 

[VirMach]

So why did you buy only the Board yourself then ?

I meant with junk boards, 10-15 years old. Not the boards you want. The math goes something like this...

PSU - $15-30
Board - $20-100
Chassis - $20-80
Fans - $5-10
Heatsink - $5-20
Trays - $3-10
Connectors - $10-30

And let's say you want the chassis, that's $20-80 in this rough example. Or, sometimes you'll have the option to get the PSU + Board (Random) + Chassis + Fans + Heatsink + Trays + Connectors for $30 to $90. It can just work out to your benefit.

And to answer your question, I bought only the boards because I bought 200-300 of what I'm suggesting here for like $20 each a couple years ago.

Well, I didn't quite understand WHAT the Solution would be. Was it in a previous Post or ?

I don't want to waste everyone's time so let me verify it first. Don't worry if it doesn't work I'll also let you guys know (and why I was dumb.)

 

[luckylinux]

So, some updates from my Side. And I would appreciate if somebody had an Idea about how to save Output of UEFI Shell Scripts (*.nsh Files) to a Variable. I tried redirecting to File then using dmpstor -l <file_name> <var_name> but that doesn't work at all, maybe it's only intended for Files that have been saved by dmpstor and/or UEFI Environment Variables that are accessible later on by the OS. Unsure .

IMPORTANT #1: UEFI Shell Scripts (*.nsh Files) MUST be saved in UTF-8 Encoding

IMPORTANT #2: setup_var.efi Configuration Files as outlined in GitHub - datasone/setup_var.efi: UEFI command-line tool for read/write access of variables, MUST be saved in UTF-16LE Encoding (if you use those, I prefer a single Script to be honest though)

POSSIBLE ISSUE #3: I removed the Several ########## in Comments and added a space after the First # as in # My Long Comment). I'm not sure if this was (part of the Issue), but between the Encoding that was set to UTF-16LE and these Comments, the UEFI Shell Script just didn't run AT ALL

SSP\patch.nsh

# Tested with BIOS Version L2.09 on ASROCK B450D4U-V1LQ5
# Tuned based on File_DXE_driver_CbsSetupDxeSSP_CbsSetupDxeSSP_body.fbd.0.0.en-US.ifr.txt

# Disable printing of Commands
@echo -on

# Enable Overclock & Accept Risk
setup_var.efi AmdSetup(0x5000):0xD4=0x01

# Enable Combo CBS
setup_var.efi AmdSetup(0x5000):0x20=0xFF

# Enable ECC Memory
setup_var.efi AmdSetup(0x5000):0x11B=0x01

SSP\read.nsh (this is where I'd like some help to save output of setup_var.efi Command to Variable, see commented Examples with > and dmpstor to see what I tried and failed):

# Tested with BIOS Version L2.09 on ASROCK B450D4U-V1LQ5
# Tuned based on File_DXE_driver_CbsSetupDxeSSP_CbsSetupDxeSSP_body.fbd.0.0.en-US.ifr.txt

# Enable printing of Commands
@echo -on

# Enable Overclock & Accept Risk
# setup_var.efi AmdSetup(0x5000):0xD4=0x01
# setup_var.efi AmdSetup(0x5000):0xD4 > result
# dmpstor -l result enable_overclock
# echo Enable Overclock: AmdSetup(0x5000):0xD4=%enable_overclock%
echo Enable Overclock Setting:
setup_var.efi AmdSetup(0x5000):0xD4
echo Expected AmdSetup(0x5000):0xD4=0x01
echo ...................................................................

# Enable Combo CBS
# setup_var.efi AmdSetup(0x5000):0x20=0xFF
# setup_var.efi AmdSetup(0x5000):0x20 > result
# dmpstor -l result enable_combo_cbs
# echo Enable Combo CBS: AmdSetup(0x5000):0x20=%enable_combo_cbs%
echo Enable Combo CBS Setting:
setup_var.efi AmdSetup(0x5000):0x20
echo Expected AmdSetup(0x5000):0x20=0xFF
echo ...................................................................

# Enable ECC Memory
# setup_var.efi AmdSetup(0x5000):0x11B=0x01
# setup_var.efi AmdSetup(0x5000):0x11B > result
# dmpstor -l result enable_ecc_memory
# echo Enable Overclock: AmdSetup(0x5000):0x11B=%enable_ecc_memory%
echo Enable ECC Memory Setting:
setup_var.efi AmdSetup(0x5000):0x11B
echo Expected AmdSetup(0x5000):0x11B=0x01
echo ...................................................................

@RolloZ170: the good Thing, if there is one that is, is that the Settings seem to survive after a Reboot, at least a very fast one (no unplug Power Cord, Shutdown completely, etc):
1. Boot into UEFI Shell
2. Run patch.sh
3. Reboot with reset Command
4. Boot into UEFI Shell
5. Run read.sh



EDIT 1:
But no extra Menu Visible in BIOS .

I'll try to Boot Ubuntu and if not try RV, RN and ZP instead, since SSP didn't work.

EDIT 2: SSP Patch did NOT work

EDIT 2: RV Patch did NOT work

EDIT 3: RN Patch did NOT work

EDIT 4: ZP Patch did NOT work


So I think it's a no go after all

EDIT 5: To be Clear, there is NO BIOS MENU unlocked, and dmidecode and dmesg show the same Output as before. So, the RAM is running without ECC being Enabled .

 

[VirMach]

Another Option could be to get a proper KVM switch (4 Port or 8 Port, for HDMI/VGA and USB Keyboard/Mouse) and then use the standard PiKVM there, instead of Just the VGA/HDMI Switch. Price is NOT that much higher at least for 4 Ports on Aliexpress.

https://www.aliexpress.com/item/1005005244233383.html

"Only" Modification would be the soldering of the Change Switch/Button, some GPIO Configuration and some Interface on PiKVM to pre-select the Target Device. And that Switch is a bit of a PITA since you can only Cycle using the same Button.

Much more expensive Solutions for 8 or 10 Ports might also have some Remotre / RS232 Feature, but I think at that Point you better do it yourself anyways.

That was my backup option versus actually doing the proper PiKVM/PiKVM Switch Multiport. It's great in a lot of ways, but also iffy in others. Basically, it could theoretically be much better for us, but would take a long time to do properly and without getting deep into it, I wouldn't even know if it would be better or save us money. Endless possibilities, which means more potential for it to be awesome and exactly what we need, or for things to go wrong and at the very least drag us into many more hours of work....

I guess I'll just go ahead and let you know what I'm doing, you've pretty much went down the same rabbit hole. But this is for @luckylinux's eyes only, I'd feel really awful if anyone else tried this out and it became a headache for them.

Raritan Dominion KX2. It's the only KVM switch I could find that has a "proper" control center AND isn't insanely expensive (anymore, it's old) AND can be used remotely (not all of them can at this pricepoint) AND has enough ports for us AND doesn't have some weird required insanely priced license to actually use the product AND is new enough to where the dongles actually use USB (yes, I'm comparing it to models so old they're PS/2.)

Only problem? The "official" dongles for it (DC2CIM-xxx) can also be used for the next version/model and are quite pricey ($15-30 each.)

Well okay, not the only problem, but the one that makes it not attractive on pricing. So I started looking at all dongles that were cheap, then going back and matching them to the KVM switches, and figuring out why it wouldn't work. I'd always end back up at the KX2. If only there was a way to use a dongle not meant for it with it...

The information on all this is pretty lackluster, but I eventually figured out one of the dongles meant for the Raritan Paragon II should work with the KX2. The Paragon II is also a cheaper option if you don't need the specific features on the KX2. The dongle for the Paragon II is P2CIM-USB or P2CIM-AUSB-C (this is the specific newer sub-version I'm looking at using.)

Some potential issues on top of the features you lose is weird compatibility problems in specific scenarios, lag, having it potentially drop off. These are theoretical until I get my hands on it. No official support. Depends on specific firmware version of the KX2, resolution/refresh rates, certain specific models and ports on the KX2 may have issues (there's different number of port sub-models for the KX2.)

Cost?

• $100-160 for the KX2-464 or KX2-864 (64 Ports, I bought mine $100~ plus $20~ shipping for the KX2-864.)
• Other options exist with lower port counts but essentially drives up cost per port. KX2-108, 116, 132, 216, 232, 416, 432, 832
• $2-3 per dongle for the P2CIM-AUSB-C (I bought mine $2.50~ each shipped, versus $1.92 + $0.70 shipping other option.)

Optionally you can go with the DCIM-USB/USBG2 which has similar limitations as the cheaper one except with better/more "official" support. You can get these for maybe $9-12 each. Or of course the actual dongles for it, which are $15-30 each. One important thing this also gives you is virtual media support.

And the proper one to get is technically the KX3, which is newer, better in every way like with improved simultaneous user support, HTML5, and so on. However this also means they cost $500-800+

In the end, there's a reason they're so cheap, and I feel like I'm about to find out about many of those reasons (that I missed) very soon.

In conclusion, I just looked it up for EU and I don't think this specific one will work for you given the prices I'm seeing pop up.

 

[luckylinux]

@dbram: is ECC Memory working with your 2000 and 3000 Series CPUs ?

I think 5000 Series CPUs is out of Luck concerning ECC Support. I tried everything I could think of short of Reverse Engineering the X86 Part of the BIOS as @RolloZ170 mentioned (but I don't think I have the Skills to do that TBH).

Can you see if your ECC is actually working ?

At any Time using dmidecode:

dmidecode | grep -i "Error correction"

If it does NOT work it will report as follows, with one Line having NO ECC Correction (related to the RAM, the L1/L2/L3 Caches always have ECC working):

    Error Correction Type: None
    Error Correction Type: Multi-bit ECC
    Error Correction Type: Multi-bit ECC
    Error Correction Type: Multi-bit ECC

After a fresh Boot you could try grepping dmesg:

dmesg | grep -i edac

If it ONLY reports this, then ECC is not Enabled:

[    0.650269] EDAC MC: Ver: 3.0.0

Thank you

 

[dbram]

Aren’t we forgetting the nanokvm (lite) ? Put them in a fenced vlan or so…

 

[luckylinux]

Unless you @RolloZ170 have other Ideas. I don't think I can manage on my own to disassemble the X86 Hardcoded part of the BIOS ...

And if asking ASRock Rack is not an Option according to what some User said on Reddit (since this is effectively a Hetzner Motherboard), then I really think we are out of Luck.

I guess now we know why they were so eager to get rid of them. They are e-Waste. If the AMD Ryzen 3700X works with ECC working then it might be OK, but with these 2 x 5700X and ECC being disabled feels like a waste of Money .

 

[VirMach]

Something more realistic would be IMHO at around 140 EUR + 20 EUR Shipping, but unsure if the KX2-132 1x1x32 USB - DKX2-132 would have the same Features

My understanding of the differences between the models, outside of the number of ports, is that there are limitations on the number of users allowed to log in at once, and some vague real-world limitation on how much it's capable of doing simultaneously in terms of using the console based on the underlying hardware. So more RAM, etc. As well as single/dual PSU and single/dual network.

Cheapest P2CIM-AUSB-C I found is 20 EUR each on eBay. Where did you find yours ?

Also eBay, but sellers in the US, not abroad. One of them has high quantities and happens to be a few states away. Sometimes eBay doesn't do a great job at representing the actual cost when it's for multiple units based on how they set up shipping, so I do know one of them will deliver to Europe (I picked France) and without any customs/duties, just the cost of the items and shipping, it looks like it would be about $4.70 each at 24 total quantity.

I'll test them out and I might end up with extras as I'm working out a deal with one of the sellers (I'm getting more to cover all 88 boards even though I only have 64 ports on the first device right now, and still haven't even tested it.) If you're interested, they do end up working, and you don't find a better option by then, I wouldn't be opposed to figuring out how much it'd cost to send them to you and sell any extras at-cost.

Otherwise maybe I'd try to use some Unofficial Radxa Zero 3E / Radxa 3C SBC as a ~ half Price Alternative to the Raspberry Pi 4 (GPU/Codec Support kinda sucks but not sure that's relevant if I use the cheap HDMI 2 USB Capture Device, but at least there is some Mainline Kernel Support and it's ~ 19 EUR for the Radxa Zero 3E or 23 EUR for the Radxa 3C, Radxa 2A would be even better at 17 EUR with normal USB instead of USB Type C of the Radxa Zero 3E, but almost no Mainline Support right now ).

What's your actual use case/minimum requirements? I've seen other options along the way that I've just thrown out because it didn't work for us, maybe it would make sense for you.

If only the Raspberry Pi 3 would also work, then there would be quite a few better Deals for those, but then I'd need the Raspberry Pi Pico to emulate Mouse & Keyboard and that adds quite a bit of Wiring & Components to the List.

I'm only saying this after your edit about custom PCB and talking about putting them in a 1U enclosure.

There's apparently also a custom build of PiKVM that runs on x86. And theoretically... network switches do sure look similar to what we need for the other part of it. You could start a multi-year project on slapping those together. Of course you still need the OTG/capture, anything else, and this wouldn't save any money unless you get investors, buy everything in bulk, and then take it to market to compete with these other $7,000+ brand new multi-port KVM devices.

 

[Cruzader]

20-25 USD each ????

Yeah i offered on a bunch of lots like these they had asking prices in the 200-400 area, but i offered 20$ per x16 without bracket, 25$ per x16 with bracket and 15$ per other card (connectx3 2x qsfp and connectx4 lx 2x sfp28).
Biggest "discount" was 80$ accepted on a 300 something listing.

Tho i think the offer/accept im the most happy with sofar this year is this one.
23$ accepted for 8x 16gb lot, so a nice 253$ for 88x16gb (1408gb) before shipping/import.

Would that be a Supermicro RSC-RR1U-E16 Riser ? And are you also using the Supermicro CSE-512 ?

I belive that was the riser, im not using CSE-512 but have a box of angled risers.
A seller ive bought supermicro boards from regularly just leaves them sitting on the board when pulling them from 1u cases it seems.

 

[luckylinux]

here's mine :
Model name: AMD Ryzen 7 3700X 8-Core Processor

# dmidecode | grep -i "Error correction"
        Error Correction Type: Multi-bit ECC
        Error Correction Type: Multi-bit ECC
        Error Correction Type: Multi-bit ECC
        Error Correction Type: Multi-bit ECC

# dmesg | grep -i edac
[    0.687160] EDAC MC: Ver: 3.0.0
[    4.203283] EDAC MC0: Giving out device to module amd64_edac controller F17h_M70h: DEV 0000:00:18.3 (INTERRUPT)
[    4.203288] EDAC amd64: F17h_M70h detected (node 0).
[    4.203293] EDAC MC: UMC0 chip selects:
[    4.203294] EDAC amd64: MC: 0:     0MB 1:     0MB
[    4.203297] EDAC amd64: MC: 2:  8192MB 3:  8192MB
[    4.203302] EDAC MC: UMC1 chip selects:
[    4.203303] EDAC amd64: MC: 0:     0MB 1:     0MB
[    4.203306] EDAC amd64: MC: 2:  8192MB 3:  8192MB

Thanks .

That looks more or less like my ASUS with the 5950X. So yeah, ECC is working with the 3700X.

I guess I'll have to order that CPU going forward and try to get rid of these 5700X that won't work with ECC Memory on this Particular Board at least .

 

[dbram]

@dbram : do you think your solution with the I/O Shield would also work with the Supermicro CSE-512L ? I found some very good Deals on those (not sure if they ship to Denmark though) but not sure if the I/O Shield is replaceable and if there is enough room for everything (Motherboard, Fans, ...) overall since the Chassis is significantly shorter Depth.

could work, depends how the default io shield looks like looks like.
i think so since the original io shield design was for X470 in a supermicro.

as long as the original shield from the case has this 1u shield format size (don't mind the connectors on this supermicro X9SCL-F shield).


according to this ebay listing they're similar Supermicro Server Chassis IO Shield SC116 SC512L-200 SC512-200B SC512L-260 SC815 | eBay
but of course no warranty from the designers side

 

[dbram]

Another small update:

vga to hdmi cable, these are very picky, especially wen entering the bios. got a very cheap vga to hdmi from aliexpress, where only the boot logo for the NIC's PXE is shown, nothing else, no bios, no boot, no post, nothing. according to a video from Techno Tim, who tried something similar on a couple of Supermicro boards, the adapter cables are a bit shady, some work some don't. I ordered one of his recommendations 'Benfei vga to hdmi' and will test with these.

The benfei vga to hdmi adapter arrived.

what works right now:
using a laptop and cheapish usb3 hdmi capture stick using OBS on Windows 11 I can see the bios screen, F11 brings the boot selection menu, and I can see the proxmox console now. that should be enough for my use-cases for the time being.

Still have to try the Raspberry PI / PiKVM route...
ISO mounting is not really a prio right now, using netboot.xyz with local mirrored images for netbooting the installers
downside is, now I really need to get up and running with the esp32-s3


Maybe a brainfarth but a potential lowcost approach would be:
for the hdmi capture, csi2 vs usb3 hdmi stick, the usb hdmi seems cheaper. especially since it could be setup using one of the B450 nodes to act as the remote controller node

if you own e.g; 12+1 nodes you can use :

• 12 keyboard hid simulators and atx controls (pi pico, esp32-s3, leonardo, ... ) , still need a pcb for the hardwired atx controls
• 3 usb hdmi sticks,
• 3 benfei vga to hdmi adapters,
• 3x vga 4 to 1 switches,
• a ryzen 3100 cpu and small memory modules or so in that kvm controller node is more then enough, the encoding is done in the hdmi sticks chips (if I'm not mistaken).

 

[VirMach]

Jokes aside, I'd say at the very least:
- Be able to access BIOS
- Be able to Issue remote reboots/start/shut-down (need ATX Power Control Board).
- Preferably (at least partially, e.g. Groups of 2/4/8) independent Systems (in your Case if the main Unit gets broken, you lose access to EVERY one of your 88 Devices)
- Target Devices: 24 Devices or Higher (can be 8x4, 4x8, 3x12, 2x12, etc Configuration)
- Preferably HTML5
- Single user, Homelab, nothing too fancy

There's no good solution for your requirements, the closest one seems to be the one you're already exploring which is PiKVM mixed in with whatever else you need. There's a MergePoint KVM switch that's fairly cheap and supports HTML5, but won't do your power controls and the dongles on those are crazy high priced. Only works for single user, for multiple you need an expensive license, which is why it was ruled out for us.

The one I mentioned using, there's technically a way to not use Java but only from Windows (they have some other program.) I do not know for certain if it uses HTML5 or not for that, just that it's not Java so I'd assume it's probably HTML5.

I don't know what you're doing for power on these but if you have a proper PDU set up, then that should end up getting you your power cycling with some type of API/control panel which you should be able to integrate with the KVM switch in most cases, and it should end up costing less per server with a PDU f you get the right one used (at least in the US, we can get 30-48 port PDUs, switched, compatible with our setup, for $100-200.)


----

Not sure what was discussed for I/O shields but if anyone wants an STL file to 3D print some for the SuperMicro Chassis (I think they use the same screws across SC512 and others I have) let me know, not that it's very difficult to just make one yourself. There's also a spray you can get to simulate any benefits you'd be getting in terms of actual shielding versus using metal ones, just make sure to wear proper N95 or whatever when spraying it. Please note, huge disclaimer, I'm not a professional on shielding, I could be totally wrong but the spray is called Total Ground Carbon Conductive Paint, 12 Oz Aerosol - MG Chemicals

You guys could also definitely use the shields they sent and flatten them and cut it to work in the 1U format but I found that method to take longer than it's worth even though I have all the equipment to do it.

I haven't actually found your solution for it so I have no idea if it's way more sensible than mine or not.

 

[luckylinux]

another long shot:

• did you use the latest microcode updates ?
• are you using bios 2.12 ?

it's a step i took when installing proxmox nodes, apply microcode updates, one of the proxmox community scripts, you never know

No, NOT the latest BIOS, I'm on L2.09 (as the Board came with), but as @ALinden reported, it doesn't seem to Matter (even after applying the patching with setup_var.efi, in both cases (for him)).

One Reddit User got stuck after trying to update so I'm not sure if it's a good Idea to update. Like ... should we even try ?

About microcode, that is blacklisted / blocked by Default on Ubuntu (Desktop and the LiveUSB I tried as well). I seem to recall Proxmox blocks microcode Updates by default as well. But even that is too late compared to when ECC Initialization must occur (during POST, BIOS must clear all ECC Memory, for it to be "valid", not sure that's the best Terminology to explain it though).

Any by the Way, I think somebody Mentioned that latest BIOS supported only 5000 Series CPUs. Or it also included 3000 Series CPUs ?

 

[VirMach]

This is the STL that I'm using, I left a little slot on top where you can insert a CPU shroud into it should you want to also print one of those, I can give you guys a version without our logo on it.

You can print 4 of them at a time on something like a Bambulab A1 mini 0.4 nozzle at 0.16mm Optimal, in 1 hour 16 minutes.

Versus 2 hours 17 minutes for the one you guys seemed to have found, plus that one needs some work. I actually used that one several years ago, it has some issues getting it in and getting the screws in, the bottom three "clips" kind of get stuck. And I think the VGA port for it was too tight where it'd be more difficult to kind of align it properly in all cases but I might be remembering wrong.

I have a bunch of different versions if you don't want hexagons, logo, or the hole at the top just let me know, this is just the version that prints faster.

EDIT -- the screw holes are definitely the right size and position for SC512/SC813 though, confirmed, same with 3 clips at bottom (aligns properly.)

 

[dbram]

it has some issues getting it in and getting the screws in, the bottom three "clips" kind of get stuck. And I think the VGA port for it was too tight where it'd be more difficult to kind of align it properly in all cases but I might be remembering wrong.

Ah yes, that's exactly why I created a new version for it
B450d4u 3mf

and this is the changelog of yesterday

• no holes yet for screws. : fixed in v1.5
• the connectors at the bottom need to be slimmer, fixed in v1.5
• the margin around the vga connector needs to be increased fixed in v1.5
• add opening for the top case cover clip : fixed in v1.5

Welcome to contribute editing further on the onshape b450 iohsield project


FYI: this prints in PETG in under 24 minutes on a K1-max

 

[luckylinux]

not sure this is e-waste : good enough for me

For the 5700X it's kind of e-waste Really, if I cannot get ECC to work. I might use them as some Distcc building Server at best, but I wouldn't really trust them ...

For the 3700X I agree, it's still a very good Deal. Now that I have like 60 DDR4 UDIMMs with ECC I'm considering upgrading the entire Homelab.

But I still cannot see how either this ASRock board or the Fujitsu TX1320 M3 I was upgrading to is really better in all aspects compared to the "Venerable" Supermicro X10SLM-F/X10SLL-F where at least there is a Decent PCIe Slot connected to the PCH/DMI/Chipset (thus I can use my Mellanox ConnectX-4 NIC with ASPM working correctly, NOT so in any CPU-connected PCIe Slot), had more Slots available and somehow better BIOS too (Fujitsu TX1320 M3 does NOT support SR-IOV for one Thing).

 

[VirMach]

Ah yes, that's exactly why I created a new version for it
B450d4u 3mf

and this is the changelog of yesterday

• no holes yet for screws. : fixed in v1.5
• the connectors at the bottom need to be slimmer, fixed in v1.5
• the margin around the vga connector needs to be increased fixed in v1.5
• add opening for the top case cover clip : fixed in v1.5

Wow, I'm kind of impressed I remembered that correctly but I can't remember what I did an hour ago.

@dbram : do you think your solution with the I/O Shield would also work with the Supermicro CSE-512L ? I found some very good Deals on those (not sure if they ship to Denmark though) but not sure if the I/O Shield is replaceable and if there is enough room for everything (Motherboard, Fans, ...) overall since the Chassis is significantly shorter Depth.

There's these Supermicros that are even shorter than the one you mentioned, and we can still fit these boards into them. Found the name for you, although for these I had to take a dremmel and cut out the I/O shield because it was "built in" to the chassis.

SYS-5019S-L

Reference photo:



The CSE-512L works fine, we use a ton of them mainly for the X470D4U brother to this board. I also have some fan bracket STLs for them if you want, because the 512 will come with one blower fan that isn't good enough for most Ryzens. Might be OK for your 5700X though, but iit won't blow directly onto it like what it's designed for so you'll need a shroud. Okay maybe not "need" but advised.

You could 3D print a shroud but I found it's easier to get these thin plastic sheets for crafting and origami them.

 

[VirMach]

I just got a quote from Poland. 22 EUR each plus around 8 EUR each Shipping, batches of 5x. Not too bad a 30 EUR everything included I guess ...

From some Pictures online I see a couple of screw and also those latches of the I/O Shield, so it *should* be removable.

It is removable, no worries there for the 512. Have you looked at the rails though? Usually out of stock and expensive. They "droop" a little if you only attach them from the front.

When I say expensive I don't mean direct from Supermicro although more than what you paid for the chassis, they're expensive because they're always out of stock so the market prices.