Converting HP Z238 C236 mainboard for homelab use

[oneplane]

I'm planning on using some (extremely) cheap Z238 workstation boards (<30 USD each, ordered 2, with taxes and shipping still below €100) to create Xeon E3 v5 (essentially an i5 with ECC turned on) based servers. I'm planning on either running HarvesterHCI or Ceph+Proxmox and Kubernetes VMs on them. On top of getting two of those boards, I'm getting E3-1240v5 + DDR4 ECC UDIMMS and basic tower coolers (Coolermaster Hyper or Noctua), ATX to HP-custom-bs power adapters (essentially splitting the power delivery and control signals + fake fan feedback) and used uATX cases to setup nodes to add to an existing setup of Supermicro X10-based, Raspberry PI 3+4 and Dell Wyse D10 hosts.

The interesting thing about this (and why I'm writing about this at all) is that those C236 chipset based boards are so extremely cheap yet have plenty of features. They support the Pentium Gold processors too, so with a lower core count you still get ECC so just for storage nodes they make nice base boards. The only real downside is that it doesn't do registered DIMMS, but that is to be expected of everything sub-E5. The C236 chipset is also well-supported by coreboot, this means that if you don't need all the weird custom stuff HP puts in their UEFI images, it is every easy to get base operations working using firmware that's not completely messed up. Their service manual contains a block diagram showing the relevant parts, and because it doesn't have Intel SPS (but the CSME instead) this should be fairly easy to port.

Net result: 4c/8t 3.5Ghz, 32GB DDR4 with ECC, mainboard (with huge coreboot potential), used case, power supply and ATX adapter all for less than €290 including shipping and taxes! Not bad when you consider this is all done in western EU and surplus IT is not really cheap around here.

If I want to get a bunch of drives for OSDs and Ceph running on 3 nodes, I'll probably get a bunch of the infiniband cards + optics or DACs referred in a different thread, but other than that, this should be more than capable for a reasonable homelab.

 

[Stephan]

Have you considered a board with say an Aspeed 2400 or 2500 for remote management? Because while cheap, if you get the wrong CPU you can't even use AMT KVM, provided it is licensed for the board by manufacturer. That being said, C236/C246 are really low power and good solutions to start a cluster. Make sure you have PCIe expandability for things like U.2 SSD and e.g. a Mellanox 40/56G ethernet card to run the Ceph replication over.

 

[oneplane]

Have you considered a board with say an Aspeed 2400 or 2500 for remote management? Because while cheap, if you get the wrong CPU you can't even use AMT KVM, provided it is licensed for the board by manufacturer. That being said, C236/C246 are really low power and good solutions to start a cluster. Make sure you have PCIe expandability for things like U.2 SSD and e.g. a Mellanox 40/56G ethernet card to run the Ceph replication over.

I did, but every time just a mainboard alone would cost more than the entire set I scraped together. Even X10 boards are priced quite high for their age. I already have a few of those, and it's not like I don't already have colocated servers or a stack of AWS accounts, but I really want to see what a hacked-together cluster can do (for a low price).

I have also looked at OpenBMC and Google's old SGABIOS thing, those seem to be avenues worth exploring. I don't actually need KVM or storage sharing, a serial console would be plenty since I plan on PXE-booting everything. If I do get coreboot working I can also configure any settings from the OS so that'd be a neat thing too. That just leaves remote power control, not sure what I'll do there. Maybe just a Pi zero and some GPIO-to-frontpanel hackery.

 

[paf]

Have considered something like a refurbished HP Z240?

The E3-1240v5 processor has no built-in graphics, so you need a graphics board.
Or using it as a server you don't need a graphics card?

 

[oneplane]

Have considered something like a refurbished HP Z240?

The E3-1240v5 processor has no built-in graphics, so you need a graphics board.
Or using it as a server you don't need a graphics card?

Yeah, using it as a lower power server, and only got the mainboards (for 30 usd), no video required. I do have PCIe GPUs available already just in case. Refurbished z240 systems are relatively expensive (one lower power box would be equal to three of the ones I combined). I do see them listed in the US for lower pricing but shipping and tax to the EU would make it much more expensive again

 

[oneplane]

So, just as an update; this construction has been running great for a few weeks now, I've been running Proxmox and Harvester to see how those would run, and I'm currently looking at 10GbE+ network cards to try out Ceph.

There are some interesting things to note:
- It has AMT 11, so there is integrated KVM, Serial over LAN, power control etc.
- This can be disabled with the HAP bit as well as me_cleaner.
- They have HP SureStart which essentially is a CSME on top of the Intel CSME, but implemented in the Nuvoton SuperIO, it blocks some kinds of firmware modification, but you can easily circumvent that by flashing a firmware table that points to 'nothing' causing the SureStart firmware on the SuperIO's EC to leave the firmware alone, allowing you to use coreboot and me_cleaner.

At the end of the day, very cheap method to get a somewhat recent Xeon E3 node up and running.

 

[unmesh]

What's the size of this mainboard? I'm wondering if I can use it in an existing minitower chassis.

 

[oneplane]

What's the size of this mainboard? I'm wondering if I can use it in an existing minitower chassis.

As far as I can tell it's micro-ATX. I've fit one in a standard PC case and it reaches to the 4th PCI slot cover. One of the models HP uses this mainboard in (Spitfire Rev. A) is the HP ProDesk 600 G2 Small Form Factor as well as 800 G2. There is a somewhat usable sketch on page 31 at http://h10032.www1.hp.com/ctg/Manual/c04831203

To use it with a standard ATX power supply you do need a connector and PWM conversion kit, but those are cheap and readily available. (https://www.amazon.com/24Pin-Female...efix=hp+elite+800g1+power+cabl,aps,119&sr=8-7)

 

[paf]

I think that the C236 chipset is not used on the HP ProDesk 600 G2 Small Form Factor, and is not used also on the 800 G2.

One uses the Q150 (600) and the other (800) uses a Q170 chipset.

Info:
https://www.hardware-corner.net/desktop-models/HP-ProDesk-600-G2-SFF/
HP EliteDesk 800 G2 SFF – Specs and upgrade options

(edit for more info)
The more common machines with the C236 chipset are:
HP Z240 & Z240 SFF
Dell T3420 and 3620
Lenovo P310 and P320
Fujitsu W550, J500 and J550/2

 

[oneplane]

I think that the C236 chipset is not used on the HP ProDesk 600 G2 Small Form Factor, and is not used also on the 800 G2.

One uses the Q150 (600) and the other (800) uses a Q170 chipset.

Info:
https://www.hardware-corner.net/desktop-models/HP-ProDesk-600-G2-SFF/
HP EliteDesk 800 G2 SFF – Specs and upgrade options

(edit for more info)
The more common machines with the C236 chipset are:
HP Z240 & Z240 SFF
Dell T3420 and 3620
Lenovo P310 and P320
Fujitsu W550, J500 and J550/2

That is correct, HP uses the Spitfire Rev A. version with the C236 in the Z238 system, but the mainboard has multiple versions (all named "Spitfire Rev A.") with different configurations. But because it's the same physical PCB used in all those systems the chipset itself is just a different parameter per 'release' of the same PCBA.

I have the schematics for the board and the chipsets are actually drop-in replaceable between them. It appears the series for this board have a few PCHs;

Spitfire base = Q170
Scorpion = Q170
Toledo = Q150
Spitfire Bunun = C236

Bunun also has no DDR3 options (only DDR4) while the Skylake PCH configuration does. It also has an M.2 option.

The HP/Foxconn schematic part number for all of them is the same: 795207-000. The PBC P/Ns are the same too, only the PBA P/Ns differ, which makes sense since the assembly depends on the options selected for the final model.

Edit: it appears from the die and package that the C236, Q150 and Q170 are nearly identical, perhaps only a few features are fused off and some I/O options and firmware enablement disabled between them. Hopefully it's just a result of binning and not yet another crappy market segmentation choice Intel made.