NUC Like Server?

[tjk]

Hey Folks,

I've been testing a few of the https://store.minisforum.com/products/hx90 for a purpose built application I am working on, but it is missing two things:

1. IPMI/OOB Port
2. 10G Interface

I'd like to find something pre-built if I can, but also happy to build my own if I have to. But I'm running into issues on both counts.

Anyone ever build anything like this with or know any vendor that may have something available we can buy 50 to 100 units from?

 

[i386]

Closest I can think of: E300-9D-8CN8TP | Mini 1U | SuperServers | Products | Super Micro Computer, Inc.
has 10GBE and ipmi, but also 40mm fans ._.

 

[Stephan]

I'd say "doesn't exist" in this form factor. Server to me means ECC RAM, IPMI, and the lowest entry I found viable would be UDIMM DDR4 on some micro ATX or ATX board like B550D4-4L or whatever Supermicro has. You have to look at this from a TDP standpoint: IPMI is 5 W and 10G dual port say 10 W so 15 W total, which is the TDP of some low-end i3 NUC already. Defeats the purpose. Also formfactor and cooling problem because a single laptop fan construction would no longer suffice.

If you really need that many machines best bet imho would be refurbished 2U servers like HPE DL380 or Lenovo x3650 M4/M5. I suppose you can pack 20 into a rack, so 5 racks and you are done. You'd need 5 dedicated 3-phase 11 kW PDUs. 50-100 units is another problem, because right now new servers in 2U are backordered in lots of places.

Original Intel NUCs with i5 usually have AMT enabled, also Visual BIOS can be set up to enable display port emulation so KVM will not black out once i915 driver is loaded. Would leave the 10 Gbps networking problem, also no ECC.

There is a story that Hetzner started hosting in Gunzenhausen with midi tower PCs on wooden racks. Depends on how much space you have.

 

[tjk]

There is a story that Hetzner started hosting in Gunzenhausen with midi tower PCs on wooden racks. Depends on how much space you have.

Yea, they are building the AX101 Ryzen in an open case with ASUS motherboards, I have a number of these running already.

The HX90 works for us, but I'd like to get 10G and having IPMI/OOB for MAAS provisioning would also be a nice-to-have.

I have a large number of the ASRockrack Ryzen servers running today, they work well. The new requirement doesn't need ECC or RPSU's since it will be a cluster of a large number of machines that I don't care if 1 or 10 of them fail, for this use case only.

 

[Sean Ho]

Which NUC-like aspects are you looking for, and why? Form-factor? Power draw? TDP for cooling? Noise? Does it need to be a uSFF/TMM box, or would 1U work? Xeon-D is (somewhat) low power and has builtin 10GbE, and usually comes with IPMI. Perhaps the venerable HP EC200a?

 

[tjk]

1U would work, the easy option is to buy more ASrockrack 1U ryzens, but something like the HX90 is about half the cost of the equiv ASR 1U build.

 

[Oarman]

Have you tried Intel AMT / AMD DASH to see if it might be a substitute for real IPMI?

Lots of TMMs have AMT or DASH. I don't recall seeing any that have 2.5GbE yet, let alone 10, though. You could do a 5GbE dongle off a USB-C port but I don't think I'd trust a server to that.

This also brings up the question of why IPMI is such a power hog, I have a hard time seeing them being that much more demanding than a Raspberry Pi and those are more like 5w max. I wonder if that's the logical reverse of Stephan's comment, i.e. in a server consuming x00 watts that ideally never sits idle, who cares about a 5w parasitic load?

 

[tjk]

Have you tried Intel AMT / AMD DASH to see if it might be a substitute for real IPMI?

I haven't, the problem is two fold.

I need to have maas being able to use ipmi to reboot a node to provision, and 2nd is these will be deployed in many geo locations, and need ipmi for OOB access when/if needed.

Appreciate all the feedback so far folks!