EXPIRED HYVE Edge Metal G10 - Epyc 7642 - 250£/offer

[luckylinux]

Actually they have some more pieces available.

Now you are giving me some BAD Ideas .

 

[luckylinux]

All 6 gone already

 

[luckylinux]

Wow, next time… don’t think, just buy. I believe I paid 100 pounds per machine, when buying 8.

Same ... around 120 GBP / Piece shipped to Denmark actually . I also have 8 Pieces here but I was thinking of getting a few extras .

Main learning for me was to limit the Frequency via cpupower, the noise went down A LOT and Performance didn't change much.

 

[luckylinux]

Not sure why I get emails immediately after somebody replies but their Post does NOT show up here ...



@flobert: for the cpupower Command (must install linux-cpupower, i.e. apt install linux-cpupower on Debian/Ubuntu), it's not very complicated at all, just something that IMHO it's better to do unless you don't mind the Fan Noise .

# Get the List of Supported CPU Frequencies and Governors
cpupower frequency-info

# Set CPU Governor to Conservative (avoid too fast Frequency Changes, both increase/decrease, somewhat smoother)
cpupower frequency-set --governor conversative

# Set CPU Frequency to 2.1GHz
cpupower frequency-set -u 2.1GHz

# Check to see if Configuration has been updated
cpupower frequency-info

I believe the cpupower frequency-set -u 2.1GHz Command is correct, however check afterwards. You might try to set it back to 2.3GHz, but if the Fan Noise shoots up, you might need to reduce it again.

I did a Linux Kernel Compile later as "stress Test" and the Fans were relatively quiet. Yeah, CPU Temperature was IIRC in the high 70 / low 80, but the Fan Noise was relatively low.

Note that on these EPYCs apparently you can ONLY set the CPU Frequency to the EXACT Values that the P0/P1/P2 States let you to.
If you try to set e.g. to 2.2GHz, it will NOT update your Configuration at all ! The valid Frequencies are listed in the Output of cpupower frequency-info.

This is very different from what you can do with Intel Xeon E3 v3/v5/v6 or Intel E5 v2/v3/v4 where you can set to whatever Value between e.g. 0.8GHz and 3.2GHz.

Finally, doing a Linux Kernel Compile test I believe the Power Consumption went from 250W (without any cpupower Commands) or so down to 150W or so (2.1GHz IIRC). Performance Hit was very low though .

IMPORTANT: do **NOT** set 150W Wattage Limitation or Frequency Limitation to e.g. 2.3GHz in BIOS, your Performance will tank otherwise (3x lower Performance). I believe cpupower even at 2.1GHz is letting you turbo boost quite nicely or at least the Performance is very good compared to stock, but the Fan Noise is quite low (relatively speaking, definitively NOT NOCTUA Level Silence ).

EDIT 1: added some small Clarifications about Frequencies

 

[luckylinux]

I was lurking in this thread for a while after finding the BargainHardware listing on eBay. I bought one immediately after receiving the email from eBay. I wanted to buy more, but I was limited to a single purchase (it showed an error like "1 unit per each 10 days"). I made an offer of 149 GBP for one unit and added a note about the issue. An hour later, I managed to make another offer. I offered 100 GBP per unit for the remaining 6. In total, it came to almost 900 GBP including shipping and taxes.

Now, I'm looking for DDR4 LRDIMM, but the cheapest 512GB kit (8x64GB) I’ve found is around 600 EUR. The "sweet spot" seems to be 16GB RDIMM, but using 16x16GB lowers the speed and is limited to 256GB. Anyone knows where to get cheaper LRDIMM?

The main question is: why are they selling it so cheap? What’s the catch? As far as I know, the downsides are:

• Only 4x (or 3x?) NVMe slots
• No PCIe slot (needs a adapter)
• No PCie bifurcation (we can't add another 8x NVME)
• No BIOS/Firmware update
• Not from Dell, HP, or another known brand
• IPMI seems quite basic and not fully "plug and play"
• High noise (that is expected from 1U, not exactly true)
• CPU is 'fused/locked' to the motherboard (maybe we can't change the CPU too?!)
• No rails (previous posts seem to have found a compatible one).

What else? I don’t see why these disadvantages would drop the price to 250 GBP or even lower. The same seller is selling a DELL R640 (with 2x 6138) and 64GB for 233 GBP, (price without VAT!), seems a good option too, but it's not EPYC.

Pretty much spot on the Remarks, except:

• You will always need a Riser for 1U Chassis, no matter which Brand (although these are a bit special indeed, also expensive)
• High Noise IMHO only at bootup (see my Post above with 2.1GHz Frequency set via cpupower, Noise is quite good while Performance is almost the same, I mean ... for a 1U Chassis of course )
• IIRC you have 6 x NVMe Slots (2 x M.2 on the Motherboard, 2 x M.2 on top of the Steel Plate, 2 x M.2 on the bottom of the / lower part of the Steel Plate). Possibly the Steel Plate Slots you might be able to convert to U.2 if you don't mind taking out the Steel Plate, drilling it, screwing the Drive from the bottom, then remounting the Steel Plate on top. I linked some Ideas about the M.2 Adapter (just received the "standard" Solution which typically works for ASUS Motherboards, maybe it works here too. This Motherboard is THICK though !

Apples and Oranges ? 40% of the Performance (60% lower Score) than EPYC at the same Price just for the Brand ?

Intel Xeon Gold 6138 vs AMD EPYC 7642 [cpubenchmark.net] by PassMark Software

www.cpubenchmark.net

 

[luckylinux]

It's a Dual Xeon Gold 6138, not a single. It still worse... But, that is the point: Older Xeon + DELL = same price of HVYE EPYC server.

True, I always forget this EPYC is "just" a single Socket Model .

Although if you look in BIOS, it seems they (or at least Tyan does) probably have a dual Socket as well (in BIOS you can find Memory Info/Settings for Socket 2, obviously empty on this one).

EDIT 1: I'm just NOT used to have 48 Cores in one Socket . It would be fun to get the ARM Ampere Motherboard once Hetzner get rid of them . Those have even more Cores I believe .

 

[luckylinux]

Id assume its the typical 80C or 2x64C.

For me the fun/novelty ended at the software support, was given a few 2x64C storage appliances and they did not last long in my lab.

Is it really that bad ? I know many SBC Software Support is ****, but for an enterprise Solution I would have expected better.

Or are you talking in Terms of Application Support ? I believe pretty much whatever Open-Source Solution should work OK on these.

Proxmox VE is not Officially supported but there is/was an unofficial Port I believe. The other Thing missing might be OPNSense.

EDIT 1: so what did you do with those Appliances ? Sold them for a few thousand EUR ?

 

[luckylinux]

The package sat at the GSP warehouse for over a week, though I do believe the tracking showed as delivered.

So you received it in the End ???

 

[luckylinux]

Regarding NVME: there are 4 u.2 connectors, but only 3 U.2 -> M.2 converter boards. I'm working on a 3d print to house 4 U.2 nvme's in the front.
For the nvme's in the back I used these with success: https://nl.aliexpress.com/item/1005003346953721.html

I ordered a similar one from Aliexpress but it felt very loose when I put it into the Hole in the Motherboard.

Granted I didn't put a Screw in yet so maybe that will widen/expand the Holder in the Back, but as it is now I'd say it's barely holding if at all.

Another Option could be to simply spread it before inserting it. That's kinda counterintuitive to how I'm used to spread though .