EXPIRED HYVE Edge Metal G10 - Epyc 7642 - 250£/offer

[Cruzader]

Why would you "upgrade" to Mikrotik, from what I read their "Network OS" (Router OS) is not that great.

Id upgrade to mikrotik since its a significant upgrade from what there is.

It didn't seem you cared about Power Consumption at all.

im fine with using and paying for the power, but i will still be renting out majority of the basement so it cant be ear deafening through the 2 walls seperating the racks from the appartment.
That is why im usualy avoiding 1U, the jump up to 2U fans helps so much on the noise.

 

[luckylinux]

Damn i missed that. I wonder if he has more stock..shipping to AU is just as much as the server but the way you described it against the ryzen is giving my fomo for a cluster lol

Well maybe ask the Seller if you care about Rails just to be 100% sure.

To me it looks like at least an Inner Rail, because the other Side already has the Rack Ears.

FYI Power Consumption seems to be >= 100W at IDLE:

https://www.servethehome.com/amd-epyc-7642-benchmarks-and-review-48-cores-of-power/

I wasn't been able to get my 2nd gen EPYC 7642 idle under 100W. Most of that pow... | Hacker News

news.ycombinator.com

I know it's NOT an accurate Figure but that's the only Thing I found about this CPU

So if you look at Idle 2 x Ryzen Systems are probably going to fair better especially at Idle. Much more expensive RAM and the CPU on top of it though ...

 

[Damo]

Anyone got one and tested yet? Struggling to get into it, USB ports don't seem to be working, presume the IPMI default password has changed too.

 

[Terexxor]

Anyone got one and tested yet? Struggling to get into it, USB ports don't seem to be working, presume the IPMI default password has changed too.

I set one up yesterday with Proxmox over iPXE, it seems to be running fine so far but have not done much testing other than loading a few VM's.

Only the Internal USB header works which is enough to connect a keyboard and load the OS over iPXE. Once OS is installed you can reset IPMI password for admin user 2, IPMI is set to DHCP by default.

The IPMI is running OpenBMC, KVM works and it seems like you can load media also. There is no Fan Control and BIOS is very limited, overall it seems to be missing features seen on the S8030 board.

Idle in Proxmox is around 72-85W with 2 x 32GB 2933Y DIMMS and 1 NVME, it stays fairly quiet around 15%/4500rpm idle. Highest I've seen in proxmox is around 6500rpm. A quick test running a few VM's using 48 cores total took around 125w from the wall idle.

 

[luckylinux]

There is always a catch isn't there! I guess it's because it has 6x CCDs...

Yeah but maybe you can disable a few of them and it would still be great Value . Not sure if it would lower Power Consumption though.

IIRC on Ryzen if you don't go overboard with Memory Frequency (<= 3200 Mz ???) then the Infinity Fabric Power Consumption is "reasonable".

Unsure about EPYC here, but I guess it would follow a similar Principle.

 

[Damo]

I set one up yesterday with Proxmox over iPXE, it seems to be running fine so far but have not done much testing other than loading a few VM's.

Only the Internal USB header works which is enough to connect a keyboard and load the OS over iPXE. Once OS is installed you can reset IPMI password for admin user 2, IPMI is set to DHCP by default.

The IPMI is running OpenBMC, KVM works and it seems like you can load media also. There is no Fan Control and BIOS is very limited, overall it seems to be missing features seen on the S8030 board.

Idle in Proxmox is around 72-85W with 2 x 32GB 2933Y DIMMS and 1 NVME, it stays fairly quiet around 15%/4500rpm idle. Highest I've seen in proxmox is around 6500rpm. A quick test running a few VM's using 48 cores total took around 125w from the wall idle.

Is there a fix to the USB on the back? Not played around in BIOS yet. Also I can't get the two LAN Ethernet ports to be detected on Linux.

 

[Terexxor]

Is there a fix to the USB on the back? Not played around in BIOS yet. Also I can't get the two LAN Ethernet ports to be detected on Linux.

There is no option to enable the rear USB in BIOS, I think they might be locked down for security reasons as VGA works fine. I'm using the onboard Broadcom 1GBe NICs in proxmox, they need re-enabling in the BIOS.

 

[luckylinux]

Im expecting these to be a 85-90w idle since a bit more slimmed down spec.

But by how i see it (rough prices with import/shipping);
- B450D4U-V1LQ5, 50$
- 5700x, 125$
- psu, 30$
- case, 120-150$ (so il go by 135)
- cooler, 20$
- 4x 32gb, 150-180$ (so il go by 165)
- nic, 25$
- assumed idle-ish 30w at 0,085$kwh, 44$ for 2yrs.
595$

The hyve just missing ram;
- server itself, 200$
- 4x 32gb, 80$
- assumed idle-ish 90w at 0,085$kwh, 132$ for 2yrs.
412$

Its not low power, but its still gone be cheaper for me to run for 2years.
The hyve is also priced so far below its real value that id have no problem reselling it at the same cost in 2 years (probably also turn a profit), while the ryzen build will likely have lost 30-50% of its value.

Most epyc mobos alone are close to 3x what they wanted for this server now, even the cheap route of HUANANZHI H12D-8D is 2x this (and their bundle with a quivalent cpu is 4x the hyves cost).

Yeah, I 100% agree that this EDGE HYVE is MUCH CHEAPER than the Ryzen Alternative. It also has management although not sure how good it is (besides the Fan Issue). The Linked Thread didn't seem conclusive on the possibility of flashing the BMC / BIOS on a brand new Chip and if that worked correctly. Unless I missed it somehow.

I think your comparison is a bit wrong, although I agree with your Conclusions :
- The B450D4U-V1LQ5 we got for 25-30 USD
- The Case I got SC512 for around 27 EUR including PSU (not sure about the exact Amount, but below 30 EUR IIRC)
- In order to get the same Benchmark Score, you'd need 2 x Ryzen System (probably even 2.5 x) - while your figures are a bit off IMHO for the Ryzen System particularly the Chassis, it's probably going to be 2x 500 USD or so ~ 1000 USD for an equivalent System
- The Hyve I got for around 150 USD Shipped & delivered included Danish VAT

Plus the Hyve you can install 8-16 DIMMs (if you wanted to). Not apples to apples anymore, but it has scalability.

Power Consumption not an Issue for me in Summer with Solar System ... In Winter on the other Hand it will be bad for sure .

 

[Terexxor]

First of all, does anybody know the exact Part Number of the Motherboard and the matching TYAN Part Number ?

I'm not quite sure where I can find the Manual for this one ...

For rear USB Ports I'd expect at least a Jumper or a BIOS Setting to be honest. Why put the Cost of the Part if you are they aren't going to use it, even though they customized the BIOS & BMC ? Hardware Standardization with TYAN original Motherboard maybe ?

It's branded as a Hyve board in BMC and dmidecode, on boot it is using a Hyve splash screen with no Tyan branding. No info online about this specific board from what I can see.

root@prx-1:~# dmidecode -t 2
# dmidecode 3.6
Getting SMBIOS data from sysfs.
SMBIOS 3.3.0 present.

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: HYVE
Product Name: HS-1811DLite1
Version: 5411J0070002
Serial Number:
Asset Tag:
Features:
Board is a hosting board
Board is removable
Board is replaceable
Location In Chassis:
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0

 

[Damo]

If anyone gets an upgraded BMC image they can share please do. It's over 5 years old, doesn't seem very good functionally.

 

[luckylinux]

Its roughly the build i was considering doing for some nodes.

I am starting to feel a increasing need to grab these tyans/hyves for my next storage nodes now, atleast ordering one of that model to see how they are on consumption.

Ive spent probably 550-600$ per node sofar on ryzens.
If those are just 30-50w above the ryzens im tempted to grab a stack by seafreight and move over to them.

But why would you want to take those HYVES with Xeon Scalable ?

Seems to have the same Benchmark as an AMD Ryzen 3700X or so, but with RDIMM/LRDIMM cheaper Memory Support.

The EPYC seems a much better deal to be honest.

 

[Damo]

You guys got any major issues testing these yet? My system keeps crashing I have tried using both

Samsung 16GB 2Rx4 (M393A2G40EB1-CRC0Q) x8
and
SK Hynix RDIMM 2Rx8 (HMAA4GR7AJR8N-XN) x2


After about 10-20 minutes the system just restarts. Tried other disks, CPU is not overheating, no ideas. I have another one of these maybe I got a bad batch will try other tomorrow.

No event logs

 

[Cruzader]

First of all, does anybody know the exact Part Number of the Motherboard and the matching TYAN Part Number ?

Its this board - S8036GM2NE-LE

 

[luckylinux]

Just curious how TYAN GT86C-B5630 are cooled? Couldn't see a single fan and given it's 1U if smaller fans are used noise would be an issue for some

Right in the Middle staring at you





EDIT 1:

EDIT: now i see those 6 smaller fans after disks. Pulling warm air from 12 disks to cool CPU and RAM doesn't seem optimal either

That's how it's usually done anyways. Supermicro SC216, SC825, SC826, SC846, SC847, etc .... Intake at the Front, go through HDDs/SSDs, then Cool CPU/RAM/VRM/other Components on Motherboard etc. Pretty sure most Servers work this Way.

Usually you don't want HDD Temperatures going above 40 if you can avoid. RAM/CPU can easily go to 70-80 °C, you have a better Temperature Margin there.

 

[Cruzader]

The -LE model is the board used, with a OEM firmware for hyve.

 

[Damo]

Any ideas yet on compatible rails?

 

[Damo]

You are the one that already received it ...

As I said a few Posts back, aren't there already (at least inner) Rails in Place ? From the Seller Picture it looked so ...

Only markings I can see are King Slide

 

[Damo]

Isn't it (or at least the Inner Rail) already included ?

Bottom of the Picture:
View attachment 45104

Yes inner rail is and they are marked "King Slide"

 

[Damo]

Cannot you just "slide them in" to something like a Supermicro 01-sc81362-xx00c002 Bracket ?

These outer rails are very thin. They do fit technically but I would not trust it staying there.

 

[luckylinux]

They would be as storage nodes, so their 12x 3.5" 2x 2.5" bays is what im after.

Ah OK.

The epyc hyves sold out yesterday and still not restocked, might be all gone so setting as expired.

Dude, you (and me too ) just bought 8.

Are you setting up a Datacenter at Home or what ?

 

[Damo]

Didn't you say it was inner Rail just before ? I'm getting confused like this ...

But to be honest, if you have half the Load supported in the Front and half in the Back, it shouldn't be that Bad. It's a 1U server without a bunch of 3.5'' HDDs ...

Sorry ignore stupid me. The inner rails not compatible with anything Supermicro (outer) I have here. There is a big circular metal part sticking out the side.

If I was to just use the ears you'd need a very short rack.

 

[luxxywastaken]

hello

my friend just got this server yesterday from this seller it come and we put ram in it 2 x 16gb either side basically we been trying to get into the openbmc with no luck and now trying to get display from vga port with no luck again seems the vga port just displays nothing bit confused how we manage this server at the moment seems we can't do anything

you got any suggestions to get anywhere with this server

the ram is 16gb 2Rx4 PC4 btw

 

[luckylinux]

I might have fallen for the temptation to grab another 8 after first order... so getting 16.

So I wasn't the only one that Doubled the Order. Went from "only" 4 to "only" 8 .

il probably put 8 in my own lab and 8 in my labrack at work.
The lab hosts there are getting old without much of a budget to get decent stuff.
So probably just suggest a trade of 8 against 3-4tb ddr4 out of old blades or something like that, so the incoming modest lab budget for next year can go towards replacing the old networking with some 25/100g mikrotiks instead.

Why would you "upgrade" to Mikrotik, from what I read their "Network OS" (Router OS) is not that great.

It didn't seem you cared about Power Consumption at all.

Personally at least lately I had issues with IPv6 Neighbor Discovery broken between Switches for which I had to add Static Routes pretty much everywhere . It's usually not "major" Issues but lots of small ones, but definitively NOT what I expect from some new Enterprise grade Equipment.

10gbe Routers I've been buying from Aliexpress for 3x cheaper and flashing OpenWRT on them. Too bad there's no Option for higher Bandwidth yet .

 

[ribroc]

Cannot you just "slide them in" to something like a Supermicro 01-sc81362-xx00c002 Bracket ?

That's an interesting idea, but I wouldn't bother.
Kingslide rails in my experience are extraordinarily unique on the inner rails - particularly on the case-specific mounting mechanisms, however they are extraordinarily generic on the outer rails. I have a few of these same hyve chassis on the easily found 3561-660PZZ0GM outer rail, the only modification needed is a pair of pliers to bend the rear retaining lip on the outer rail out of the way. This could be a problem for cases where the rackmounted equipment had no ears to limit further rearward movement, as then the equipment could continue sliding, but that's usually not a problem. All other function is fine.

you got any suggestions to get anywhere with this server

I don't have this motherboard but I would suggest giving the system a very, very long time to boot. 15-20m if possible. Some of these systems take a long time to become responsive.

 

[Damo]

hello

my friend just got this server yesterday from this seller it come and we put ram in it 2 x 16gb either side basically we been trying to get into the openbmc with no luck and now trying to get display from vga port with no luck again seems the vga port just displays nothing bit confused how we manage this server at the moment seems we can't do anything

you got any suggestions to get anywhere with this server

the ram is 16gb 2Rx4 PC4 btw

Try different RAM. By default overclocking is enabled highest setting in BIOS.

 

[ribroc]

Are you sure the Part Number you gave is correct ?

Yes, I provided that one purposefully. Norco/Chenbro/Hyve are all vendors of the same style of whitebox chassis and generally something that fits one of them will fit most all of them. You do not need to be beholden to brand-name for kingslide either, I've seen their same rails under different branding at some cabinetry supply shops. The main concern when being particular about them is the retention latching mechanism.

 

[luckylinux]

No, it will likely be another 2 Weeks before I get them unfortunately. And I'm not sure when I'll have some Time to play with them either.

 

[TrevorH]

hello

my friend just got this server yesterday from this seller it come and we put ram in it 2 x 16gb either side basically we been trying to get into the openbmc with no luck and now trying to get display from vga port with no luck again seems the vga port just displays nothing bit confused how we manage this server at the moment seems we can't do anything

you got any suggestions to get anywhere with this server

the ram is 16gb 2Rx4 PC4 btw

Stick a network cable in the BMC network port and hook it up to a box that you can run wireshark on then power cycle the machine at the wall and capture all traffic it sends out on boot to a file that you can load into wireshark GUI and see what it sends on boot. Somewhere in that lot you'll likely find its current ip address. I doubt if the BMC depends on the server RAM to operate, they're usually self-contained.