EXPIRED HYVE Edge Metal G10 - Epyc 7642 - 250£/offer

[luckylinux]

Anyone got one and tested yet? Struggling to get into it, USB ports don't seem to be working, presume the IPMI default password has changed too.

Ideally you should just boot a Linux (e.g. Debian/Ubuntu) LiveUSB, let it aquire an IP via DHCP, SSH into it, then reconfigure IPMI to use DHCP & reset the IPMI Password.

Alternatively use TFTP/PXE Boot (Network Booting) and you could do the same Thing:

# Configure IPMI to use DHCP IP Addressing
# Supermicro: use channel 1
ipmitool lan set 1 ipsrc dhcp

# Fujitsu TX1320 M3: use Channel 2
# ipmitool lan set 2 ipsrc dhcp

# Reboot the BMC
ipmitool bmc reset cold

# Reset ADMIN Password
# Admin is typically User #2
ipmitool user set password 2

# Print LAN Information
ipmitool lan print

EDIT 1: note that the exact "Channel" to use for the various Configurations, User Number, etc, will depend on the Manufacturer (and hopefully it's close to "stock" Tyan).

 

[ribroc]

What about the 2nd Part Number which is slightly different, would that also Fit you believe ?

3561-660EPZZ0GM and 3561-660PZZ0GM both work with the pliers modification I mentioned. I have two of the "E"s in reserve actually. I do not know why there is that labeling distinction. I usually just search for 3561-660, sorry for the confusion!

 

[Cruzader]

For those that did get your servers, how were they packed? one unit per box or multiple per box?
Wondering how big of a mountain i need to bring with me from the store as they arrive...

 

[Cruzader]

You of all People are wondering about that ? Aren't you regularly receiving Pallets full of Servers anyways ?

For pallets and large shipments sent "normally" all the delivery drivers in my area know what garage is used for deliveries and just put it there.

The ebay packages get sent as postnord to a small grocery store close to where i live, while im storing all the hardware 45min away close to work.
If its 2 per box i can fit one shipment of 8 in the car, if its 1 per box i need to bring a trailer.

Ebay is showing me 5 shipments for each order of 8, but not all are tracking and ebay shows max 5 (plus tends to mix up shipments to their hub and from their hub).
The shipping cost was also way lower than 8 regular "full size" servers tend to be.

 

[luckylinux]

Is there a fix to the USB on the back? Not played around in BIOS yet. Also I can't get the two LAN Ethernet ports to be detected on Linux.

That Issue sounds familiar to me at least since I saw it on the ASROCK B450D4U-V1LQ5 with an AMD Ryzen 2700X, where rear USB Ports and even onboard VGA (via ASPEED 2510) would NOT work. It seems like some AGESA and BIOS Version incompabitiliy (BIOS was too recent / new, therefore only AMD Ryzen 3000 and 5000 series, e.g. 3700X and 5700X were officially supported, although 5000 Series had lots of Quirks).

Most likely try with a newer or older BIOS Version, that might help.

What do you mean with:

Also I can't get the two LAN Ethernet ports to be detected on Linux.

Try some of these Commands, they should list what Interface is available:

dmesg | grep -E " eno[0-9]+| eth[0-9]+| enp[0-9]+| ens[0-9]+" # Possibly try also other combinations

journalctl -xb --no-pager | grep -E " eno[0-9]+| eth[0-9]+| enp[0-9]+| ens[0-9]+" # Possibly try also other combinations

lshw -class net # requires lshw Package to be installed

ifconfig -a # requires net-tools Package to be installed

ip link

ip addr

What are you running as OS ? Debian Trixie / Proxmox VE 9 ?

EDIT 1: it could also be that the NIC is disabled in BIOS or that there is a Jumper, always check

 

[Cruzader]

Since I did 2 Orders of 4 Pieces each, it could be they managed to squeeze 4 Servers in one Package, so that would be 6U and I guess around 60 kg ? Probably too heavy for most Pakkeshops. I think my Local Pakkeshop can only accept up to 30kg Items.

Ebays policy is maximum 30kg, but they do not reject if they are a bit over either.

Postnord handling them in Norway do not accept normal packages over 35kg (i had a 36-37kg package demonstrably converted to freight by Postnord and delivered home on a pallet rather than store as normal package).

 

[cybertinus]

I received 4 boxes today. Great little machines!

The 2 USB ports on the back didn't work indeed. But this is easily fixable by just enabling the USB controller in the BIOS. So, when enabling the 2 onboard NICs, also just enable the second USB controller and the ports on the back work!

 

[luckylinux]

First of all, does anybody know the exact Part Number of the Motherboard and the matching TYAN Part Number ?

I'm not quite sure where I can find the Manual for this one ...

For rear USB Ports I'd expect at least a Jumper or a BIOS Setting to be honest. Why put the Cost of the Part if you are they aren't going to use it, even though they customized the BIOS & BMC ? Hardware Standardization with TYAN original Motherboard maybe ?

 

[Johan Kooijman]

Actually I just looked at my Postnord App and if I trust what it says, I should receive 4 Packages for 8 Servers in Total.

So that would be 2 Servers per each Package.

Assuming of course that all the Packages have already been communicated to Postnord (it might be that another 4 Packages appear later).

Can confirm, 2 servers per package, very neatly packaged!

 

[Johan Kooijman]

89 x 15 x 58 cm per 2 servers.

 

[ItsZigi]

For all those with non-working rear USBs you can infact enable them. If you go into bios with keyboard plugged into the internal usb

Can go to AMD CBS > FCH Common Options > USB Configuration Options > then set XHCI Controller 1 to enabled

 

[luckylinux]

If anyone gets an upgraded BMC image they can share please do. It's over 5 years old, doesn't seem very good functionally.

There was another Thread about it linked in the 1st Page, it does NOT seem so easy to do if possible at all ...

 

[Johan Kooijman]

Not yet, will order risers and test later this week.

 

[luxxywastaken]

Stick a network cable in the BMC network port and hook it up to a box that you can run wireshark on then power cycle the machine at the wall and capture all traffic it sends out on boot to a file that you can load into wireshark GUI and see what it sends on boot. Somewhere in that lot you'll likely find its current ip address. I doubt if the BMC depends on the server RAM to operate, they're usually self-contained.

hello

we did get openbmc ip and got to web interface but other then that defualt login won't work and still not display from server vga port it self

 

[luckylinux]

You guys got any major issues testing these yet? My system keeps crashing I have tried using both

Samsung 16GB 2Rx4 (M393A2G40EB1-CRC0Q) x8
and
SK Hynix RDIMM 2Rx8 (HMAA4GR7AJR8N-XN) x2


After about 10-20 minutes the system just restarts. Tried other disks, CPU is not overheating, no ideas. I have another one of these maybe I got a bad batch will try other tomorrow.

No event logs

If you have the Chance, easiest is probably to use Serial Logging using a "Null Modem" Cable (if you can get it to work, I tried troubleshoot an early boot Kernel Panic but never managed until now) and setup serial Logging continuosly on another PC.

Another possible Alternative could netconsole (also I never managed to get it to work, I always tried using netcat, but some Users reported that a "real" syslog Server is required, because also they couldn't get netconsole to work over netcat, despite some Tutorials showing so).

Did you check Memory Errors and Temperatures ? Is the NIC or other Components possibly overheating ?

I guess we have no chance in Hell for a BIOS Update eh ?

 

[Johan Kooijman]

Order where ? The only one I found is the R16 from China for 30 USD. Is there something else ? I couldn't find the left at all.

I think the Risers will end up being more expensive than the Server itself .

I have a few in other machines in the lab, S8036 mainboards, so let's hope it works. I can't find anything about bifurbication in the bios, I guess that won't be an option.

 

[Johan Kooijman]

As far as I can see, you can hook up 4 u.2 drives in the front of the machine. There are only 3 U.2 -> M.2 converter boards, but there are 4 connectors. Will try that as well.

 

[luckylinux]

Its this board - S8036GM2NE-LE

Is this the EXACT Motherboard or the non-OEM one ? Just to make sure if we could use those BIOSes .

 

[Cruzader]

As far as I can see, you can hook up 4 u.2 drives in the front of the machine. There are only 3 U.2 -> M.2 converter boards, but there are 4 connectors. Will try that as well.

By spec the 2 cables each split into a pair of U.2 ports.

Cloudflare mentions in their blog using 3 drives, so i assumed there was atleast one u.2->m.2 pcb on the underside of the plate also.

All 4 in front along with m.2 on mobo should be usable.

 

[Cruzader]

What I think I'll miss mostly is the Riser. I don't think I'll use more than 2 x U.2 and/or 2 x M.2 Drives in these ...

Just to open up another wormhole, you could connect a riser to one of (or both) the ports used for the u.2

 

[luckylinux]

The -LE model is the board used, with a OEM firmware for hyve.

Alright, so no BIOS and no BMC Firmware Update possible at all, basically.

(the BMC was discussed in another Thread in the 1st Page, but I don't believe the User managed to get a new BMC Firmware uploaded, even with a brand new BIOS Chip)

EDIT 1: unless if the BIOS Chip is "socketed", then we could order a new BIOS Chip and flash the TYAN Firmware on it ?

Same principle for the BMC Chip ... maybe .

 

[Buraz39]

I opened the thread about this server, flashing tyan firmware causes error 0022 which by the looks of it means cpu is bound to cloudflare/hyve bios or bmc firmware, the cpu is effectively locked to the chassis, changing only bmc firmware causes no boot, changing only bios same thing, theyre loud, the standard setting for fans was 15% then when loaded 100% then 15% i found no way to regulate it ipmi doesnt work with the known fan commands, maybe there are different ipmi commands that work with openbmc, havent tried compiling openbmc for tyan to see if that would work

 

[Cruzader]

Picked up my boxes but not gotten as far as unpack all of power one on.
Are the riser options available in the cloudflare/OEM bios out of the box?



For the combo link it looks like SLIMSAS1 and SLIMSAS2 are the slots used for the 4x U.2 in front.

31. PCIE/SATA 74P SLIMSAS Connector (SLIMSAS1)
30. PCIE/SATA 74P SLIMSAS Connector (SLIMSAS2)

27. PCIE 74P SlimSAS Connector (SLIMSAS 3 removed on -LE SKU)
26. PCIE 74P SlimSAS Connector (SLIMSAS 4 removed on -LE SKU)
25. PCIE 74P SlimSAS Connector (SLIMSAS 5 removed on -LE SKU)
23. PCIE 74P SlimSAS Connector (SLIMSAS 6 removed on -LE SKU)
24. PCIE 74P SlimSAS Connector (SLIMSAS 7 removed on -LE SKU)
22. PCIE 74P SlimSAS Connector (SLIMSAS 8 removed on -LE SKU)

 

[luckylinux]

Any ideas yet on compatible rails?

You are the one that already received it ...

As I said a few Posts back, aren't there already (at least inner) Rails in Place ? From the Seller Picture it looked so ...

 

[Johan Kooijman]

No, those options are not available. No bifurbication

 

[Cruzader]

The only other "Deal" I found so far on eBay was from China at 30 USD / Piece (DOUBLE !!) of the TYAN M8036-R16-1F one.

Asked about 8 or 16 and they would do 25/ea, not amazing but lower than 30 i suppose.

 

[luckylinux]

Only markings I can see are King Slide

Isn't it (or at least the Inner Rail) already included ?

Bottom of the Picture:

 

[Johan Kooijman]

For those interested ...

Rails

Offered 10 USD / Set for 8 Sets, got an Invoice and after contacting the Seller, he adjusted Shipping to approx. 23 USD (to Jetcarrier US).

This one probably calls for Sea Shipping since it's 2kg / Piece so 16kg in total for me ...

List Price is 18 USD / Set (already discounted from 24 USD / Set) so I'd say it's quite good

Only 3 Left. I was pretty sure the Seller had another offer which he took down that had more than 11 Pieces. Might be worth to contact him and ask if you are interested in more .

Pardon Our Interruption...

Riser

I just found this Riser which was added Recently:

TYAN M8036-L16-1F - 75 USD for 5 Pieces (15 USD / Piece) - 7 Sets currently remaining

Lot of 5 TYAN M8036-L16-1F Riser card | eBay

Find many great new & used options and get the best deals for Lot of 5 TYAN M8036-L16-1F Riser card at the best online prices at eBay! Free shipping for many products!

www.ebay.com

So 15 USD / Piece plus another I guess ~ 40 EUR for a Package of 2 Sets (10 Risers) to be shipped to Denmark via Jetcarrier, so probably around 20 EUR / Piece overall.

Not a super good Deal compared to the entire Server but considering that Supermicro standard x16 Risers go for about 10-15 USD/EUR it's not too bad either considering these are/might be a bit special. Indeed the PCIe Slot Alignment is different compared to Supermicro PCIe Risers so the Supermicro Risers would likely NOT work in this HYVE Edge Chassis without some Modification.

I just ordered 2 Sets, so I will have 2 extras here in Europe if somebody else is Interested .

The only other "Deal" I found so far on eBay was from China at 30 USD / Piece (DOUBLE !!) of the TYAN M8036-R16-1F one.

EDIT 1: added remaining Quantity for Rails (3 Pieces) and Risers (7 Pieces)

Please let us know if the riser indeed is the correct one. I tried contacting the seller about shipping to The Netherlands, but haven't heard a thing..

 

[Cruzader]

Did you also order the LEFT one I got for 15 USD / Each (plus shipping via Jetcarrier & VAT of course) ?

Have not gotten around to it, but i saw my 10$/ea offer for the 10rails they had left was accepted.