EXPIRED HYVE Edge Metal G10 - Epyc 7642 - 250£/offer

[Cruzader]

EDIT 1: I'm just NOT used to have 48 Cores in one Socket . It would be fun to get the ARM Ampere Motherboard once Hetzner get rid of them . Those have even more Cores I believe .

Id assume its the typical 80C or 2x64C.

For me the fun/novelty ended at the software support, was given a few 2x64C storage appliances and they did not last long in my lab.

 

[luckylinux]

Ebays policy is maximum 30kg, but they do not reject if they are a bit over either.

Postnord handling them in Norway do not accept normal packages over 35kg (i had a 36-37kg package demonstrably converted to freight by Postnord and delivered home on a pallet rather than store as normal package).

Actually I just looked at my Postnord App and if I trust what it says, I should receive 4 Packages for 8 Servers in Total.

So that would be 2 Servers per each Package.

Assuming of course that all the Packages have already been communicated to Postnord (it might be that another 4 Packages appear later).

 

[inkey]

Has anyone here had Pitney Bowes/eBay Global Shipping Program (GSP) lose your shipment?

There haven't been any tracking updates in 2 weeks since it left the GSP warehouse in the UK. The last location is an exit scan in CITY, STATE. I've opened an Item Not Received case with eBay but I'd much rather have what I ordered than a refund.

How long it took to receive and send? One of my order is flag as "Delivered" to the Shipping Center, 5 days ago. However, the GSP didn't report it as received, so it was not even processed by GSP.

It's my first time using GSP.

 

[hmartin]

How long it took to receive and send? One of my order is flag as "Delivered" to the Shipping Center, 5 days ago.

The package sat at the GSP warehouse for over a week, though I do believe the tracking showed as delivered.

 

[Johan Kooijman]

I was lurking in this thread for a while after finding the BargainHardware listing on eBay. I bought one immediately after receiving the email from eBay. I wanted to buy more, but I was limited to a single purchase (it showed an error like "1 unit per each 10 days"). I made an offer of 149 GBP for one unit and added a note about the issue. An hour later, I managed to make another offer. I offered 100 GBP per unit for the remaining 6. In total¹, it came to almost 900 GBP including shipping and taxes.

Now, I'm looking for DDR4 LRDIMM, but the cheapest 512GB kit (8x64GB) I’ve found is around 600 EUR. The "sweet spot" seems to be 16GB RDIMM, but using 16x16GB lowers the speed and is limited to 256GB. Anyone knows where to get cheaper LRDIMM?

The main question is: why are they selling it so cheap? What’s the catch? As far as I know, the downsides are:

• Only 4x (or 3x?) NVMe slots
• No PCIe slot (needs a adapter)
• No PCie bifurcation (we can't add another 8x NVME)
• No BIOS/Firmware update
• Not from Dell, HP, or another known brand
• IPMI seems quite basic and not fully "plug and play"
• High noise (that is expected from 1U, not exactly true)
• CPU is 'fused/locked' to the motherboard (maybe we can't change the CPU too?!)
• No rails (previous posts seem to have found a compatible one).

What else? I don’t see why these disadvantages would drop the price to 250 GBP or even lower. The same seller is selling a DELL R640 (with 2x 6138) and 64GB for 233 GBP, (price without VAT!), seems a good option too, but it's not EPYC.

EDIT:
¹ "In total" means the total of 7 servers (1 server in one purchase + 6 servers in another one).

Regarding NVME: there are 4 u.2 connectors, but only 3 U.2 -> M.2 converter boards. I'm working on a 3d print to house 4 U.2 nvme's in the front.
For the nvme's in the back I used these with success: https://nl.aliexpress.com/item/1005003346953721.html

I'm still looking for rails & risers. I found them, but no answer on the question if they can ship to The Netherlands.

 

[luckylinux]

Can confirm, 2 servers per package, very neatly packaged!

Can you give us the Measurement of each Package, for me, @Cruzader and potentially other People here ? Thanks in Advance .

 

[gb00s]

Funny how these are showing up on 'Kleinanzeigen' for double the price with the identical specs and all testing 'without problems'.

 

[gb00s]

I was referring to someone who offered 5 for 450 Eur each this morning .... and boom it's deleted. Well ...

Edit: even with 128GB you can max add 100 Eur i got mine for 130 Eur (2x) incl shipping. So ...

 

[Cruzader]

I was referring to someone who offered 5 for 450 Eur each this morning .... and boom it's deleted. Well ...

Id expect that to easily sell, ive had higher offers for mine.

Im considering parting with some of them after getting 3k for 4pcs type offers.
I could pay for a C6400 with 4xC6425 nodes with that, id have taken that offer as a straight swap.

 

[luckylinux]

Did anybody check if PCIe Bifurcation works ? And if we can use that "weird" x24 (?) PCIe slot without some super expensive Riser (just a regular x16 one) ?

 

[epi1337]

@luckylinux
I am interested in the TYAN M8036-L16-1F riser. Do you still have one available, and would you be willing to ship it to Germany? Have you already tested it with the HYVE Edge Metal G10, and does it work?

 

[cybertinus]

In the front of the servers you can place 3 M.2 NVMe's, via those convertor prints Hyve has developed, to convert from U.2 to M.2. But, I rather just plug in U.2 devices directly. Those are faster and then I can use 4 devices instead of 3. The issue is that you have to get those devices attached some how, which is hard because of all the M.2 stuff. But you can easily remove it. Just 4 screws and then you can remove the top plate. Then just pull up a pin and then you can remove the bottom plate. And then I had a 3D holder designed, for 4 U.2! And I'm sharing this file now with all of you, so you can print this holder yourself, if you want to. I have permission from the actual designer to do so.

This holder can hold 4 NVMe's, both 7 and 15mm thick. And it slides over the pins the bottom plate was attached to as well, and then it is in place.
The 15mm thick NVMe's you should place in the lowest screwholes, and the 3rd one from the bottom. The 7mm thick NVMe's should be placed in the second and forth screwhole from the bottom. Otherwise you will hit a bump in the chassis and the connector won't fit.

Don't print this in PLA, because the inside of the server can become too hot for it. Use PETG.
I have tested this print on a Bambu Labs P1S and it printed fine.

I hope this helps somebody!

 

[luckylinux]

Not yet, will order risers and test later this week.

Order where ? The only one I found is the R16 from China for 30 USD. Is there something else ? I couldn't find the left at all.

I think the Risers will end up being more expensive than the Server itself .

 

[cybertinus]

For those who are curious on how the holder looks exactly, I made some pictures
The holder itself. As you can see it can hold 4 storage devices easily. In the top left holder you can see a small hole between the 3rd and 4th screw hole counted from the bottom. This was a small error in the 3D drawing. This has been resolved in the .step file I shared in my previous post






And this is the holder nicely placed in the server itself, with the original plates with the U.2->M.2 convertors removed. It fits nicely.

I hold the server in a 90 degree angle, to try to make the holder fall from the pegs. It stayed in place. If you really want to, you can add double sided tape to the bottom of the holder (very thin stuff, a millimeter thick at the most!).


I like this holder!

 

[Johan Kooijman]

Nice! Will certainly print a couple of these, thanks!

 

[luckylinux]

hello

we did get openbmc ip and got to web interface but other then that defualt login won't work and still not display from server vga port it self

Then boot a LiveUSB using the Internal USB Header and reset the IPMI Password like I said in my Previous Post ...

 

[Cruzader]

They have some more units available now

 

[Cruzader]

Looks like all gone again

 

[Damo]

I have a few can sell £220 based in UK, can ship worldwide (you pay).

 

[jd456]

Anyone got any measurements and close up photos of ICS on the card for reverse engineering the riser cards? I have experience getting some of these made and assembleing them

GitHub - TobleMiner/HPE-FlexibleLOM-adapter: Adapter for using HPE FlexibleLOM cards in full height PCIe slots

Adapter for using HPE FlexibleLOM cards in full height PCIe slots - TobleMiner/HPE-FlexibleLOM-adapter

github.com

I am sure this experience would transfer well, as I cant find any risers in stock

 

[luckylinux]

What I think I'll miss mostly is the Riser. I don't think I'll use more than 2 x U.2 and/or 2 x M.2 Drives in these ...

 

[epi1337]

if you need a riser for reverse engineering here are some in stock:

Pardon Our Interruption...

I still wait for my delivery of M8036-R16-1F to confirm that they are working.

 

[cybertinus]

I ordered a few from there as well. Also waiting for delivery. they should be here between the 21st and the 27th... Then I can test them as well.

 

[luckylinux]

Are you sure it will NOT work if I simply use the Supermicro Riser (x16) ? Of course I'll lose a x8 "Part" (which I wasn't planning on using anyways), but what is so special about the x24 Slot otherwise ? The Signals are routed in a Proprietary Way ? The Height of the Supermicro Riser wouldn't fit the Chassis ?

 

[cybertinus]

If you hate the build in IPMI, then this might be a good solution too: Introduction - Sipeed Wiki
Since a riser card for 1 of the slots is very hard to come by, I'm going to use that slot for this KVM solution. I will report on the results of it .
I bought the full version (so with both PoE and wifi). I don't need wifi, but the difference between the only PoE version and the PoE+Wifi version was $2 ($69 vs. $71), so I saw no reason to not have Wifi. I will use PoE to power the KVM, because I can't power it via PCIe ofcourse, because I don't have a riser there. This also gives me the advantage to power on the server when it is off, because via PoE the KVM has power as well, so it can simulate a press on the power button .
This also solves the issue of the machines hanging during the reboot. Then via the KVM you can powercycle the machine, without having to go to the server, which can be useful if you are accessing them remotely (which is my plan).

 

[luckylinux]

For those interested ...

Rails

Offered 10 USD / Set for 8 Sets, got an Invoice and after contacting the Seller, he adjusted Shipping to approx. 23 USD (to Jetcarrier US).

This one probably calls for Sea Shipping since it's 2kg / Piece so 16kg in total for me ...

List Price is 18 USD / Set (already discounted from 24 USD / Set) so I'd say it's quite good

Only 3 Left. I was pretty sure the Seller had another offer which he took down that had more than 11 Pieces. Might be worth to contact him and ask if you are interested in more .

Pardon Our Interruption...

Riser

I just found this Riser which was added Recently:

TYAN M8036-L16-1F - 75 USD for 5 Pieces (15 USD / Piece) - 7 Sets currently remaining

Lot of 5 TYAN M8036-L16-1F Riser card | eBay

Find many great new & used options and get the best deals for Lot of 5 TYAN M8036-L16-1F Riser card at the best online prices at eBay! Free shipping for many products!

www.ebay.com

So 15 USD / Piece plus another I guess ~ 40 EUR for a Package of 2 Sets (10 Risers) to be shipped to Denmark via Jetcarrier, so probably around 20 EUR / Piece overall.

Not a super good Deal compared to the entire Server but considering that Supermicro standard x16 Risers go for about 10-15 USD/EUR it's not too bad either considering these are/might be a bit special. Indeed the PCIe Slot Alignment is different compared to Supermicro PCIe Risers so the Supermicro Risers would likely NOT work in this HYVE Edge Chassis without some Modification.

I just ordered 2 Sets, so I will have 2 extras here in Europe if somebody else is Interested .

The only other "Deal" I found so far on eBay was from China at 30 USD / Piece (DOUBLE !!) of the TYAN M8036-R16-1F one.

EDIT 1: added remaining Quantity for Rails (3 Pieces) and Risers (7 Pieces)

 

[inkey]

if you need a riser for reverse engineering here are some in stock:

Pardon Our Interruption...

I still wait for my delivery of M8036-R16-1F to confirm that they are working.

Looking the pictures it seems like a ordinary x16 riser. Would something like that or that work? My servers arrived recently, but I didn't setup everything yet.

 

[epi1337]

You can try. The lower-cost PCIe 3.0 M7106 riser should also be compatible.

https://xyzinformatica.com.br/datasheet/DataSheet_S8036.pdf