EXPIRED HYVE Edge Metal G10 - Epyc 7642 - 250£/offer

[luckylinux]

Did anybody else experience DIMM Channel Issues ?

On 2 separate Servers populated with 16 x 32GB DIMMs each (RDIMM PC4-2666V), I get an entire Channel NOT detected neither in BIOS nor dmidecode.

In one Case it seems to be Channel B (2x32GB missing), in another Case it seems to be Channel F0 (2x32GB missing).

Sure some DIMMs might be bad (I tried to reseat them once already), but this seems to be a bit of a Coincidence.

In BIOS I set RAM Overclock to enabled and manually set 1333MHz as the target Frequency (so the DIMM will run at their rated Speed, i.e. 2x1333MHz = 2666MHz).

EDIT 1: reseating the affected DIMM (and also saving BIOS Settings forcing RAM Frequency to 1333MHz, i.e. 2x1333MHz = 2666 MHz Operation) seems to have fixed that on one Server (the one where Channel B was affected).

I still need to try on the other Server where Channel F was affected.

 

[luckylinux]

Did anyone find a way to simply reboot these machines from OS? Mine goes to blank screen, fans 100% and just sits untill I do a power cycle.

No, same issue .

You need to remove Power.

Not sure if you can do that remotely with ipmitool using e.g. a Raspberry Pi (or whatever remote machine by sending the Command via ipmitool over the LAN Network).

Otherwise only Option is to have a Relay/Contactor/Smart Plug or possibly (better) a Controllable PDU.

 

[luckylinux]

Whenever i see a host getting stuck there on reboot i log on BMC and do the "reboot BMC" then both BMC and host come back up.
So guessing its BMC glitching that prevents it from booting back up.

Got 3-4 different firmware versions across the ones i got, its not all of them that this happends with but have not looked further at what version those that do tend to get stuck run.
If its just some versions doing it or not.

I did notice that they did not like lrdimm tho.
And with my luck 14tb out of 16tb in the 2 bladecenters i gutted was lrdimm...

Really ? I had no Issues with 8x64GB LRDIMM.

I didn't try with 16x64GB LRDIMM (yet ?) though.

EDIT 1: Complain about your "Luck". With the current RAM Prices you can still make a HUGE Profit out of those. I think I have less than 8tb of DDR4 .

EDIT 2: These RAMs I tested and work

LRDIMM: 8 x Micron DDR4-RAM 64GB PC4-2400T ECC LRDIMM MTA72ASS8G72LZ-2G3B2

RDIMM: 8-16 x SAMSUNG DDR4-RAM 32GB PC4-2666V ECC RDIMM M393A4K40CB2-CTD6Q or CTD6Y or CTD7Y etc

EDIT 3: @Cruzader isn't it maybe an Issue with EPYC not liking SK Hynix RAM ? I remember reading something about it before at least ...

 

[luckylinux]

Might be, it was all from blades bought at the same time with same model/revision but i dont remember what brand it was.

Took 4-5 attempts to get a boot and would freeze after sitting just 5-10minutes with proxmox booted.
Consistently across the 4 i had on the workbench prepping then.


Amusingly the 2 bladecenters are just jammed stuck in the racks in one of our DCs, the replacement hardware above both were left with all its weight down on them.

Otherwise they would have been thrown away with the 16tb in them a long time ago.

We did not need the space and nobody wanted to start shifting the weight above so they were just left there unplugged full of blades.
I pulled the blades and the chassises are still sitting there, probably will be intil we replace racks.

I didn't try Proxmox, just Ubuntu (or was it Debian, I cannot remember).

Could be you are one of those "rare Cases" as Proxmox Forum Staff usually replies, who experience an Issue they cannot replicate (and is probably related to Ubuntu Kernel and/or one of their Patches on top of it).

 

[luckylinux]

For what it's worth.. on my boxes proxmox was stable. Which makes sense since it's simply debian 13 + ubuntu kernel.

"Simply". Way too many Ubuntu Kernel Bugs seen on Proxmox.

Sure, they don't always affect the same Users, but they DO happen.

When everything else fails, running the Debian Kernel usually fixes the Issue (even though it's not supported / recommended, it's not like they give Users any Alternative).

 

[luckylinux]

OK so I just tried the King Slide Rails I ordered and ... #### .

The Inner Rails already fitted to these Servers don't seem to work with the King Slide Outer Rails . Not only that, this King Slide Screwless release Latch (even when you fit the single "Backup" Screw), automatically unlatched as soon as I try to slide the original Inner Rails in, they jam 5 cm in, then while pulling out one of the outer Rails completely unlatches at the front and almost falls down.So the Backup Screw is basically not a Screw since it comes out together with the holder as soon as the Latch is lifted (either by Hand or when trying to get a jammed Server out).

On the other Hand it would seem that the King Slide inner Rails do NOT fit the Server mounting Holes and are also quite short.

What am I doing wrong ? Incompatible Rails ?

https://forums.servethehome.com/index.php?threads/hyve-edge-metal-g10-epyc-7642-250-offer.50844/page-5#post-480891

I thought also @Cruzader bought them. Did you have any Issues with yours or you didn't try at all ?

Also a Note ... when releasing the back Side Latch (if you want to get the outer Rails out for repositioning them etc), don't lift the tap too much, just a tad is enough. I lifted it all the way and since they used some cheap plastic instead of a real Spring or something, it wouldn't spring back the Way it's supposed to be. I kinda straightened out by force, reinstalled and it kinda works, but it's one thing to keep in mind.

EDIT 1: Picture



The inner King Rails as part of the Inner+Outer Rail Set are also like double the Thickness (well not of the material, but rather the Profile), even though they are shorter.

To say these were compatible was such a huge overstatement .

EDIT 2: inner rails that came with the server are on the Outside, inner rails as part of the King Rail inner+outer kit are on the inner Side

EDIT 3: do I need to start from Scratch, drill Holes in the Server and/or inner Rails ? I'm just tired I'd like to get SOMETHING going without getting stuck again and again .

EDIT 4: @ribroc, any Suggestion ?

I was re-reading your Post at https://forums.servethehome.com/ind...-epyc-7642-250-offer.50844/page-4#post-479866 but I'm not quite sure how the inner Rails being extremely specific to the Chassis is even possible with fully generic Outer Rails.

What should I exactly do ? You mentioned something about using a pair of Pliers .... do I need to use the inner Rails that are part of the Kit (thus throwing away the inner Rails that came with the Sever) ? Or do I need to use the inner Rails that came with the Server, even though they don't seem to fit ? I'd say they are slightly too large to fit correctly in the outer Rails though, unless I'm doing something very wrong .

 

[luckylinux]

@ribroc, @Cruzader: any Idea about the Rails ?