Gigabyte MJ11-EC1 EPYC 3151 Mystery

[BadMax02]

EPYC Zen1 have problems with PCIe ASPM. You need to deactivate it in "/etc/kernel/cmdline" with "pcie_aspm=off". After this you need to do a "update-initramfs -u" otherwise the change will not used by the system. In UEFI the ASPM settings are unfortunately missing.

ive messed around with this a lil more after installing proxmox a few weeks ago,
for me those spamming errors went away when i turned the sata hotswapping into mode 2, others worked for me aswell but i cant recall which ones made the console spam those errors and which one didnt.

I followed the kernel parameter found here: AMD EPYC Server with Ubuntu - Enable SATA Hot-Swap - Thomas-Krenn-Wiki-en

that way u still retain the advancedpcie error reporting but dont get a spammed console, seems to be working perfectly fine for me with mode 2, but i didnt test hot swap since i dont really need it

 

[toni3131]

@hmartin

I can't stop the board with the power button. It seems to be starting automatically once the PSU receives voltage. Probably some setting in the BMC which I didn't find ?

I've tried resetting the BMC configuration to default. Also tried flashing the F09 default BIOS I've found here, it uploads but then nothing happens (stuck at Uploaded 100% for more than 15-20min).

Provide some screenshots. What can you do? Have you tried factory resetting the BMC from the web management interface?

I have the usual BMC options from the Power Control menu. The BMC sees that the board is powered on and if I choose any of the other options (Power Off, Hard Reset, etc) and click Perform Action, nothing happens. Keeping the power on physical button pressed for about 20s does nothing.

 

[ekke]

Where can I find new bios for this MB the 126117.bin ? I tried searching for it here but no luck, bios is f09 according to the bios post screen

I can only get ram two work in the first two slots, the blue ones. anyone got quad rank to work ? I have 4x8GB reg ecc 2400 that Id like to use

 

[hmartin]

I can't stop the board with the power button. It seems to be starting automatically once the PSU receives voltage. Probably some setting in the BMC which I didn't find ?

Do you have a multimeter? Verify that the ATX to 4 pin adapter (the tiny black connector) has all the wires connected. There have been previous posts from people who received adapters with loose/broken connections.

It sounds like you either have:

1. Defective ATX adapter (e.g. some signal like PSU power good is spotty)
2. Defective PSU
3. Defective motherboard

 

[toni3131]

I tested with 2 different PSUs (One brand new and one older). It has the same behaviour.

For the ATX adapter, I receive from PSU without the 4pin small connector connected to MB:

- Black pin -> Ground
- Red pin -> 5V
- Grey pin -> 0V
- Green pin -> 5V

When the 4pin connector is connected and power supply button is ON:

- Black pin -> Ground
- Red pin -> 5V
- Grey pin -> 5V
- Green pin -> Ground

Also tested the continuity on the adapter at both ends and looks ok.

 

[benis]

SlimSAS-4i is already populated, you just need to cross-flash the EC0 BIOS to be able to configure it as PCIe (I believe there is also a modded EC1 BIOS mentioned previously in this thread).

In summary:
NVMe (PCIe x4, adapter required if you plan to use standard PCIe instead of M.2)
SlimSAS-4i (PCIe x4, cable and adapter required)

The SlimSAS-4i is for the additional SATA or PCIE lanes (the connector close to the SATA-ports)? If yes, do I need to cross-flash the bios to enable the PCIE lanes?

 

[dreunion61]

The SlimSAS-4i is for the additional SATA or PCIE lanes (the connector close to the SATA-ports)? If yes, do I need to cross-flash the bios to enable the PCIE lanes?

Yes, just take it from Gigabyte site: MJ11-EC0 (rev. 1.2) | Server Motherboard - GIGABYTE Global

And flash the rbu file via BMC. After that you can also unlock the other FAN headers curve controls by applying this:

https://forums.servethehome.com/index.php?threads/gigabyte-mj11-ec1-epyc-3151-mystery.41395/post-407547

EDIT: Before you do anything of that above, you should update the BMC first:

https://download.gigabyte.com/FileList/Firmware/server_firmware_ast2500_AMI_12.x.zip?v=dd2b8770005fd089b339935977271d64

For instance, I could not dump my OG bios before the BMC update.

 

[benis]

Yes, just take it from Gigabyte site: MJ11-EC0 (rev. 1.2) | Server Motherboard - GIGABYTE Global

And flash the rbu file via BMC. After that you can also unlock the other FAN headers curve controls by applying this:

https://forums.servethehome.com/index.php?threads/gigabyte-mj11-ec1-epyc-3151-mystery.41395/post-407547

EDIT: Before you do anything of that above, you should update the BMC first:

https://download.gigabyte.com/FileList/Firmware/server_firmware_ast2500_AMI_12.x.zip?v=dd2b8770005fd089b339935977271d64

For instance, I could not dump my OG bios before the BMC update.

Thanks a lot! All seems to work fine with dumping and flashing. Now just waiting for adapter and cable.

 

[benis]

How far have you managed to push the ram speed? I just tried 2666 (on 2400 ram). dmidecode -t memory do report higher speed but GB6 scores are not changed at all, hum. Wanna break that 1K single core score (just for fun).

Update: Yeah, 2666MHz is max. Not possible to overclock the ram as far as I can tell.

 

[Zenfan90]

Hey fellow home server enthusiasts

I have bought a Gigabyte Mj11-EC1 previously bought at Ram König for 35 Euro and 5 Euro shipping a good deal and probably working because sold via Ram König previously but without the ATX adapter.

That is a bit of a problem because I can not find the adapter anywhere.

It seems Ram König makes the adapter for the boards they will sell but yeah this one was not so lucky to have it and they say they do not sell it separately and gave me that instruction

https://files.ram-koenig.de/information/649/power-guide_de.pdf

Well I can not solder and I have another idea so if you would be so kind and tell me that my idea is not crazy.

I can just buy an ATX 24 pin extension male to female and then cut of or rip the not needed wires off and the 4 I need insert directly to the 4 pin socket and secure with electric tape for security and never probably touch it again hopefully?

 

[binarycodes]

Hello fine folks of the internet,

I ordered the MJ11 from the ebay (same seller), have gone through the threads about twice now. But I still have a question.
Pardon my ignorance about all things SAS.

I saw that it is already feasible to convert the M.2 nvme to a PCIex4 and then connect a 10GB SFP+ X520-DA2 to it. ( Not enough bandwidth to push two 10GB ports but one should be fine. And I wont have enough IOPS to push 10GB anyways at the moment ... planning 1 vdev)

Where I need some help/guidance/confirmation is this - The slimsas 8i to PCIex8 adapter should be able to run this LSI hba (https://amzn.eu/d/c8cjO6d) to get me another 16 drives, right?

 

[etorix]

A HBA is a PCIe device. If you have the right cable and adapter, you can drive it from this SFF-8654 8i.
But, frankly, at the point where you're adapting both SFF-8654 ports into PCIe slots, you'd be better building your NAS from a MC12-LE0: That will be easier and cheaper.
Or a Supermicro X10SDV (-7TP4F), Supermicro A2SDi-H-TF (-TP4F), Gigabyte MB10 board.

Other tip: If you need a -16i HBA, the 9305 is better than the 9300: Single controller, smaller card, less power.

 

[binarycodes]

Yea this board is just too esoteric but its cheap! Even with the adapters here in Finland it turns out cheaper than those Supermicro ones. The MC12-LE0 would have been nice but yea I don't know what madness drove me to MJ11. Maybe its the thrill of fighting the board.

Thanks for the tip on the 9305. This is my first time charting SAS waters (just the cables though) and this thread have been immensely helpful. Without this thread I would have bought the MC12-LE0 ...

PS: I have my eye on the MB10-DS4 (the Datto OEM one) as the fallback option should my experiments with this board fail. Since I can re-use the same RDIMM sticks without spending a fortune on the supermicro boards.

 

[etorix]

Without this thread I would have bought the MC12-LE0 ...

You really should have. HBA in x16, NIC in x4, boot from M.2 x1… done.
Maybe reading the MC12-LE0 thread first would have helped?

I have my eye on the MB10-DS4 (the Datto OEM one)

Which is not really a DS4, but the sellers take this shortcut. This board reportedly has the drawback of having Java-based IPMI.

This is my first time charting SAS waters (just the cables though)

Mind that "MiniSAS", "SlimSAS" are just fancy names for connectors; no protocol is implied.
Your build list appears to suggest a NAS build with more than 8 drives but no proper SAS backplane with expander—possibly a build in a big ATX tower, or in a "rack for non-rack users" (e.g. Silverstone). This is a road with many ambushes, notably around cabling and cooling (major issue).

 

[toni3131]

I did not solve the issue with the board. The seller sent me a new one which works out of the box with the same RAM, PSU, etc.

I've tried to look up any difference between the working one and broken one, pins, swiches, broken pins, etc but did not find any issue.

Maybe the BIOS chip may be broken.

 

[binarycodes]

Mind that "MiniSAS", "SlimSAS" are just fancy names for connectors; no protocol is implied.
Your build list appears to suggest a NAS build with more than 8 drives but no proper SAS backplane with expander—possibly a build in a big ATX tower, or in a "rack for non-rack users" (e.g. Silverstone). This is a road with many ambushes, notably around cabling and cooling (major issue).

That bit was hitherto undecided, but just ordered the Jonsbo N5. So its a little less hassle power wise (fingers crossed - everything fits nicely)
I am not trying to add 8+16 drives upfront, since this whole SAS/HBA/Backplane thing is new to me, I was trying to figure out what my options are.


And, yes that DS4 thingy is an odd Datto OEM board. Some complaints on the internet about IPMI calling home, etc. so not really keen on it. Lets hope this board works out fine. I am worried about the memory working since we have had so many complaints in this thread itself. Fun times once the board arrives.

 

[etorix]

just ordered the Jonsbo N5

Ohh, that's a funny one. E-ATX, no less. Plenty of air on the top; I'm more skeptical about the lower 120 mm fans moving air over their 8 HDDs, which are quite far away. As for the other 4 HDDs stacked vertically in another carrier in front of the PSU , I'm quite confident they are NOT going to get some air. I also would NOT want that on my desk, filled as in some advertising pictures with a dual Xeon motherboard and a full complement of GPU cards…

Please report how cooling goes.

 

[Th0mas51]

Hello guys, I managed to replace the stock cpu heatsink of this motherboard!

I use this mb in a jonsbo N1 case, running with the stock heatsink fanless the CPU temps were around 60°C using the lowest frequency possible (1.2GHz).
So inspired by this blog post: HOME LAB PART 5: The Ultimate Small Form Factor Server for Home Lab
I modified his design to fit a noctua NH-L12S to this SP4r2 socket and had it 3D printed. To mount it on I used the screws and springs harvested from the original heatsink, as the backplate does not look easily removable.

The material used for the print is PA 12 (nylon), which may not the best, but should perform better than PLA (and was cheap).
The best material for this use case would be PC (polycarbonate) for better heat and stress resistance, but is more expensive to have it printed..

I'm happy with the result, the mb and heatsink fit like a glove in the case, and the CPU temperature went down under 40°C (fan spinning at 1000rpm barely audible, ambient temperature of 27°C).

I leave the zipped 3D model here if anyone is interested, it should work with other noctua heatsinks of similar design, since they look like they have pretty much the same cpu contact surface layout.

Hi !

Thanks for your 3d model.

I got the heatsink adapter printed using ABS with carbon fiber, and it looks like either Noctua has slightly increased the width of the heatsink, or maybe I printed in a material that is not flexible enough, as I had to force the adapter onto the Noctua heatsink and the plastic broke.

Sadly I don't know how to edit an STL file, but I think it would only need maybe 0.5 mm extra in order to fit correctly without breaking.

If by any chance you are able to edit the wholes to add 0.5mm, that would be amazing !

Thanks.

Thomas

 

[iridial]

hey, good to know it was useful for someone else
wrong size is either due to printing material or heatsink different size,
I added 0.5 mm to the cavity height, you can see in the screenshots the measures before and after.
My advice however is to try to print it once in PLA, to see how it fits. Also by how the crack looks in your photo, it shows that was printed in the wrong orientation.
It should be printed with the cavity facing upwards, for better strength.

 

[Th0mas51]

Thanks for the new model, @iridial !

I'll ask the person that printed the previous model to print this new one with the correct orientation.

If someone is curious, I have measured the size of the part of the heatsink that needs to fit in the cavity.

I will let you know how the new model fits.