Gigabyte MJ11-EC1 EPYC 3151 Mystery

[user9482389]

Hello,

Attached is the original F09 Bios that I dumped from my system and made into a flashable .rbu image. It is not possible to flash back the dump as is via the WEBGui or AFUxxx. I compared to the MJ11-EC0 image from Gigabyte site and found the difference. There are a lot of system specific NVRAM variables in the dump. In a hex editor I exchanged these with FFs. Empty space is all bits set. Adding the rbu 10 bytes at the end with the right checksum made it flashable. You can check this with your own BIOS dump. From hex 00294000 they should be identical. There is a part that UEFITOOL calls padding that is not empty and that differs. I have flashed BIOS for one motherboard to another without problems.

If you instead prefer to modify your dumped BIOS I can give instructions to do so.

You should flash this from the BMC with the main CPU off. After the flashing when you restart, the system will reboot several times before coming to the BIOS screen.

Of course be warned that all flashing can potentially go wrong and you will have to take all responsibility

I have uploaded my BIOS to dropbox. here is the link

Dropbox

www.dropbox.com

The sha256 checksum is f7eeaeefa17fe98ec56007c24aa50e6e392b2fca862b4330a33a42d91e5b42f3

Peter

I tried flashing this file via Maintenance > Firmware Update > Selected your rbu file > Update type: BIOS > Proceed to flash.
It just uploads to 100% and does not start the flash process. Is there something I am missing?
Also tried flashing via AFUWIN, but that did not work either.

 

[PeterF]

I tried flashing this file via Maintenance > Firmware Update > Selected your rbu file > Update type: BIOS > Proceed to flash.
It just uploads to 100% and does not start the flash process. Is there something I am missing?
Also tried flashing via AFUWIN, but that did not work either.

Hello,
Are you on an old BMC version or one of the latest?
I would also try flashing from the UEFI prompt with AFU for EFI.
If flashing with AFU you need to remove the last 16 bytes of the file to transform it to a .bin file

BR
Peter

 

[user9482389]

Hello,
Are you on an old BMC version or one of the latest?
I would also try flashing from the UEFI prompt with AFU for EFI.
If flashing with AFU you need to remove the last 16 bytes of the file to transform it to a .bin file

BR
Peter

Thank you for your fast response!

I already did truncate those 16 bytes for AFUWIN, but got the verification failed error.
With those 16 additional bytes, I obviously got a size mismatch error.

I am running on BMC version 12.49.06. I will update the BMC and try again to flash.


The reason for reverting to the EC1 BIOS is an issue I have with this board.
I have an ASM2812X PCIe Switch for 4 NVMe SSDs, but the system refuses to boot with this card installed, even though the SSDs are recognized in the UEFI.
I'm hoping that maybe the EC1 that came with a PCIe Switch (I think the ASM2824) supports this.

 

[user9482389]

Updating the BMC to 12.61.21 fixed the flashing issue from the web interface.
The ASM2812X PCIe switch card worked after flashing back your original F09 image (it did not with just the updated BMC).

So there is a drawback of running F02 on this board, it will brick compatibility with the ASM2812X and possibly other PCIe switch cards.

I now have 4x NVMe devices running in this Sabrent PC-P3X4, which is connected to the M.2 port via a riser (ADT-Link R43UH 15cm, which fits absolutely perfectly!).
The SFF-8654 8i runs a cheap 2.5Gb card, which is also connected via a riser. Could easily be upgraded to 10Gb when necessary.
Payed around a total of 60€ for both riser cards and the SFF-8654 8i cable, which is not too bad in my opinion!

Thanks @PeterF for your help and for solving the rbu file mystery!

 

[PeterF]

Updating the BMC to 12.61.21 fixed the flashing issue from the web interface.
The ASM2812X PCIe switch card worked after flashing back your original F09 image (it did not with just the updated BMC).

So there is a drawback of running F02 on this board, it will brick compatibility with the ASM2812X and possibly other PCIe switch cards.

I now have 4x NVMe devices running in this Sabrent PC-P3X4, which is connected to the M.2 port via a riser (ADT-Link R43UH 15cm, which fits absolutely perfectly!).
The SFF-8654 8i runs a cheap 2.5Gb card, which is also connected via a riser. Could easily be upgraded to 10Gb when necessary.
Payed around a total of 60€ for both riser cards and the SFF-8654 8i cable, which is not too bad in my opinion!

Thanks @PeterF for your help and for solving the rbu file mystery!

Good to know that you got it working! and your cards work correctly!

 

[iridial]

Hello guys, I managed to replace the stock cpu heatsink of this motherboard!

I use this mb in a jonsbo N1 case, running with the stock heatsink fanless the CPU temps were around 60°C using the lowest frequency possible (1.2GHz).
So inspired by this blog post: HOME LAB PART 5: The Ultimate Small Form Factor Server for Home Lab
I modified his design to fit a noctua NH-L12S to this SP4r2 socket and had it 3D printed. To mount it on I used the screws and springs harvested from the original heatsink, as the backplate does not look easily removable.

The material used for the print is PA 12 (nylon), which may not the best, but should perform better than PLA (and was cheap).
The best material for this use case would be PC (polycarbonate) for better heat and stress resistance, but is more expensive to have it printed..

I'm happy with the result, the mb and heatsink fit like a glove in the case, and the CPU temperature went down under 40°C (fan spinning at 1000rpm barely audible, ambient temperature of 27°C).

I leave the zipped 3D model here if anyone is interested, it should work with other noctua heatsinks of similar design, since they look like they have pretty much the same cpu contact surface layout.

 

[Grinchy]

Was someone able to change fanspeed for this board with ipmitool commands?

Was thinking about creating an fan script for Linux, but I can‘t get it working.


this is what I found at Reddit, but it doesn‘t seems to work :-(

For fan settings we have below information:

Below IPMI command format for fan control.

Set Fan Control 0x3c 0x16

Byte1: 0x02

Byte2: FAN_NO

Byte3: Duty1

Byte4: Duty2

Byte5: Duty3

Byte6: Temp1

Byte7: Temp2

Byte8: Temp3

Get Fan Control 0x3c 0x16

Byte1: 0x03

Byte2: FAN_NO

Ex. Get fan 1 status

$ Ipmitool -I lanplus -H 192.168.50.43 -U admin -P admin raw 0x3c 0x16 0x03 0x01

– 01 0a 32 64 14 50 5f

Ex.Set fan 1

$ipmitool -I lanplus -H 192.168,50.43 -U admin -P admin raw 0x3c 0x16 0x02 0x01 0x14 0x32 0x64 0x14 0x50 0x5f

- 01 14 32 64 14 50 5f



Regards,

GIGABYTE

 

[BudSpencer]

Hello, recently I have acquired this server and have followed the instructions to power it according to this link: https://files.ram-koenig.de/information/649/power-guide_de.pdf. I have a doubt whether it is necessary to use the 4-pin connector as indicated to power the motherboard, or if an 8-pin PCIe connector can be used instead. This is mainly because the power supply that I want to use does not have the 4-pin connector, and I wanted to avoid having to buy some Molex or PCIe to 4-pin adapter. Thanks in advance for your help!

 

[Grinchy]

Hello, recently I have acquired this server and have followed the instructions to power it according to this link: https://files.ram-koenig.de/information/649/power-guide_de.pdf. I have a doubt whether it is necessary to use the 4-pin connector as indicated to power the motherboard, or if an 8-pin PCIe connector can be used instead. This is mainly because the power supply that I want to use does not have the 4-pin connector, and I wanted to avoid having to buy some Molex or PCIe to 4-pin adapter. Thanks in advance for your help!

Just like writen there, you need the 4pin Port for 5v (taken from the PSU ATX Port), and the 12v 4/8 pin CPU Connector.

 

[BudSpencer]

Hello, as I mentioned earlier, I recently acquired this system along with 32 GB RAM (2x16GB). I installed TrueNAS Scale on an M2 drive and I don't detect any issues. By default, it came with BMC version 12.49.06 and BIOS G431-MM-OT 9/14/2021 F9 installed. I have not updated them yet in case I find an error and need to return the product.

I tested the memory using MemTest86 and it found errors in both modules. Below are the details of the tests performed:

RAM modules:

SK Hynix China 16GB 2Rx8 PC4 - 2666V - RE1 - 11
SK Hynix Korea 16GB 2Rx8 PC4 - 2666V - RE1 - 11

Motherboard slots labeled as per manufacturer documentation:

DIMM_P0_B1
DIMM_P0_B0
DIMM_P0_A1
DIMM_P0_A0

Memory test results:

DIMM_P0_B1 => SK Hynix China => ERROR
DIMM_P0_B1 => SK Hynix Korea => ERROR

DIMM_P0_A1 => SK Hynix China => OK
DIMM_P0_A1 => SK Hynix Korea => ERROR

From the manufacturer's manual, page 26, I found the following note:

"When populating DIMMs into a channel, slot numbers having the suffix '1' must be populated first, followed by slot numbers having the suffix '0'."

If I try to populate RAM modules in channels 0, none of them are detected correctly and the memory test can't be performed:

DIMM_P0_B0
DIMM_P0_A0

I can't determine if the problem is caused by the motherboard, the RAM modules, or both. In theory, if the RAM module "SK Hynix China" works with no errors on DIMM_P0_A1, it should have the same result on DIMM_P0_B1, but it fails.

This is an example of the errors on MemTest86:



I have initiated the return process, but if anyone knows why these types of errors are occurring or if there are any tests I can perform that would provide the necessary guarantees that the product is not defective, I would stop the return process.

Thanks,

 

[Grinchy]

I would try to put the Speed to 2133MHz instead of 2400MHz. Also Samsung Ram seems to works better with this Board.


Someone got Hot Swap working with this Board? Trying it with TrueNAS Scale, but it won't recognize the HDD until restarting.

 

[jmy90]

Hey guys, I have been using this board for my home NAS(TrueNAS Scale) But I am experiencing some random NIC issues I am unable to troubleshoot.
When I have some traffic either spike or torrent download for instance my IGB I think i210 just disapears and I lose connection... The only way to restor it is to reboot the system. No driver or interface reload is helping only hard reset.
This is pretty anoying as it happens randomply and it may not accure for days.
Anyone faced such issues? TrueNAS scale is running debian 12

 

[Grinchy]

Are you using something like powertop auto-tune or pcie_aspm.policy=powersave?

It seems like this can cause problem with PCIe and NICs

I’m using it with TrueNAS for some time now, and never had problems with the nic.

 

[jmy90]

I have aspm off and no policy

Are you using something like powertop auto-tune or pcie_aspm.policy=powersave?

It seems like this can cause problem with PCIe and NICs

I’m using it with TrueNAS for some time now, and never had problems with the nic.

GRUB_DISTRIBUTOR="TrueNAS Scale"
GRUB_TIMEOUT=10
GRUB_DISABLE_RECOVERY="true"
GRUB_CMDLINE_LINUX_DEFAULT="libata.allow_tpm=1 amd_iommu=on iommu=pt kvm_amd.npt=1 kvm_amd.avic=1 intel_iommu=on zfsforce=1 nvme_core.multipath=N pcie_aspm=off pci=nomsi pci=noaer"
GRUB_TERMINAL_INPUT="console"
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX=""

In fact if i dont use pcie_aspm=off pci=nomsi pci=noaer I cannot even boot it I get tons of pci errors that scrolls very fast
could be the nvme that i have installed that is causing the issue ?

what test i can run to check it

 

[Grinchy]

In fact if i dont use pcie_aspm=off pci=nomsi pci=noaer I cannot even boot it I get tons of pci errors that scrolls very fast

This errors are normal with epyc and Linux.

I would highly recommend to use pcie_aspm=off and not noaer, cause this will just turn off error messages for pci. So if errors occur, you wouldn’t even see them.

I’m using this:

midclt call system.advanced.update '{"kernel_extra_options": "pcie_aspm=off ahci.mobile_lpm_policy=0"}'

I never heard that SSDs can cause this kind of problems.

 

[jmy90]

It is a mystery to me. The thing is that it may not happen for weeks. When it happens the only way to recover is to connect to the mgmt and do a restart ...
What I have noticed is that it happens when I download torrents with multiple files. Like a whole show season.
Maybe I should try to update or reinstall drivers.

 

[jmy90]

I was able to resolve the issue. It was the PSU... swapped the psu and no more random reboots...

 

[ekke]

The board I got, uses a standard 6 pin PCIe.

https://www.aliexpress.com/item/1005005670883185.html

BTW. I ran some iperf3 tests and did ZFS replication of most of my data, no issues so far on ConnectX-3.

Hi! Could you post a picture of how you mounted the NIC on the pcie adapter in the chassi plz


Btw, do you guys think that this adapter would work:
SFF8654 to PCIe x16 Adapter: SFF-8654 8i to PCIe 4.0 x16 Adapter for Mainboard DIY Computers Accessories | eBay

 

[Grinchy]

Hi! Could you post a picture of how you mounted the NIC on the pcie adapter in the chassi plz


Btw, do you guys think that this adapter would work:
SFF8654 to PCIe x16 Adapter: SFF-8654 8i to PCIe 4.0 x16 Adapter for Mainboard DIY Computers Accessories | eBay

i’m using This for Months now. It’s still working Great.

 

[cookiemonsteruk]

Hi. I've seen someone asked on page 14 if SAS drives are supported by the board and I saw the answer is no. However I guess he was asking about the onboard SATA connectors.
I would want to put an HBA, an IBM M1015 flashed to LSI9211-IT that I have into the PCIe slot to connect to the backplane of my case that holds 24 disks. They are 3.5" spinners SAS and I only happen to have about 8 disks populating the caddies. This configuration should work right?
The MB doesn't have to worry about the disks, just to be able to identify the HBA, and that in turn deals with the disk and SAS commands.
Am I overlooking something?
p.s. regarding power, I'd be using the CPUs of the case and they are enough for the load. Thanks for the assistance.
Edit: I've found my answer by looking again at the QVL. There are HBAs there so all is good.