X11DPH-T / Please help understand and update BIOS/BMC

[Kitem]

So, after some troubleshooting, I finally reached the light out of the tunnel.


And the solution was so easy that I felt ashamed.

1. This is my first double CPU MB. I usually connect only the 24-pin cable from the PSU to the MB. But this MB has also two 8-pin power supply connectors. Once connected to the PSU, I had no more issues in booting or with the FANs.
2. Sensors. I'm quoting a statement found on the web
   

   the IPMI sensors never worked while the computer sat in BIOS Setup; they did work when booted to FreeDOS on a USB flash drive. Several of the Supermicro FAQs address this issue: only 1 program / service can directly monitor the hardware sensors at a time.

   Now that I boot the system from USB and loaded the proxmox installer, I can see the sensors being shown in the IPMI (well, not all of them but it's a great improvement )
   
   

   http://imgur.com/oTFJMnU

Thank you all for your precious help.
I will for sure bother you again while on my learning curve

EDIT: I forgot to ask: Does this mean that the RAM module is defective and I have to replace it?

http://imgur.com/a/9oAvbeM

 

[RolloZ170]

EDIT: I forgot to ask: Does this mean that the RAM module is defective and I have to replace it?

bad contact (DIMM slot, CPU socket) bad RAM, wrong RAM (need ECC, and registered for 2nd gen)

 

[Kitem]

Strange....I have 8 modules...all the same

P1-DIMMC1

    Max Capable Speed: 2666 MHz
    Operating Speed: 2666 MHz
    Size: 32768 MiB
    Serial No.: 14A4EAB3
    Part No.: M393A4K40BB2-CTD
    Manufacturer: Samsung

P1-DIMME1

    Max Capable Speed: 2666 MHz
    Operating Speed: 2666 MHz
    Size: 0 MiB
    Serial No.: 14A4FD61
    Part No.: M393A4K40BB2-CTD
    Manufacturer: Samsung

 

[RolloZ170]

Strange....I have 8 modules...all the same

so bad RAM, bad contact (DIMM slot, CPU socket)
swap the C1 with E1
if error persists in E1:
reseat CPU

 

[Kitem]

how do I reset CPU?

 

[RolloZ170]

how do I reset CPU?

reseat CPU (remove and insert carefully again)

 

[Kitem]

I reset the CPU, changed place, and tried installing it to the other CPU....... The same message

P1-DIMMEx

    Max Capable Speed: 2666 MHz
    Operating Speed: 2666 MHz
    Size: 0 MiB
    Serial No.: 14A4FD61
    Part No.: M393A4K40BB2-CTD
    Manufacturer: Samsung

It's linked to the RAM module, not to the socket/CPU

I guess I have to change it.

 

[RolloZ170]

It's linked to the RAM module, not to the socket/CPU

sure if you insert this DIMM in other slot, and the error moves.

 

[Kitem]

While I wait for the new RAM module, I have another question about the I-SATA 0-3/4-7 ports.

As far as I understand, the cable I need to connect to the 4/8 Sata disk is an SFF-8643.
However, it's too small to fit the I-SATA port on the MB.

Do you know if I need an adapter or if the SFF-8643 is wrong?
In this case, which one do I have to use?

Thanks

 

[RolloZ170]

Do you know if I need an adapter or if the SFF-8643 is wrong?

there are cables SFF-8087 to 4x Sata

 

[Kitem]

I just lost another RAM module. I bought them on eBay, but do you think these are just bad modules or something else destroying them?

http://imgur.com/a/Jj8aLe7

 

[CyklonDX]

they could've been bad; or its pumping too much voltage/amp into them.
*(best to run memoy test on new ram to be sure there are no issues with it)

 

[nexox]

If two DIMMs from a batch failed that quickly I would return the whole set.

 

[Kitem]

they could've been bad; or its pumping too much voltage/amp into them.
*(best to run memoy test on new ram to be sure there are no issues with it)

What software do I have to use to run the memory test? Can I do it by installing Linux or Windows on a VM?

EDIT:

I'm running MemTest86 from an external drive.
It tests 191 GB of memory out of the 256G installed.

If I understand correctly, 256 - 191 = 65 GB

The system uses one, and 64GB are the damaged ones, which will not be tested

EDIT2: After 4h it completed the first Pass.
I know I have to run at least 7/8 passes, and I may run them tonight.
However, how can I test the failed ones?

http://imgur.com/a/kLDQ5nj

 

[CyklonDX]

You don't need to test failed one as its already on post returning too many errors and system is disabling it for you so you don't crash.

Typically strict people run 100 loops to be sure if memory is good or not. Normal just wait for their system to disable the ram (see it in logs) before they replace it. (or report of increased error rate if its ecc)

TechPowerUp

MemTest64 is a free system memory stability and error detection software for all Intel and AMD processors. It runs on 64 bit Windows without installation.

www.techpowerup.com

here's a live one for windows.

Unraid has its own live memory tester
You can test memory using dd in loops to mem too, but you may miss some parts.


Best way to test memory is to load stuff in them, so run a kvm or zfs on linux and assign all memory to it; for windows hyper-v should do. If there are issues with sticks they should pop right away with increased memory errors.

 

[Kitem]

When I replace the RAM modules, I know I have to buy the same GB/Mhz/DDR.
However, can the brand be different?

 

[CyklonDX]

For server ecc ram, yes (they all share the same timings, and Hz stages); For non-std consumer grade ram you have to buy same as they can have all kinds of timings, at different Hz stages.

 

[Kitem]

ok thanks
Since they are ECC, I can buy a different brand.
But what if the size/MHz are different?
Can it cause a problem?

I have another question (sorry, but still learning )

I see that the Xeon Gold 6138 can handle up to 48 PCi Express lanes.
Since I have two of them, am I right assuming I can use up to 96 lanes?

Well....I have a total of 16x3 + 8x4 = 80 lanes
Can I use all of them?

Again....thank you very much for your patience and support

 

[CyklonDX]

But what if the size/MHz are different?
Can it cause a problem?

You can still manage - sometimes - depending on motherboard support, its not recommended at all though.
You need to make sure different sizes are in different channels, if clocks are different its likely going to result in running at the slowest ram speed.
You should never mix same memory channels with different sizes or clocks. (you will either not post, post but it will take long time for memory training, some channels might get disabled and you will loose available memory, and/or have all kinds of odd issues.)

All this will likely impact your system performance; from just booting to actual ram performance - and hinder your cpu performance as well due to uneven memory channels. Best to get matching memory in size and clocks.

Since I have two of them, am I right assuming I can use up to 96 lanes?

You should look at your motherboard manual, before making assumptions.
While cpu's can have x amount of pcie lanes - doesn't mean motherboard will use them or allow you to use it.

https://www.supermicro.com/manuals/motherboard/C620/MNL-1912.pdf

*page 12 for pcie slot
*page 17 for full layout.

 

[Kitem]

Thanks again for taking the time to answer my (noob) questions

You should look at your motherboard manual, before making assumptions.
While cpu's can have x amount of pcie lanes - doesn't mean motherboard will use them or allow you to use it.

https://www.supermicro.com/manuals/motherboard/C620/MNL-1912.pdf

*page 12 for pcie slot
*page 17 for full layout.

I did check the manual before writing, but at a second look, I realised that CPU2 will manage the 3 slots X16 while CPU1 will manage the 4 slots X8
So, I guess I can use all of them