note if the ECC data is faulty written in the RAM, it comes faulty out.
if bits are flipped by radiation, the quality doesn't matter.
LGA 1700 Alder Lake "Servers"
- Thread starter tjk
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
if bits are flipped by radiation, the quality doesn't matter.
What I meant was: If I have a module with one correctable error per year it is much better than a module giving one correctable error per week. Obviously, due to environmental conditions this must always be seen in comparison to other modules in the same machine. Radiation should hit all modules in more or less the same way while quality of RAM cells might differ between modules.
On a side note, do you mind checking IOMMU groups? I am wondering how well separated they are on that motherboard.
Has anybody a confirmation on that? On the SuperMicro page, the board is still listed as "12th/13th generation".
Since 14th generation is just a speed bump to 13th generation without any fundamental changes (except the core configuration on some SKU has changed), I'm curious what happens when inserting a 14th gen CPU into X13SAE with Bios 2.x. Has anybody tried that?
included MCUs just updated. but 14th gen is same stepping than 13th.
2.1
90672(90675,B06F2,B06F5) rev.2C
90671 rev.113
3.0
90672(90675,B06F2,B06F5) rev.32
90671 rev.11E
SM support just answered that BIOS 3.0 supports the latest generation of CPUs... e.g., Raptor Lake Refresh.
However, for those interested in X13SAE-F, according to web site, 3.0 is (so far) only released for X13SAE. For X13SAE-F, it still offers 2.1.
Since I do not need BMC (it only increases idle power), this is no problem for me so my next workstation will be X13SAE based.
However, for those interested in X13SAE-F, according to web site, 3.0 is (so far) only released for X13SAE. For X13SAE-F, it still offers 2.1.
Since I do not need BMC (it only increases idle power), this is no problem for me so my next workstation will be X13SAE based.
I'm very disappointed of x13sae-f, for iommu group and other strange behaviors. coming from a X10SRA-F (test on pve 8.0.9 clean install)
USB is all on the same group, is not possible to pass only usb3 or a part of it to a VM. Same for realtek audio.
now the real problem:
tried to pass the SAS controller to a vm, at beginning in cpu slot 4 ok (group 4 and 6 if i remember well), but disconnect problem after a few hours of passing slot 4 and cpu slot 7 to different vm.
tried pch slot 2 (group 25 and 26), the 4x bottom one... basically the 2.5gbps network card disconnects from the bus when the VM starts (the lan is not passed to any vm, and is on a different group)
so i tried the upper slot (same 25/26 group) and work for the moment, i don't know if other components are compromised.
I just know that it's very strange that if I pass group 25/26 to a VM, group 19 disconnects, and it doesn't seem like consistent behavior to me.
I didn't find the option to split slots 4 and 7 in the bios (8x8x)
now the build is a mess, everything stuck together.
i forgot this
Anyone have the iommu group list of the Gigabyte MW34-SP0 Rev.1.0 ?
Code:
Group 0: [8086:4680] [R] 00:02.0 VGA compatible controller AlderLake-S GT1
Group 1: [8086:4668] 00:00.0 Host bridge 12th Gen Core Processor Host Bridge/DRAM Registers
Group 2: [8086:460d] [R] 00:01.0 PCI bridge 12th Gen Core Processor PCI Express x16 Controller #1
Group 3: [8086:464d] [R] 00:06.0 PCI bridge 12th Gen Core Processor PCI Express x4 Controller #0
Group 4: [8086:467d] 00:0a.0 Signal processing controller Platform Monitoring Technology
Group 5: [8086:7ae0] 00:14.0 USB controller Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller
USB: [1038:1361] Bus 001 Device 018 SteelSeries ApS Ideazon Sensei
USB: [2516:014f] Bus 001 Device 017 Cooler Master Co., Ltd. CK351 Gaming Optical Switch Keyboard
USB: [050d:3201] Bus 001 Device 016 Belkin Components F1DF102U/F1DG102U Flip KVM
USB: [0424:2524] Bus 001 Device 015 Microchip Technology, Inc. (formerly SMSC) USB MultiSwitch Hub
USB: [05e3:0610] Bus 001 Device 014 Genesys Logic, Inc. Hub
USB: [2109:2817] Bus 001 Device 013 VIA Labs, Inc. USB2.0 Hub
USB: [0b1f:03ee] Bus 001 Device 006 Insyde Software Corp. RNDIS/Ethernet Gadget
USB: [0557:9241] Bus 001 Device 004 ATEN International Co., Ltd SMCI HID KM
USB: [1d6b:0107] Bus 001 Device 002 Linux Foundation USB Virtual Hub
USB: [051d:0002] Bus 001 Device 005 American Power Conversion Uninterruptible Power Supply
USB: [1d6b:0002] Bus 001 Device 001 Linux Foundation 2.0 root hub
USB: [0781:5588] Bus 002 Device 007 SanDisk Corp. Extreme Pro
USB: [05e3:0616] Bus 002 Device 006 Genesys Logic, Inc. hub
USB: [2109:0817] Bus 002 Device 005 VIA Labs, Inc. USB3.0 Hub
USB: [1d6b:0003] Bus 002 Device 001 Linux Foundation 3.0 root hub
[8086:7aa7] 00:14.2 RAM memory Alder Lake-S PCH Shared SRAM
Group 6: [8086:7acc] 00:15.0 Serial bus controller Alder Lake-S PCH Serial IO I2C Controller #0
[8086:7acd] 00:15.1 Serial bus controller Alder Lake-S PCH Serial IO I2C Controller #1
Group 7: [8086:7ae8] 00:16.0 Communication controller Alder Lake-S PCH HECI Controller #1
[8086:7aeb] 00:16.3 Serial controller Device 7aeb
Group 8: [8086:7ae2] 00:17.0 SATA controller Alder Lake-S PCH SATA Controller [AHCI Mode]
Group 9: [8086:7ac8] [R] 00:1a.0 PCI bridge Alder Lake-S PCH PCI Express Root Port #25
Group 10: [8086:7ab8] [R] 00:1c.0 PCI bridge Alder Lake-S PCH PCI Express Root Port #1
Group 11: [8086:7ab9] [R] 00:1c.1 PCI bridge Alder Lake-S PCH PCI Express Root Port #2
Group 12: [8086:7abb] [R] 00:1c.3 PCI bridge Device 7abb
Group 13: [8086:7ab0] [R] 00:1d.0 PCI bridge Alder Lake-S PCH PCI Express Root Port #9
Group 14: [8086:7a88] 00:1f.0 ISA bridge Device 7a88
[8086:7ad0] 00:1f.3 Audio device Alder Lake-S HD Audio Controller
[8086:7aa3] 00:1f.4 SMBus Alder Lake-S PCH SMBus Controller
[8086:7aa4] 00:1f.5 Serial bus controller Alder Lake-S PCH SPI Controller
[8086:1a1c] 00:1f.6 Ethernet controller Ethernet Connection (17) I219-LM
Group 15: [10de:21c4] [R] 01:00.0 VGA compatible controller TU116 [GeForce GTX 1660 SUPER]
[10de:1aeb] 01:00.1 Audio device TU116 High Definition Audio Controller
[10de:1aec] 01:00.2 USB controller TU116 USB 3.1 Host Controller
[10de:1aed] 01:00.3 Serial bus controller TU116 USB Type-C UCSI Controller
Group 16: [1344:51c4] [R] 02:00.0 Non-Volatile memory controller 7450 MAX NVMe SSD
Group 17: [1987:5016] [R] 03:00.0 Non-Volatile memory controller E16 PCIe4 NVMe Controller
Group 18: [1283:8893] [R] 04:00.0 PCI bridge IT8893E PCIe to PCI Bridge
Group 19: [8086:15f2] [R] 06:00.0 Ethernet controller Ethernet Controller I225-LM
Group 20: [1a03:1150] [R] 07:00.0 PCI bridge AST1150 PCI-to-PCI Bridge
[1a03:2000] [R] 08:00.0 VGA compatible controller ASPEED Graphics Family
Group 21: [10b5:8724] [R] 09:00.0 PCI bridge PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA
Group 22: [10b5:8724] [R] 0a:00.0 PCI bridge PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA
Group 23: [10b5:8724] 0a:08.0 PCI bridge PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA
Group 24: [10b5:8724] 0a:09.0 PCI bridge PEX 8724 24-Lane, 6-Port PCI Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA
Group 25: [1000:0097] [R] 0b:00.0 Serial Attached SCSI controller SAS3008 PCI-Express Fusion-MPT SAS-3
Group 26: [1000:0097] [R] 0d:00.0 Serial Attached SCSI controller SAS3008 PCI-Express Fusion-MPT SAS-3now the real problem:
tried to pass the SAS controller to a vm, at beginning in cpu slot 4 ok (group 4 and 6 if i remember well), but disconnect problem after a few hours of passing slot 4 and cpu slot 7 to different vm.
tried pch slot 2 (group 25 and 26), the 4x bottom one... basically the 2.5gbps network card disconnects from the bus when the VM starts (the lan is not passed to any vm, and is on a different group)
so i tried the upper slot (same 25/26 group) and work for the moment, i don't know if other components are compromised.
I just know that it's very strange that if I pass group 25/26 to a VM, group 19 disconnects, and it doesn't seem like consistent behavior to me.
I didn't find the option to split slots 4 and 7 in the bios (8x8x)
now the build is a mess, everything stuck together.
i forgot this
Code:
[ 0.000000] x86/split lock detection: #AC: crashing the kernel on kernel split_locks and warning on user-space split_locks
[ 83.928807] x86/split lock detection: #AC: CPU 2/KVM/4546 took a split_lock trap at address: 0x7ef1d050
[ 83.928807] x86/split lock detection: #AC: CPU 3/KVM/4547 took a split_lock trap at address: 0x7ef1d050
[ 83.928807] x86/split lock detection: #AC: CPU 1/KVM/4545 took a split_lock trap at address: 0x7ef1d050
[ 228.797980] x86/split lock detection: #AC: CPU 3/KVM/5054 took a split_lock trap at address: 0x7ef1d050
[ 228.798042] x86/split lock detection: #AC: CPU 1/KVM/5052 took a split_lock trap at address: 0x7ef1d050
[ 228.798051] x86/split lock detection: #AC: CPU 2/KVM/5053 took a split_lock trap at address: 0x7ef1d050
[ 727.758714] x86/split lock detection: #AC: CPU 1/KVM/6483 took a split_lock trap at address: 0x7ef2d050
[ 727.758720] x86/split lock detection: #AC: CPU 2/KVM/6484 took a split_lock trap at address: 0x7ef2d050
[ 1012.470571] x86/split lock detection: #AC: CPU 0/KVM/6482 took a split_lock trap at address: 0xfffff80770442fb3
[ 1022.437950] x86/split lock detection: #AC: CPU 0/KVM/5051 took a split_lock trap at address: 0xfffff8030ca42fb3
[ 3309.099363] x86/split lock detection: #AC: CPU 2/KVM/13570 took a split_lock trap at address: 0x7ef1d050
[ 3309.099414] x86/split lock detection: #AC: CPU 3/KVM/13571 took a split_lock trap at address: 0x7ef1d050
[ 3309.099424] x86/split lock detection: #AC: CPU 1/KVM/13569 took a split_lock trap at address: 0x7ef1d050
[ 3402.136678] x86/split lock detection: #AC: CPU 0/KVM/13568 took a split_lock trap at address: 0x6d15ed1dAnyone have the iommu group list of the Gigabyte MW34-SP0 Rev.1.0 ?
Attachments
Last edited:
As far as i can tell only the X13SAE has the 3.0 update. X13SAE-F is only 2.1. What board are you running, and how many memory sticks is installed?
Regards
@brained can't speak to the issues with virtualized IOMMU you are having, but the split-lock warnings are a VM hypervisor problem. Split-locks by default are logged in the kernel, because they are highly discouraged at this point. There is a setting you can change to silence this, but this is really a code issue.
See Why is "x86/split lock detection" spamming my syslog when Steam is running? for a good synoopsis. (In this case Steam is discussed, but it applies to any software that is doing split-locks.)
In particular The search for the correct amount of split-lock misery [LWN.net] is relevant.
See Why is "x86/split lock detection" spamming my syslog when Steam is running? for a good synoopsis. (In this case Steam is discussed, but it applies to any software that is doing split-locks.)
In particular The search for the correct amount of split-lock misery [LWN.net] is relevant.
Thanks James,
allready silenced it with "split_lock_detect=off" in grub.
I ordered a pci 1x usb3 controller to test it on the bottom open ended pci 4x slot (phc slot 2), i want to verify if is in the same group (25/26) of the other 4x slot. i report soon the result.
Another one...
For my tests, is impossible boot with only the AST video output. i blacklisted the intel igpu, "dmesg | grep i915" is clean, passed to a vm, works in the windows vm, but no monitor recognised & no signal.
Last edited:
tried add usb controller on pch slot 2.
new group:
unfortunately eno2 pci disconnect whenthe vm start
new group:
Code:
Group 22: [1106:3483] [R] 09:00.0 USB controller VL805/806 xHCI USB 3.0 Controller
....
Group 27: [1000:0097] [R] 0c:00.0 Serial Attached SCSI controller SAS3008 PCI-Express Fusion-MPT SAS-3
Group 28: [1000:0097] [R] 0e:00.0 Serial Attached SCSI controller SAS3008 PCI-Express Fusion-MPT SAS-3Attachments
So,
a summary for pci slot vm passtrouth:
a summary for pci slot vm passtrouth:
Code:
CPU SLOT7 PCI-E 5.0 X16 : Can
PCH SLOT5 PCI-E 3.0 X4 : Can
CPU SLOT4 PCI-E 5.0 X8 (IN x16) : Can ONLY on the same VM of SLOT7
PCH SLOT2 PCI-E 3.0 X4 : NOT - Ethernet Controller I225-LM disconnect from the hostThanks good info. Wondering why the X13SAE-F version haven't been upgraded to 3.0 by SuperMicro. Anyone knows if there could be any technical reason to this?
Update with respect to X13SAE BIOS 3.0 (the version with no BMC):
According to SM, BIOS 3.0 supports Raptor Lake refresh, e.g., 14th generation CPUs. However, boards sold today will most likely have version 2.1. Therefore, the question yet unanswered is: Will a 2.1 BIOS boot with a 14th generation CPU?
Since I have access to a 12th (and a 13th) generation CPU, I just tried out. My X13SAE arrived with BIOS 2.1 (the old one), but it booted with no issue with an i9 14000K (14th generation), even showing the right name of the processor. Therefore, I was able to update the BIOS to 3.0 with no need for an older CPU.
This just in case somebody else has this question...
According to SM, BIOS 3.0 supports Raptor Lake refresh, e.g., 14th generation CPUs. However, boards sold today will most likely have version 2.1. Therefore, the question yet unanswered is: Will a 2.1 BIOS boot with a 14th generation CPU?
Since I have access to a 12th (and a 13th) generation CPU, I just tried out. My X13SAE arrived with BIOS 2.1 (the old one), but it booted with no issue with an i9 14000K (14th generation), even showing the right name of the processor. Therefore, I was able to update the BIOS to 3.0 with no need for an older CPU.
This just in case somebody else has this question...
14th gen is the same stepping/cpuid than 13th gen. uses same microcode.
the BIOS reads the Brand-String out of the processor.
I'm aware of that. However, a research on the internet revealed (for other mainboard manufacturers) that in some cases board do not boot with an BIOS that officially only supports 13th gen. Most boards do, but not all (maybe they check the string provided by the processor). Since I have not found anything related to X13SAE, I just posted by success story for others looking for that.
I read for some hours through many threads in many forums... obviously I have not saved all the links. Therefore, I leave it to you to search for it
(although it might be possible the the board in question did not boot for some other reason)