Hi folks,
I posted this over at L1T last week, so if you already saw my post there, feel free to scroll on...
I have a SuperMicro X10SRH-CF board (UP, based on the C612 chipset) configured with a 12-core Xeon E5-2678 v3 processor and eight DIMMs in the proper slots to activate all the controllers and channels. I enabled Cluster-on-Die in the BIOS, which to the best of my understanding should expose two NUMA nodes to the kernel–each with six cores and two memory channels. Unfortunately, I can only see a single NUMA node with all 12 cores and all the RAM.
I spent a couple hours trying every conceivable permutation of BIOS settings that seem related to NUMA, interleaving, snooping, etc. No combination of options exposes more than one node to the kernel. The motherboard manual specifically calls out settings for NUMA and node interleaving, but I don’t actually see those in the latest BIOS where they’re supposed to be.
I’m trying to figure out where to go from here. I know my CPU is a strange OEM part and it’s definitely possible that it has some features fused off. It’s also possible that I am completely misunderstanding how this is supposed to work and appear from within Linux… maybe the kernel doesn’t support “little” NUMA?
I reached out to SuperMicro to try to get some support and here's what they said:
Thank you in advance for any insight you can offer!
Mike
I posted this over at L1T last week, so if you already saw my post there, feel free to scroll on...
I have a SuperMicro X10SRH-CF board (UP, based on the C612 chipset) configured with a 12-core Xeon E5-2678 v3 processor and eight DIMMs in the proper slots to activate all the controllers and channels. I enabled Cluster-on-Die in the BIOS, which to the best of my understanding should expose two NUMA nodes to the kernel–each with six cores and two memory channels. Unfortunately, I can only see a single NUMA node with all 12 cores and all the RAM.
I spent a couple hours trying every conceivable permutation of BIOS settings that seem related to NUMA, interleaving, snooping, etc. No combination of options exposes more than one node to the kernel. The motherboard manual specifically calls out settings for NUMA and node interleaving, but I don’t actually see those in the latest BIOS where they’re supposed to be.
I’m trying to figure out where to go from here. I know my CPU is a strange OEM part and it’s definitely possible that it has some features fused off. It’s also possible that I am completely misunderstanding how this is supposed to work and appear from within Linux… maybe the kernel doesn’t support “little” NUMA?
I reached out to SuperMicro to try to get some support and here's what they said:
That can't be right... can it?
Thank you in advance for any insight you can offer!
Mike