Mellanox Connect X3 does not work with Alder Lake/Raptor Lake

[TheMaster]

Alder Lake + Mellanox CX-3 10 GbE NIC = Weird Happenings

I have the same Problem as in the Reddit Post above.

I have a Intel Core I5 13500 with a Mellanox X3 "MCX354A-FCB_A2-A5" Card linked up to a 10 Gbit Switch.

System:

CPU: Intel Core I5-13500
Motherboard: Gigabyte Z690M AORUS ELITE AX DDR4
RAM: Crucial 32GB DDR4
OS: Server 2022 or Windows 11 (Same Problem)

Now the Problem:

If i enable the 8 E Cores from the CPU i get a Bluescreen at the start of the OS.

„==================================================
Dump File : 020923-9906-01.dmp
Crash Time : 09.02.2023 19:42:39
Bug Check String : SYSTEM_THREAD_EXCEPTION_NOT_HANDLED
Bug Check Code : 0x1000007e
Parameter 1 : ffffffff`c0000005
Parameter 2 : fffff802`4fd932d9
Parameter 3 : fffff680`611e74a8
Parameter 4 : fffff680`611e6cc0
Caused By Driver : mlx4eth63.sys
Caused By Address : mlx4eth63.sys+84860
File Description :
Product Name :
Company :
File Version :
Processor : x64
Crash Address : mlx4eth63.sys+132d9
Stack Address 1 :
Stack Address 2 :
Stack Address 3 :
Computer Name :
Full Path : C:\Windows\Minidump\020923-9906-01.dmp
Processors Count : 20
Major Version : 15
Minor Version : 20348
Dump File Size : 1 819 988
Dump File Time : 09.02.2023 19:44:25
==================================================
„

If i disable the E-Cores in the Bios the Boot is without a Problem.

Motherboard Bios and Windows Drivers are up to Date and the Mellanox Card is Working fine without the E-Cores


---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


I talked to the Nvidia Support but they only answer Questions to a Enterprise Users with a entitlement.

Is here someone with connections to Nvidia? Maybe we could get a out of Bound Support for this Cards. Looks like all X3 Cards have the same Problem.

Or does someone know a woraround for this driver Bug.
(ok with disabled e cores it is working but then i could have a 12500)

And for the Record the Card is working fine in a Ryzen 5900 System.

Thanks.

 

[i386]

I would expect a "fix" from intel. If e cores break a driver it will also break other software...

 

[TheMaster]

I dont think so because i found out that on Linux there is no Problem with the e-Cores so it sounds like a driver Bug from Mellanox/Nvidia.

 

[Stephan]

This will IMHO never get fixed. Neither from Intel's side, which is firing people left right and center right now. Nor from Nvidia's side, because CX3 in their mind is EOL.

Options: Leave E-Cores disabled. Switch to a different card. Switch to Linux. Switch to Ryzen. Or to 12400/12500/12600 which turbo like crazy because they don't have too many cores for the heatspreader's thermal transfer capacity.

E-Cores were painful not only on Windows but also for Linux for a longer time. Goes to show what a bad job Intel has done here. Sold chips with limited software compatibility.

 

[i386]

Options: Leave E-Cores disabled. Switch to a different card. Switch to Linux. Switch to Ryzen. Or to 12400/12500/12600 which turbo like crazy because they don't have too many cores for the heatspreader's thermal transfer capacity.

I was thinking the same and wrote something similar but then deleted it because it was passive aggressive/too negative

 

[Stephan]

@i386 If the Windows driver was open source, 10 bucks that someone would have already fixed this particular bug and published a new driver. But it isn't and Windows driver signing snake oil is needlessly standing in the way to boot. The issue is akin to the Pope decreeing that all Porsche 911 from 1989 need to be recycled, because too old. Makes no sense. Wouldn't sell new CX6 if you could just use those unbreakable CX3 at 56 Gbps forever like on Linux. Only obsolescence one could imagine is people wanting PCIe 4.0 x16 instead of PCIe 3.0 x8 because PCIe bandwidth is some bottleneck. 10 years ago, Mellanox really was 5-10 years ahead of its time.

 

[TheMaster]

Thanks for the Info.

1) That i could disable the e-cores -> I knew that already
2) That CX3 is EOL does not mean that it is garbage or obsolete
3) Maybe NVidia and Intel do not know about this Problem
4) Maybe CX4 and CX5 have the same Problem? I dont know.

--------------------------------------------------------------------------------------------------------

And i came here because i know that sth have good connection to the big players So maybe ... maybe the could talk to them *little hope*

 

[mach3.2]

2) That CX3 is EOL does not mean that it is garbage or obsolete

Agreed, I'm still using one. But the direct implication is nvidia won't spend resources fixing it since they already EOL'd that lineup years ago.

Also windows driver signing as mentioned in post #6. Yeah doesn't look likely that it's getting fixed from my pov.

 

[manfri]

An just when i bought some, i've discovered that aren't compatible with VmWare 8..

But these NIC are awesome even right now.

Aside from VMWare I do not think there is a more compatible and capable 10GB nic (and the single port version is also tiny)

HPE has the audacitity to sell 10GB nic with 57810S chipset from 2010 (i talk about PCI 2.0 folks) like hot newss

Hot for sure, it's the only NIC that declare who need special fan configuration (at least the 10BaseT version) on HPE Server..

 

[TonyArrr]

4) Maybe CX4 and CX5 have the same Problem? I dont know.

I’ve just gotten a CX4 and popped it in my Alder Lake desktop without issue. Updated firmware, installed win-OF2, all working normally.

I’m still waiting for the new switch that will connect to it to arrive, however, so it doesn’t have an active link yet, but it doesn’t seem to share the same problem with e-cores

 

[TheMaster]

@TonyArrr

Thanks for the info that a CX4 is working on a Alder Lake CPU.