So I slapped a Mellanox MCX516A-CCAT ConnectX-5 EN into PCIe5 x16 Slot on my ASUS Pro WS W680M-ACE SE motherboard and booted up into Proxmox expecting to see a couple of new happy ports to play with... there was nothing. However the old names for the original ports incremented - like Proxmox saw a new network something. Weird. I started running commands:
Nothing
No IB devices found
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
Unloading MST PCI module (unused) - Success
Unloading MST PCI configuration module (unused) - Success
MST modules:
------------
MST PCI module is not loaded
MST PCI configuration module is not loaded
No MST devices were found or MST modules are not loaded.
1: lo: UNKNOWN mode DEFAULT
2: enp6s0: UP mode DEFAULT <--port0
3: enp7s0: DOWN mode DEFAULT <---port1
4: enx6a85cfe79de1: UNKNOWN mode DEFAULT <--- IPMI
5: vmbr0: UP mode DEFAULT <--- bridge
Where the NIC is installed ---> 00:01.0 PCI bridge [0604]: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 [8086:460d] (rev 02)
Subsystem: ASUSTeK Computer Inc. 12th Gen Core Processor PCI Express x16 Controller [1043:8694]
Kernel driver in use: pcieport
[ 0.602002] pcieport 0000:00:01.0: PME: Signaling with IRQ 122
[ 0.602040] pcieport 0000:00:01.0: AER: enabled with IRQ 122
06:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-LM (rev 06)
Subsystem: ASUSTeK Computer Inc. Ethernet Controller I226-LM
Kernel driver in use: igc
07:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-LM (rev 06)
Subsystem: ASUSTeK Computer Inc. Ethernet Controller I226-LM
Kernel driver in use: igc
. .. bonding_masters enp6s0 enp7s0 enx6a85cfe79de1 lo vmbr0
Hmmm ...
I plugged in a couple DACs to the ports and couple switches. The port lights on the NIC lit up solid (not flickering) green. The switch's ports blinked like NY City woke up - green.
So weird.
I didn't see a UEFI setting I flubbed or the NIC listed anywhere in the BIOS.
I plopped in a known non-working CX512 and while it's definitely borked (no FW) -
still listed it.
That CX516A is a dead card right? Like 100% dead call the ME and next of kin. Send it to Ebay as "open box".
Bash:
lspci -v | grep Mellanox
Bash:
ibv_devinfo
Bash:
mst startLoading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
Unloading MST PCI module (unused) - Success
Unloading MST PCI configuration module (unused) - Success
Bash:
mst status -v------------
MST PCI module is not loaded
MST PCI configuration module is not loaded
No MST devices were found or MST modules are not loaded.
Bash:
ip link2: enp6s0: UP mode DEFAULT <--port0
3: enp7s0: DOWN mode DEFAULT <---port1
4: enx6a85cfe79de1: UNKNOWN mode DEFAULT <--- IPMI
5: vmbr0: UP mode DEFAULT <--- bridge
Bash:
lspci -nnkSubsystem: ASUSTeK Computer Inc. 12th Gen Core Processor PCI Express x16 Controller [1043:8694]
Kernel driver in use: pcieport
Bash:
sudo dmesg | grep pcieport[ 0.602040] pcieport 0000:00:01.0: AER: enabled with IRQ 122
Bash:
lspci -k | sed -n '/Ethernet/,/driver in use/p'Subsystem: ASUSTeK Computer Inc. Ethernet Controller I226-LM
Kernel driver in use: igc
07:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-LM (rev 06)
Subsystem: ASUSTeK Computer Inc. Ethernet Controller I226-LM
Kernel driver in use: igc
Bash:
ls -a /sys/class/netHmmm ...
I plugged in a couple DACs to the ports and couple switches. The port lights on the NIC lit up solid (not flickering) green. The switch's ports blinked like NY City woke up - green.
So weird.
I didn't see a UEFI setting I flubbed or the NIC listed anywhere in the BIOS.
I plopped in a known non-working CX512 and while it's definitely borked (no FW) -
Bash:
lspci -v | grep MellanoxThat CX516A is a dead card right? Like 100% dead call the ME and next of kin. Send it to Ebay as "open box".
