Mellanox ConnectX-3 Showing wrong bus type

jriker1

New Member
Sep 27, 2016
8
0
1
54
I just installed a Mellanox ConnectX-3 two port in my Windows 2016 server. Haven't installed the fiber cable or SFP+ transceivers yet, however the Bus Type is showing as x8 when I know it's x16 slot any idea what's up or is that just because it's the max the card supports so the "card" is x8?

1645229555703.png

Also I've read some things that pci-e 2.0 can support full speed of 10Gbe but only on one port connection. Is that accurate?

Thanks.

JR
 

Brandon_K

New Member
Jan 17, 2021
26
17
3
Pittsburgh, PA
The card is only a x8 card, so yes, it's only going to report a x8 connection. It's not physically long enough to be a x16 card.

PCIe2.0 x8 would give you 32gbps of bandwidth, so significantly more than a dual port 10gbe card would be able to push. The ConnectX3 is a PCIe3.0 card though, not 2.0.
 
  • Like
Reactions: fohdeesha

i386

Well-Known Member
Mar 18, 2016
2,993
959
113
33
Germany
5 GBit/s per lane
8x 5 Gbit/s = 40GBit/s

PCIE 2.0 uses 8b/10b encoding meaning for transfering 8 databits you will need 10 bits on the media (pcie lane)

40GBit/s / 10 * 8 = 32 GBit/s
 

jriker1

New Member
Sep 27, 2016
8
0
1
54
Thanks. So is there anything I can do with my current older systems to get the hard drives not bottleneck things? So take for example my oldest self build. An Asus Rampage III Extreme. Have it connected to the Marvell SATA 6.0 slot for one. That is coming back with a horrible.

picture3.png

Then I have an LSI 9260-8i raid card with three SAS Seagate ST33000650SS drives in Raid 5
picture3.png

Both seem slow to me. Was going to replace everything with SSD Drives but not sure they will perform. Or will installing a SATA card in one of the PCI-E slots help instead of the onboard?
 

Attachments

Last edited:

LodeRunner

Active Member
Apr 27, 2019
385
158
43
What drive is connected to the single Marvell SATA6 port? If it's a 7200 RPM disk, then those transfer rates are expected. If it's an SSD, then something is very wrong. That RAID-5 speed is also about where I would for 3 spinning drives.

A SATA 6 SSD should max out the port at 500-550 MByte/s. So a RAID5 array of those would theoretically be (N-1)*(drive speed); depending on how good the RAID card is. If that board supports NVMe at all, then M.2 drives in PCIe brackets would be an alternate solution.

Burning a PCIe slot for a plain SATA 6 card is a waste of money and a slot, unless you're installing a HBA for FreeNAS or somesuch.
 

jriker1

New Member
Sep 27, 2016
8
0
1
54
Ordered a Samsung 870 EVO for the boot drive to replace the spinning boot drive which is also 2.5inch.

I'm curious as I troubleshoot some initial connectivity issues. Looking at MTU values I ran a

netsh interface ipv4 show subinterfaces

and got

1645580379830.png

So thinking something may be wrong as the Mellanox is obviously Local Area Connection 3 but is returning two entries with two different MTU values. I see no VLANS or anything if that would cause it. This is on Windows 7. My Windows 10 computer, though with a different card, shows one line with MTU 9000.
 

LodeRunner

Active Member
Apr 27, 2019
385
158
43
That's still slower than I would expect for a 870 EVO on a SATA 6 port under most circumstances.

As far as the network oddness in Win 7; I can't say as I no longer run Win 7. Seeing that, I would go into Device Manager (make sure to enable Show Hidden Devices) and purge all network interfaces, then go through the registry as well and cleanup the interface entries there. After a reboot and driver reinstall, see if you still have sub interface entries.

Edit:
This is from a much abused 1TB 860 EVO so not an entirely equal comparison:
1647820512095.png
System is a Ryzen 7 5800X with a MSI X570E board, so whatever SATA chip is on that.
 
Last edited: