PCIe 2.0/3.0 4x Lane 10Gb SFP+ card?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Jepong

Member
Nov 2, 2017
44
1
8
38
Well it's not fully clear. Actually : a 1 port 10gb nic card running pcie2.0 x4 , if i plug into a pcie3.0 x1 : Will this work at about 5gb speed ? But work without error, just a bit slower.. ?
thanks
I think it will run at pcie 2.0 x1
 

lunadesign

Active Member
Aug 7, 2013
256
34
28
It seems clear that a PCI-E 2.0 x4 (2000 MB/s) will handle a 10G 1 port card (1250 MB/s) but I'm curious what would happen with a 2 port card (2500 MB/s).

1) Would I be able to hit 10G on each port individually (assuming the other port is not busy at the same time)? Or does the card detect the PCI-E bandwidth limitation and cap one or both ports?

2) Also keeping in mind that the PCI-E speed is 2000 MB/s in each direction, does that mean that I could theoretically have one port sending at 10G at the same time the other one is receiving at 10G?
 

Docop

Member
Jul 19, 2016
41
0
6
45
I guess a Mellanox 2 dual nic into a pcie3.0 x1 will most likely run at 4gb each. But those mellanox are pcie2x8 card... so anyone had tested ??
A pci2x4 into a pci3x1 should give 8gb on 1 port.
 

Jepong

Member
Nov 2, 2017
44
1
8
38
I guess a Mellanox 2 dual nic into a pcie3.0 x1 will most likely run at 4gb each. But those mellanox are pcie2x8 card... so anyone had tested ??
A pci2x4 into a pci3x1 should give 8gb on 1 port.
A ConnectX-2 card runs at pcie 2.0 and will not run at pcie 3.0 even if you put it in a 3.0 slot. It will also run at x1 only if the slot is x1 electrical. So putting a pcie 2.0 x4 or x8 card in a pcie 3.0 x1 slot will run at pcie 2.0 x1. Severely limiting the bandwidth to just 500 MB/s. Someone correct me if I'm wrong
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
A ConnectX-2 card runs at pcie 2.0 and will not run at pcie 3.0 even if you put it in a 3.0 slot. It will also run at x1 only if the slot is x1 electrical. So putting a pcie 2.0 x4 or x8 card in a pcie 3.0 x1 slot will run at pcie 2.0 x1. Severely limiting the bandwidth to just 500 MB/s. Someone correct me if I'm wrong
That's correct.
 

Jepong

Member
Nov 2, 2017
44
1
8
38
I would like to put a Mellanox MNPH29C-XTR ConnectX-2 Dual Port into a x4 slot. Will the slot become a bottleneck? Considering 2x 1250MB/s is greater than 2000MB/s (pci-e 2.0 x4). ConnectX-2 is cheaper than ConnectX-3 and I'm buying multiples. I just realized I've got a similar question with lunadesign but since it has not been answered I'm asking again. I'll be putting this in a vSan host
 
Last edited:

Jepong

Member
Nov 2, 2017
44
1
8
38
Thanks for answering! Let me repeat lunadesign's questions.

1) Would I be able to hit 10G on each port individually (assuming the other port is not busy at the same time)? Or does the card detect the PCI-E bandwidth limitation and cap one or both ports?

2) Also keeping in mind that the PCI-E speed is 2000 MB/s in each direction, does that mean that I could theoretically have one port sending at 10G at the same time the other one is receiving at 10G?
 

Jepong

Member
Nov 2, 2017
44
1
8
38
Thats what i thought . 10gigabit is 1250mbyte/s so theres a little bottleneck. 985mbyte/s translates to about less than 8gigabit/s.
 
Last edited:

zack$

Well-Known Member
Aug 16, 2018
701
315
63
Is anyone able to confirm whether the Solarflare SFN6122F will work in a pcie 2.0 x4 (physical x8)? I have read the thread and am pretty certain it should, but the sfuserguide v4.12.0 is throwing me off with the following: "8 lanes Gen2 (5.0GT/s), x8 edge connector (usable in x8 and x16 slots), 127 SR‐IOV virtual functions per port."

Is the "usable" here electric or physical?
 

DanielWood

Member
Sep 14, 2018
44
17
8
Is anyone able to confirm whether the Solarflare SFN6122F will work in a pcie 2.0 x4 (physical x8)? I have read the thread and am pretty certain it should, but the sfuserguide v4.12.0 is throwing me off with the following: "8 lanes Gen2 (5.0GT/s), x8 edge connector (usable in x8 and x16 slots), 127 SR‐IOV virtual functions per port."

Is the "usable" here electric or physical?

Physical. I am unaware of ANY PCIe card that will not downgrade all the way to electrical x1. In fact, that is what is often used for mining rigs. Most of the USB3 cable connected mining risers are PCIe x1. The video card runs fine on those (assuming you have supplied it with 75W capability on the PCIe slot), it is just PCIe bandwidth limited (which is largely irrelevant for mining applications).

The question becomes where is the bottleneck. PCIe 2.0 x4 at 2GB/sec, that is a bottleneck for 2x 10G but will not bottleneck unless you are completely saturating both. PCIe 3.0 x1 is a bottleneck of around 8Gbps no matter what you have on the other side. As long as you are aware and OK with these limitations, go wild with any combo of PCIe slots and cards.

I regularly use a PCIe x1 to x16 extender (x1 to the motherboard, x16 to the card) for flashing NICs and HBAs. The only issues I've had were when I forgot to connect the external power, the Dual 40GB NIC downgraded to PCIe 2.0. As soon as I gave it that 12V boost it needed, PCIe 3.0 x1 worked with no issue.
 
  • Like
Reactions: zack$

newyork10023

New Member
Jan 19, 2016
1
0
1
The Intel (and third-party) X550-T2 using a PCIe gen3 x4 slot is available. Still unable to find a DA2 with x4 slot, Intel or otherwise.
 

chelmi

New Member
May 19, 2021
6
0
1
I was wondering too why there would be T2 (10G RJ45) cards available in PCIe gen3 x4 available and none as DA2 (SFP+)

Now however, both Mellanox ConnectX-3 and Intel X710-DA2 are available as PCIe gen3 8x.

If my math and logic is not wrong, a PCIe gen3 8x card in a 4x slot will still run at PCIe gen3 4x electrical, hence 3938 MB/s.
As 2x10Gbe would max out at 2x 1250MB/s = 2500MB/s, 3938 MB/s should be plenty of bandwidth to let both ports run at full speed.

Is anything in my logic flawed ?

By the way, I got an Intel X520-DA2 PCIe gen2 8x plugged into a PCIe gen3 4x port.
While this works, I am still only getting 6.6 Gbit/s throughput on this one on one port which is roughly 825 MB/s.
This would not be even close to PCIe gen2 4x which should give me 2000 MB/s. It almost looks like the card would fallback to PCIe gen1 4x (or PCIe gen3 1x) for lack of a gen2 electrical port.

So before I go ahead and buy yet another card that may or may not work at full speed I would love someone confirms my above logic and math is in theory correct.
 

i386

Well-Known Member
Mar 18, 2016
4,221
1,540
113
34
Germany
I was wondering too why there would be T2 (10G RJ45) cards available in PCIe gen3 x4 available and none as DA2 (SFP+)

Now however, both Mellanox ConnectX-3 and Intel X710-DA2 are available as PCIe gen3 8x.
MY guess: 10GBE SFP+ based cards share the design with QSFP+ based cards, while 10GBaseT doesn't have faster variants that require a different pcb design.
 
  • Like
Reactions: chelmi

Tommy Blom

New Member
Jan 13, 2019
1
0
1
Have à look at Mellanox ConnectX-3 MCX311A-XCAT EN 10G Ethernet SFP+
1 port pcie 3.0 x4.
Works well in Linux FreeBSD and Windows
 

chelmi

New Member
May 19, 2021
6
0
1
Have à look at Mellanox ConnectX-3 MCX311A-XCAT EN 10G Ethernet SFP+
1 port pcie 3.0 x4.
Thanks for the hint Tommy Blom, however we were discussing any potential dual port cards.
1 port is quite easy to find in PCIe 3.0 4x, but dual port for some reason is not.