Dual port 10G SFP+ card: If only one port is in use does it get "all the bandwidth"?

[Markess]

If a dual port 10G SFP+ Card (PCIe 3.0 card like ConnectX-3, Solarflare SFN7122/8522, etc) has only one port in use, can that active port use all of the PCIe slot's available bandwidth? Said another way, does the card (or drivers, etc.) apportion/reserve bandwidth for the unused port?

I want to add a single 10G SFP+ connection to a motherboard with a PCIe 3.0 slot that's only x2 electrically. Digging through my spare parts box, the single port cards are all PCIe 2.0. So they wouldn't be able to make 10G on 2 lanes (at least by my math). But, I also have some dual port PCIe 3.0 cards, which should be OK on just two PCIe lanes, assuming the active port can grab all the bandwidth?

I tried googling, but its either so obvious that nobody feels the need to write about it, or its something that just doesn't get asked very much.

Maybe because nobody but me tries silly things like this?

 

[NPS]

typically your math is totally correct. There might be NICs out there working rather strangely but I doubt, because it would be rather complicated to still be able to use ports with less than the full width of the cart at all. Apart from that: just try!

On the topic of single port gen3 NICs: MCX311A-XCAT is one. I even used it in a gen3 x2 slot and it worked nicely (but I can't remember if I measured throughput)

 

[kapone]

NICs don't split PCIe bandwidth between ports. Why would they?

 

[NPS]

There are some "strange" design decisions in chips. Take for example dual port 25GbE NICs like BCM57414 that don't support linking one port at 10GbE and the other at 25GbE. Strange enough though, mixing 25GbE and 1GbE is supported on BCM57414.

But like I said: I can't imagine a NIC that splits PCIe bandwidth exists. And yes, neither can't I answer the question "why would they?". Thing is: Things beyond my imagination DO exist!

 

[nabsltd]

I want to add a single 10G SFP+ connection to a motherboard with a PCIe 3.0 slot that's only x2 electrically. Digging through my spare parts box, the single port cards are all PCIe 2.0. So they wouldn't be able to make 10G on 2 lanes (at least by my math).

PCIe 2.0 is 400MB/sec per lane, so x2 is 800MB/sec. It's not the full 10Gbps, but it's still 8x what you would get from a 1Gbps link.

 

[Markess]

typically your math is totally correct. There might be NICs out there working rather strangely but I doubt, because it would be rather complicated to still be able to use ports with less than the full width of the cart at all. Apart from that: just try!

On the topic of single port gen3 NICs: MCX311A-XCAT is one. I even used it in a gen3 x2 slot and it worked nicely (but I can't remember if I measured throughput)

Thank you! And thank you for the recommendation. The price is nice on those cards too!

NICs don't split PCIe bandwidth between ports. Why would they?

That response leads me to believe that I couldn't find anything via Google because it is a "its obvious to the people in the know, so they don't write about it" situation. I guess I know what I need to study up on next!

PCIe 2.0 is 400MB/sec per lane, so x2 is 800MB/sec. It's not the full 10Gbps, but it's still 8x what you would get from a 1Gbps link.

Thanks for that. That is probably close enough that it really won't matter to me from a homelab standpoint. In any case, more proof that I need to study up on networking next.