Intel i350 PCIe Bandwidth

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
eduncan911

eduncan911

The New James Dean
Jul 27, 2015
648
506
93
eduncan911.com
So people help me out on my calculations here...

I have a server with 2x GPUs in it. This leaves me with 2 slots:

1x PCIe x16 Gen3 direct to CPU
1x PCIe x4 (wired as x1!!) Gen2 through the PCH (Intel C612 chipset)

I have an LSI 9211-8i in the x16 slot.

I am installing the Intel i350-T4 in the x4 slot (wired at x1 Gen2).

Now, by my calculations, this shouldn't be a problem.

x1 Gen 2 equates to around 4 Gbps of unencoded PCIe data one way, which is 8 Gbps bi-direction. (AFAIK, NIC adapters are unencoded PCIe streams, not the native 10 bit streams you get from GPUs - so there's a loss there, hence 4 Gbps and not 5 Gbps).

The INtel i350-T4 = 4x 1Gbps NICs.

I should be fine, right?

Now given, I will never max this system out except on rare occasions when I am copying files from SSD to SSD over the network, a few times a week. Even then, that's just a single 1Gbps connection that the file copy will peg (~140 MB/s). This is why I got the card in the first place: to enable Teaming of 2x ports per "Primary" VMs (I'll only have 2 primary VMs), so that I can continue to use the network when making those copies (right now, the 4 year old complains when the TV stutters when I backup a 300 GB database over the single NIC connection).

This is just a home media/backup server running about 4 VMs.
 
C

canta

Well-Known Member
Nov 26, 2014
1,012
216
63
43
eduncan911 said:
I am installing the Intel i350-T4 in the x4 slot (wired at x1 Gen2).

Now, by my calculations, this shouldn't be a problem.

x1 Gen 2 equates to around 4 Gbps of unencoded PCIe data one way, which is 8 Gbps bi-direction. (AFAIK, NIC adapters are unencoded PCIe streams, not the native 10 bit streams you get from GPUs - so there's a loss there, hence 4 Gbps and not 5 Gbps).

The INtel i350-T4 = 4x 1Gbps NICs.

I should be fine, right?
.
Click to expand...
pci express 2.X uses 8b/10b encoding. this means to transmit 8bit, pci 2.x need 10bit data. 2bit are overhead :D
(10/100)*(2) = 20% loss
one lane is 5GT/s aka 500MB/s
total B/W available per lane is 500MB/s - 20% = 400MB/s


one port 1Gb ethernet port needs 125MB/s

how much one lane pcie 2.X support eth ports? 400/125 = 3 (rounded)...

you already know the calculation since one lane can support max 3 eth port when fully utilized..

if you do not fully utitlized all 4 ports of i350, one lane could works!
have fun!!!
 
A

Aluminum

Active Member
Sep 7, 2012
431
46
28
canta said:
pci express 2.X uses 8b/10b encoding. this means to transmit 8bit, pci 2.x need 10bit data. 2bit are overhead :D
(10/100)*(2) = 20% loss
one lane is 5GT/s aka 500MB/s
total B/W available per lane is 500MB/s - 20% = 400MB/s


one port 1Gb ethernet port needs 125MB/s

how much one lane pcie 2.X support eth ports? 400/125 = 3 (rounded)...

you already know the calculation since one lane can support max 3 eth port when fully utilized..

if you do not fully utitlized all 4 ports of i350, one lane could works!
have fun!!!
Click to expand...
You subtracted the 20% twice, 500MB/s is the bandwidth available per 2.0 lane. GT is gigatransfers (10b raw) and not gigabytes (8b data), they already accounted for the overhead when converting from GT/s to MB/s.

One 2.0 lane is just about dead on for saturating 4 gig ports.
 
C

canta

Well-Known Member
Nov 26, 2014
1,012
216
63
43
ZeDestructor

ZeDestructor

New Member
Oct 3, 2015
22
5
3
32
canta said:
yap, thanks for correction
5GT/s - 20% = 4GT/s => 500MB/s :D

so toal BW is 500MB/s on one direction..

still not good on 4 saturated ethernet ports...
Click to expand...
You'll be quite good actually. Some small losses from control data between driver and NIC, but that's about it.

Meanwhile, here I am.. with an X520-2 in a miserly PCIe 2.0 1x port (got 2 GPUs, soundcard and wifi card preventing me from either putting it in the wide CPU slot and setting the 1x slot to 4x) to get around LACP limits and multi-stream SMB inconsistencies :(
 
Z

Zyborg231

New Member
Sep 5, 2023
8
3
3
Why is this card rated at 5GT/s if it can run with x4 lanes? Shouldn't it be capable of 20GT/s?
 
T

Tech Junky

Active Member
Oct 26, 2023
337
112
43
Zyborg231 said:
Why is this card rated at 5GT/s if it can run with x4 lanes? Shouldn't it be capable of 20GT/s?
Click to expand...
First - 8 YO thread
Second - 1GE ports = 4gbps total

Speed of the slot gen matters as most NICs are backwards compatible down to Gen 1 though today you can get a single 10GE / x1 for $100 if you have a Gen 4 slot to stick it in.
 
E

ericloewe

Active Member
Apr 24, 2017
295
129
43
30
I'm not sure that really answers the question, which seems to be one of semantics... PCIe links are not typically referred to by their aggregate capacity in transfers per second (good thing too, because PCIe 6.0 will carry two bits per transfer, prior to coding). The GT/s figures are only really used to indicate the speed of each lane (and again, that will break with PCIe 6.0).
You could argue that 20 GT/s is outright incorrect since it's a 4-wide 5 GT/s link, but it's not a hill I want to die on and PCIe links are significantly more complex than their banner specs.

To unambiguously describe a PCIe link, just state the generation and width, optionally add the total user bandwidth (though PCIe 6.0 is making that figure more annoying, too, due to extra overhead so that we end up with 7.5ish GB/s up from a hair under 4 GB/s, which everyone just referred to as 4 GB/s).
 
Z

Zyborg231

New Member
Sep 5, 2023
8
3
3
Just trying to wrap my head around PCIe speeds and how that translates into GT/s and Gbps. I suppose Intel probably wouldn't make a card that couldn't get full 1Gbps bandwidth on all 4 ports at the same time.

Does anyone know what the difference between I350T4V2 and I350T4V2BLK is? The BLK seems to be half height and has some additional features like on chip QoS. I don't know which one to get?
 
You must log in or register to reply here.
Top Bottom