6 port Intel i350 PCIe Card $46.99 and less for qty

MiniKnight

Well-Known Member
Mar 30, 2012
3,042
944
113
NYC
I'd probably just use 10G at this point, but if you just need NICs then here's a 6 port PCIe Card less if you need quantities of them

I used the PEG6i back in the day so maybe this is a successor.
 

Dreece

Active Member
Jan 22, 2019
506
162
43
I wonder, do these have SRIOV enabled in them? I recall some don't, this could be a nice to have (a couple or so) in the parts bin for future requirements.

Not to worry, they apparently do.


Silicom’s Six Port Copper Gigabit Ethernet PCI Express Server adapter is based on Intel i350AM4 Quad port Gigabit Ethernet MAC+PHY and Intel i350AM2 Dual port Gigabit Ethernet MAC+PHY of Intel Controller. The Silicom i350 support PCI-SIG Single-Root I/O virtualization and sharing specification (SR-IOV).
...my google-fu is improving! :cool:
 
  • Like
Reactions: Sleyk

Fritz

Well-Known Member
Apr 6, 2015
3,012
1,031
113
68
How much power do these consume? this would be an important consideration inh a PFSense box.
 
  • Like
Reactions: Sleyk

MiniKnight

Well-Known Member
Mar 30, 2012
3,042
944
113
NYC
Mine arrived today. It has CRAZY blocks from the i350's to the top heatsink. The i350's you often see on motherboards without heatsinks so I don't think this really needs much airflow
 
  • Like
Reactions: Sacrilego

RTM

Well-Known Member
Jan 26, 2014
899
336
63
Mine arrived today. It has CRAZY blocks from the i350's to the top heatsink. The i350's you often see on motherboards without heatsinks so I don't think this really needs much airflow
My guess is that the main reason for the heatsink is not the i350AM2 and AM4 controllers, but a PCIe switch chip.
 

Bjorn Smith

Well-Known Member
Sep 3, 2019
775
417
63
48
r00t.dk
Mine arrived today. It has CRAZY blocks from the i350's to the top heatsink. The i350's you often see on motherboards without heatsinks so I don't think this really needs much airflow
So I am considering buying one of these, but I need to know how many pcie devices this turn up as, i.e. if it turns up as a 4 port+2 port with two pcie devices, then its golden, because it should be possible to pass through to a VM, but if it shows up as just one device, then not so much.

So if you have it installed, can please do a
Code:
lspci -vvv
if its installed on linux or just check device manager if it is installed on windows.

Thanks a lot.
 

mimino

Active Member
Nov 2, 2018
193
75
28
So I am considering buying one of these, but I need to know how many pcie devices this turn up as, i.e. if it turns up as a 4 port+2 port with two pcie devices, then its golden, because it should be possible to pass through to a VM, but if it shows up as just one device, then not so much.

So if you have it installed, can please do a
Code:
lspci -vvv
if its installed on linux or just check device manager if it is installed on windows.

Thanks a lot.
Isn't it common for each port to be in a separate IOMMU group for this kind of devices?
At least that's how it is for all my 10G cards of any brand.
 

Markess

Well-Known Member
May 19, 2018
995
651
93
Northern California
So I am considering buying one of these, but I need to know how many pcie devices this turn up as, i.e. if it turns up as a 4 port+2 port with two pcie devices, then its golden, because it should be possible to pass through to a VM, but if it shows up as just one device, then not so much.
I'm not supper well versed in IOMMU, but can you tell if there's any IOMMU group separation/breakout just from the PCIe device count? Don't you still have list the devices by IOMMU group to see how the system groups the PCIe devices?

For example, LSPCI for my motherboard lists the onboard USB (both 2.0 and 3.0) controllers as three separate PCIe devices, but it put them all in the same IOMMU group. So even though there's three distinct PCIe devices, I can't pass any through unless I pass them all through. At least that's how I understand it?

If I went to the bother of getting a separate USB PCIe card to do passthrough and I didn't have to, that would suck! :p
 

mimino

Active Member
Nov 2, 2018
193
75
28
@Markess you're right, you do need a proper IOMMU separation working. For that your processor must support ACS. This might be of some help.

There is a workaround in case ACS is not properly implemented, by adding "pcie_acs_override=downstream" to kernel boot params, but I never had to use it and not sure how well or even if it works at all.
 
  • Like
Reactions: Markess

Markess

Well-Known Member
May 19, 2018
995
651
93
Northern California
@Markess you're right, you do need a proper IOMMU separation working. For that your processor must support ACS. This might be of some help.

There is a workaround in case ACS is not properly implemented, by adding "pcie_acs_override=downstream" to kernel boot params, but I never had to use it and not sure how well or even if it works at all.
That's a really interesting blog. Thanks for sharing it.

I was eyeing the i350 card in the OP for a system I wanted to make for "general purpose" testing/tinkering/learning, including virtualization. I'd originally planned on using an Intel C224 board & Xeon E3v3 I had on hand. But, reading through the blog entries at the link you gave, I'm now rethinking the wisdom of using E3 for this. As for this i350 card, hopefully someone who gets theirs can let us know how the ports separate out.

According to the blog, the system I mentioned above does support ACS, and is generally pretty good with IOMMU separation. Its just the USB that all got lumped together. Lucky, that was an easy fix with an add-in PCIe card.
 

mimino

Active Member
Nov 2, 2018
193
75
28
That's a really interesting blog. Thanks for sharing it.

I was eyeing the i350 card in the OP for a system I wanted to make for "general purpose" testing/tinkering/learning, including virtualization. I'd originally planned on using an Intel C224 board & Xeon E3v3 I had on hand. But, reading through the blog entries at the link you gave, I'm now rethinking the wisdom of using E3 for this. As for this i350 card, hopefully someone who gets theirs can let us know how the ports separate out.

According to the blog, the system I mentioned above does support ACS, and is generally pretty good with IOMMU separation. Its just the USB that all got lumped together. Lucky, that was an easy fix with an add-in PCIe card.
I could not get passthrough working properly on any CPU pcie bus using E3v3. Only PCH attached bus worked w/o issues but it was x8 2.0 (likely also C224) and that prompted me to dump the E3 platform altogether. These days ryzens do a much better job than E3 did, even being desktop CPUs.
 

Sacrilego

Member
Jun 23, 2016
77
75
18
47
I just got mine delivered today. Packaging was a bit lacking in my opinion, but it arrived in good condition and for the price I can't complain much.

The heatsink covers a large portion of the card, but is a bit thin. Its used to cool down the PLX PCIE switches onboard. The card does get a bit warm, so you will want to have some air blowing on the heatsink just in case it gets too hot when pushed hard.

There are three PCIE switches on the card, one is used for 4 ports of the card, a second one for the other 2 ports and the third one to tie the other 2 switches together and interface with the x8 edge. Each port is assigned 4 PCIE 2.0 lanes from the switches.

Each port came up as an independent PCIE device that I could select for passthrough and assign to different virtual machines without issue. I did this on ESXi 7 running on an ASUS Z10PC-D8 and an E5-2620 v3.

I'm quite satisfied with the card so far.

Now to test SR-IOV. No luck with SR-IOV so far. I can try to enable it and I see a max of 8 VFs per device (port), but ESXi just refuses to cooperate and resets the amount of VFs to 0 which disables it. I don't see any SR-IOV support on the HCL for it either. I'm giving up on ESXi for now.
I'll try on Hyper-V tomorrow.
 
Last edited:
  • Like
Reactions: Markess and Dreece

Bjorn Smith

Well-Known Member
Sep 3, 2019
775
417
63
48
r00t.dk
Each port came up as an independent PCIE device that I could select for passthrough and assign to different virtual machines without issue. I did this on ESXi 7 running on an ASUS Z10PC-D8 and an E5-2620 v3.
Awesome, just what I wanted to hear - so basically you could pass through 6 devices to 6 different VM's?
 

Bjorn Smith

Well-Known Member
Sep 3, 2019
775
417
63
48
r00t.dk
Sucks that I live in Denmark - I have to pay

$16.51 shipping
$17.87 import charges
so almost as much as the card itself.

A total of 81,37
But I am going to order one, just so I can get my "edge" ESXI up to speed with pfsense running in a VM.