Dual NICs on PCIE x1 slot recommendation

sofianito

New Member
Nov 9, 2011
14
0
1
Madrid, Spain
Hi,

I am looking to fill the second and last PCIE x1 slot of my HP MicroServer MB with a dual nics 1gb card that support:

- Solaris 11 natively (required)
- Link aggregation (required)
- VLAN (required)
- Jumbo Frames (Required)
- TCP checksum offload (would be a plus)
- ESX5i (would be a plus)

Any recommendations are welcome :)
 

Metaluna

Member
Dec 30, 2010
64
0
6
Check out the Intel E1G42ET dual-port card. This has the 82576 chip, which is on ESXi 5's compatibility list. Not sure about Solaris 11, but it's on Solaris 10's HCL, so I'd say it's pretty likely to be supported.

The card is PCIe x4 but I think it will fit in a x1 slot.
 

sofianito

New Member
Nov 9, 2011
14
0
1
Madrid, Spain
After googling a bit, it seems that a part of the StarTech which seems not supported on Solaris, there are not a lot of choices. Most of dual NICs need an x4 PCIe slot...

However, according to the specs of the HP NC360T, a four lane (x4) PCIe, it is compatible with x1, x4, x8, and x16 PCIe slots!. In order to plug it in the x1 slot of the MicroServer, a hardware modding needs to be done. Basically you either dremel the x1 slot of the motherboard or the card pins. In my case, I don't want to mess with the motherboard, so I'll choose option 2 ;). The end result would look like this picture (courtesy of NoGi's Random Tech Blog):


The only doubt I have is whether the HP MicroServer's PCIe v2 x1 would get saturated by two gigabit ports running at full duplex...
 
Last edited:

Metaluna

Member
Dec 30, 2010
64
0
6
That Startech card looks like it uses two Realtek chips. Definitely not a good choice for ESXi support, and probably not great for Solaris either. That HP card uses an Intel 82571 chip, so probably a good choice if you can get it for less than the Intel one I mentioned. If it's about the same price, go for the Intel as it's a newer generation chip, probably has a more powerful processing/offloading engine, and uses less power.

That mod looks pretty scary. Look how close the upper left corner of the cut area is to the signal traces and vias for the remaining part of the connector. Even if you don't physically cut into that area, if your cutoff wheel generates too much heat you could delaminate the traces and break them.

As for whether a 1x slot can support two gigabit ports without losing performance, I don't know. You're cutting it pretty close (no pun intended) bandwidth-wise. Neither of the Intel chips I mentioned support 5.0GT/s PCIe transfers, so the card would need to sustain 2.0Gbit/sec over a 2.5Gbit/sec link, including PCIe protocol overhead, etc. My guess is you'll probably lose a little performance but not anything too dire. Maybe 10 or 20% less.
 
Last edited:

cactus

Moderator
Jan 25, 2011
829
77
28
CA
I did this to two video cards and one came out working and the other I must have messed up with ESD. I had an Intel Dual Port ET in a 1x port that I cut the back out of so it will for sure work just be careful when handling the card. The Intel 82571 is only PCI-e v1 so it is cutting it close for throughput. If you are really concerned there are some Intel chips that support PCI-e v2 link.
 

sofianito

New Member
Nov 9, 2011
14
0
1
Madrid, Spain
@Metaluna: The Intel E1G42ET looks great with a lot of features for a virtualized environment, but it is expensive... Also being an x4 PCIe card, I am afraid it doesn't fit OOTB in the x1 slot. It would need some modding same as the HP NC360T... Although it has less features, the pros of the HP card over the intel one is that it is less than half of the price in ebay, and if mess with it I guess I would not pull my hair as hard as if it were the intel one... :p

@cactus:
I had an Intel Dual Port ET in a 1x port that I cut the back out of so it will for sure work just be careful when handling the card
Was the cut similar to the picture I posted? Do you have a picture of your modding?
 
Last edited:

cactus

Moderator
Jan 25, 2011
829
77
28
CA
I have 3 of the dell branded E1G42ETs from ebay for $50US each. I did not mod any of my Intel NICs, I cut the back out of the x1 PCI-e slot on the motherboard sothe 4x card would fit. The two video cards I did cut down were cut at the edge of the solder resist.
 

sofianito

New Member
Nov 9, 2011
14
0
1
Madrid, Spain
Lucky you! :). Are you willing to sell me one :p. Actually, the lowest price I found in ebay is around $140 shipping cost not included...

Did you use a dremel or a knife such as X-Acto for modding?

At the edge did you you cut along the longitudinal or lateral axis?

Forgot to mention that in my case, if I decide to mod the mdb slot x1, I would need to do 2 cuts! because in the HP MicroServer mdb, there is an internal PCIe v2 x4 slot aside thex1 slot. The x4 slot is for an internal HP IPMI card. Here is a picture:
 
Last edited:

sofianito

New Member
Nov 9, 2011
14
0
1
Madrid, Spain
I finally bought a dual link HP NC360T and modded it to fit the PCIe x1 slot on my HP MicroServer :D. I used an X-Acto X235 saw to do the cut. It was very precise and dit the job perfectly :cool:



 

cactus

Moderator
Jan 25, 2011
829
77
28
CA
sofianito, I used a fine tooth saw similar to the one you used.

How is everything working out?
 

sofianito

New Member
Nov 9, 2011
14
0
1
Madrid, Spain
So far, so good :D

Both nics are recognized under Solaris 11:


# dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net0 Ethernet up 1000 full bge0
net3 Ethernet up 1000 full e1000g0
net1 Ethernet up 1000 full e1000g1

# dladm show-link
LINK CLASS MTU STATE OVER
net0 phys 1500 up --
net3 phys 1500 up --
net1 phys 1500 up --

# for i in 0 1 3; do dladm show-linkprop net${i} | /usr/xpg4/bin/grep -E 'LINK|speed|duplex|mtu'; done
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
net0 speed r- 1000 1000 --
net0 duplex r- full full half,full
net0 mtu rw 1500 1500 1500

net1 speed r- 1000 1000 --
net1 duplex r- full full half,full
net1 mtu rw 1500 1500 1500-9216

net3 speed r- 1000 1000 --
net3 duplex r- full full half,full
net3 mtu rw 1500 1500 1500-9216

# ipadm show-addr
ADDROBJ TYPE STATE ADDR
lo0/v4 static ok 127.0.0.1/8
net0/v4 static ok 172.16.1.4/24
net1/v4 dhcp ok 172.16.1.22/24
net3/v4 dhcp ok 172.16.1.19/24
lo0/v6 static ok ::1/128


I am planning to test the following:
* Link aggregation: I need to figure out how to configure link aggregation on Solaris 11 and HP Procurve 1810g-24 switch.
* Jumbo Frames
* Measure iSCSI performance between the ESXi5 node and the HP MicroServer SAN

I got a lot of stuff to learn, but I am motivated :)

Cheers


 
Last edited:

alderaan

New Member
Apr 4, 2012
2
0
0
Hi i've a NC364t Quad-port Hp and i'd like to use it with my hp microserver. I can do the same thing with this card?
 

MiniKnight

Well-Known Member
Mar 30, 2012
2,987
892
113
NYC
Probably can. I think pci express specs say everything up to x16 can work in a x1 slot.
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,489
223
63
Assuming he is using these all at once. Using 2 max is not bad. PCI express 2 = 500MB/s for an x1.
 

Metaluna

Member
Dec 30, 2010
64
0
6
Not all multi-port controllers support the PCIe 2.0 speeds though, so he might be limited to 250MB/s.
 

alderaan

New Member
Apr 4, 2012
2
0
0
Not all multi-port controllers support the PCIe 2.0 speeds though, so he might be limited to 250MB/s.
Ok, done everything. So used 2 nic:

- NC364t 4xlan connected in pci-e x16
- intel 39Y6127 2x lan cutted and connected in pci-e following this thread.

Thank.
 

RADCOM

New Member
Aug 27, 2012
11
4
3
IBM variant of the pro/1000PT I count 14 traces after the cut on my card, yours has only 13 or is my eyesight really that bad and I have no business cutting this card anyway lol Did you cut through one?
 

TvdH

New Member
Jul 17, 2013
5
0
1
Would it not have been easier to use something like this instead of cutting the card's PCI-E interface?