SFP+ Dual NIC with 10G-BaseT and 5000Base-T support. Does it exist?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
So far the only NIC I can find that actually supports 2500, 5000, and 10000 is the intel X550-T2 which I own but is not fiber. I need such a card to connect to my modem through which I get 350MB/s with a 5GB connection, and then on to my Brocade at 10GB. Does an SFP+ board support this? I cannot find any and I have been looing hard.

I just got an HP T740 to use exclusively as a firewall router and not surprisingly the NIC overheats.

I find it strange that I cannot find an SFP+ NIC to fill the bill as I have plenty of copper "transceivers" that support multimode. It seems like the NIC's don't support this, just some switches?!?

Thanks!
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
To my knowledge there is no 5gbe standard on fiber, so if you get this connection from your ISP on fiber, I would assume the link rate is 10G but your bandwidth limited elsewhere.

10gbase-t and 5gbase-t are standards for ethernet over twisted pair cables, so not fiber.

Honestly I am not quite sure, how to understand your setup.
But I am assuming that you have something like this:

Internet Connection with a modem that has a RJ45 plug that does 5gbe
Firewall device with a yet to be decided 10G NIC
Brocade switch with one or more SFP+ cages

If this is the case, I would probably want to use a RJ45 10gbe NIC, like the X550 you mentioned, in the firewall and a SFP+ to RJ45 transceiver in the brocade.

If this is what you tried already, and your problem is the NIC is overheating, perhaps add a fan?

If your ISP modem is not RJ45, and the transceiver is something standard, then you could probably lookup its specs to help you determine if it is actually a 10gbe device.
 
  • Like
Reactions: CIR-Engineering

Sean Ho

seanho.com
Nov 19, 2019
768
352
63
Vancouver, BC
seanho.com
How about a newer 10GbaseT SFP transceiver like ipolex, FS.com, or recent S+RJ10, popped into a 10G port of your Brocade, with the other end of the cat6 directly into your modem's 5GbaseT port? Then use two SFP+ ports to send traffic through your router/firewall (isolate WAN using a VLAN). STH frontpage recently had a series of reviews on these transceivers, noting which were compatible with NBaseT.
 
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
Thanks for your reply. You are very close on your assumptions, sorry I was not more clear.

Yes, my modem has a 5GBaseT port that supports a 1GB, 2.5GB, and 5GB copper Ethernet connection. I need to run that into the HP T740 and was/am planning to use the X550-T2 (Intel claims 17.5 watts max at 10GB). That same X550T has been running on an OPNsense VM, and has been connected to the Brocade via copper transceiver for a couple years. The X550-T2 has been connecting to the modem at 5GB and the Brocade at 10GB. I regularly pull 350 MB/S download rates.

Here is my issue with overheating, the chassis is so small on the T470 and the CPU fan hardly runs because the CPU load isn't very high most of the time. Here is another thread I started regarding fan speed and overheating, it's insanely frustrating:

This is what is making me consider lower power NICs, actually a passive SFP+ cable would work here instead of fiber to the brocade, and copper SFP+ transceiver to the modem.
 
Last edited:

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
Thanks for that. I only just today discovered this was even a thing; transceivers converting 2.5/5GBaseT to 10GB SFP+. I actually have four of them here already, but my ignorance made me believe that the switch also had to support the data rates to match! I am thrilled to have discovered otherwise. This thread pointed me to the article you mentioned:

My unRAID server was hosting the OPNsense VM from which I removed the X550-T2. I purchased a MELLANOX CONNECTX-3 to run fiber between the unRAID server and the Brocade. For testing, today I put the X-3 as WAN in the H740, and setup OPNsense with it and the integrated Realtek NIC as LAN. I plugged an SFP+ copper transceiver into the X-3 and ran it to the brocade SFP+ copper transceiver to see what would happen. I tested with iperf3 to a WAN server. It works halfway...

In the past, I always have had to specify in OPNsense (or PFsense) that the X550-T2 card needs to connect at 5GB. Otherwise, it would connect at only 2.5GB. Based on my iperf3 testing, it seems that the X-3 with SFP+ copper 10GBaseT is connecting at 2.5GB instead of 5GB as well. In PFsense (I have been trying things with both these router OS's) my choices for the X-3 card:

1667611997926.png

Is there any way to tell the transceiver to connect at 5GB instead of 2.5GB? It seems logically I can only tell the X-3 10GB so I seem to be stuck.
 
Last edited:

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
Another concern. The Mellanox single port X-3 SFP+ is getting very hot as well with a coper SFP+ transceiver. It will only get hotter if I get a dual port X-3 even if the second port is only running passive SFP+. I really need to get the stupid fan in the T740 to work right. Otherwise, I'm going to return the H740 or hack it up and add a fan. I really don't want to hack it all up, it looks nice and is brand new.
 

Sean Ho

seanho.com
Nov 19, 2019
768
352
63
Vancouver, BC
seanho.com
And 10GbaseT is inherently hot and power-hungry; it's the price of running such high rates across twisted pair. Hence the suggestion to run WAN from the modem directly to the switch (which presumably has better cooling), then you can use SFP and passive DAC to/from the T740, without any 10GbaseT on the tiny thin client.
 
  • Like
Reactions: CIR-Engineering

AveryFreeman

consummate homelabber
Mar 17, 2017
413
54
28
42
Near Seattle
averyfreeman.com
So far the only NIC I can find that actually supports 2500, 5000, and 10000 is the intel X550-T2 which I own but is not fiber. I need such a card to connect to my modem through which I get 350MB/s with a 5GB connection, and then on to my Brocade at 10GB. Does an SFP+ board support this? I cannot find any and I have been looing hard.

I just got an HP T740 to use exclusively as a firewall router and not surprisingly the NIC overheats.

I find it strange that I cannot find an SFP+ NIC to fill the bill as I have plenty of copper "transceivers" that support multimode. It seems like the NIC's don't support this, just some switches?!?

Thanks!
I gleaned something from discussions about people doing this with egress throttling from the 10GbE end, albeit with 10Gbase-T (to 2.5GbE), sounded like it was necessary to get decent throughput, otherwise lots of dropped packets. I took it to mean you can use 2.5GbE with a 10GbE connection, even though most 10GbE ports don't expressly support it, but I don't actually have this kind of setup, so I could be wrong.

I don't see why you couldn't do the same thing with fiber.
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
And 10GbaseT is inherently hot and power-hungry; it's the price of running such high rates across twisted pair. Hence the suggestion to run WAN from the modem directly to the switch (which presumably has better cooling), then you can use SFP and passive DAC to/from the T740, without any 10GbaseT on the tiny thin client.
Crap, that's a good suggestion. However, I just realized that would require one more 10GB port than I already have. Ugh.
 
  • Like
Reactions: AveryFreeman

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
H740 Fan small.jpg



Stopgap until hopefully HP fixes the firmware. I really didn't want to hack up the case, so I made an acrylic panel with a hole punched out for the added fan. The Acrylic cover is held on by magnets. The fan came out of a Brocade because I thought 40mm would work well. It shoots air right to the open area at the NIC heat sink and the hot SSD drive which is practically sandwiched up against said NIC heat sink.

I will take temperature readings with the fan pushing air in as it is, but I also want to try the fan pulling air out (also might need larger fan). Pulling may work better because the design is for the CPU fan to pull air through vents in the bottom up through the chassis. This fan may add to that air flow if it were pulling, but my concern is that it may also overpower the CPU fan and actually cause air to get sucked in against the flow of the CPU fan... I'll take temperature measurements both ways.

If HP would just fix the firmware this would not be needed :mad:
 

AveryFreeman

consummate homelabber
Mar 17, 2017
413
54
28
42
Near Seattle
averyfreeman.com
Crap, that's a good suggestion. However, I just realized that would require one more 10GB port than I already have. Ugh.
The back of the NIC where the X550 chip is (all the small SMD caps) is measuring 55 degrees C. That should be fin right?
From everything I've read, that's pretty standard sounding. Watch your speeds and see if it looks like it's throttling speed for temperature, though.

re: NIC, not familiar with the 550-T2 but if you're shopping around, there's a QL41134HLCU that is low profile and has 4 SFP+ ports, saw one on eBay for $155: QLogic 10Gb 4-port SFP+ QL41134HLCU PCI Express Adapter | eBay
 

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
I grabbed this guy:

I think I can put a 40mm fan right over the heat sink and punch it through the acrylic. I got the board with two passive DAC cables. So, I'll use one DAC to the switch and an SFP+ transceiver copper for the modem. Should be 10.5 watts total instead of 17.5 watts. I think I'll be set. Right now with the fan I added everything has been very stable already.

I'll have to work out a low profile bracket :rolleyes:. Should be able to make or buy one. Anyone know of a good 'buy' source for the bracket.
 

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
This monstrosity is a flexlom (or was it ocp|mezzanine?) card with a third party adapter. Very unlikely that there are profiles for that.
For a vanilla or normal oem card there are official low and high brackets...

Edit: Fixed typo
 
  • Wow
Reactions: CIR-Engineering

prdtabim

Active Member
Jan 29, 2022
170
66
28
This monstrosity is a flexlom (or was it ocp|mezzanine?) card with a third party adapter. Very unlikely that there are profiles for that.
For a vanilla or normal oem card there are official low and high brackets...

Edit: Fixed typo
It's a Mezzanine OCP 2.0 card with a PCIe adapter.
The card is a Mellanox CX342A CX3.

I have a similar card with CX341A ( 01 sfp+ port ).
 
  • Like
Reactions: CIR-Engineering

CIR-Engineering

I am a functional adult?
Jan 14, 2021
85
30
18
48
Chicago USA
www.cir-engineering.com
This monstrosity is a flexlom (or was it ocp|mezzanine?) card with a third party adapter. Very unlikely that there are profiles for that.
For a vanilla or normal oem card there are official low and high brackets...

Edit: Fixed typo
It's a Mezzanine OCP 2.0 card with a PCIe adapter.
The card is a Mellanox CX342A CX3.

I have a similar card with CX341A ( 01 sfp+ port ).
Yeah, I'm pretty sure it's just a Mellanox CX342A CX3. A fellow member on STH figured out how to put this all together in a thread somewhere. He even had instructions on making a one port into a two port and cross flashing with Mellanox firmware.
 
  • Like
Reactions: AveryFreeman

AveryFreeman

consummate homelabber
Mar 17, 2017
413
54
28
42
Near Seattle
averyfreeman.com
H740 Fan small.jpg



Stopgap until hopefully HP fixes the firmware. I really didn't want to hack up the case, so I made an acrylic panel with a hole punched out for the added fan.
Wait, I'm trying to figure out what kind of computer you're making this out of - is it a Z2 mini? I scanned through the thread and didn't see you mention it anywhere...

I'm working on a cluster of Dell 7050 micros and HP Elitedesk 800 G4s, with A+E key and M-key to PCIe x16 risers for the networking. I am using QL45212 25GbE cards for the networking, I might swap them out of the A+E key risers for official 10GbE cards since I'm guessing I'll only get 700Mbps out of them tops (PCIe x1, maybe x2, can't remember specs)

I solved the cramped case issue by putting them in a 4U rack box I had leftover from my DJing days, they fit really well sideways, it's kind of like a ghetto blade system. You can get 6 of them in there with the NICs, AC adapters, and a rack-style power strip side-mounted carefully.

Add the USW-Pro-Aggregation, 1TB PM981s and 2GB SATA TLC SSDs for each node, and an Aten K1516A with some KVM dongles, and I think I'll be set for some vSAN.