10 and 40 Gb networking on the cheap ($5 - $10)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

RageBone

Active Member
Jul 11, 2017
617
159
43
So, i have all my 544+FLRs finally working. Settled on HPEs Flexlom FW version 2.42.5700 that seems to work flawless as far as i can tell, with the exception of a BUG relating to SRIOV that seems to have snug into version above 2.42.5000.
Boards without SRIOV Support enabled will fail to boot into the OS, and get stuck on some loading screen.

Right after flashing, mstconfig settings do survive so flash, and change config to disable SRIOV and you should be good.
 

romanboy

New Member
Jul 18, 2019
1
0
1
649283-B21 (dual-port QSFP+

Hello,

I've followed the steps, got my adapter, soldered it, however I can only get my 649283-B21 to 10GBps (QSFP), not 40 (QSFP+). Am I doing something wrong, or assuming something incorrect? Also, I couldn't find ConnectX3-rel-2_36_5000-web.tgz, nor can I find mft-oem, but most other programmes worked fine, such as mst, flint, mlxconfig. I've used them on Windows 10 and Debian 10. I'vew upgraded to the latest firmware on 2 of my cards, on both my Windows machine and my Debian machine.

Many thanks for your help.
 

flashdisc

New Member
Apr 13, 2021
7
0
1
I opened a GitHub issue and Tobias has said he will incorporate the changes into a Revision B board.
A big shout out to @tsys and @hmartin for spending time and effort into this topic.

I would like to purchase 2 adapters for HPE 533FLR-T 2-Port 10GbE PCI-Express x8 Network adapter cards (RJ45 ports).

Ideally I want the adapter to be Plug and Play - no soldering etc.

Where may I buy those? I am located in Germany.

I found this, but do not know the revision - A, B? C most like not.

 

hmartin

Active Member
Sep 20, 2017
316
243
43
37
A big shout out to @tsys and @hmartin for spending time and effort into this topic.

I would like to purchase 2 adapters for HPE 533FLR-T 2-Port 10GbE PCI-Express x8 Network adapter cards (RJ45 ports).

Ideally I want the adapter to be Plug and Play - no soldering etc.

Where may I buy those? I am located in Germany.

I found this, but do not know the revision - A, B? C most like not.

I don't know what revision sysmocom has, I previously purchased from them and it was revision A (but this was before I opened the GitHub issue). You could message them on Twitter to ask.

I have around 25 Rev B adapters (unassembled) I will sell to people at cost.

If you want pre-assembled adapters that work with the 533FLR-T, I can send you two modified Rev A adapters at cost.
 
  • Like
Reactions: RageBone

flashdisc

New Member
Apr 13, 2021
7
0
1
@hmartin

thank you for you answer

sysmocom has confirmed that it indeed is revision A

I am curious: wasn't revision B to work out of the box with the 533FLR-T?

Yes, I am interested in your offer, I will send you a private message.

Best
 

hmartin

Active Member
Sep 20, 2017
316
243
43
37
I am curious: wasn't revision B to work out of the box with the 533FLR-T?
No, only Rev C. Broadcom-based cards require A10 and A11 to be pulled to 3.3V, which Rev B does not do.

Rev B is equivalent to KCORES v1.0, while Rev C incorporates the changes to KCORES v1.1 (which is compatible with Broadcom cards).

TobleMiner did not want to tie A10/11 to 3.3V because they are software defined pins that could have different functionality for different cards.

It should be easy to modify Rev B cards by adding a wire to A10/A11, but you asked about finished adapters and for that I have only modified Rev A.
 

flashdisc

New Member
Apr 13, 2021
7
0
1
It should be easy to modify Rev B cards by adding a wire to A10/A11, but you asked about finished adapters and for that I have only modified Rev A.
Thank you for the clarification, I got it now.

If it's working with the modified Rev A. than it is good to go.
 

flashdisc

New Member
Apr 13, 2021
7
0
1
first of all a big Thank You to this forum for your patience and competence. Especially a large Thanks to @hmartin for selling me the adapter

I do have a new question:

Is it possible to set up the NIC with 2,5 or 5 GBit mixed mode?

the NIC HP FlexFabric 10Gb 2-port 533FLR-T does only offer 1 or 10 GBit speed, no 2,5 or 5 GBit inbetween in Windows 10. TrueNAS only shows a 1 GBit connection as of yet since I do not have another 10 GBit card atm.



I can report back:

the adapter is working - cutting the rear of the PC case was required, the adapter and NIC do not align.

HPE drivers and firmware update have been applied:

-----------------------------------------------------------------------------
Online Firmware Update Tool for Windows (5.2.3.0)
Copyright (c) 2011-2019 Hewlett-Packard Enterprise Development LP
-----------------------------------------------------------------------------
Upgrade library: LibQLGC_HpFwUpg.dll (ver 8.15.0.0)
Found 1 adapters

Examining NIC HP FlexFabric 10Gb 2-port 533FLR-T Adapter at Ethernet 6
Update firmware MBI from version 7.10.33 to 7.18.80? [y]es/[n]o/[q]uit/[c]ontinue (y) y
Flash file name: C:\...FW\hp_mbi_7.18.80.bin
Updating HP FlexFabric 10Gb 2-port 533FLR-T Adapter...please wait.

takes longer than 10 minutes, be patient

upon PC boot "CTRL+S" will enter the NIC setup
 

hmartin

Active Member
Sep 20, 2017
316
243
43
37
I do have a new question:
Is it possible to set up the NIC with 2,5 or 5 GBit mixed mode?

the NIC HP FlexFabric 10Gb 2-port 533FLR-T does only offer 1 or 10 GBit speed, no 2,5 or 5 GBit inbetween in Windows 10. TrueNAS only shows a 1 GBit connection as of yet since I do not have another 10 GBit card atm.
I don't believe NBASE-T is supported by the cards, the NBASE-T standard is too recent (802.3bz-2016, versus the cards which are from 2014).

the adapter is working - cutting the rear of the PC case was required, the adapter and NIC do not align.
Really? I have not had any issues in fitting the cards into the PCI slot opening in the chassis. They cannot mount like a normal PCI card because they are missing the L-bend, but I didn't have to modify the case. Can you provide some photos of what you mean?
 

hmartin

Active Member
Sep 20, 2017
316
243
43
37
If anyone in the EU is looking to purchase unassembled Rev B adapters, drop me a PM. I have around 20 available which I'm selling at cost (approx. 3€/adapter + shipping). I cannot assemble them for you, you will need to solder it yourself.

Edit: I have no more adapters
 
Last edited:

flashdisc

New Member
Apr 13, 2021
7
0
1
@hmartin

Regarding the "non-fit" of the NIC - my bad, cutting the rear of the PC case was required, the adapter and NIC do align with the PC slot after removing the NIC slot bracket.
 

RageBone

Active Member
Jul 11, 2017
617
159
43
@hmartin Thank you for the 5, arrived and are as expected.
Positively surprised that they are beveled and the ground-plane isn't showing.
Didn't look for that in the pictures.
 

notanothernick

New Member
Jan 26, 2022
1
0
1
If anyone in the EU is looking to purchase unassembled Rev B adapters, drop me a PM. I have around 20 available which I'm selling at cost (approx. 3€/adapter + shipping). I cannot assemble them for you, you will need to solder it yourself.
Hi Martin,
do you still have adapters? I'd take 5 of them if possible.
Cheers,
Chris
 

tic226

Member
Feb 2, 2017
52
29
18
122
DE
Has anyone built a bracket/baffle to securely mount the card? A friend printed the KCORES baffle[0] for me but unfortunately their adapter is taller so it doesn't fit properly with Tobi's adapter. I'd like to use the cards (544+FLR) in 2U servers which means i have to mount them in riser cages with some quick release machanism that doesn't allow using screws (no threads and no space).
I've got some strips of 2mm copper sheet and my pair of metal shears to try my luck at building a few brackets, i'll report back (if i haven't lost a finger or two...).

[0] KCORES-FlexibleLOM-Adapter/Mechanicals/PCI Baffle at V1 · KCORES/KCORES-FlexibleLOM-Adapter
 

tic226

Member
Feb 2, 2017
52
29
18
122
DE
Success, all of my nine ten fingers are intact and the two brackets fit nicely (no background in mechaninal engineering and built on my desk so they're not pretty).
However, as user Babbsdrebbler reported last year, the cards don't like being pulled when powered. Yesterday one of the cards slid from their PCIe socket when i was testing them without the brackets and now the server won't pick it up anymore, it's probably dead.
_DE50035_comp.jpg_DE50037_comp.jpg_DE50040_comp.jpg_DE50043_comp.jpg
 
  • Like
Reactions: RageBone
Jun 30, 2022
113
24
18
This is a really interesting thread and I'm glad I found it before buying some HP 561FLR-Ts.

Plan is to use them, channeled bonded for direct attachment between two desktop machines that will both be dealing with multicam 1080p video editing and transcoding. Consolidate most of the SSD storage onto one machine, Samba share, etc. Current NAS nowhere near fast enough to edit directly from.

Understand that 561FLR-T are PCIe Gen 2. How would they perform in Gen 1 slot with read workloads as opposed to theoretical bandwidth limits etc?
 

RageBone

Active Member
Jul 11, 2017
617
159
43
This is a really interesting thread and I'm glad I found it before buying some HP 561FLR-Ts.

Plan is to use them, channeled bonded for direct attachment between two desktop machines that will both be dealing with multicam 1080p video editing and transcoding. Consolidate most of the SSD storage onto one machine, Samba share, etc. Current NAS nowhere near fast enough to edit directly from.

Understand that 561FLR-T are PCIe Gen 2. How would they perform in Gen 1 slot with read workloads as opposed to theoretical bandwidth limits etc?
why not get Mellanox CX3+ and a DAC? Cheaper and faster.