HPE 621SFP28 and Marvell QLogic QL41212HLCU

[vangoose]

Got this card to play with, haven't installed it yet. I'll install it in my storage ESXi server so SR-IOV/NPAR will definitely help a lot. All the offloading features will help NSX-T as well. I don't have 20gb switch so it has to run at 10gb speed for now.

Based on FastLinQ 41000 chipset, I think it's same as Qlogic QL41212. I have CX-3 but SR-IOV is not supported anymore, I have one CX-4 too and will see which one works better.

Interesting feature
I/O Virtualization and Multitenancy
• SR-IOV (up to 192 virtual functions)
• Switch-independent NPAR, NPAReP
• GRE and NVGRE packet task offloads
• VXLAN packet task offloads
• GENEVE packet task offloads

RDMA Universal RDMA
• RoCE
• RoCEv2
• iWARP
• Storage over RDMA: iSER, SMB Direct, S2D, and NVMe-oF
• NFSoRDMA
• iSCSI extensions for RDMA (iSER)
• Low latency

RDMA Use Cases
• S2D
• PVRDMA
• Live Migration
• SMB Direct
• NVMe-oF
• NFS
• CEPHS over RDMA

 

[Freebsd1976]

which is better QL41212 or QL45212 ? read the docs but still not know differenence

 

[vangoose]

which is better QL41212 or QL45212 ? read the docs but still not know differenence

Depending on packaging
41000 is 10/25 gbe
45000 is 10/25/40/50/100 gbe

 

[vangoose]

Reading user guide, qL41000/45000 Linux driver comes with NVMEoF target mode driver, not sure how that works with VMware's NVMEoF support.
You need physical nvme devices and can export the namespace through nvmeof.

Will probably give it a try later.

 

[vangoose]

Gone is the old easy to use CCM bios utility in 57810.

There are few ways to manage, cli, poweshell and vcenter plugin.

Took me a while to figure out how to get vcenter plugin work. Once it's working, it's easy to use and has temp monitoring as well.

Got NPAR and SR-IOV turned on now, default is off.

 

[vangoose]

HP 621sfp28 doesn't support NPAR on Supermicro server, maybe it only works on HPE servers.

Qlogic QL41212 has all features available. Once you have NPAR turned on, SRIOV needs to be reconfgured.

Once RDMA is turned on ,lots of BSOD due to VMware driver issue. The latest is 3.11.16 has BSOD, but earlier version is 3.11.17 is fine. What's going on with version number? 3.11.17 is older because from release notes, its internal fw is older than 3.11.16 and release date is also earlier.

CX 4 is rock solid and stable. What I like QLogic card is NPAR, you can divide the card into multiple physical cars, some with rdma turned on and some off.

 

[AveryFreeman]

Sorry to revive necrothread, but I got a handful of QL45212s and am wondering what kind of (hopefully inexpensive) modules I can use with them.

I'm using ESXi 7.0U3g, which supports the card natively (which was nice, as Linux support was rather limited). ESXi recognizes the NICs are there, but I don't get any lights when I plug any of my modules in.

I've tried with a Dell 10Gb-S, Intel 10Gb and 1Gb RJ45 modules, and a Cisco H10G-CU DAC, and so far nothing seems to be recognized by the Qlogic card.

Other end is a USP-Pro-Aggregation switch, I have all Ubiquiti branded modules in that, and I can see the switch-end description of the modules in the Unifi controller when it's connected to the QL45212 - if only one end of line connected the switch presents as if there's nothing plugged in.

What are you using, and is your card HPE branded? Mine are all Dell. Thanks

 

[vangoose]

Sorry to revive necrothread, but I got a handful of QL45212s and am wondering what kind of (hopefully inexpensive) modules I can use with them.

I'm using ESXi 7.0U3g, which supports the card natively (which was nice, as Linux support was rather limited). ESXi recognizes the NICs are there, but I don't get any lights when I plug any of my modules in.

I've tried with a Dell 10Gb-S, Intel 10Gb and 1Gb RJ45 modules, and a Cisco H10G-CU DAC, and so far nothing seems to be recognized by the Qlogic card.

Other end is a USP-Pro-Aggregation switch, I have all Ubiquiti branded modules in that, and I can see the switch-end description of the modules in the Unifi controller when it's connected to the QL45212 - if only one end of line connected the switch presents as if there's nothing plugged in.

What are you using, and is your card HPE branded? Mine are all Dell. Thanks

They are 25gb cards and you need sfp28 for it. My cables are from fs.com. Mines are hpe branded but you can code for qlogic.

 

[vangoose]

They are 25gb cards and you need sfp28 for it. My cables are from fs.com. Mines are hpe branded but you can code for qlogic.

It should work with 10G sfp, but it might need to be coded for qlolgic. I haven't touched them for 2 years.

I have them connected to Arista 100G ports with these cables
Arista Networks CAB-Q-4S-100G-3M QSFP28 to SFP28 Breakout DAC Cable - FS

 

[AveryFreeman]

It should work with 10G sfp, but it might need to be coded for qlolgic. I haven't touched them for 2 years.

I have them connected to Arista 100G ports with these cables
Arista Networks CAB-Q-4S-100G-3M QSFP28 to SFP28 Breakout DAC Cable - FS

Very interesting. Have you been able to get them to work with any Cisco branded modules? I had a reply from 10GTek when I asked if they had compatible modules saying I should use Cisco, but all my DACs are all Cisco-branded (both ends, not compatible but made by Cisco) and they're only recognized by my switch AFAICT.

That FS.com ad mentions something about a configuration utility?