HP NC523SFP / QLogic QLE3242 10Gb NICs SFP+/DAC compatibilities and Linux?

Discussion in 'Networking' started by BLinux, Jan 31, 2018.

  1. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,866
    Likes Received:
    523
    Does anyone have any thing to share regarding HP NC523SFP / QLogic QLE3242 10Gb NICs and their SFP+/DAC compatibility? And how about driver support in Linux? Is the native kernel driver okay? are there benefits to use 3rd party drivers from QLogic or HP? Are there any firmware issues?
     
    #1
  2. _alex

    _alex Active Member

    Joined:
    Jan 28, 2016
    Messages:
    851
    Likes Received:
    89
    i got my hands on a couple of 3442, both rj45 and sfp+
    haven't had a chance for a closer Look, but these were used under a 3.1 Kernel with - how i got it - in-tree drivers for iscsi.
    so i'd guess in-tree drivers forthe older 3242 should be mature.

    for DAC/sfp's i'm not sure, can try with finisar sfp's and a mixed bag of dac's / breakout-cables
     
    #2
  3. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,866
    Likes Received:
    523
    do you know which linux driver these use? is it the qlcnic? do you know if that driver supports sr-iov?

    please let me know if/when you test them out and if you find that it's picky about SFP+ or DACs. thanks...
     
    #3
  4. _alex

    _alex Active Member

    Joined:
    Jan 28, 2016
    Messages:
    851
    Likes Received:
    89
    no, they are in proprietary os atm, just had a quick Look over dmesg that was accessible. but will install smth more convenient soon. guess the 9442 should do sr-iov, can try this. but not sure for 9242, best would be looking at specs and Kernel Module parameters to get an idea.

    will need some more days to get things sorted, the nics are only small part of the whole thing i'm currently digging through ...
     
    #4
  5. _alex

    _alex Active Member

    Joined:
    Jan 28, 2016
    Messages:
    851
    Likes Received:
    89
    Obviously bnx2x is used there ...
     
    #5
  6. BLinux

    BLinux Well-Known Member

    Joined:
    Jul 7, 2016
    Messages:
    1,866
    Likes Received:
    523
    maybe i'm missing something here, but it's not obvious to me it is bnx2x.

    from what i can tell searching, it does appear to be instead, qlcnic:

    http://driverdownloads.qlogic.com/Q...t.aspx?ProductCategory=325&Product=1159&Os=65

    Drivers & Software - HPE Support Center.

    And, although I don't know if this info is current, looks like sr-iov support in the driver is in "tech preview":

    Network Adapter Feature Support in RHEL - Red Hat Customer Portal

    At least on RHEL/CentOS.

    I'm curious about the QLE3242 because I've seen it for quite cheap lately and wondering how good of an option it would be for 10GbE in a VM environment, as an alternative to Mellanox options which cost a bit more.
     
    #6
  7. arglebargle

    arglebargle H̸̖̅ȩ̸̐l̷̦͋l̴̰̈ỏ̶̱ ̸̢͋W̵͖̌ò̴͚r̴͇̀l̵̼͗d̷͕̈

    Joined:
    Jul 15, 2018
    Messages:
    315
    Likes Received:
    84
    I picked one of these up for $15 on eBay last week to test SR-IOV, unfortunately I haven't found vendor driver source anywhere so I'll need to install RHEL on a USB drive to really test it.

    One note first, I'd like to jump right into testing this card but it runs hot and I mean H. O. T. -- I burned my hand swapping it out after it'd run for 15 minutes last night. I need to find some way to cool it before I do any real testing. The heatsink literally smells like a space heater does in winter, it's absurd how much heat this card produces.

    That PCIe 2.0 19W note in the dump is 100% accurate, the card idled at 16-17W with no DAC or module plugged in.

    I dumped some info in the short time I had the card booted and @BLinux was right, this card uses qlcnic:

    Code:
    01:00.0 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev 54)
        Subsystem: Hewlett-Packard Company NC523SFP 10Gb 2-port Server Adapter
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 37
        Region 0: Memory at fe600000 (64-bit, non-prefetchable) [size=2M]
        Region 4: Memory at fe890000 (64-bit, non-prefetchable) [size=64K]
        Expansion ROM at fe840000 [disabled] [size=256K]
        Capabilities: [40] MSI-X: Enable+ Count=32 Masked-
            Vector table: BAR=0 offset=00090000
            PBA: BAR=0 offset=00090800
        Capabilities: [80] Power Management version 3
            Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
            Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [88] Vital Product Data
            Product Name: HP NC523SFP 10Gb 2-port Server Adapter
            Read-only fields:
                [PN] Part number: 593715-001
                [EC] Engineering changes: A-5115
                [SN] Serial number: MY51170CXU
                [V0] Vendor specific: 19W PCIe Gen2
                [V2] Vendor specific: 5117
                [MN] Manufacture ID: 4e 45 33 32 31 30 34 30 34 2d 32 31 20 20 46
                [RM] Unknown: 30 31 30 38 30 32
                [YA] Asset tag: NA
                [RV] Reserved: checksum good, 0 byte(s) reserved
            End
        Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+
            Address: 0000000000000000  Data: 0000
        Capabilities: [c0] Express (v2) Endpoint, MSI 00
            DevCap:    MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
            DevCtl:    Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
                RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
                MaxPayload 128 bytes, MaxReadReq 256 bytes
            DevSta:    CorrErr+ UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
            LnkCap:    Port #0, Speed 5GT/s, Width x8, ASPM not supported, Exit Latency L0s <64ns, L1 <1us
                ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
            LnkCtl:    ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
            LnkSta:    Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
            DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
            DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
            LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                Compliance De-emphasis: -6dB
            LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
            UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
            UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
            UESvrt:    DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
            CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
            CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
            AERCap:    First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140 v1] Device Serial Number 59-69-46-61-6e-48-73-75
        Kernel driver in use: qlcnic
        Kernel modules: qlcnic
    01:00.1 Ethernet controller: QLogic Corp. cLOM8214 1/10GbE Controller (rev 54)
        Subsystem: Hewlett-Packard Company NC523SFP 10Gb 2-port Server Adapter
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 37
        Region 0: Memory at fe400000 (64-bit, non-prefetchable) [size=2M]
        Region 4: Memory at fe880000 (64-bit, non-prefetchable) [size=64K]
        Expansion ROM at fe800000 [disabled] [size=256K]
        Capabilities: [40] MSI-X: Enable+ Count=32 Masked-
            Vector table: BAR=0 offset=00091000
            PBA: BAR=0 offset=00091800
        Capabilities: [80] Power Management version 3
            Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
            Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [88] Vital Product Data
            Product Name: HP NC523SFP 10Gb 2-port Server Adapter
            Read-only fields:
                [PN] Part number: 593715-001
                [EC] Engineering changes: A-5115
                [SN] Serial number: MY51170CXU
                [V0] Vendor specific: 19W PCIe Gen2
                [V2] Vendor specific: 5117
                [MN] Manufacture ID: 4e 45 33 32 31 30 34 30 34 2d 32 31 20 20 46
                [RM] Unknown: 30 31 30 38 30 32
                [YA] Asset tag: NA
                [RV] Reserved: checksum good, 0 byte(s) reserved
            End
        Capabilities: [a0] MSI: Enable- Count=1/32 Maskable- 64bit+
            Address: 0000000000000000  Data: 0000
        Capabilities: [c0] Express (v2) Endpoint, MSI 00
            DevCap:    MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
            DevCtl:    Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
                RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
                MaxPayload 128 bytes, MaxReadReq 256 bytes
            DevSta:    CorrErr+ UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
            LnkCap:    Port #0, Speed 5GT/s, Width x8, ASPM not supported, Exit Latency L0s <64ns, L1 <1us
                ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
            LnkCtl:    ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
            LnkSta:    Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
            DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
            DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
            LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
            UESta:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
            UEMsk:    DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
            UESvrt:    DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
            CESta:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
            CEMsk:    RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
            AERCap:    First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [140 v1] Device Serial Number 59-69-46-61-6e-48-73-75
        Kernel driver in use: qlcnic
        Kernel modules: qlcnic
    
    Code:
    ethtool -i <nic>
    driver: qlcnic
    version: 5.3.66
    firmware-version: 4.8.20
    expansion-rom-version:
    bus-info: 0000:01:00.0
    supports-statistics: yes
    supports-test: yes
    supports-eeprom-access: yes
    supports-register-dump: yes
    supports-priv-flags: no
    root@ted:~# modinfo qlcnic
    filename:       /lib/modules/4.15.18-4-pve/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko
    firmware:       phanfw.bin
    version:        5.3.66
    license:        GPL
    description:    QLogic 1/10 GbE Converged/Intelligent Ethernet Driver
    srcversion:     1ACB70F0C897B41034DC5BC
    alias:          pci:v00001077d00008440sv*sd*bc02sc00i00*
    alias:          pci:v00001077d00008040sv*sd*bc02sc00i00*
    alias:          pci:v00001077d00008C30sv*sd*bc02sc00i00*
    alias:          pci:v00001077d00008830sv*sd*bc02sc00i00*
    alias:          pci:v00001077d00008430sv*sd*bc02sc00i00*
    alias:          pci:v00001077d00008030sv*sd*bc02sc00i00*
    alias:          pci:v00001077d00008020sv*sd*bc02sc00i00*
    depends:
    retpoline:      Y
    intree:         Y
    name:           qlcnic
    vermagic:       4.15.18-4-pve SMP mod_unload modversions
    parm:           qlcnic_mac_learn:Mac Filter (0=learning is disabled, 1=Driver learning is enabled, 2=FDB learning is enabled) (int)
    parm:           use_msi:MSI interrupt (0=disabled, 1=enabled) (int)
    parm:           use_msi_x:MSI-X interrupt (0=disabled, 1=enabled) (int)
    parm:           auto_fw_reset:Auto firmware reset (0=disabled, 1=enabled) (int)
    parm:           load_fw_file:Load firmware from (0=flash, 1=file, 2=POST in fast mode, 3= POST in medium mode, 4=POST in slow mode) (int)
    
    If you have a machine that supports ARI forwarding on the upstream PCIe switches and bridges you might want to look at the NC552SFP instead. It's the same price and the in-box linux kernel driver supports SR-IOV, the only catch being that it requires ARIFwd+ for SR-IOV to function. The NC552 does use the bnx2x driver, BTW.
     
    #7
    Last edited: Sep 13, 2018
Similar Threads: NC523SFP QLogic
Forum Title Date
Networking Xsigo IS24 F-X430083 or QLogic SilverStorm 9024-CU24-ST2-DDR Sep 1, 2017
Networking Old Brocade (QLogic) BR-1020 CNA on Windows 10 Jul 19, 2017
Networking Qlogic QLE3142/Netxen NX3-20G Nic Issues Feb 4, 2017
Networking QLogic QLE8152 Networking setup Feb 29, 2016
Networking Qlogic QLE3142/ Netxen NX3-20G Ubuntu drivers Jun 21, 2015

Share This Page