Solarflare SFP+ 10Gbe cards

[heromode]

solarflare activation keys are not free. They sell the cards at different price points depending on what features are activated, or sell the keys after the fact.

Thanks. I bought the cards because of the price, about 35 EUR per card. Just for standard 10Gbe networking, so i assume i don't need openonload, as i'm not running databases or webservers.

 

[heromode]

Old thread, but seems like the most appropriate title for generic questions about Solarflare network cards. I've bought a few S7120 adapters, first time dealing with these, tried updating the firmware, and got confused.

First loaded sfupdate in Windows 10, latest version I could find on Xilinx website. All seemed to go fine, firmware updated from v4.0.2 to v6.6.0, boot ROM from v.4.0.0.6589 to v5.0.7.1000, and "adapter" from v4.0.1.6625 to v6.2.7.1000. Then I booted to CentOS and out of curiosity tried sfupdate there. And those turned out to be newer versions, firmware updated to v.8.0.1, controller firmware to 6.2.7.1001, and boot ROM to 5.2.2.1006. But linux version seems to only "update" the firmware of the "1st port", outputting "ens3f1np1: firmware is not accessible, please use ens3f0np0". Just like in Kei-0070 post from 2019:



I booted back to Windows, and this is how it looks there:
View attachment 19484
After the first update (to v6.6.0, done with Windows version of sfupdate) both ports showed the same firmware, it looks like sfupdate in Linux does it differently and for some reason leaves one port's (?) firmware unrecognized.

The card seems to work fine anyway (both ports), so I'm just curious whether it's normal and why the Linux version does this?

Hi, i'm about to attempt a firmware update on my 7022F's, and first question here is, did you have multiple cards installed in multiple slots at the same time when you ran into this issue?

i'm thinking i should do one card at a time just based on this.

 

[dontwanna]

Hi, i'm about to attempt a firmware update on my 7022F's, and first question here is, did you have multiple cards installed in multiple slots at the same time when you ran into this issue?

i'm thinking i should do one card at a time just based on this.

Hi. No, I didn't have multiple cards installed, I was also updating one card at a time. Since these are dual port cards - that's how they're displayed in sfupdate, as 2 separate adapters, but it's just 1 card (with 2 ports, each called an "adapter"). I've still no idea why linux sfupdate does that - "updating" the firmware only for "one port". Doesn't seem to cause any problems, just looks weird.

 

[heromode]

Running Debian 11 Live boot USB stick:

Same issue phenomena here: (MAC addresses edited out)

wget https://www.xilinx.com/content/dam/xilinx/publications/solarflare/oem-driver-downloads/dell/8522/linux-utilities/sf-utils-v6.5.0.1003.x86_64.rpm.zip
apt install unzip
unzip sf-utils-v6.5.0.1003.x86_64.rpm.zip
apt install rpm2cpio
rpm2cpio sfutils-6.5.0.1003-1.x86_64.rpm | cpio -idmv
cd usr/sbin/

./sfupdate
    Solarstorm firmware update utility [v6.5.0]
    Copyright Solarflare Communications 2006-2017, Level 5 Networks 2002-2005

    ens2f0np0 - MAC: XX-XX-XX-XX-XX-XX
        Controller type:    Solarflare SFC9100 family
        Controller version: v4.6.3.1001
        Boot ROM version:   v4.4.0.1002

    This utility contains more recent Boot ROM firmware [v5.0.5.1002]
    - run "sfupdate --write" to perform an update
    This utility contains more recent controller firmware [v6.2.7.1000]
    - run "sfupdate --write" to perform an update

    ens2f1np1 - MAC: XX-XX-XX-XX-XX-XX
        Firmware cannot be accessed via this interface,
        please use ens2f0np0.

./sfupdate --write

    Solarstorm firmware update utility [v6.5.0]
    Copyright Solarflare Communications 2006-2017, Level 5 Networks 2002-2005
    ens2f0np0: updating controller firmware from 4.6.3.1001 to 6.2.7.1000
    ens2f0np0: will be disabled during controller firmware update
    To continue, press Y then Enter
    To skip this update, press Enter
    y
    ens2f0np0: writing controller firmware
    [100%] Complete                                                            
    [100%] Complete                                                            
    ens2f0np0: updating Boot ROM from 4.4.0.1002 to 5.0.5.1002
    ens2f0np0: writing Boot ROM
    [100%] Complete                                                            
    ens2f0np0: writing version information
    ens2f1np1: writing version information                                      
    [100%] Complete                                                            
    ens2f1np1: Firmware is not accessible, please use ens2f0np0.

./sfupdate
    Solarstorm firmware update utility [v6.5.0]
    Copyright Solarflare Communications 2006-2017, Level 5 Networks 2002-2005

    ens2f0np0 - MAC: XX-XX-XX-XX-XX-XX
        Firmware version:   v6.5.1
        Controller type:    Solarflare SFC9100 family
        Controller version: v6.2.7.1000
        Boot ROM version:   v5.0.5.1002

    The Boot ROM firmware is up to date
    The controller firmware is up to date

    ens2f1np1 - MAC: XX-XX-XX-XX-XX-XX
        Firmware cannot be accessed via this interface,
        please use ens2f0np0.

I'm assuming this is normal, and everything is ok.

 

[dontwanna]

I'm assuming this is normal, and everything is ok.

In Windows, sfupdate doesn't do that, it updates "both" ports. But the fw version is older, and I don't know whether it's possible to somehow extract newer fw from the Linux sfupdate and use it for updating in Windows. But yeah, cards work fine just like that, so I never bothered tinkering with it.

 

[heromode]

well, there is one controller, one ROM, and 2 ports. there is not two SFC9120 controllers, or two ROM chips. One can verify by peeking under the cooler My guess is it just can't READ the controller and boot ROM via the second interface. It's all good.

 

[heromode]

I was able to verify that it's all ok:

wget https://www.xilinx.com/content/dam/xilinx/publications/solarflare/oem-driver-downloads/dell/8522/linux-utilities/sf-sfreport.tgz
tar -zxvf sf-sfreport.tgz
./sfreport.pl

results in a sfreport-2022-04-15-01-51-49.html file.

opening the file in a browser reveals:

Network interfaces for Solarflare adapters
name    address    driver_version    controller_version
ens2f1np1    0000:01:00.1    5.10.0-8-amd64    6.2.7.1000 rx2 tx1
ens2f0np0    0000:01:00.0    5.10.0-8-amd64    6.2.7.1000 rx2 tx1

 

[Harmony]

Anyone else keep getting stuck on resetting adapter?

 

[heromode]

Anyone else keep getting stuck on resetting adapter?

Should not do that. Only thing i can think of is make sure you didn't accidentally download the OEM drivers and firmware, which are at the bottom of the same page as standard software here: NIC Software and Drivers

 

[dstrawsb]

Card arrived today. Tested it on one of the windows machines first and all signs were good, although the drivers would not install. Kept causing an NDIS SYSTEM_THREAD_EXCEPTION_NOT_HANDLED bluescreen. Not sure on the cause. But the card showed a link with my intel SFP installed and windows recognised the device which was good enough for me. Swapped it back out with my X710 card and put it into the server and surprisingly, it worked straight from the get go. SFC9120 driver present and a 10Gb link. Yay. Now all that I need to sort out is running some fibres.



Boot ROM firmware can be updated and there is a slightly newer controller firmware. This is the output from sfupdate

Solarstorm firmware update utility [v7.5.4]
Copyright Solarflare Communications 2006-2018, Level 5 Networks 2002-2005

enp1s0f0np0 - MAC: xx
    Firmware version:   v6.6.0
    Controller type:    Solarflare SFC9100 family
    Controller version: v6.2.7.1000
    Boot ROM version:   v5.0.7.1000

This utility contains more recent Boot ROM firmware [v5.2.2.1002]
   - run "sfupdate --write" to perform an update
This utility contains more recent controller firmware [v6.2.7.1001]
   - run "sfupdate --write" to perform an update

enp1s0f1np1 - MAC: xx
    Firmware cannot be accessed via this interface,
    please use enp1s0f0np0.

Does anyone know what that rear edge connector is used for?

 

[sp_00n]

Bought two of these. it is the very first adapter that does not work out of the box in Ubuntu for me. It is Solarflare SF432-1012. Maybe its dead. As I am not super experienced in linux, how should I proceed with driver installation and verification? cant see any ethernet interfaces using "lshw" except the one that is on the motherboard. edit: same goes for lspci

 

[ziggygt]

I just bought a pair of full height SF432-1012-R3 S7120 Solarflare cards on ebay for $30. They seem to work OK after a few struggles. They are an upgrade for my SFN6122F Dual and SFN5152F Single cards. I had hoped to use the half height brackets from the old cards. Of course they do not fit. The spacing between the transceivers is different and the bracket mounting holes are in a slightly different position. Might be able to hack them.

Biggest issue so far is the DAC compatibility. Arista Cisco and Mellanox work. Molex does not work. My Mellanox SPF+ transceivers work. I was surprised because the Molex DACs worked well in the SFN6122F. I wonder if there is something in the driver setup that can adjust for compatibility?

 

[mattventura]

I bought an SFN7501. Works fine on Linux, never got it to work on Win10, always just BSOD'd on startup.

 

[ziggygt]

I bought an SFN7501. Works fine on Linux, never got it to work on Win10, always just BSOD'd on startup.

I am getting BSOD with Win10 on a HP Z400 with a SFN6122F but I have similar card running on a Supermicro X8 motherboard. I hate to mess with it, but I’ll try to get a report out of the working config. The Mellanox cards I have run fine in win 10 so I am matching solarflare for TrueNas and Mellanox-2 for windows targets.

 

[etherbadger]

Bought two of these. it is the very first adapter that does not work out of the box in Ubuntu for me. It is Solarflare SF432-1012. Maybe its dead. As I am not super experienced in linux, how should I proceed with driver installation and verification? cant see any ethernet interfaces using "lshw" except the one that is on the motherboard. edit: same goes for lspci

i have several of these from ebay, listed as "Solarflare SF432-1012-R3 S7120 PCIe Dual SFP+ Ports"

I use them in HP elite desk 800gen4 SFF pcs running proxmox. I had to disable secureboot and fiddle around with the option rom settings to get them to show up to proxmox, and i think i also ended up setting their default personality to ethernet via the solarflare management tools in windows.

despite the sales detail, they show up in linux via lspci -v as

01:00.1 Ethernet controller: Solarflare Communications SFC9120 10G Ethernet Controller (rev 01)
        Subsystem: Solarflare Communications SFN7x22F-R3 Flareon Ultra 7000 Series 10G Adapter

The the solarflare firmware update tool in windows calls them "SFN5162F" as does the Solarflare Boot Manager menu.


I recall seeing them briefly in the boot sequence for "solarflare boot manager." do you see them in your box?

I did nothing for the driver, on proxmox or ubuntu when it had it in a different machine. currently it looks like

root@virt1:~# ethtool -i enp1s0f0np0
driver: sfc
version: 5.15.104-1-pve
firmware-version: 6.2.5.1000 rx1 tx1
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
root@virt1:~# modinfo sfc
filename:       /lib/modules/5.15.104-1-pve/kernel/drivers/net/ethernet/sfc/sfc.ko
license:        GPL
description:    Solarflare network driver

 

[heromode]

First thing you should do with these cards when buying is basically boot up a debian live USB, and check them out with the sfboot utility. I have the 7022's, so not sure about your older versions. The PDF is very detailed, but basically you should reset them to factory settings first, then if the OS doesn't see them, start by changing the 'boot image' setting, that can be 'all|optionrom|uefi|disabled' Mine work fine as UEFI only in proxmox / c612 chipset xeon E526xxv4's

you adapters might have a nonstandard amount of MSI-X interrupt limits, or might have physical port partitions, which is why it's important to reset them first using 'sfboot -c' or 'sfboot --clear', and then also update firmware using sfupdate.

Anyways, as you have different versions of the cards, i'll just paste what i have documented about downloading and installing the sfboot util on a live debian, incase it saves someone time. You should use the in-tree kernel drivers, and the 'sfcutil' is nothing more than a renamed 'ethtool' basically, so no need for those.

Just make sure to adapt the commands below in case your cards use a different set of utils, than my 7x22 series.

apt install dkms unzip alien
mkdir sfc
cd sfc/
wget https://www.xilinx.com/content/dam/xilinx/publications/solarflare/drivers-software/linux/SF-107601-LS-72_Solarflare_Linux_Utilities_RPM_64bit.zip
unzip SF-107601-LS-72_Solarflare_Linux_Utilities_RPM_64bit.zip
alien -c sfutils-8.2.4.1004-1.x86_64.rpm
dpkg -i sfutils_8.2.4.1004-2_amd64.deb

also as a note from the PDF:

Using the Solarflare Utilities bootable Linux disk image If you are unable to boot the system, you can instead use the Solarflare Utilities bootable Linux disk image. This is available from: https://support.solarflare.com/index.php?id=1960&option=com_cognidox This package is supplied as a ISO 9660 disk image. It contains thew same utilities as the Solarflare Linux Utilities RPM, as well as a 4.0 driver for Solarflare adapters. CAUTION: The driver contained in this package has only undergone QA for Utilities use, and should therefore not be used outside of this package or to carry production network traffic

edit: my lspci output is the same as @etherbadger 's above so he has 7x22 series.

 

[sp_00n]

I will try to investigate a little bit more but on two systems they are not listed via lspci nor I cant see them in the server's bios or during boot time. I asked here as it was really wierd for me that two cards I bought were dead.

 

[heromode]

I will try to investigate a little bit more but on two systems they are not listed via lspci nor I cant see them in the server's bios or during boot time. I asked here as it was really wierd for me that two cards I bought were dead.

just boot with a debian live USB, and install the sfboot utility as described above, then simply run 'sfboot'.
If that doesn't see the cards then you have issues.

 

[christt]

Help pls, i bought an X2522 plus for testing its onload,efvi and tcpdirct.From the doc i got "In all cases an Onload AppFlex activation key is required on the adapter.",does it mean i should buy an activation key to install or the x2522plus has already had one key when produced?

 

[aero]

Help pls, i bought an X2522 plus for testing its onload,efvi and tcpdirct.From the doc i got "In all cases an Onload AppFlex activation key is required on the adapter.",does it mean i should buy an activation key to install or the x2522plus has already had one key when produced?

You do not need any activation key for onload, efvi, tcp direct, or ptp with the PLUS version

you can check the licensed features once you install it and run "sfkey"