Brocade 1020 CNA 10GbE PCIe Cards

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tg2

New Member
Sep 21, 2015
2
1
3
39
I would just like to add that on all versions of ubuntu after 12.04 the version of brocade drivers that are included with the kernel is problematic and will result in rx queue kernel panics under high network and CPU load.

The issue has been resolved by the inclusion off 3.2.5.1 drivers in the 4.2+ linux kernels.

4.1 has the new driver and firmwares, but is broken and will not initiate the cards at boot time:

Code:
[   15.069887] scsi host4: Brocade FC/FCOE Adapter, hwpath: 0000:41:00.1 driver: 3.2.23.0
[   15.071153] bnad_pci_probe : (0xffff88061e4a5000, 0xffffffffc00d10c0) PCI Func : (2)
[   15.071718] bar0 mapped to ffffc90004100000, len 262144
[   15.072094] BNA: Initialization failed err=1
[   15.072156] bnad_pci_probe : (0xffff88061e4a6000, 0xffffffffc00d10c0) PCI Func : (3)
[   15.072207] bar0 mapped to ffffc90004180000, len 262144
[   15.072535] BNA: Initialization failed err=1

So, if you are experiencing the bna rx kernel panic stacktrace you can try updating to the newest kernel and firmwares as outlined below:

-----------------

The kernel files for can be installed for 64 bit systems as follows:

To get the firmwares:
(as root):

Code:
cd /lib/firmware
wget http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain/ctfw-3.2.5.1.bin
wget http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain/ct2fw-3.2.5.1.bin
wget http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/plain/cbfw-3.2.5.1.bin

To install 4.2.0 kernel (ubuntu team hasn't posted 4.2.1 yet - which is the current stable, although marked unstable I have been using it for a while without any stability issues - much less crashing than with the old 3.2.3.0 drivers/3.x combo)
(as root):

Code:
mkdir /tmp/420kernel/; cd /tmp/420kernel
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.2-unstable/linux-headers-4.2.0-040200_4.2.0-040200.201508301530_all.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.2-unstable/linux-headers-4.2.0-040200-generic_4.2.0-040200.201508301530_amd64.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.2-unstable/linux-image-4.2.0-040200-generic_4.2.0-040200.201508301530_amd64.deb
dpkg -i linux-headers-4.2.0*.deb linux-image-4.2.0*.deb

Once done, reboot and things should come up as normal.

A quick check should reveal that you are now running the newest firmware and drivers:
(as root):
Code:
modinfo bna; uname -a; cat /etc/*release*
filename: /lib/modules/4.2.0-040200-generic/kernel/drivers/net/ethernet/brocade/bna/bna.ko
firmware: ct2fw-3.2.5.1.bin
firmware: ctfw-3.2.5.1.bin
version: 3.2.25.1
description: QLogic BR-series 10G PCIe Ethernet driver
license: GPL
author: Brocade
srcversion: 82BFE8B363B3EB37FB14E65
alias: pci:v00001657d00000022sv*sd*bc02sc00i*
alias: pci:v00001657d00000014sv*sd*bc02sc00i*
depends:
intree: Y
vermagic: 4.2.0-040200-generic SMP mod_unload modversions
signer: Build time autogenerated kernel key
sig_key: B3:D5:4D:D8:43:77:6E:54:6A:4C:A7:1D:D4:58:67:FC:94:56:69:2F
sig_hashalgo: sha512
parm: bnad_msix_disable:Disable MSIX mode (uint)
parm: bnad_ioc_auto_recover:Enable / Disable auto recovery (uint)
parm: bna_debugfs_enable:Enables debugfs feature, default=1, Range[false:0|true:1] (uint)
Linux hostname 4.2.0-040200-generic #201508301530 SMP Sun Aug 30 19:31:40 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.3 LTS"
NAME="Ubuntu"
VERSION="14.04.3 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.3 LTS"
VERSION_ID="14.04"
HOME_URL="The leading OS for PC, tablet, phone and cloud | Ubuntu"
SUPPORT_URL="Official Ubuntu Documentation"
BUG_REPORT_URL="Bugs : Ubuntu"

If something doesn't work or you wish to revert - you can go back by removing the 4.2.0 kernel packages (grub should rebuild after)
(as root):
Code:
apt-get remove 'linux-headers-4.2.0*' 'linux-image-4.2.0*'
update-initramfs -uk all
update-grub
As a note, if your dpkg or apt script complains that the disk is full - you have to remove some older kernels taking up space in /boot (often its own partition of very small size) before you can install/remove any kernels. You probably have some kernels for 3.13, 3.16 or 3.19 in /boot that are no longer being used and could be moved/deleted.

Now you can reboot and the old kernel will show up with the old drivers.
 
Last edited:
  • Like
Reactions: Kristian

homelab74656

New Member
Jul 22, 2015
17
0
1
58
I recently got a Brocade 1020 for $40 including shipping on Ebay (I guess prices have gone up). Anyways, I have it installed in an HP DL180 G6. I am running vmware ESXi 6.0. The adapter is detected, but I have an issue. It is only showing up in my storage adapters as a Fiber Channel type. I cannot add it under networking. I know this is a converged adapter, but how do I get it to be detected so that I can use it for ethernet? I have two Mellanox connect x2 cards in my other HP G6 servers and they are detected as normal ethernet adapters.
 

kirlcheah

Member
Jan 24, 2015
67
2
8
42
I recently got a Brocade 1020 for $40 including shipping on Ebay (I guess prices have gone up). Anyways, I have it installed in an HP DL180 G6. I am running vmware ESXi 6.0. The adapter is detected, but I have an issue. It is only showing up in my storage adapters as a Fiber Channel type. I cannot add it under networking. I know this is a converged adapter, but how do I get it to be detected so that I can use it for ethernet? I have two Mellanox connect x2 cards in my other HP G6 servers and they are detected as normal ethernet adapters.
You will need to add in another driver from esxi. You will have to look Ethernet drivers instead of the fibre channel. I did the same last time and reload the drivers after downloading them and it works fine with both drivers. You can either run Ethernet or fibre channel.
 

homelab74656

New Member
Jul 22, 2015
17
0
1
58
I got the ethernet driver installed in ESXi for the 1020. I hooked it up to the two Mellanox cards with a 1m brocade active sfp+ direct attach cable going to each.

Unfortunately all hosts show the link is down. If I hook the two Mellanox cards together, they detect each other fine. The brocade doesn't work at all. The backs of the cards don't even light up.

Any suggestions?
 

kirlcheah

Member
Jan 24, 2015
67
2
8
42
I got the ethernet driver installed in ESXi for the 1020. I hooked it up to the two Mellanox cards with a 1m brocade active sfp+ direct attach cable going to each.

Unfortunately all hosts show the link is down. If I hook the two Mellanox cards together, they detect each other fine. The brocade doesn't work at all. The backs of the cards don't even light up.

Any suggestions?
Hi,

You will need to run it in VMware with the vswitch with ip. Set it in the same subnet such as 172.100.1.2 in box a and 172.100.1.3 in box b. You will then have the vm correpond to the IP address. It will work. However, I am only assuming this as you did not tell me your configurations.
 

homelab74656

New Member
Jul 22, 2015
17
0
1
58
Hi,

You will need to run it in VMware with the vswitch with ip. Set it in the same subnet such as 172.100.1.2 in box a and 172.100.1.3 in box b. You will then have the vm correpond to the IP address. It will work. However, I am only assuming this as you did not tell me your configurations.
Unfortunately I can't even get that far. I have the Mellanox cards connected right now and they are working fine, so I'm pretty sure it's not a software configuration issue in vmware. Unless I forgot to install something driver related. But the ESXi host sees the 1020 just fine, it just reports the link is down.
 

kirlcheah

Member
Jan 24, 2015
67
2
8
42
what optics are you using? by the way, are you connecting brocade to brocade 1020 instead of brocade 1020 to mellanox?

I am not sure with the mellanox cards but when I put in both side 1020 with active cable, it just light on. i am unsure as you are talking about mellanox and brocade 1020 together. so i am just assuming.
 

homelab74656

New Member
Jul 22, 2015
17
0
1
58
I am using Brocade branded active 1M direct attach twinax.

The cable goes directly from each 1020 port to each Mellanox card. If I unplug the Brocade, and connect the Mellanox together instead they work fine.
 

RTM

Well-Known Member
Jan 26, 2014
956
359
63
Crazy enough, 1020 to 1020 works great too!

So Connectx2 to Connectx2 works great. 1020 to 1020 works great. But Connectx2 to 1020 doesn't even light up!
You could try updating the firmware of the NICs, I seem to remember that there have been reports that updating the firmware of Brocade 1020 cards improved its compatibility with other NICs (although that was with SFP+'s and fiber optic cables), if you are a little lucky it might just do the trick.
 

homelab74656

New Member
Jul 22, 2015
17
0
1
58
At this point I think you're right. I was thinking that perhaps I got one of the cards that apparently only works with fiber SFP's and not direct attach, but after this latest test, that doesn't appear to be the case.
 

gunit

New Member
Mar 28, 2018
1
0
1
44
Hey guys,

I got the cards, but my Win7 pro machine refuses to recognize the cards even with drivers from Qlogic 2012 server install files.

Can anybody please share a link to drivers I could try or these cards are no good in 2018?
 

Marta

New Member
Mar 29, 2018
2
1
3
47
Hello!
If anybody has problem with modules for these cards, for distances more than a few meters, cheap $22 OPTION SFP+ modules (SR 300m MM) works with them without problems after firmware flash.
 

Milos

New Member
Feb 22, 2018
25
2
3
Black hole with internet
Are these card any good for using today ? Because they are like dual port for 10$ on ebay + some cable. It would be used in dell r710/720/r520 and win 10 pro machines.

Best regards
 

kirlcheah

Member
Jan 24, 2015
67
2
8
42
Hi,

It depends on your usage and OS. If you use a non compliant OS, you will need to troubleshoot. If you are using compliant OS, should work out of the box. However, with other cards dropping in price to usd 30 you might want to look elsewhere. This could be hot to some people liking