Fibre channel HBA running at slower speed?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

dswartz

Active Member
Jul 14, 2011
611
79
28
For my homelab, I ebay'ed a qlogic 2562 and 2 2560 HBAs. Idea is that the 2 hypervisors will have their storage on the san. SAN is an esos run from memory distro. So the HBAs are 8gb because that seemed adequate (hyper-v doesn't need that much disk power, AFAIK). The ESOS appliance automatically puts the 2562 in target mode, and everything works. Except the speed is too slow. The default qlogic setting in the BIOS is 'auto', but the 2 units seem to sync up at 4gb, not 8gb. If I try to force 8gb in both units, they don't sync up at all (the 3 speed leds, 2, 4 and 8 on the IO panel just blink continuously). I've tried a 10gb twinax cable, and a multimode fibre cable I got off amazon, leveraging the 2 SFP the 2562 came with. Before I get into things like flashing newer firmware, does anyone have any ideas? I mean 4gb isn't *bad*, but I'd love to get the 8gb the units supposedly provide. Thanks!
 

tinfoil3d

QSFP28
May 11, 2020
901
426
63
Japan
Never touched FC. Did you confirm that SFP modules are indeed 8gb?
4gb may also sound like a pcie lanes shortage if you're lucky to be running them in pcie2 age motherboard and they're "downgraded" to single lane. Well they might also be pcie 2.0 devices and downgraded to x1 link due to lack of 4 or 8 available lanes.
 

MountainBofh

Beating my users into submission
Mar 9, 2024
321
239
43
While I've never dealt with Fiber Channel myself...when I've had similar issues with SAS HBA's in the past, firmware updates often do wonders.
 

dswartz

Active Member
Jul 14, 2011
611
79
28
Never touched FC. Did you confirm that SFP modules are indeed 8gb?
4gb may also sound like a pcie lanes shortage if you're lucky to be running them in pcie2 age motherboard and they're "downgraded" to single lane. Well they might also be pcie 2.0 devices and downgraded to x1 link due to lack of 4 or 8 available lanes.
Well, I also tried a 10gb DAC that works fine as an ethernet cable. No difference. Both cards are seated in 3.0 X8 slots, so this is a mystery. I may go the route of upgrading FW in both cards, but then again 400MB/sec isn't awful either :)
 

tinfoil3d

QSFP28
May 11, 2020
901
426
63
Japan
Yes they may be in x16 and be x16 devices but in reality downgraded to x1 if you have many other pcie devices. These are pcie gen 2 devices so it truly looks as if one of your systems is overloaded with pcie devices and one of HBAs gets only a single lane, which is 5GT/s, if we remove the protocol overhead that's roughly gonna come out at 400-something MB/s.
What are your systems and OS? Its generally super easy to boot live linux and check dmesg | grep -i limited to figure out and solve any anomalies
 

dswartz

Active Member
Jul 14, 2011
611
79
28
Okay, so a bunch of digging: one box is a supermicro 1U with a single x8 (but seems to be electrically x4? - maybe the riser did that? the manual seems to say 3.0, but ok.). I had no spare server, so I pulled out my old gaming desktop that has 1 X8 3.0 slot and a couple of x1 slots (unused). Also an x16 for the graphics card. I did the '-limited' as you suggested, and both HBAs are being reported as being limited to 16gb/sec, so that's a mystery. Might end up settling for 4gb - still decent performance....
 

dswartz

Active Member
Jul 14, 2011
611
79
28
Go figure. Out of paranoia, pulled both HBAs, reseated them, and... 8gb/sec. Time to knock off for a stiff drink lol.
 
  • Like
Reactions: nexox