Unique OEM Dual LSI 2308 + Mellanox ConnectX-3 VPI Combo Expansion Cards

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
B

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,216
1,608
113
I pulled these out of a Data Direct Networks appliance last year and never got around to playing around with them. I think as far as expansion cards go, these are pretty unique. You have 2 x LSI 2308 HBAs (under the top left and top right heatsinks), which are each hooked up to pair of SFF-8614 external SAS ports (4 total). On top of that, there is also a Mellanox ConnectX-3 VPI NIC with single QSFP port (bottom left heatsink). To make all the magic happen, there is a 32 lane PCIe switch (PLX 8732), so no dealing with PCIe bifurcation or the likes (bottom right heatsink). As such, they show up as 3 independent devices and take the standard drivers so far as I can tell.

Under Linux, everything is recognized, however I get errors on the HBAs for some reason. Maybe they need to be flashed with a current IT/IR firmware?

Code: 
bluefox@test1:~$ dmesg | grep mpt2sas
[    2.317399] mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (8139932 kB)
[   17.736004] mpt2sas_cm0: sending diag reset !!
[   48.925580] mpt2sas_cm0: diag reset: SUCCESS
[   53.899130] mpt2sas_cm0: _base_spin_on_doorbell_int: failed due to timeout count(10000), int_status(c0000000)!
[   53.900847] mpt2sas_cm0: doorbell handshake int failed (line=4625)
[   53.902538] mpt2sas_cm0: _base_get_ioc_facts: handshake failed (r=-14)
[   53.904383] mpt2sas_cm0: failure at /build/linux-uI6OXA/linux-4.15.0/drivers/scsi/mpt3sas/mpt3sas_scsih.c:10768/_scsih_probe()!
[   53.920618] mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (8139932 kB)
[   69.298519] mpt2sas_cm1: sending diag reset !!
[  100.508994] mpt2sas_cm1: diag reset: SUCCESS
[  105.482682] mpt2sas_cm1: _base_spin_on_doorbell_int: failed due to timeout count(10000), int_status(c0000000)!
[  105.484531] mpt2sas_cm1: doorbell handshake int failed (line=4625)
[  105.486350] mpt2sas_cm1: _base_get_ioc_facts: handshake failed (r=-14)
[  105.488336] mpt2sas_cm1: failure at /build/linux-uI6OXA/linux-4.15.0/drivers/scsi/mpt3sas/mpt3sas_scsih.c:10768/_scsih_probe()!
I have not had time to tinker with them further and likely won't anytime soon (seems telecommuting full time just means I work more). NIC appears to work as expected at least.

 
Last edited:
  • Like
Reactions: Sleyk, Patrick, rocketpanda40 and 5 others
B

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,216
1,608
113
It is. About the same size as my desktop GPU, sans the heatsink of course.
 
B

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,216
1,608
113
Unfortunately I don't recall (and can't check now). Maybe one of the people that bought one can answer once USPS delivers them.
 
C

Crazywatermelon

New Member
May 18, 2016
9
7
3
Received mine yesterday, no issues with the card and as reported the NIC works no issue but I can't get the 2308 chips to work. I have had no success flashing the card, sas2flash complains when running any command and says it needs firmware but it rejects any firmware even when using the older sas2flash (P14) that's usually fine with anything and Megarec won't detect it. If anyone winds up being able to flash it I'd appreciate a hand.
 
F

Freebsd1976

Active Member
Feb 23, 2018
404
73
28
Crazywatermelon said:
Received mine yesterday, no issues with the card and as reported the NIC works no issue but I can't get the 2308 chips to work. I have had no success flashing the card, sas2flash complains when running any command and says it needs firmware but it rejects any firmware even when using the older sas2flash (P14) that's usually fine with anything and Megarec won't detect it. If anyone winds up being able to flash it I'd appreciate a hand.
Click to expand...
the nic is qcbt or fcbt？
 
B

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,216
1,608
113
Quite the write-up! Have to say, it's a bit disappointing that they did that considering the server I pulled these out of was not one of their proprietary ones, but in fact a Supermicro 6037R-TXRF (only deviation from stock was adding a second fan to cool the cards). I bought it off eBay last year, but as expected, the drives had been pulled. Photo if you're curious: https://drunkencat.net/misc/DSC_0892.jpg

I think the only thing left to try would be to use an EEPROM programmer to flash the LSI chips directly. Presumably they're the 8 pin ones with the yellow dot directly to the right of the respective heatsinks. CH341A + SOIC8 clip (+ maybe a 1.8V adapter depending on the chip) only runs ~$10 on eBay. Not sure how much more effort you'd want to put into these though?
 
  • Like
Reactions: cesmith9999, Sleyk and JoshDi
D

Dave Corder

Active Member
Dec 21, 2015
326
217
43
42
I know this is an older thread, but I just stumbled upon it this past weekend. I'm intrigued a bit by this card - it'd save me a slot in my somewhat constrained whitebox FreeNAS build, if only it actually worked

I'm kinda tempted to pick one of these up and try flashing a new SBR directly to the EEPROM - I think I have all the pieces I'd need for that, but I've not done it before. Is there anyone who might be able to assist with that if I wanted to give it a try?
 
B

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,216
1,608
113
Shouldn't be too much work. EEPROM for the HBAs is the 8 pin chip to the right of each heatsink with a yellow dot on it. With a CH341A programmer and SOIC8 clip, you should be all set.
 
D

Dave Corder

Active Member
Dec 21, 2015
326
217
43
42
Well, just snagged one of these cards from a seller on eBay for 20 bucks. Looks like I have a weekend project coming up.

I've used my lunch break to brush up on LSI flashing and SBRs, which took me to the threads on cross-flashing the Dell integrated storage slot cards. And there's some info there on a procedure to cross-flash using megarec and then lsirec/lisutil to flash the card in Linux instead of sas2flash. I might be able to use some of that info with this card...will have to do some testing before I go the EEPROM route, just to see if I manage to stumble on anything that @Sleyk didn't.

Actually, scratch that, first step will be to dump the SBR with the CH341A so I have a backup in case I accidentally brick something, and then deconstruct it to see if it can reveal any secrets.
 
sparx

sparx

Active Member
Jul 16, 2015
326
120
43
Sweden
You should probably be able to flash this card with the right binary file. For 3008 based cards there is a check if you are doing a correct upgrade for the card, and since you dont have a firmware file for this specific card but trying to run a "vanilla" card in it, it will not pass the sa2flash check and plainly refuse to upgrade.
I would download all the 2308 releases that are available. If you can get the binary from the card itself you should be able to hex compare the binary from the card with a vanilla card 2308(of same release version) and see what changes are made. There will be a few. But it could be manageable to make the corrections in a newer vanilla binary to get it working.
 
You must log in or register to reply here.
Top Bottom