10G network cards that work in a EMC/Lenovo Storcenter PX12-350r

[TheUnnamedNewbie]

I have a storcenter px12-350r that I'm using as my main nas. It has got 8 1gig connections at the moment (4 by default, 4 more through a network card I had laying around and didn't need for anything else, so under the motto "more ports = more better" decided to put it in even though there is pretty much no point).

I have a few Mellanox ConnectX2 cards, but they don't seem to get recognized. (I think it's driver related - makes sense that they don't include drivers for them in their custom debian-based OS).

Can anyone, with reasonable certainty, tell me if there are other 10g network cards that do work in this NAS? I was thinking of trying an intel card since they pretty much use one big driver lib iirc, but before I go spend another 100 bucks on a second hand intel nic, I would like to hear what people here have to say about it.

Other suggestions related to this (like, someone might know how to get those connectx2 cards to work) are also welcome.
I'm already using the linux active load balancing on the NAS, but, of course, this only helps for multiple connections, and doesn't impact the speed of single file transfers.

 

[Samir]

I know this is a big time bump on this thread, but I just got one of these and wanted to share the answer.

The official card which seems to work straight out of the box is actually a Qlogic/Broadcom BCM957712A1202G. It was sold as Iomega part number 36040. You can find the card on ebay under its Broadcom part number for a reasonable price. I haven't tried to upgrade mine yet, but may in the future.

It would be interesting to know if any other Qlogic/Broadcom cards might work that are in the same family as the BCM957712A1202G if anyone knows what the other Qlogic/Broadcom part numbers might be.

 

[Stephan]

Wisdom of the Ancients

xkcd.com

Never too late to share insights. In three months a homelab enthusiast from Bokhtar, Tajikistan will run up and down the road with joy because you answered the question that has tormented him in his sleep for eight years.

 

[Samir]

So digging a bit more now that I've got to play with this PX12-400R. When looking at dmesg from the console, there's an Intel and Broadcom driver that's there, so I think some of the popular Intel nics should work too. In fact, I recall the driver name is 'ixgbe' I believe which in a search led me to various nics in the x520/x540 line. I should have also remembered the Broadcom driver name as that might have led to some other part numbers--rats, I'm going to have to get that info. Time to power it back up.

Another interesting thing about the PX12-400R is that it natively supports sas drives even though there's nothing about it in the documentation or marketing material. You would think they would put this out as a selling point. It only crossed my mind when I noticed the backplane connectors were sas, not sata. And I didn't think anything of it as synology does this on some of their units. But to my surprise, a 400GB 15k Cheetah drive powered right up and was available in the webui for configuration. Very cool box since it's probably very easy to remove the stock usb dom and put something else, especially with a vga connector and serial port built-in too. I guess the hard part would be finding support for the sas controller, but I'm sure the usual linux flavors can probably handle it.

Okay, so it's booted, ixgbe is the confirmed the driver name for Intel, and the Broadcom one is bnx2x, which has a nice page here that goes into detail on the driver: 10. BNX2X Poll Mode Driver — Data Plane Development Kit 24.03.0 documentation

I think this will definitely open the doors to finding some cheap 10Gb nics for the PX12 series.

 

[JumpingJackFlash]

I own two Lenovo PX12 units. The PX12-450r has a Broadcom 57810S dual-port SFP+ 10 Gbps Fiber NIC (using the 'bnx2x' Linux driver), and the PX12-400r has an Intel X520-DA2 (82599) dual-port SFP+ 10 Gbps Fiber NIC (using the 'ixgbe' Linux driver). Both loaded with no issues. You should be able to utilize any NIC supported by Debian 7.11 (last/latest internal OS with the latest FW update). One adapter that DID NOT work for me was an Emulex OCE11102 dual-port SFP+ Fiber NIC.

 

[Samir]

Thank you for posting your research! I hadn't updated from my test results so I'll do that now.

I've installed a dual 10GBase-T card that's the HP 656594-001 530T. It's broadcom based and was plug and play. I haven't tried to see if it will max out the 10Gb yet, but it maxed out 1Gb in read tests using some old 300GB SAS drives set up RAID0.

 

[JumpingJackFlash]

I typically see 400-500MB/s transfers between clients and the units, and that's for about the first 8GB transferred. After that, they drop to 280-300MB/s. I maxed out the RAM in each unit at 32GB, but I have not had the most success tweaking the underlying configuration to get the arrays to cache more data. I have done a couple of test copies between the units using the internal apps, and they peaked at about 700MB/s. I should also note that I am using all traditional SATA HDDs in the units in four-drive banks configured each as a RAID5. One has all 4TB drives, and the other has 2/3/4TB arrays respectively. The latest/last firmware does allow for larger drives, but I have not been able to afford the upgrades yet.

 

[Samir]

Thank you for sharing additional details! Did you use 8GB DDR3 ECC UDIMMs to get to 32GB? Also, I wonder if SAS drives will be far faster than SATA. My 'free to me for testing' 300 and 450GB SAS drives could probably break the 1Gb limit just as a single drive if they were in perfect condition (these drives have bad sectors, clicking and other issues so they're far from optimal but are still functional hence why they were free.

 

[JumpingJackFlash]

SAS drives are technically backward-compatible with the SATA bus, but I have not tried that. The controllers in the PX12 units are SATA and are only capable of SATA-II speeds (which is probably why the transfer rates drop to ~300MB/s after the buffering runs out). As for the RAM, you have to order unbuffered ECC DIMMs. The type I ordered last was Samsung 8GB M391B1G73QH0-YK0 PC3L-12800E 2Rx8. It's ridiculous how the RAM market has turned upside down. I paid $6.99 each over three years ago; the cheapest price as of this post on E-Bay is ~$20 each plus shipping.