Lenovo 430-8i 430-16i HBA (LSI 3408, 3416) with ZFS

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

jena

New Member
May 30, 2020
28
6
3
Hi all,

In my previous post, I mention that our research lab's mini server room and needs a small storage server running TrueNAS.
I got initial quote from Lenovo and will also try DELL and ixSystem later.

I think I might get a more accurate answer here and people are generally very knowledgeable about varies different brand of servers.

In my scenario, buying from three big brand (DELL, HPE, Lenovo) yields much better discount and support over Supermicro for a research lab in the University (I personally like Supermicro myself for its simplicity and flexibility).

Lenovo SR655 has only two HBA options: 430-8i, 430-16i, based on LSI 3408, 3416 respectively.
In the datasheet, it says:
Non-RAID (JBOD mode) support for SAS and SATA HDDs and SSDs (RAID not supported);
End-to-End CRC with Advanced Error Reporting; "
Configuration through
  • XClarity Provisioning Manager UEFI interface
  • XClarity Controller web interface
  • XClarity Administrator Configuration Patterns
  • StorCLI command-line interface
  • LSI Storage Authority (LSA) GUI interface
  • UEFI Human Interface Infrastructure (HII)
I read the forum discussion about Lenovo 540-8i RAID card, also based on LSI 3408
Lenovo 530 datasheet

Question o_O : Based on both information about 430(HBA) and 530(RAID), Does the datasheet already indicate that 430 HBA is running under IT mode and thus can be used in ZFS without cross-flash LSI vanilla IT firmware?

The reply at TrueNAS forum seems to suggest that cross-flash is necessary
Q: 430-8i HBA based on LSI 3408
A [by senior member#1] : Both are supported under the mpr driver but you will likely have to flash them from the OEM firmware to HBA or IT firmware. Ensure that this is possible and available from your OEM, or there are instructions on how to flash your specific OEM card into its LSI equivalent (LSI 9400-8i or 9400-16i I believe)

Q: For 430-8i, if TrueNAS already sees individual drives, could I assume that I don't need firmware flash?
A [by senior member#2]: Unfortunately not. The problem with unsuitable storage controllers is not so much that they "simply don't work". It is instead edge-case situations when things may go south. For more details please check "Don't use RAID" from the recommended readings in my signature.
I get his point. But I cannot and Lenovo will not cross-flash the firmware to LSI vanilla IT mode version. (it will void the warranty)

It kind of means that the only two options are Supermicro and ixSystem's server.
Basically, it almost like using new servers from other vendors automatically disqualify the use of TrueNAS.

Just my 2 cents thought:
  • Large business or institution are probably using large scale proprietary solutions, like IBM Spectrum Scale, or other fancy stuff. So TrueNAS is not appealing to them.
  • Customers who would consider TrueNAS are medium/small business or individual research labs (I got this impression from a few use case published on ixSystem website, like mentioning of a research lab at UCSD).
  • The server option (R, X, M series) that ixSystem offers are quite limited and specs and pricing are not transparent (also old, still on 2nd gen Xeon). If these options are not suitable, or the customer sometimes have to order from a particular vender due to an agreement, or If ixSystem's server option is 30-50% more expensive than the equivalent specs from the big three, the customer most likely simply give up using TrueNAS or ZFS.
  • I understand that testing and certification take a lot of effort. But testing an already HBA card like Lenovo 430 seems to be a low hanging fruit (maybe it is very close to a vanilla LSI card). I would love to see that ixSystem to sell subscription for TrueNAS support on limited selections of other OEM brand server. In fact, they did certify a server from Asigra (based on Supermicro).
Thanks!

Summary of post at TrueNAS forum

Our research lab is considering a TrueNAS storage server to store and feed bulk data to our computation node via dual 10G SFP+ or single 40G QSFP.
It will be 8 x 3.5inch 18TB HDD in RAID-Z2, with lz4 compression; future expansion is another 8 HDD RAID-Z2 vdev

Work load
SMB:
  • 12TB of bulk "cold" data storage
  • Proxmox VM reads some data from storage server, compute, and writes data back. Most file reads are ~10MB image files or other large binary data files and a couple small 1-100kB configuration ASCII text files.
Syncthing backup for members file on their PCs
Maybe a few other jail plugins or Dockers

Lenovo SR655 2U server with 12 x 3.5in SAS front bay + 4 x 3.5in SAS mid bay + 4 x 2.5in rear SAS bay (mirrored boot drive)
1. EPYC Milan 7313 16 Core 3.0GHz
2. RAM 128GB 3200MHz
3. Mellanox ConnectX-4 Lx 10/25GbE SFP28
4. HBA 430-8i (LSI SAS3408), 430-16i (LSI SAS3416)
5. No slog
6. L2ARC (optional)
7. Mirrored Intel S4610 240GB as boot drive
 

jena

New Member
May 30, 2020
28
6
3
The Lenovo 430-8i/16i series are based on the interim gen, SAS3.5 generation chipsets, so they are definitely compatible with IT mode.

These cards actually are already full HBA cards, and they will show up by their chipset, so your good to purchase if you can get them cheaply.

The 430-8i and 16i will show up essentially as a 9400-8i and 9400-16i.

The senior forum member on the Truenas forum just didnt want to tell you advice since they probably never worked with a 430-8i/16i so they didnt want to give you a definite answer. However, I can tell you as far as ZFS and passthrough, Lenovo has already given the answer that it will work with all passthrough cases as it is a pure HBA :.)

IF ZFS/Truenas supports the LSI 9400-8i/16i, then the 430-8i/16i will work.

To fully answer your question, the 430 series was made STRICTLY as HBA cards, no RAID at all.

The 530 series was made as RAID compatible.

So think along the lines of the 430 series as a LSI 9211-8i card from the 2nd generation. It is a full IT mode HBA card.
The 530 series are like the LSI 9240-8i or the Dell H310. It has a RAID software ROM on it, but is fully flashable to IT mode, as it uses the same chipset as the 430 series.

Of note, these newer interim gen cards only update and flash through STORCLI, etc, so the flashing process is alittle unclear for now. Broadcom is seeking to do away with crossflashing to different vendors currently, so it remains yet to be seen what will happen, but for now, besides the good work the guys have done in crossflashing the 530-8i. However, the 430-8i and 16i will work in any application an LSI 9400-8i/16i would work in. :.)
Thank you so much!
I knew that I would get a good answer here.

TrueNAS forum sometimes could be a bit too "religious" about cross-flash vanilla IT-firmware:p.
I understand that they want to be on the "safe" side for ZFS newbies, so they insisted on vanilla LSI firmware or LSI HBA cards.

BTW, I found that LSI vanilla HBA card doesn't work with HPE's gen8 drive caddy (the activity LED doesn't work).
The LED works under HP branded LSI HBA or RAID card.

But on the other hand, IMO, this greatly impact commercial deployment of TrueNAS.
I don't think ixSystem could compete with the big three in hardware (not only in pricing, but more importantly in R&D and support).
Don't get me wrong, ixSystem is good at some special servers like their X and M series with high availability feature.
 
  • Like
Reactions: Sleyk