ZFS with Broadcom/LSI 3108 in JBOD mode

[TechTrend]

At a remote location we'll need to deploy a ZFS storage server using an OmniOS VM with napp-it over VMware ESXi. The only available server there has a SuperMicro X10DRH-CT motherboard with an onboard Broadcom 3108. In normal circumstances we would add a Broadcom 3008 to that motherboard for ZFS. Yet that change is not feasible for several weeks due to travel logistics.

Some blogs mention successfully using a Broadcom 3108 in JBOD mode for ZFS. It is certainly not an ideal solution, but would it be a viable until we get a 3008 controller installed? Pool would use 4 mirrors, each with two WD HC530 14TB SATA drives plus one spare. Risk of power interruptions is very low.

Thanks.

 

[ano]

I think you can reflash it to normal It, or run JBOD mode

 

[gea]

BroadCom 3108 is a hardware Raid 5/6 controller. You can't reflash to a simple HBA with the for ZFS wanted raidless IT firmware. What should be possible is to use single disks in a Raid-0 manner but do not use the adapter Raid 1/5/6 functions.

Another option may be to use the 3108 as bootcontroller for ESXi and passthrough Sata (if disks are Sata and not SAS)

 

[TechTrend]

Thanks for your responses. I configured the BroadCom 3108 in JBOD mode with single disks and no RAID. ZFS sees the individual drives. Created a pool with 4 mirrors and configured an iSCSI LUN as storage for 2 low activity VMs. It seems operational so far with those workloads. Will report if any problems come up as more workloads are added.

 

[Sean Ho]

Yes, others have reported that 3108 in JBOD is essentially IT mode, complete with SMART data, and the drives can be read on other HBAs, so there shouldn't be any RAID headers.

 

[gea]

IT mode is equal to flashed with a raidless IT firmware and the controller is connected via the minimalistic mpt_sas driver
This is not the case here. The firmware comes with the full raid stack and the controller connects via a different raid driver.

But if you see the individual disks and smart is working everything should be ok. Real IT mode and mpt_sas driver is more common (defacto standard for ZFS) has less functions with possibly less bugs and maybe a slightly better performance

 

[Sean Ho]

Yes, I agree that real IT mode is still preferable. Perhaps "3108 JBOD is essentially IT" is a little too strong of a claim; I only meant that it addresses the most basic prerequisites for using zfs or other software raid.

 

[lastb0isct]

Issue with the 3108 is a 16 drive limit on this motherboard =(

 

[bandit8623]

Issue with the 3108 is a 16 drive limit on this motherboard =(

disable and add a pci card. 9361-8i
or go it 9300-8i

 

[anniesteuber]

BroadCom 3108 is a hardware Raid 5/6 controller. You can't reflash to a simple HBA with the for ZFS wanted raidless IT firmware. What should be possible is to use single disks in a Raid-0 manner but do not use the adapter Raid 1/5/6 functions.

Another option may be to use the 3108 as bootcontroller for ESXi and passthrough Sata (if disks are Sata and not SAS) Hello Neighbor

Thanks for your responses.

 

[BackupProphet]

In bios/controller settings, you can set the device to jbod. Works great! I run several X10drh-ct with ZFS

 

[gea]

You need the Raid-1/5/6 driver that is not as common or tested as the basic HBA mrt driver for it/ir mode.
So not a best use case but a ok, at least it works.

 

[BeTeP]

BroadCom 3108 is a hardware Raid 5/6 controller. You can't reflash to a simple HBA with the for ZFS wanted raidless IT firmware.

SAS3108 is no more "hardware raid" than SAS3008. As in it uses the same general compute cores for all its processing needs. And yes, it absolutely can run the same firmware as 3008.

 

[gea]

This is not as I understand it.
Where have you seen the IT firmware or an info about IT firmware on 3108?

 

[ericloewe]

I'm not sure I've seen it done with the SAS3108, but the process has been documented for some SAS2 controllers. It's definitely not officially supported and Broadcom's tools will fight you every step of the way, but it's also true that the hardware is essentially compatible (which is also how they get MegaRAID firmware onto SAS3008s).

 

[BeTeP]

Where have you seen the IT firmware or an info about IT firmware on 3108?

In my own servers. Before yesteryear's influx of dirt cheap Inspur branded adapters I was routinely buying used 3108 controllers at half price of 3008 ones and flashing them to IT mode.