Broadcom 9600-24i -> TRIM unsupported

[NablaSquaredG]

Hey folks,

I'm having issues with a Broadcom 9600-24i controller and ZFS reporting "TRIM unsupported"

Specs:

• H12SSL-NT + EPYC 7713 @ BIOS 2.8
• HBA: Broadcom 9600-24i (Newest Firmware 8.9.1.0-00000-00002)
• OS: Proxmox VE 8.2.4 @ Kernel 6.8.8-2-pve
• SSDs: 24x Micron 5300 PRO (All Firmware D3MU001) @ Backplane Supermicro BPN-SAS3-216A-N4 (Direct Attach)

I tried both the in-kernel driver and the Broadcom 8.9.1.0.0-1 driver to no avail.

ZFS reports "(trim unsupported)"

lsblk reports

NAME        DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda                0        4K       0B         0
├─sda1 0 4K 0B 0
└─sda9 0 4K 0B 0

Any ideas? I'm already in contact with the Broadcom support, but they are sloooooow. Somebody at Level1Techs forum had the same issue: LSI 9600 24i no TRIM support for sata SSD?

I'm aware that I can force TRIM to enabled via

/etc/udev/rules.d/10-trim.rules

ACTION=="add|change", SUBSYSTEM=="scsi_disk", ATTRS{model}=="Micron_5300_MTFD", ATTR{provisioning_mode}="unmap"

but that's a hack...

 

[BLinux]

I don't know that particular Micron SSD, but a lot of Broadcom/LSI controllers require certain SATA features on the SSD to support TRIM. In particular:

1. Data Set Management TRIM supported (limit 8 blocks)
2. Deterministic read ZEROs after TRIM

Do you know if the Micro 5300 have these features?

 

[NablaSquaredG]

I don't know that particular Micron SSD, but a lot of Broadcom/LSI controllers require certain SATA features on the SSD to support TRIM. In particular:

1. Data Set Management TRIM supported (limit 8 blocks)
2. Deterministic read ZEROs after TRIM

Do you know if the Micro 5300 have these features?

Yes, of course.. It's an Enterprise SSD after all

sudo hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
Model Number: Micron_5300_MTFDDAK1T9TDS
...
Firmware Revision: D3MU001
[...]
* WRITE BUFFER DMA command
* READ BUFFER DMA command
* Data Set Management TRIM supported (limit 8 blocks)
* Deterministic read ZEROs after TRIM
Security:
[...]

 

[dapouet]

Hi!
I'm the one who posted on level1, did you heard back from Broadcom or find another solution?

I admit I was fed up and reverted to using my 9300 16i with a hacky solution to connect my 24 drives, but I still have the card so...

 

[NablaSquaredG]

I'm the one who posted on level1, did you heard back from Broadcom or find another solution?

Broadcom is tracking it under an issue number, but not fixed yet...

 

[dapouet]

Broadcom is tracking it under an issue number, but not fixed yet...

Thanks for the update!

 

[NablaSquaredG]

Thanks for the update!

Broadcom replied..

Hello Robin,

Engineering just got back to me and said that "Unmap/Trim is supported only for SAS/NVMe "

It looks like you are using SATA SSD's:
"Controller is used with 24 disks Micron 5300 PRO (Firmware D3MU001) on a direct attach backplane (Supermicro BPN-SAS3-216A-N4)"

Thank you and let me know if you have any questions,
Storage and Ethernet NIC Technical Support
Broadcom Inc.
Joseph

9600-24i only support TRIM for SAS/NVMe? WTF is this...

My expectations were low, Broadcom. But holy f***.

 

[dapouet]

Broadcom replied..



9600-24i only support TRIM for SAS/NVMe? WTF is this...

My expectations were low, Broadcom. But holy f***.

Thanks for the news, this is pretty incredible that a basic feature like that is unsupported and it doesn't seem to bother them...