Help getting M1015 / LSI SAS2008 to work on Red Hat 8

Tinkerer

Member
Sep 5, 2020
45
14
8
Hey all, I am trying to get my old trusty M1015 to work in Red Hat 8. As you may not be aware, Red Hat in their infinite wisdom removed support for LSI SAS2008/2108/2116 and other controllers and they have since been included in elrepo.org. The controller is in IT mode without a BIOS.

I installed the module from that repo, loaded it but I don't think its working. I suspect because the firmware P20 I am running isn't detected by the module? Im guessing ...

I think I once had to do a similar thing by making a module "see" a piece of hardware that was compatible but not detected due to these codes [1000:0072]. A simple test was to echo something to /proc/... but I'm not sure if that's actually the problem and I can't really find good source to help me figure it out.

If anyone knows anything I'd love to hear it! Thanks in advance!

Code:
# lsmod | grep mega
megaraid_sas          172032  0

The card is identified as:
Code:
0000:65:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
        Subsystem: Broadcom / LSI Device [1000:3020]
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 32 bytes
        Interrupt: pin A routed to IRQ 11
        NUMA node: 0
        Region 0: I/O ports at b000 [size=256]
        Region 1: Memory at 9a500000 (64-bit, non-prefetchable) [size=16K]
        Region 3: Memory at 9a480000 (64-bit, non-prefetchable) [size=256K]
        Expansion ROM at 9a000000 [disabled] [size=512K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [68] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 4096 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
                DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq-
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Exit Latency L0s <64ns
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s (ok), Width x4 (downgraded)
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range BC, TimeoutDis+ NROPrPrP- LTR-
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS- TPHComp- ExtTPHComp-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis- LTR- OBFF Disabled,
                         AtomicOpsCtl: ReqEn-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        Capabilities: [d0] Vital Product Data
                Not readable
        Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [c0] MSI-X: Enable- Count=15 Masked-
                Vector table: BAR=1 offset=00002000
                PBA: BAR=1 offset=00003800
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
                        MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
                HeaderLog: 00000000 00000000 00000000 00000000
        Capabilities: [138 v1] Power Budgeting <?>
        Capabilities: [150 v1] Single Root I/O Virtualization (SR-IOV)
                IOVCap: Migration-, Interrupt Message Number: 000
                IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
                IOVSta: Migration-
                Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00
                VF offset: 1, stride: 1, Device ID: 0072
                Supported Page Size: 00000553, System Page Size: 00000001
                Region 0: Memory at 000000009a4c0000 (64-bit, non-prefetchable)
                Region 2: Memory at 000000009a080000 (64-bit, non-prefetchable)
                VF Migration: offset: 00000000, BIR: 0
        Capabilities: [190 v1] Alternative Routing-ID Interpretation (ARI)
                ARICap: MFVC- ACS-, Next Function: 0
                ARICtl: MFVC- ACS-, Function Group: 0
# modinfo megaraid_sas
Code:
# modinfo megaraid_sas
filename:       /lib/modules/4.18.0-305.19.1.el8_4.x86_64/weak-updates/megaraid_sas/megaraid_sas.ko
description:    Broadcom MegaRAID SAS Driver
author:         megaraidlinux.pdl@broadcom.com
version:        07.714.04.00-rh1
license:        GPL
rhelversion:    8.4
srcversion:     52E4854F97510342B42CD66
alias:          pci:v00001000d000010E7sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E4sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E3sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E0sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E6sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E5sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E2sv*sd*bc*sc*i*
alias:          pci:v00001000d000010E1sv*sd*bc*sc*i*
alias:          pci:v00001000d0000001Csv*sd*bc*sc*i*
alias:          pci:v00001000d0000001Bsv*sd*bc*sc*i*
alias:          pci:v00001000d00000017sv*sd*bc*sc*i*
alias:          pci:v00001000d00000016sv*sd*bc*sc*i*
alias:          pci:v00001000d00000015sv*sd*bc*sc*i*
alias:          pci:v00001000d00000014sv*sd*bc*sc*i*
alias:          pci:v00001000d00000053sv*sd*bc*sc*i*
alias:          pci:v00001000d00000052sv*sd*bc*sc*i*
alias:          pci:v00001000d000000CFsv*sd*bc*sc*i*
alias:          pci:v00001000d000000CEsv*sd*bc*sc*i*
alias:          pci:v00001000d0000005Fsv*sd*bc*sc*i*
alias:          pci:v00001000d0000005Dsv*sd*bc*sc*i*
alias:          pci:v00001000d0000002Fsv*sd*bc*sc*i*
alias:          pci:v00001000d0000005Bsv*sd*bc*sc*i*
alias:          pci:v00001028d00000015sv*sd*bc*sc*i*
alias:          pci:v00001000d00000413sv*sd*bc*sc*i*
alias:          pci:v00001000d00000071sv*sd*bc*sc*i*
alias:          pci:v00001000d00000073sv*sd*bc*sc*i*
alias:          pci:v00001000d00000079sv*sd*bc*sc*i*
alias:          pci:v00001000d00000078sv*sd*bc*sc*i*
alias:          pci:v00001000d0000007Csv*sd*bc*sc*i*
alias:          pci:v00001000d00000060sv*sd*bc*sc*i*
alias:          pci:v00001000d00000411sv*sd*bc*sc*i*
depends:       
name:           megaraid_sas
vermagic:       4.18.0-305.el8.x86_64 SMP mod_unload modversions
sig_id:         PKCS#7
signer:         ELRepo.org Secure Boot Key
sig_key:        E9:D4:71:CF:B4:FE:13:6C
sig_hashalgo:   sha256
signature:      57:F6:90:AB:71:87:5A:DD:F0:91:C3:13:70:45:47:D3:3E:27:8E:DB:
...
parm:           lb_pending_cmds:Change raid-1 load balancing outstanding threshold. Valid Values are 1-128. Default: 4 (int)
parm:           max_sectors:Maximum number of sectors per IO command (int)
parm:           msix_disable:Disable MSI-X interrupt handling. Default: 0 (int)
parm:           msix_vectors:MSI-X max vector count. Default: Set by FW (int)
parm:           allow_vf_ioctls:Allow ioctls in SR-IOV VF mode. Default: 0 (int)
parm:           throttlequeuedepth:Adapter queue depth when throttled due to I/O timeout. Default: 16 (int)
parm:           resetwaittime:Wait time in (1-180s) after I/O timeout before resetting adapter. Default: 180s (int)
parm:           smp_affinity_enable:SMP affinity feature enable/disable Default: enable(1) (int)
parm:           rdpq_enable:Allocate reply queue in chunks for large queue depth enable/disable Default: enable(1) (int)
parm:           dual_qdepth_disable:Disable dual queue depth feature. Default: 0 (int)
parm:           scmd_timeout:scsi command timeout (10-90s), default 90s. See megasas_reset_timer. (int)
parm:           perf_mode:Performance mode (only for Aero adapters), options:
        0 - balanced: High iops and low latency queues are allocated &
        interrupt coalescing is enabled only on high iops queues
        1 - iops: High iops queues are not allocated &
        interrupt coalescing is enabled on all queues
        2 - latency: High iops queues are not allocated &
        interrupt coalescing is disabled on all queues
        default mode is 'balanced' (int)
parm:           event_log_level:Asynchronous event logging level- range is: -2(CLASS_DEBUG) to 4(CLASS_DEAD), Default: 2(CLASS_CRITICAL) (int)
parm:           enable_sdev_max_qd:Enable sdev max qd as can_queue. Default: 0 (int)
parm:           host_tagset_enable:Shared host tagset enable/disable Default: enable(1) (int)
 

Tinkerer

Member
Sep 5, 2020
45
14
8
As far as im aware these cards use the mpt_sas driver in IT mode. Have you tried installing that?
Damn that was 2 days worth of searching my ass off. Thanks!

For those reading and wondering, there are 2 mpt3sas drivers available, one in baseOS-rpms and from ELrepo. I picked the one from ELrepo, same version but 1 minor point release newer. So that would be `dnf install kmod-mpt3sas`, then `modprobe mpt3sas` and `lsblk` nicely showed the disk attached to it.

Awesome! Means I'm set to test passthrough of that card to a VM on ESX and (hopefully) do a successful P2V of my homeserver.

Thanks again!
 

jabuzzard

New Member
Mar 22, 2021
26
9
3
The reason for the removal is because Broadcom won't provide RedHat with support for the life cycle of RHEL8. RedHat clearly felt it was therefore best to remove support. Noting that some cards have been added back in

That said with RHEL 8.4 the 2008 along with the 2108 and 2004 where added back in, albeit uncertified and hence unsupported. I know for certain the 2108 works. At work we have gone down the route of replacing cards and servers that won't work with RHEL8. Mostly that has been HP branded Qlogic 10Gb cards.

An alternative is to pinch yourself very very hard and use Oracle Linux with their "Unbreakable" kernel which adds support back in for lots if not all the removed devices. For us that was mostly not an option due to hard requirements to use the RHEL kernel for GPFS.
 

Tinkerer

Member
Sep 5, 2020
45
14
8
Thanks for the explanation and the alternative suggestions.

After I got the above to work and tested it, I decided not to go forward with the passthrough option. After some more searching I came across XCP-NG, which turns out to be a really awesome alternative for ESX that supports zfs for a storage backend natively.