LSI 9362-8i in a Thunderbolt enclosure + Windows = no go?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

hys17

New Member
Mar 18, 2023
4
1
3
Hello everyone,

Hope I'm posting in the right place, as I was literally pulling my hair on this dilemma and really hope someone could point me to the right direction...



-------Lengthy post alert...----------

Bought an OWC Thunderbay Flex 8 last year and been using their SoftRaid, with a RAID5 8x8TB setup. Also built a new PC with Asus H670-Pro WiFi D4 and ThunderboltEX 4 adapter, to support this enclosure. I'm aware that 8 bays on a RAID5 is pretty alarming but the Windows version only supports RAID 1/0/5 even till now.



Fast forward to last month, I finally pulled the trigger and bought an LSI Megaraid 9271-8i card, since this enclosure has spare PCIE slots that can be tunneled via Thunderbolt. Hoping to turn this enclosure into a hardware-raid system (which is under their advertisement).

That's when the nightmare begins. For the life of me, I just couldn't get the card to be detected at all in Windows (10/11). Even though I've managed to boot into the ROM BIOS and set up the raid and everything, the Windows Device Manager just doesn't see it at all. I even took the card out and installed it directly into my PC's motherboard, to replace the thunderbolt adapter, still no luck.

From my research, this is a pretty old card that the latest driver and firmware stopped around 2016 or so. Alright, returned it.



This month, I got a Megaraid 9362-8i (set up a RAID10), which supposedly has newer support. This time, it'd get detected in Windows, if it's directly installed on the motherboard, but not in the thunderbolt enclosure.

Alright, gotta be something with the thunderbolt PCIE tunneling I suppose, progress!

Then I started fiddling around with the motherboard's thunderbolt-related settings in the BIOS, e.g. Discrete Thunderbolt Support, DTBT Controller Configuration, Control IOMMU Pre-boot Behavior, etc. Toggling everything I could and check if it makes it better.



On one instance, I accidentally discovered, that after a period of time after booting to Windows, the enclosure would turn to Standby mode (as it's not being detected/used for a while I guess). But if I open the Device Manager and Scan for hardware changes, all of the sudden, it'd "wake up" the card/enclosure and it shows up in the Device Manager finally!

However, it comes with big caveats, as it'd only work after each fresh reboot of Windows (and not 100% for some reason); the Raid volume shows as a "removable storage", and I'd have to change the Removal Policy in the Device Manager every time to make it "Better performance";



Last and the biggest caveat is, probably due to it's detected after booting, neither MegaRAID Storage Manager, nor the LSA software would find the controller host. So there's no way to monitor the raid while using it.

At this point, I was almost certain that it has something to do with the Thunderbolt PCIE tunneling. But for the life of me, I couldn't find the right settings/tweak (if there's one). Funny thing is, I have a 10G PCIE ethernet card installed in the enclosure, and it works perfectly regardless when I connect the enclosure to the PC. Of course I've swapped the PCIE slots of the 10G card and the Megarad, just to rule out factors.

I even tried a Virtual Machine just to see if virtual tunneling via IOMMU does anything, but I've already reached my knowledge/limit on this.



Wondering if anyone can share some insight or should I just give up and go back to SoftRAID with RAID5?



Summary of specs:

Windows 11 Pro

i7-12900KF

Asus H670-Pro WiFi D4

128GB DDR4

ThunderboltEX 4 add-on


OWC Thunderbay Flex 8, 8x8TB HDD

Avago/LSI Megaraid 9362-8i


Little background: I do video editing and currently have a customized PC, plus a MacBook Air M2. The thing I'm looking for in a storage system, is fast and reliable and relatively easy to maintain. Judging by my experience so far however, such thing is hard to nail down. I work mostly in Windows, with often tap in macOS, but very limited knowledge/experience on Linux.


Thanks in advance for your help!
 

Moopere

New Member
Mar 19, 2023
26
3
3
Hmm. Theres a lot going on here. I have no experience with Thunderbolt, let alone an external cabinet of this type. My vague understanding is that Thunderbolt is basically PCIe over the wire - but I'm sure there is more to it than that.

Sounds like your original 9271 RAID card was DOA - I use these as well as 9260's and they run just fine even in modern hardware systems with Windows Server 2019. You say you couldn't get it working with windows 10/11 ... but which of those two is it? I believe that Windows Server 2019 uses a base of Windows 10 1809. I cannot get my cards to work with Server 2022 which has a base of Win10 22H2 and I suspect Windows 11 wouldn't work either. There appears to be a driver incompatibility.

The LSI 9362-8i isn't on the supported list either afaik. My guess is that it would work, but would be great to see some confirmation.

I'd definitely concentrate on getting one of these cards working in your main system before moving it to the thunderbolt system. It need to be discovered at boot time _every_ time you switch on _and_ successfully load up its driver during Windows boot. If we can't do both of these on the main motherboard then we have basic compatibility problems that I'd be inclined to sort out before moving the card to the Thunderbolt enclosure.

The flakey business surrounding discovery of the card - I've seen this with my older PCIe2 LSI cards. They don't much like being put into a PCIe 4 system. If your motherboards BIOS allows you to set the PCIe speed this has helped (me) to get stability out of my RAID cards - I set PCIe to Gen2 and they are discovered immediately every time.
 

hys17

New Member
Mar 18, 2023
4
1
3
Hmm. Theres a lot going on here. I have no experience with Thunderbolt, let alone an external cabinet of this type. My vague understanding is that Thunderbolt is basically PCIe over the wire - but I'm sure there is more to it than that.

Sounds like your original 9271 RAID card was DOA - I use these as well as 9260's and they run just fine even in modern hardware systems with Windows Server 2019. You say you couldn't get it working with windows 10/11 ... but which of those two is it? I believe that Windows Server 2019 uses a base of Windows 10 1809. I cannot get my cards to work with Server 2022 which has a base of Win10 22H2 and I suspect Windows 11 wouldn't work either. There appears to be a driver incompatibility.

The LSI 9362-8i isn't on the supported list either afaik. My guess is that it would work, but would be great to see some confirmation.

I'd definitely concentrate on getting one of these cards working in your main system before moving it to the thunderbolt system. It need to be discovered at boot time _every_ time you switch on _and_ successfully load up its driver during Windows boot. If we can't do both of these on the main motherboard then we have basic compatibility problems that I'd be inclined to sort out before moving the card to the Thunderbolt enclosure.

The flakey business surrounding discovery of the card - I've seen this with my older PCIe2 LSI cards. They don't much like being put into a PCIe 4 system. If your motherboards BIOS allows you to set the PCIe speed this has helped (me) to get stability out of my RAID cards - I set PCIe to Gen2 and they are discovered immediately every time.
Thanks so much for the reply!
9271 didn't work on my Windows 10 nor 11. The card's BIOS could show up at boot time but not inside Windows. After a while of troubleshooting, the card wouldn't show BIOS at all so I returned it...
I never thought about the PCIE Gen thing actually. After playing with the BIOS, it turns out this motherboard only has Gen3 PCIE other than the GPU one. Since 9362-8i is Gen3 also, I just set all the PCIE slots to Gen3. Didn't change anything though.
I haven't tested the card on my motherboard extensively, but it seems to be working well if directly installed.

I know 9362-8i's official documents don't say it supports windows 11. However, the latest firmware is dated 2022 and from what I've researched, it should work. Just the thunderbolt enclosure is giving me trouble and OWC is no-help on this one. The tech agent basically told me that this enclosure isn't officially supported on Windows, while their official website says otherwise....Really wish I've known all this before getting this enclosure...
 

Moopere

New Member
Mar 19, 2023
26
3
3
Such things bother me (smile) - I wouldn't be able to let this rest until I'd found the answer :)

Its probably too disruptive to your workflow but if you had the time I wonder how (or if) this would work under Linux? It may be down to the thunderbolt driver/card you have in the new PC.

I suspect there would be quite some shenanigans going on at the hardware level, mostly hidden by drivers, to make such a system work and appear to Windows as if its just a big PCIe bus.
 

hys17

New Member
Mar 18, 2023
4
1
3
Such things bother me (smile) - I wouldn't be able to let this rest until I'd found the answer :)

Its probably too disruptive to your workflow but if you had the time I wonder how (or if) this would work under Linux? It may be down to the thunderbolt driver/card you have in the new PC.

I suspect there would be quite some shenanigans going on at the hardware level, mostly hidden by drivers, to make such a system work and appear to Windows as if its just a big PCIe bus.
Funny thing: Linux was the first other OS I thought of when either those cards wasn't working, as I suspect it's more of an OS problem since the card shows up in BIOS just fine. But because the way my PC is set up - dual booting with Windows 10 and 11, while I was slowly migrating from 10 to 11 in the past year or so, without having to disrupt my work, I didn't have a spare internal storage to fully install a Linux.
I searched portable Linux systems and tried a few on a USB key but the only one could successfully boot with the Thunderbolt attached is Ubuntu Game Pack. It failed to install the driver and management utility though (probably due to the portable system's limitation), with my limited knowledge when it comes to Linux and programming (also why I didn't go with the TrueNas or ZFS approach in the first place), I had to ditch the entire approach.

I honestly thought the same thing with Thunderbolt that it's just a big PCIe bus. But the experience and later on research showed otherwise. For example, IOMMU was implemented at TB4 (I think?) to deal with a security issue with previous generation when tunnelling PCIe in VM. From what I understand, such thing shouldn't happen or would require a totally different approach if it's just a PCIe bus with a cable.

Talking about sleepless nights... I had got up at 2am and thought about some ideas that I haven't tried and stayed up till 7...

At this point, I'm pretty exhausted (and sick actually) and just want a way that could work with my current hardware and provide something like RAID 10, with good performance, doesn't matter hardware or software raid. Currently, I'm looking at the HighPoint RocketRaid 3720C. It supports both Windows and macOS so might work better for my usage. The lack of cache memory might be some hit but can be easily complimented by something like PrimoCache (trying the demo and liking it so far). Really hope this works... (face-palm)
 
  • Like
Reactions: Moopere

hys17

New Member
Mar 18, 2023
4
1
3
Just want to circle back with updates.

The HighPoint card didn't work at all on my PC that it doesn't detect any hard drives (weirdly worked perfectly on my MacBook...)

Then I had to go back to SoftRaid for over a week while waiting on Amazon deliveries. The RAID volume would disappear at some reboots and my computer would hang on reboots. It turns out now the unit hates the 7 HGST drives that I had in there since forever (luckily I had a Toshiba drive in there to actually discovery the problem). The drives would show up in the raid but with I/O error and no SMART values. Googled around and found the 3.33v thing. Tried the tape method (with a bunch of different tapes, including Kapton one) and no luck. Weirdly, they were working fine just a month ago.

The same issue with then Adaptec 71605 controller. Though this time the card gets detected in Windows without a problem, which is what I'm using temporarily (with a few IronWolf and Toshiba).

I'm getting the Thunderbay Flex 8 RMAed and hopefully they could at least replace it. Then I've pulled the trigger of getting an Areca Thunderbolt Hardware RAID DAS since I'm pretty fed up with Flex 8's support on Windows. I know a lot of people are saying hardware raid is dead in 2023 and software raid makes a lot of sense and gives so much functionality and flexibility, at many scenarios. But for certain usage, people just need something easy and just works, with good performance, and hardware raid can still shine at those areas I believe.

Anyway, hope my experience can help someone and hopefully no one has to go through so many hoops...
 

Docop

Member
Jul 19, 2016
41
0
6
45
You try many thing at same time. you had issues on your raid card, the asus tb extension is just alone a problem generator on his own. And booting the eeprom of the raid card, after the tb.. many time the pc do reset as the tb is not detected... and too what is the cable used for the enclosure.. many aspect. Anyhow, Synology do not have any raid hrdw and when issues, it's a 1 click button and not had to fetch another older card in order to restore something. Best is just go with 8644 link on 9300 and then you have your full external enclosure and full speed nvme for upgrade.