Thunderbolt 4 or USB4 PCIe expansion cards as HBA?

[Partofthesolution]

Hi all,

I am wondering if TB4 or USB4 PCIe expansion cards can be used as some kind of HBA-alternative for external drive enclosure.

TB4 and USB4 can transfer 32Gbps (4GB/s) payload (40G in total). Both can tunnel PCIe. This means, it could be possible to tunnel M.2 / U.2 / other PCIe drives in an external drive enclosure via an PCIe switch.

I think this could be a nice alternative for NAS systems and an alternative to traditional HBAs with external SAS connectors. I am thinking about extending my ZFS based storage system with it if its working.



As far as I know, there are not much TB4 controller chips available on the market. Most devices use Intel 8000 series TB4 controller chips. These chips are available for different kind of applications:

Intel Goshen Ridge (JHL8440) for docking stations
Intel Maple Ridge (JHL8540 and JHL8340) host controllers – for use in computers, tablets, and other client devices




There are few PCIe TB4 / USB4 expansion cards on the market. Here are two examples:

MSI e-Shop | Offiziell von MSI Deutschland

Im offiziellen e-Shop von MSI Deutschland finden Sie alles rund um Gaming. Mainboards, Desktop-PCs, Monitore, All-in-One-PCs, Mini-PCs, Laptops, Wasserkühlungen und vieles mehr. Ihr erster Schritt zum erfolgreichen eSports-Spieler. Aktionen exklusiv für Mitglieder - sicherer Käuferschutz -...

de-store.msi.com

ThinkStation Thunderbolt 4 PCIe Expansion Card with HP Bracket | 4XF1L53431 | 4XF1L53431

Expand the capabilities of your ThinkStation with this Thunderbolt 4 PCIe Expansion Card. Comes with HP bracket. Great offers available!

www.lenovo.com

ASRock > Thunderbolt 4 AIC

As far as I know, most of these cards have some kind of display connectors, because Intels TB4 controller have DisplayPort 1.4a inputs, so you can connect a video signal to these inputs and transfer it over the USBC connector to a monitor.

Because most of these cards list some requirements like: "Only support Intel 500/600/700 series motherboard with JTBT+JUSB header " (the MSI one). Of course you need a mainboard with such connectors to use the DP 1.4a input. Its a nice feature, but is it mandatory ?

Here you can find a block diagram of the Intel 8000 series and some more details: Intel's Maple Ridge (JHL8540) Thunderbolt 4 Controller Now Shipping





There are already external drive enclosures with TB3 or TB4 connectors available for different drive form factors like M.2, 2,5", 3,5" drives:

Hard Drives & SSDs Enclosures with Connection external: Thunderbolt Price Comparison Skinflint UK

compare prices and read user reviews for Hard Drives & SSDs Enclosures with Connection external: Thunderbolt

skinflint.co.uk

Does anyone already have experience with this topic?

 

[Tech Junky]

I use the gigabyte one on an ASRock board because it offers higher USB pd output. Both the ASRock and gigabyte have the same cable requirements.
. MSI has been spamming news outlets for a couple of years for USB4 and finally just released it with their boards only.

IRL TB 4 can hit 3.2GB and USB4 apparently unlocks additional bandwidth since it's pcie 4 and hits 3.8GB/s. Then only enclosure though right now is the zikedrive that's made for USB4. On the TB side you would be looking for enclosures using the JHL8540 and there's a handful of ensures. there's a ton of JHL7540 options but those don't have USB fallback.

 

[Falloutboy]

I use the gigabyte one on an ASRock board because it offers higher USB pd output. Both the ASRock and gigabyte have the same cable requirements.
. MSI has been spamming news outlets for a couple of years for USB4 and finally just released it with their boards only.

IRL TB 4 can hit 3.2GB and USB4 apparently unlocks additional bandwidth since it's pcie 4 and hits 3.8GB/s. Then only enclosure though right now is the zikedrive that's made for USB4. On the TB side you would be looking for enclosures using the JHL8540 and there's a handful of ensures. there's a ton of JHL7540 options but those don't have USB fallback.

Realize this is quite a dated post but be aware that with the Maple Ridge cards they will not work on GIgabyte systems with newer systems as an example the Aorus X670E xtreme will run a Maple Ridge card with the V10 Bios BUT the same card will fail with the latest V30 bios , it seems Gigabyte learnt of this happening and just took recognition out of BIOS so now it will only work up to Titan Ridge ( Thunderbolt 3 ), oddly the maple ridge card will still work on an ASRock Steel Legend X670E as a TB4 card.

BTW TB3 is actually better than TB4 - go figure.

 

[Tech Junky]

@Falloutboy

lspci | grep Maple
05:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:01.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:02.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:03.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
07:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
09:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]

 lsmod | grep thund
intel_wmi_thunderbolt    16384  0
thunderbolt           524288  0
wmi                    28672  3 video,intel_wmi_thunderbolt,wmi_bmof

 

[Falloutboy]

@Falloutboy

lspci | grep Maple
05:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:01.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:02.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
06:03.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)
07:00.0 USB controller: Intel Corporation Thunderbolt 4 NHI [Maple Ridge 4C 2020]
09:00.0 USB controller: Intel Corporation Thunderbolt 4 USB Controller [Maple Ridge 4C 2020]

lsmod | grep thund
intel_wmi_thunderbolt    16384  0
thunderbolt           524288  0
wmi                    28672  3 video,intel_wmi_thunderbolt,wmi_bmof

What flavor of Linux are you running and are you using the Shorted pins on the card? I am running ubunu 18.04LTS atm in live mode. If I run it up without the wire patch on the 5 pin connector on the card I can boot into Ubuntu but lspci does not show anything like that, if I leave the wire patch on I get the lights come up on the RGB keyboard and a black screen from there.
I suspect that the driver for the card I have presently plugged in which the Titan Ridge card is not presently part of the kernel.

 

[Tech Junky]

No hacks or anything but, I do push the limits with the kernel to stay up to date and manually install them weekly.

 (Linux Mint 21.3 64bit / Linux 6.10.0-061000rc4-generic)

It works fine though with Ubuntu as well. I have my 5G WWAN plugged into one of the ports and it's rock solid.

 

[Falloutboy]

No hacks or anything but, I do push the limits with the kernel to stay up to date and manually install them weekly.

 (Linux Mint 21.3 64bit / Linux 6.10.0-061000rc4-generic)

It works fine though with Ubuntu as well. I have my 5G WWAN plugged into one of the ports and it's rock solid.

Interesting ill try running with live mint if there is one, not quite ready to actually install an OS yet as was hoping to install trunas and run all os's via VM

 

[Tech Junky]

live mint

Pretty sure it's the same prompts as Ubuntu. Most o the debian based options run the same way with minor tweaks to the desktop environment. The only thig that Mint does is split up the install media into different options. I went with the XFCE one because that's what I use to RDP into the box and manage it. The native desktop in Ubuntu doesn't wok for this for some reason and requires adding packages to get it working. Not a huge deal but, still another step to deal with.

List of Linux distributions - Wikipedia

en.wikipedia.org