USB3.2x2 10GbE adapters

[colinb]

Very simply, are there any?!

Asking as I'm trying to juggle the limited expansion options on a Z690 motherboard. Posting current plan below (also going in DIY/Builds) for ease of reference, as I'm very open to any other ideas in terms of reshuffling what is where in order to squeeze a 10 or 40Gb/s NIC in there and actually be able to use the network bandwidth...!

ASRock Z690 Pro RS w/ i7-13600 + 4x16Gb DDR4
m.2-1 4.0 x4 - NVMe SSD (SSD pool)
m.2-2 3.0 x4 - NVMe SSD (Boot + VM images)
m.2-3 4.0 x4 - NVMe SSD (mirror m.2-1)
m.2-E key - spare
PCIe 5.0 x16 - 4x NVMe SSD PCIe switch card in ZFS mirrors (VM boot and data disks, torrents, Usenet staging) - oversubscribe if not saturating bandwidth
PCIe 4.0 x4 - PERC H330 (8x SSD/HDD in ZFS mirrors)
PCIe 3.0 x4 - DC P3600 SLOG
PCIe 3.0 x1 - Coral Dual TPU on switched adapter card
PCIe 3.0 x1 - spare
8x SATA spare
2 x USB 2.0 Headers (Support 4 USB 2.0 ports)
2 x USB 3.2 Gen1 Headers (Support 4 USB 3.2 Gen1 ports)
1 x Front Panel Type C USB 3.2 Gen2x2 Header (20 Gb/s)
2 x USB 3.2 Gen2 Ports (10 Gb/s) (ReDriver) (Rear)
2 x USB 3.2 Gen1 Ports
2 x USB 2.0 Ports
1 x RJ-45 LAN Port (2.5GbE)

 

[BlueFox]

If you haven't already bought the motherboard, why not just get a similar one with Thunderbolt? There are plenty of Thunderbolt 10GbE NICs. You might also want to consider a higher end platform if you're needing 36+ PCIe lanes.

Alternatively, ditch the SAS HBA and just use on-board SATA?

 

[colinb]

Already have the mobo, so that's a nonstarter unfortunately...making the best of this hamfisted attempt!

Could indeed ditch SAS HBA. Will have to have a think.

Any sense moving the SLOG to PCIe3 x1 and drop the QSFP NIC into the PCIe PCIe x4? I've never used a SLOG before so have no sense of what it best needs

 

[DavidWJohnston]

I had the same question a while back about non-Thunderbolt 10G USB3 NICs, and stumbled across this:

Is there a technical reason no 10GbE USB 3.1 Gen 2 adapters exist yet?

For example, it is practically impossible to create FireWire adapters over USB because FireWire needs DMA which USB doesn't provide. Is there a similar blocking technical reason as to why no 10 gig...

superuser.com

I totally agree with the previous poster's suggestions. In addition, one of these may be possible:

You could use your spare PCIE x1 with a regular 10G nic, that wouldn't quite get you to 10Gbps but it might be decent.

You could also use the m.2-E slot (if it supports PCIe and not just USB) and use one of those little m.2 to PCIe adapters with a 10G card cleverly-mounted in your case, like this:

https://www.aliexpress.com/item/1005005922616975.html

You could also move your Coral TPU to the spare m.2-E, then use 2x10G NICs in the x1 slots in a LAGG. Using something like this:

M.2 Accelerator with Dual Edge TPU | Coral

Integrate two Edge TPUs into legacy and new systems using an M.2 (E key) interface.

coral.ai

 

[colinb]

Thanks. That superuser post answers it for me. For anyone else in future:
Because of overheads, 10GbE over USB looks more like 6Gb/s, so manufacturers just make cheaper 5GbE adapters
Additionally, USB controllers >10Gb/s tend to be bottlenecked upstream by PCIe

 

[UhClem]

...
You could also move your Coral TPU to the spare m.2-E ((, then use 2x10G NICs in the x1 slots in a LAGG)). Using something like this:

M.2 Accelerator with Dual Edge TPU | Coral

Integrate two Edge TPUs into legacy and new systems using an M.2 (E key) interface.

coral.ai

Not likely to be successful...
Note this footnote (from your Coral link):

* Although the M.2 Specification (section 5.1.2) declares E-key sockets provide two instances of PCIe x1, most manufacturers provide only one. To use both Edge TPUs, be sure your socket connects both instances to the host.

"The large print giveth, and the small print taketh away." --Shyster proverb

 

[colinb]

Not likely to be successful...
Note this footnote (from your Coral link):

"The large print giveth, and the small print taketh away." --Shyster proverb

While it'd be nice if manufacturers actually stuck to specifications, as you note, most don't.

I found an imperfect solution in the form of a PCIe x1 card with a switch chip that will at least let both TPUs be used, even if it transpires it's not at 100% capacity:
Dual Edge TPU Adapter - PCIe x1 Low Profile

 

[chx]

As I noted under the answer to my superuser question (hi servethehome!) that answer is not quite satisfactory: benchmarks note over 8Gbps rather than 6Gbps.

I rather believe the patents encumbering 10GbE made it uneconomical to do it. However, they expired last summer so I expect to see development in this area finally. Stay tuned for CES, I'd say.