ATTO 3102- Thunderbolt Mellanox ConnectX-4 Lx (Dual Port 25GbE) - $189 or less

[i386]

they do not have any problems, its a silly myth spread by people who has no idea what smb or rdma is.

if you have the right version of windows, it works out of the box.

Like windows 10/10 pro for workstations?
Connected directly or via a 6036 switch to a windows server 2022 using the same nic, firmware and driver version?
With the switch configured and unconfigured especially for roce over smb?
And still no changes in rdma counters in perfmon when copying the 10+ GByte veam replications iso?

Works fine and out of the box when server 2022 is booted on that very same machine

 

[bugacha]

Like windows 10/10 pro for workstations?
Connected directly or via a 6036 switch to a windows server 2022 using the same nic, firmware and driver version?
With the switch configured and unconfigured especially for roce over smb?
And still no changes in rdma counters in perfmon when copying the 10+ GByte veam replications iso?

Works fine and out of the box when server 2022 is booted on that very same machine

Are you asking a concrete question or just engaging in demagogical discussion ?

 

[kapone]

demagogical

ooh. I like that word!

 

[i386]

Are you asking a concrete question or just engaging in demagogical discussion ?

Both
I'm frustrated regarding rdma + smb on windows client os for years now. I even bought licenses for Pro for workstations and for a short time it worked on one workstation, but since then I couldn't get it working again following microsoft, nvidia, sth forum member instructions or my own notes.

 

[Civiloid]

What is the power requirement for the unit? Their spec sheet says 12v center positive (I assume 5.5mm barrel jack) but it doesn't mention the current capacity needed.

I'll have a look at mine (I've decided to get one even though I don't really need it), but from what I know it should be about 3A adapter, ConnectX-4 Lx doesn't draw too much power, desktop cards are less than 30W.

 

[autoturk]

That would be 10G only (Atto 3102 is reflashable to 25G). And I won't trust Realtek to have proper HW offloads. I know for the fact that on ConnectX-4 Lx offloads are working just fine and therefore your CPU usage to transfer full line rate of 25G is negligible*.

* - unless you use macos and apple built-in driver which disables all offloads and you can enjoy driver consuming 3 cores worth of CPU on M4 Max under iperf3 doing 15 gbps (limit of my current TB3 PCIe enclosure).

wait, so if I understand you correctly using this dock at 25gbps on macOS will essentially eat up tons of CPU cycles, and there's currently no non-Apple driver for modern MacOS versions?

 

[Civiloid]

wait, so if I understand you correctly using this dock at 25gbps on macOS will essentially eat up tons of CPU cycles, and there's currently no non-Apple driver for modern MacOS versions?

From atto's website:
Supports the following macOS Versions: macOS 15 Sequoia, macOS 14 Sonoma & macOS 13 Ventura.

 

[autoturk]

From atto's website:
Supports the following macOS Versions: macOS 15 Sequoia, macOS 14 Sonoma & macOS 13 Ventura.

ah, I see. I misunderstood. Here's the page (hope this link works) with the relevant software download: Software Downloads - ATTO Technology, Inc.

 

[Civiloid]

What is the power requirement for the unit?

My unit had a 60W PSU (12V, 5A), if that matters it is GlobTek TR9CE5000LL9P-N(RVB) - it have positive inner contact, barrel connectort 2.5mm ID x 5.5mm OD x 11mm

 

[Civiloid]

I actually cross flash that card to the Atto FW?

I've connected my CX 4 Lx to a PC and used mstflint on Linux to do that. I don't know if it would work over Thunderbolt - in theory it should, however mstflint doesn't really compilable for OSX, so you would need Windows or Linux (VM might work if OSX can passthrough TB3 device into VM).

 

[Civiloid]

Ok so using the ATTO 3252 'fw_TLNS3252_32.1010.bin' firmware to flash over an originally

You should use firmware from "ATTO FFRM-N322-000" as that is PCIe version. 3252 likely have some special handling around PCIe -> TB conversion chip.

 

[Civiloid]

In case people wonder what is the Aliexpress/Amazon TB -> 25G NIC - I bought one from aliexpress and it is Thunderbolt 3 -> OCP 2.0 and OCP 2.0 ConnectX-4 Lx (MCX4411A-ACAN)

No fan inside, so I'm not sure if it won't overheat, as heatsinks on it are not so great.

 

[nexox]

I don't have any better ideas for how to use one than a ConnectX 4, but the thunderbolt to OCP 2.0 adapter is pretty neat.

 

[Civiloid]

I'm not so convienced about temperature of that thing, I'd like to see a small quiet 40mm fan inside, doesn't need to be high rpm.

And if some one is interested, QR code just says "tb3_ocp2_12d9f8_0079"

 

[Civiloid]

And quick test - just plugged it in and put a transciever in it, it kinda gets uncomfortably hot, so I wouldn't use it (maybe with a DAC, but even there its a huge maybe) without a fan, and there is no easy way to put a fan in there.

 

[fringfrong]

i bought the dual-port 25g version from aliexpress and can confirm the MLX5 driver that loads on macos by default is terrible, chews CPU and max throughput ~8gbps; on linux i saw closer to 20.

i tried flashing a couple of atto firmwares (fw_FFN322_32.1010.bin and fw_FFN322_32.1010.bin), both lead to nonfunctional ports as another posted said. i compared the config of these firmwares and they have different serde/lane setup, i don't think they are compatible. they also set power level to 4 (it's 1 in the original fw) which leads to the device getting really hot

 

[Civiloid]

i tried flashing a couple of atto firmwares (fw_FFN322_32.1010.bin and fw_FFN322_32.1010.bin),

In general, if PCB is different, you can't just flash another driver. I would try swapping IMAGE_INFO section in stock mellanox firmware with what is in FFN322 - that probably would be enough to make it work.

 

[Jeggs101]

@nimik I did the flint on this

flint -d mt4117 --image fw_TLNS3252_32.1010.bin -allow_psid_change b

And it kept failing. Is there a jumper or something I need to do?

 

[bob_dvb]

The AliExpress ones are on Back To School discount, I'm being offered £174.59 with £1.50 delivery.

https://a.aliexpress.com/_EGsA054

I'm trying to resist and the warning about them being hot is currently holding me back.

 

[reasonsandreasons]

i bought the dual-port 25g version from aliexpress and can confirm the MLX5 driver that loads on macos by default is terrible, chews CPU and max throughput ~8gbps; on linux i saw closer to 20.

Wanted to echo the "mlx5 is terrible" sentiment. I have a ConnectX-4 in an external enclosure and in my testing it performs worse than my built-in 10 Gigabit port; the ConnectX-4 reads at 641.9 MB/s from a NVMe mirror in my main server where the built-in NIC does 820.1 MB/s. Speeds don't indicate that I'd benefit from a 25Gbe connection regardless, but still a disappointing result.