Search results

P2P works just fine, and by the cascading the PLX88096 has less bandwidth when using all cards at the same time and also moving data to the CPU (not very common though, inference or train can be kept inside the switch). Note that diff architectures have to do an extra jump besides PCIe anyways...

It is a PM50100 switch (100 lanes), which has 4x5090 connected, and the last 2 MCIO downstream ports are used to connect a PLX88096 switch, where I connect the other 4 GPUs.

4x5090, 2x4090, 1xA6000 and 1xA40. llamacpp nowadays with -fit (which is enabled by default) makes the partial offload better than my manual -ot one haha. About 220GB RAM, 270GB VRAM (it uses more VRAM because context, cache, etc)

At least on my case, when running Kimi K2 Q3_K_M (a mix between RAM on a consumer 9900X and 272GB VRAM), I get about 300-400 t/s PP and 12-14 t/s TG, using llamacpp with: ./llama-server -m '/run/media/pancho/MyDrive/models_llm_2tb/Kimi-K2.5-Q3_K_M-00001-of-00011.gguf' -c 32768 --no-mmap -mg 0...

@josh, for 16 GPUs on the same board, you would need a PLX88096 switch with 10 SFF8654 ports. The one you linked would work, setting the dip switch to X4, which would let you connect up to 20 cards at X4 4.0. You would need some SFF8654 to 2*SFF8654 4i cables and some SFF8654i device...

I used the retimer on the MB PCIe slot and it works without issues. Pic of a NIC with a fan on top of it, and behind there's the MCIO Retimer.

Finally connected all the switches and GPUs on my PC. That took a while (like 3 days lol), multiple cables and such, but it works! Basically it is like this: PM50100 Switch (01:00.0) ├── Port 02.0 → GPU2 (5090) direct ├── Port 03.0 → PLX88096 (cascaded) │ └── Complex internal structure: │...

Finally connected all the switches and GPUs on my PC. That took a while (like 3 days lol), multiple cables and such, but it works! Basically it is like this: PM50100 Switch (01:00.0) ├── Port 02.0 → GPU2 (5090) direct ├── Port 03.0 → PLX88096 (cascaded) │ └── Complex internal structure: │...

Just a head up that some PLX88XXX switches from this store are quite cheap. They seem to have a good reputation so far. Also there are some offers + coupons for summer/winter sale just now. https://es.aliexpress.com/item/1005010379686637.html PLX88024 (X8 4.0 to 4*X4 4.0) for 108 USD...

Just checked and prices are quite high! I guess I have to wait until a new offer. Now I wonder, you buy the CPU separate from the motherboard right? (Like there is no bundle for Mobo + CPU)

Ohh I can see it now. It doesn't ship to Chile, that's why haha. But amazing price! I would get 2 if I could lol.

I searched that but only found one for 100USD and one for 200USD. Can you share the link?

Yes I think, basically on the main X16 5.0 slot. I'm using 26cm cables but even then I get a card at X4 5.0 or random dropouts, so I hope the MCIO Retimer helps.

Sure, here are the links. The PLX88096 with switches above: https://es.aliexpress.com/item/1005010306067997.html (though if you search for PLX88096 on aliexpress, maybe there's other cheaper ones. This one was cheaper some weeks ago) Microswitch Switchec 100 lane Gen 5: PCIe gen5 MCIO Switch...

I have gotten a MCIO Retimer that should arrive in the next 2 weeks. ATM with a passive MCIO host I get some issues (not with the switch but with a normal X8/X8 bifurcation). I'm using an AM5 board.

And another PLX88096 one, this one took a while to get here but for the price (400USD) I can’t complain much. I thought it would be a scam lol, but it works just fine, tested just quickly at X16 and X8X8, works without issues. But I'm on standby until some cables and adapters arrive. No...

And another PLX88096 one, this one took a while to get here but for the price (400USD) I can’t complain much. I thought it would be a scam lol, but it works just fine, tested just quickly at X16 and X8X8, works without issues. But I'm on standby until some cables and adapters arrive. No...

Christian answered me with the updated c-payne tool for anyone that has any of his switches (including PCIe 4.0 ones) https://c-payne.com/c-payne-tool He told me he added all of my configs and a bit more. So I requested this for those interested: * Default: All X16 (so 5*X16 5.0...

Christian answered me with the updated c-payne tool for anyone that has any of his switches (including PCIe 4.0 ones) https://c-payne.com/c-payne-tool He told me he added all of my configs and a bit more. So I requested this for those interested: * Default: All X16 (so 5*X16 5.0...

Christian shared me the manual If someone wants it, DM or check it here on level1techs forums A Neverending Story: PCIe 3.0/4.0/5.0 Bifurcation, Adapters, Switches, HBAs, Cables, NVMe Backplanes, Risers & Extensions - The Good, the Bad & the Ugly