New Chinese PCIE Switch Board GPU Testing

[panchovix]

@TrashMaster wondering something, at the end I got a similar board like you show in the original post, but with added 2 SlimSAS 8i downstream (so like this one), that should arrive in some weeks.



I wonder, is there a switch or software way to change the "bifurcation" of individual slots? Like i.e., using a X16 4.0 slot as X8/X8 4.0 instead.

Or is that reserved only for PEX/PLX88096 that use full SlimSAS?

If it's not possible, I guess a PLX88048 switch on a X16 4.0 slot of the PLX88096 would work, maybe?

 

[TrashMaster]

You can program the PLX switch to chop up the lanes almost however you want. I dont know which thread we talked about that software maybe check the pcie clearing house on L1T? I havent messed with that because im not a huge firmware guy

 

[panchovix]

You can program the PLX switch to chop up the lanes almost however you want. I dont know which thread we talked about that software maybe check the pcie clearing house on L1T? I havent messed with that because im not a huge firmware guy

You mean from here?

The board is configured by default to operate only in x16 mode, so you can use a passive m.2 splitter it doesnt dynamically reconfigure itself to bifurcate any of the slots.

As discussed in the other thread, you can actually reprogram the switch yourself (if you are feeling brave) to handle splitting into x8x8 or x8x4x4 or x4x4x4x4... the toolkit from broadscum is over at PCI/PCIe Software Development Kits

Users over at a certain chinese site were discussing this kind of thing (as well as gen5 versions of the switch chip and board) however if you are really looking for a storage switch card, check out the pictures i dropped into imgur:

http://imgur.com/a/tmuL8Kp

I haven't tested the software yet as I have yet to get a PLX88096 chip, but if it's that one I will try once I get it!

 

[panchovix]

Sorry to bother again, but it seems this board has a switch to bifurcate here?



Do you have a closer pic on that part?

EDIT: Also did you had to connect both 8 pin + 24 pin? Or 2*8 pin is enough?

 

[TrashMaster]

Nope, those are for PWM fan control:



If you want to program the PLX switch to bifurcate a slot or slimsas connector, its firmware. You can do it but I am not that talented or brave to flash a working pcie switch running my home-grown DGX station.

 

[panchovix]

Nope, those are for PWM fan control:

View attachment 46990

If you want to program the PLX switch to bifurcate a slot or slimsas connector, its firmware. You can do it but I am not that talented or brave to flash a working pcie switch running my home-grown DGX station.

Ohh interesting, thanks! I also got the 88096 switch that has 10 Slimsas ports but it will take more to arrive, prob next year. I know that one has a switch.

 

[TrashMaster]

You should talk to the seller to understand its out of the box firmware config. Cards like that tend to be storage-centric and might already be bifurcated to 4 lane channels (2x4 lanes per slimsas cable).

If you are hoping to connect a single GPU at x16 or two at x8 you might need to flash the switch to get there. Hopefully they are nice enough to provide the file and instructions.

 

[panchovix]

Oof I hope! The one I got with 10 Slimsas has this picture on the description and images



Hope it's true.

 

[TrashMaster]

That's interesting, the switches change the behavior of all the channel groups? Huh. Cant wait to see.

Make sure to grab a lspci -vvv and nvidia-smi topo -m out of it for us!

 

[panchovix]

For sure! Yes, it seems the switch changes all 10 ports. So you could do 20*X4 4.0, or 10*X8 4.0, or 5*X16 4.0. I got this one

https://s.click.aliexpress.com/e/_c3ZQVOh9

Some other images




The black one (the one you mentioned on last page) doesn't seem to have that switch, but it's cheaper.

https://a.aliexpress.com/_mLZluEt

 

[nexox]

The black one (the one you mentioned on last page) doesn't seem to have that switch, but it's cheaper.

https://s.click.aliexpress.com/e/_c3ZQVOh9

It does have three versions with different lane configurations though, could be different firmware for each, could be they just have three sets of contacts that they short with 0 ohm resistors to set the same thing as the dip switches.

 

[panchovix]

Just received a PEX/PLX88048 switch. It is X16 4.0 to 12*X4 4.0 (8 NVMe slots and 2 Slimsas 8i downstream that I think are 4x4 each).

I got this one: https://es.aliexpress.com/item/1005009881058457.html

I will put it on a X4 5.0 slot for now (so X4 4.0) to do some tests, not storage but M2 NVMe to PCIe adapters.

I have a SlimSAS X16/X8 host card but not sure if it will work with the switch set at X4/X4 on the SlimSAS ports.

The host SlimSAS I mean looks like this, but not sure if it will work:

My other PLX88096 switches should arrive either next week or next next week.

 

[panchovix]

Now, also got a PLX88096 switch/plate! It arrived and tested it shortly at X16 4.0, worked without issues.

I removed it temporary as I got a Gen 5 100 lane switch from C-payne (the microchip switchec one, not arrived yet) and I will connect that one to a X16 5.0, and one downstream connect this switch and other one, the pending PLX88096 Slimsas switch that haven’t arrived yet.

Some pictures.

Man my experience with AliExpress have been so good lol.

 

[egn]

I am also interested in the PLX88096 board. What PCI-E card did you use for connection?
I assume it has a MCIO connector and you need two MCIO to SFF-8654 connection cables.

I am thinking about a retimer card to avoid issues.

Do you think the mounting holes are matching part of an ATX motherboard?
I am thinking about mounting it instead of an ATX motherboard in the Jonsbo N5 case.

 

[TrashMaster]

I am also interested in the PLX88096 board. What PCI-E card did you use for connection?
I assume it has a MCIO connector and you need two MCIO to SFF-8654 cable.

I am thinking about a retimer card to avoid issues.

If you want the auto power on/off to work properly do not use a retimer, i tested this use case specifically.

I fell back to using the shinreal passive adapter for 2x mcio to x16 and it worked flawlessly even with 1m mcio to slimsas cables. just cross em over as in the video and diagrams.

 

[egn]

If you want the auto power on/off to work properly do not use a retimer, i tested this use case specifically.

I fell back to using the shinreal passive adapter for 2x mcio to x16 and it worked flawlessly even with 1m mcio to slimsas cables. just cross em over as in the video and diagrams.

I need a half height half length card, as my computer is a Minisforum MS-02.

 

[panchovix]

I am also interested in the PLX88096 board. What PCI-E card did you use for connection?
I assume it has a MCIO connector and you need two MCIO to SFF-8654 connection cables.

I am thinking about a retimer card to avoid issues.

Do you think the mounting holes are matching part of an ATX motherboard?
I am thinking about mounting it instead of an ATX motherboard in the Jonsbo N5 case.

For the PLX88096 board (I got this one https://es.aliexpress.com/item/1005009298773917.html, for some reason it is quite more expensive now) with this SlimSAS passive adapter (so no retimer) and it worked fine, this one https://es.aliexpress.com/item/1005010313245878.html.

But if you search by "PLX88096" on Aliexpress you will find the same board but cheaper.

For the Microswitch Switchtec PCIe 5.0 switch, I did get a MCIO retimer but I don't have it yet (either the switch or the retimer). Prob it will arrive next week or next next week.

Mounting holes for the PLX88096 board do match the part on an ATX motherboard yes.

EDIT: Also you can get a MCIO passive host card and some MCIO 8i to SlimSAS 8i 8654 cables, that should also work yes.

 

[panchovix]

Got this beauty today, PCIe Gen 5 switch with 100 lanes.

I could test it with a passive MCIO host adapter to it, but if I had to guess it won’t work really good without a retimer.

So gonna wait until I get the retimer (sadly since I want to use it so much right now lol)

EDIT: Also I wonder, which ones are the 2 MCIO X8 uplinks?

 

[TrashMaster]

Put the passive mcio card in slot 1 closest to the CPU and i bet it will work fine with a good mcio cable or two.

Are you reconfiguring it custom, or leaving all the mcio mappings default x16x16x16x16x16x16x4?

 

[panchovix]

@TrashMaster I can probably try, but I'm not sure which ones are the 2x8i uplinks haha. Asked via mail and waiting for now.

I plan first to try at 5.0 X16 * 5, but Christian told me he can send me these configs next week