PCIe Bifurcation functions and possibilities

[Allan74]

While this may not directly relate to a specific board by model or name, it may very well apply to more than I know, so I apologize in advance for this hypothetical, as I am trying to better understand the Bifurcation option(s) on my Desktop board.

While I understand that Bifurcation on a motherboard (without applying to risers) is capable of assigning PCIe lanes to a dormant slot (ie. SLi or Crossfire, x8 + x8 from an x16 with 2 already present x16 physical slots).........[continued below]

Does PCIe Bifurcation have the ability to re-assign physical board slots originally provisioned from the PCH, to then switch to the direct CPU PCIe lane pool ?

thanks.
Allan

ps. No PLX present.

 

[Allan74]

Rather than playing coy, I will simply just put it out there so that hopefully someone can tell me if my little art project actually makes sense.

Goal: I don't care about Video throughput, as I don't game. I want to run CPU DIRECT NVME M.2 RAID0 with 2 or 3 drives.

BIOS Bifurcation Setting options: x16/0/0/0, x8/x8, x8/x4/x4, x4/x4/x4/x4

(MSI x570 MEG ACE Motherboard, Ryzen9 3900x CPU)

I have done extensive testing on my current video card (RTX 2070 Super 8GB) with x16, x8 and x4 and have found the actual video game performance to be negligible (4% from x16 down to x4), while my priority with the card (NVENC h.265 Encoding) is unaffected, even down to x1, similar to that of a Bitcoin Mining x1 card setup. So, mission accomplished with neutering the video card/slot.

I hope the lane diagram makes sense, as it's all the info I could glean from the useless manual and tech support.



I figure this would be the best place to ask, since it's clearly over the heads of most gamer forums.

thanks again,
Allan

 

[name stolen]

No, it (PCIe bifurcation) does not have that capability. Bifurcation allows, as an example, the 16 PCIe lanes from the CPU (in this example) to be split between two PCIe x16 slots on the mobo. If you plug in one GPU and nothing else, you'll get x16 to the GPU. If you plug in two GPUs, each will get x8. If you plug in a GPU and a PCIe SSD add-in-card, you might get x8 and x4. Bifurcation just allows some flexibility, but the slots are either wired to the CPU or the chipset. Same with the lanes from the chipset, just usually topping out at x4 per slot, and with a lot more onboard devices.

found the actual video game performance to be negligible

I think you'll find the same with 3-way NVMe RAID, but YMMV.

If you want to try CPU-based PCIe NVMe RAID, and I'm still slightly perplexed, I suppose I would look for M.2 NVMe-to-PCIe slot adapters, where the adapters are x8, and have two M.2 slots each. And then put two adapters in your two CPU-connected PCIe slots. At x8 and x8, you have 16 lanes of CPU-connected storage. Plus, Ryzen has another x4 for storage built-in.

However, I've run into a few occasions where the consumer mobo won't boot with the GPU in the chipset-connected x16 slot, even low-power stuff, nevermind a 2070.

 

[Allan74]

With the board's current Bifurcation options, I can understand the x16/0/0/0 and the x8/x8 modes and possibly even the x8/x4/x4 mode, as that is the mode that MSI recommends when using an NVME add-in card.

What I don't understand clearly and why I came to the conclusion in the little colorful image, is, the x4/x4/x4/x4 CPU mode and the fact that there are only 2 physical slots in that board region, the other 2 being x4 M.2 slots, for a total possibility of (4) x4 device slots/ports. The bottom x16 slot is clearly represented as x4 to the PCH and is out of the 'assignment' zone of the board.

 

[name stolen]

Yeah, I don't understand the quad x4 option from the CPU for the two physical slots, unless one of two things. 1. It means bifurcating the 16 to dual x8, and within the slot, sub-bifurcating to dual x4 (like in the NVME adapter example, although I would think the adapter card would handle that, but maybe not?) 2. Maybe it's just written because it's technically possible, but it isn't actually physically possible with the board design. It doesn't SEEM possible without sub-bifurcating. And it doesn't seem possible without putting the GPU in the chipset x16 (physical, not electrical).

 

[Allan74]

There seems to be SOOooo much 'lost in translation' from the Engineer to the Tech Support and Manual writing personnel and the fact that this doesn't seem to be a very popular topic for most 'Gamers' makes it even more frustrating. None the less, I will try MSI once again, perhaps a different country's tech support email and see if I can get any more clarity.

I really had my hopes up high for this one, as not only does it make sense from their 'explanation' in the manual, it seemed to adequately provision all 16 PCIe lanes to 4 obvious physical locations..... but if Bifurcation can not re-assign PCH provisioned physical slots/devices, then it's obviously something that HAS gotten lost in translation.

Unfortunately, I can't justify a Threadripper and it seems that they (board makers) can't justify PLX functionality without removing far too many other features in this price and PCIe lane-limited class.

 

[Mithril]

So, from the CPU/bios perspective bifurcation doesn't *require* any additional slots, it's a logical change to how the lanes are grouped and what the CPU presents to the rest of the system. IIRC, what does the actual switching of those lanes is the motherboard via a MUX which has some overlapping functionality with multiplexers but does NOT allow more lanes to connect than it has upstream. So if it is a 8 lane MUX upstream then it is either 8 lanes from one slot or another for example (or 2 groups of 4). The MUX is essentially acting as a high speed switchboard but it does NOT get involved with the traffic so it has reduced complexity, cost and virtually no latency compared to a multiplexer.

That being said, this doesn't NEED to happen. What you DO need per PCIe device are the clock and other non lane signaling which could be handled entirely on an add-in card to, for example, put 4 NVMe devices into a single 16x slot. So long as the CPU and bios support 4x/4x/4x/4x then you don't need a MUX or a multiplexer or even separate slots.

Note that some server boards are designed with the clock drivers on the board for having an entirely passive riser that splits a 16x or 8x slot but those will not work in boards without the clock drivers.

In *theory* you could have a MUX change if a slot went to the CPU or the PCH but I don't think such an animal exists in wide production.

 

[pyro_]

the bifurcation option is normally used to allow a single slot to be split up for use on a single card such as when using an add in card that can hold multiple nvme m2 drives as a number of these do not have pcie switches built into them

i am going to take a guess that your pcie-1 would be the one use for this setting and if you selected 4/4/4/4 you would loose the use of slot 3 completely.

this would allow you to add in 4x nvme drives in Slot 1 and have them direct to cpu. Your video card would then need to go into one of the slots connected to your chipset rather than Cpu

 

[pyro_]

Basically you would be looking for something like this to use in slot 1 M.2 XPANDER- AERO

 

[Allan74]

Here is a few BIOS screenshots thrown together for my PCIe available settings. Generation/Bandwidth settings for PCIe_1, PCIe_3 (the x4 PCH slot) and the Chipset link to/from the CPU. The Bifurcation settings are shown as Auto (what I referred to as x16/0/0/0), x8/x8, x8/x4/x4 and x4/x4/x4/x4. I will add my original art project for lane/device numbering reference.



From what I know, a desktop chipset (x570 in this case) will only see 1 NVME/M.2 drive per slot, even if used in a 4-drive capable expansion card such as the MSI Xpander Aero above, it will only support a single drive on this and similar desktop platforms.

 

[pyro_]

It should see all 4 drives if split to 4/4/4/4. Alternatively you could get a add in card that uses a pcie switch on it and it would do the spitting rather than having the bios do the bifurcation however these cards cost a lot more than the dumb ones like the onemlinked above

 

[pyro_]

Even in your second pic It lists m2 cpu raid as the use for 4/4/4/4 which tells Me that it should see all 4 nvme drives. Also in your first pic it is referencing the m2 xpander z so it should work without any issues as that is a supported config

 

[Allan74]

This has been the only reference that I could find, but it's not a direct comparison unfortunately, as I can't guarantee that Asus and MSI wire their boards the same, as my MSI has 3 onboard M.2's already, as the Asus x570 boards only come with 2x M.2 slots.

[Motherboard] Compatibility of PCIE bifurcation between Hyper M.2 series Cards and Add-On Graphic Cards | Official Support | ASUS Global

(x570 chipset board 2/3 of the way down the page)

I may be wrong in terms of single drive support per slot, as Asus lists 2 drive support for the 2nd x16 PCIe slot on their x570 boards, assuming the video card in the top slot (when using my generation of CPU, 3rd Gen Ryzen)

Perhaps this, in relation to my board would be the x8/x4/x4 setting (x8 video card in primary PCIe x16 and 2x x4 M.2 drives in secondary PCIe x16 slot).

 

[pyro_]

how Many it can see will depend on the bifurcation support of the motherboard

yes you could try to do it that way and see if both cards would work. Worst case you would just need to use one of the chipset connected pcie slots for the video card

 

[Allan74]

Worst case you would just need to use one of the chipset connected pcie slots for the video card

I think that is what I will end up doing, as I am going to switch my vertical mount/cable for the video card and simply plug it into the bottom x16/x4 PCH slot. Once I change this over, it will give me a bit more freedom to, at the very least, test out which config will actually work in the end.

thanks for all the info.
Allan