x10sdv & Riser Cards

[Rand__]

Hi,
just obtained a SM X10SDV-6c board (used) which I am looking to use with a Riser Card since I basically need (want) 2 (or more) PCIe Slots.
Now my reseller offered an RSC-R2U-2E8 which is a PCIe Gen 2 card (2x8); o/c I'd prefer a Gen3 card as the slot is a Gen3 slot.
Now its not so simple to find out whether SM cards fit standard PCIe slots so I was looking for some community input.
Or do other vendors have matching/working cards?
Or is the whole idea absurd and I stick to the single slot on this ?

Thanks

 

[Patrick]

That is a great question. I have never tried this.

 

[Netwerkz101]

https://www.supermicro.com/support/resources/Riser.cfm

Most I see are passive and rate for "Gen 2" and "Gen 3" depending on motherboard.
My only personal experience is with: RSC-RR1U-E8

It sits in a Gen3 PCIe slot on my X10SDV-TP8F board.

 

[Rand__]

I have studied that page, even tried to pry out some info from support regarding the various values (eg can i use GPU type for non-GPU cards) but not much success.

And is the RSC-RR1U-E8 providing a Gen3 output slot as expected?

 

[Netwerkz101]

I have studied that page, even tried to pry out some info from support regarding the various values (eg can i use GPU type for non-GPU cards) but not much success.

And is the RSC-RR1U-E8 providing a Gen3 output slot as expected?

Well .. now how would one test this?
I have an OCZ PCIe NVMe AIC (Gen3 x4) in a PCIe Gen3 x4 slot.

My assumption is/was that the riser card itself is passive and only extends existing electrical signals.
"Depending on motherboard" - I interpret as if the riser card is in a Gen2 x4 or x8 slot, the device plugged in will operate at Gen 2 x4 or x8 specs ...
Likewise if the riser card is plugged into a Gen 3 x4 or x8 slot the device plugged into it gets Gen 3 x4 or x8 specs.

 

[pgh5278]

Hi,
just obtained a SM X10SDV-6c board (used) which I am looking to use with a Riser Card since I basically need (want) 2 (or more) PCIe Slots.
Now my reseller offered an RSC-R2U-2E8 which is a PCIe Gen 2 card (2x8); o/c I'd prefer a Gen3 card as the slot is a Gen3 slot.
Now its not so simple to find out whether SM cards fit standard PCIe slots so I was looking for some community input.
Or do other vendors have matching/working cards?
Or is the whole idea absurd and I stick to the single slot on this ?

Thanks

Hello this has been covered 15 months back, here are some snippets from tehis forum..
Riser Card Knowledge required....
My question to SM.. Supermicro advise (see below) the X10SDV-TLN4F can take a riser card to have 2@ x8 PCIe Gen 3 slots, to facilitate further options until other board come with different mixes of slot.. Now which riser will do the trick per SM comments below.....???

from Supermicro....Yes! PCIe Bifurication is supported in the latest BIOS on our website [.527]. However, you will need to source your own riser card that can provide additional clock to support multiple PCIe devices.
Regards, Supermicro
-------------------------------------------------------------------------------------------------------------

These are used by some forum members
PCIe bifurcation worked well. Just enable the setting in the BIOS. In this case I used legacy mode.

For reference, the riser I used is the ARC1-PELY423. I've also seen this card listed with the model number GH-PELY423.

 

[Rand__]

Ah thanks for the search terms I will have a look at that thread

 

[Rand__]

Well .. now how would one test this?

Well I would assume that you should be able to see the connection speed on lspci /pci info ?
At least I remember seeing that when I was setting up 10GBe in Freenas a while back and I was using the wrong slots...

 

[Rand__]

So just to wrap this up,
-there is one generic SM riser fitting in a regular slot (RSC-R2U-2E8) which is PCIe2 only but readily available. There will most likely not be a newer version of this since they moved to specialized variants (according to support).
-there is the ameri-rack series of risers (ARC1-PELY423+ variations) which would support bi or trifurcation on PCIe3 but seem hard to find.

Edit: R2032 2U 3-slots PCIE/PCI combo riser card might be feasible as well - has any one used these?

Edit2: I see There are some IBM Riser Cards as well - I assume those should work in a SM board?

 

[Rand__]

So I have tried to buy a R2034 (2U 1 x PCIe X 16 to 2-slots PCIe X 8 w/ 7cm ribbon cable) from RackmountMart.com ($65+p&p)
They have confirmed that it should be PCIe3 capable so it looked very promising.
Don't be surprised (if you go looking) if its not on the webpage - I found it only in an old yahoo store page of theirs, but it still is orderable.

Unfortunately after a lot of back and forth it turned out that it might only be sold to the US - so no go for me:/

 

[Netwerkz101]

I am not sure if these guys ship world-wide or not but have a look:

PLinkUSA / RackBuy 1U / 2U Rackmount Chassis IPC Riser Card, (PCI / AGP / PCI-Express) (Flex Ribbon / Extension)

 

[Rand__]

Hm those need one slot per PCIe slot they are converting don't they?
I am looking for a splitting (bi-/tri-furcation) solution, eg one PCIe3x16 slot to 2x PCIe3x8 or PCIe3x8x4x4

 

[Netwerkz101]

You are correct ...they won't help.
I guess this is a science that I am just not knowledgeable on.
Obviously OEMs do it with their 2u servers..but oh well.

 

[Rand__]

 

[Rand__]

So, after reading this thread over on [H] I now have the following:

You need either
-Bios PCI-e Bifurcation support
or a
-PLX chip on the riser.

There were various cards in (successful) use
-ARC1-PELY423 or ARC2-PELY435-C5V3 (v3 might mean pcieV3 support). Some people said Ameri-Rack would only supply to resellers, some said they got single cards easily (I had no response to my mail) (Obtainable here - quite expensive though)
-RSC-R2UT-2E8R (passive)
-rsc-r2ug-a2e16-a (Active Card with PLX chip)

Most used it for multiple GPUs so added flexible PCIe extension cables to it - the really cheap ones seemed to cause issues.

I have now ordered
rsc-r2ug-a2e16-a
RSC-R2UT-2E8R
RSC-R2UG-2E4E8 (3x)

to test them on my X10SDV-6c+ and will let you know the results

 

[Rand__]

So just a small update.
rsc-r2ug-a2e16-a - works fine, have not tested in non-bifurcation board yet but I guess it would work to (PLX chip). Both cards (NVME) recognized, slight performance loss (couple of percent). Shown as 2xPCIe 16x in HWInfo. Gets rather warm

RSC-R2UT-2E8R - Works fine as well, with bifurcation. Have not tested further since it places the cards directly over the CPU. Did not have PCI-extender to place elsewhere.

RSC-R2UG-2E4E8 - No boot

I will keep the A2E16 since its the only card fitting correctly at the moment and I assume it will be reusable in other boards.


Will add a couple of pics/perf results later

 

[nj47]

Can confirm RSC-R2UT-2E8R works well with my X10SDV-4C-TLN2F-O.

This seller Supermicro RSC-R2UT-2E8R 2U PCI-E to PCI-E x8 Riser Card | eBay accepted my best offer at $25

 

[Rand__]

So another update - It looks like the A2E16a does not work properly after all.

Under normal/low load everything is working as expected, but under heavy load the box crashes/reboots:
(vmksummary.log)
2017-03-08T12:35:49.143Z cpu7:65572)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...
0:00:00:09.390 cpu0:65536)WARNING: PCI: 751: Non-PF mem (0x0 - 0x3f) out of range
0:00:00:09.390 cpu0:65536)WARNING: PCI: 886: Out of resources on parent: Root for device: 0000:ff:12.0
0:00:00:09.390 cpu0:65536)WARNING: PCI: 940: No resources for device: 0000:ff:12.0, BAR[4]: 0x0, size: 64, type: 0x2, flags: 0x0, status: Already exists
0:00:00:09.451 cpu0:65536)WARNING: Device: 1316: Failed to register device 0x430409e41c10 s00000000.00 10b5874700000000060400 (parent=0x18c9430409e42f38): Already exists
0:00:00:09.451 cpu0:65536)WARNING: PCI: 260: dlBusAddress s00000000.00 for PCI device 0000:05:00.0 not unique; using sbdf
0:00:00:09.460 cpu0:65536)WARNING: Serial: 274: logPort initialization failed
0:00:00:09.460 cpu0:65536)WARNING: Serial: 283: gdbPort initialization failed
0:00:00:09.460 cpu0:65536)WARNING: Serial: 300: shellPort initialization failed
0:00:00:09.460 cpu0:65536)WARNING: Serial: 317: consolePort initialization failed
2017-03-08T12:41:14.105Z cpu0:65956)WARNING: xhci_pci_attach:208: xhci devid 0x8c318086: Success


Note I run a NVMe SSD and a Mellanox Connectx3 in that riser, this might be too much for it.
Talked to SM support which indicated resourcing issues - ie non compatibility with the riser (as expected).
Tried playing around with bifurcation (setting to x16 or x8/x8) but no difference.

Too bad
Looks I need to go full board after all.

 

[Rand__]

Another update:
I have the same error on a X10SRI (same installation of ESX), so its not the riser card ...
Sigh - just when I got a new board

 

[PnoT]

I have a X10SDV-4C at the moment and would like the ability to add an X520 network card and an HBA if possible. In looking at the options, that you guys have posted works, and the side mounting of RSC-R2UT-2E8R would not fit in any of my current chassis. My chasiss consist of the 503/4-203B, 512L, SC826, or SC846. The dream would be to pack 2 PCIe cards into one of 1U flavors some how!

What chassis are you using with this riser?

The ARC2-PELY423 flavors looking interesting but haven't seen those in any of the SC series chassis yet. Thoughts?