1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

x10sdv & Riser Cards

Discussion in 'Processors and Motherboards' started by Rand__, Dec 20, 2016.

  1. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    Hi,
    just obtained a SM X10SDV-6c board (used) which I am looking to use with a Riser Card since I basically need (want) 2 (or more) PCIe Slots.
    Now my reseller offered an RSC-R2U-2E8 which is a PCIe Gen 2 card (2x8); o/c I'd prefer a Gen3 card as the slot is a Gen3 slot.
    Now its not so simple to find out whether SM cards fit standard PCIe slots so I was looking for some community input.
    Or do other vendors have matching/working cards?
    Or is the whole idea absurd and I stick to the single slot on this ?

    Thanks:)
     
    #1
    Patrick likes this.
  2. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    9,297
    Likes Received:
    2,857
    That is a great question. I have never tried this.
     
    #2
  3. Netwerkz101

    Netwerkz101 Active Member

    Joined:
    Dec 27, 2015
    Messages:
    134
    Likes Received:
    29
    #3
  4. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    I have studied that page, even tried to pry out some info from support regarding the various values (eg can i use GPU type for non-GPU cards) but not much success.

    And is the RSC-RR1U-E8 providing a Gen3 output slot as expected?
     
    #4
  5. Netwerkz101

    Netwerkz101 Active Member

    Joined:
    Dec 27, 2015
    Messages:
    134
    Likes Received:
    29
    Well .. now how would one test this?
    I have an OCZ PCIe NVMe AIC (Gen3 x4) in a PCIe Gen3 x4 slot.

    My assumption is/was that the riser card itself is passive and only extends existing electrical signals.
    "Depending on motherboard" - I interpret as if the riser card is in a Gen2 x4 or x8 slot, the device plugged in will operate at Gen 2 x4 or x8 specs ...
    Likewise if the riser card is plugged into a Gen 3 x4 or x8 slot the device plugged into it gets Gen 3 x4 or x8 specs.

    [​IMG]
     
    #5
    Last edited: Dec 20, 2016
  6. pgh5278

    pgh5278 Active Member

    Joined:
    Oct 25, 2012
    Messages:
    383
    Likes Received:
    95
    Hello this has been covered 15 months back, here are some snippets from tehis forum..
    Riser Card Knowledge required....
    My question to SM.. Supermicro advise (see below) the X10SDV-TLN4F can take a riser card to have 2@ x8 PCIe Gen 3 slots, to facilitate further options until other board come with different mixes of slot.. Now which riser will do the trick per SM comments below.....???

    from Supermicro....Yes! PCIe Bifurication is supported in the latest BIOS on our website [.527]. However, you will need to source your own riser card that can provide additional clock to support multiple PCIe devices.
    Regards, Supermicro
    -------------------------------------------------------------------------------------------------------------

    These are used by some forum members
    PCIe bifurcation worked well. Just enable the setting in the BIOS. In this case I used legacy mode.

    For reference, the riser I used is the ARC1-PELY423. I've also seen this card listed with the model number GH-PELY423.
     
    #6
  7. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    Ah thanks for the search terms :) I will have a look at that thread:)
     
    #7
  8. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    Well I would assume that you should be able to see the connection speed on lspci /pci info ?
    At least I remember seeing that when I was setting up 10GBe in Freenas a while back and I was using the wrong slots... :p
     
    #8
  9. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    So just to wrap this up,
    -there is one generic SM riser fitting in a regular slot (RSC-R2U-2E8) which is PCIe2 only but readily available. There will most likely not be a newer version of this since they moved to specialized variants (according to support).
    -there is the ameri-rack series of risers (ARC1-PELY423+ variations) which would support bi or trifurcation on PCIe3 but seem hard to find.

    Edit: R2032 2U 3-slots PCIE/PCI combo riser card might be feasible as well - has any one used these?

    Edit2: I see There are some IBM Riser Cards as well - I assume those should work in a SM board?
     
    #9
    Last edited: Dec 23, 2016
    DaveHope likes this.
  10. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    So I have tried to buy a R2034 (2U 1 x PCIe X 16 to 2-slots PCIe X 8 w/ 7cm ribbon cable) from RackmountMart.com ($65+p&p)
    They have confirmed that it should be PCIe3 capable so it looked very promising.
    Don't be surprised (if you go looking) if its not on the webpage - I found it only in an old yahoo store page of theirs, but it still is orderable.

    Unfortunately after a lot of back and forth it turned out that it might only be sold to the US - so no go for me:/
     
    #10
  11. Netwerkz101

    Netwerkz101 Active Member

    Joined:
    Dec 27, 2015
    Messages:
    134
    Likes Received:
    29
  12. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    Hm those need one slot per PCIe slot they are converting don't they?
    I am looking for a splitting (bi-/tri-furcation) solution, eg one PCIe3x16 slot to 2x PCIe3x8 or PCIe3x8x4x4:)
     
    #12
  13. Netwerkz101

    Netwerkz101 Active Member

    Joined:
    Dec 27, 2015
    Messages:
    134
    Likes Received:
    29
    You are correct ...they won't help.
    I guess this is a science that I am just not knowledgeable on.
    Obviously OEMs do it with their 2u servers..but oh well.
     
    #13
  14. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
  15. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    So, after reading this thread over on [H] I now have the following:

    You need either
    -Bios PCI-e Bifurcation support
    or a
    -PLX chip on the riser.

    There were various cards in (successful) use
    -ARC1-PELY423 or ARC2-PELY435-C5V3 (v3 might mean pcieV3 support). Some people said Ameri-Rack would only supply to resellers, some said they got single cards easily (I had no response to my mail) (Obtainable here - quite expensive though)
    -RSC-R2UT-2E8R (passive)
    -rsc-r2ug-a2e16-a (Active Card with PLX chip)

    Most used it for multiple GPUs so added flexible PCIe extension cables to it - the really cheap ones seemed to cause issues.

    I have now ordered
    rsc-r2ug-a2e16-a
    RSC-R2UT-2E8R
    RSC-R2UG-2E4E8 (3x)

    to test them on my X10SDV-6c+ and will let you know the results;)
     
    #15
    Last edited: Jan 10, 2017
  16. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    So just a small update.
    rsc-r2ug-a2e16-a - works fine, have not tested in non-bifurcation board yet but I guess it would work to (PLX chip). Both cards (NVME) recognized, slight performance loss (couple of percent). Shown as 2xPCIe 16x in HWInfo. Gets rather warm

    RSC-R2UT-2E8R - Works fine as well, with bifurcation. Have not tested further since it places the cards directly over the CPU. Did not have PCI-extender to place elsewhere.

    RSC-R2UG-2E4E8 - No boot

    I will keep the A2E16 since its the only card fitting correctly at the moment and I assume it will be reusable in other boards.


    Will add a couple of pics/perf results later
     
    #16
    Blinky 42 likes this.
  17. nj47

    nj47 New Member

    Joined:
    Jan 2, 2016
    Messages:
    10
    Likes Received:
    0
    #17
  18. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    So another update - It looks like the A2E16a does not work properly after all.

    Under normal/low load everything is working as expected, but under heavy load the box crashes/reboots:
    (vmksummary.log)
    2017-03-08T12:35:49.143Z cpu7:65572)WARNING: NMP: nmp_DeviceRequestFastDeviceProbe:237: NMP device "mpx.vmhba32:C0:T0:L0" state in doubt; requested fast path state update...
    0:00:00:09.390 cpu0:65536)WARNING: PCI: 751: Non-PF mem (0x0 - 0x3f) out of range
    0:00:00:09.390 cpu0:65536)WARNING: PCI: 886: Out of resources on parent: Root for device: 0000:ff:12.0
    0:00:00:09.390 cpu0:65536)WARNING: PCI: 940: No resources for device: 0000:ff:12.0, BAR[4]: 0x0, size: 64, type: 0x2, flags: 0x0, status: Already exists
    0:00:00:09.451 cpu0:65536)WARNING: Device: 1316: Failed to register device 0x430409e41c10 s00000000.00 10b5874700000000060400 (parent=0x18c9430409e42f38): Already exists
    0:00:00:09.451 cpu0:65536)WARNING: PCI: 260: dlBusAddress s00000000.00 for PCI device 0000:05:00.0 not unique; using sbdf
    0:00:00:09.460 cpu0:65536)WARNING: Serial: 274: logPort initialization failed
    0:00:00:09.460 cpu0:65536)WARNING: Serial: 283: gdbPort initialization failed
    0:00:00:09.460 cpu0:65536)WARNING: Serial: 300: shellPort initialization failed
    0:00:00:09.460 cpu0:65536)WARNING: Serial: 317: consolePort initialization failed
    2017-03-08T12:41:14.105Z cpu0:65956)WARNING: xhci_pci_attach:208: xhci devid 0x8c318086: Success


    Note I run a NVMe SSD and a Mellanox Connectx3 in that riser, this might be too much for it.
    Talked to SM support which indicated resourcing issues - ie non compatibility with the riser (as expected).
    Tried playing around with bifurcation (setting to x16 or x8/x8) but no difference.

    Too bad :(
    Looks I need to go full board after all.
     
    #18
  19. Rand__

    Rand__ Active Member

    Joined:
    Mar 6, 2014
    Messages:
    972
    Likes Received:
    103
    Another update:
    I have the same error on a X10SRI (same installation of ESX), so its not the riser card ...
    Sigh - just when I got a new board ;)
     
    #19
Similar Threads: x10sdv Riser
Forum Title Date
Processors and Motherboards X10SDV-TLN4F 1U Case and Riser Oct 6, 2015
Processors and Motherboards PCIe Bifurcation in X10SDV-6C+-TLN4F Jun 9, 2017
Processors and Motherboards How much airflow for X10SDVT-LN4F (Xeon D 1541)? Jun 7, 2017
Processors and Motherboards SM X10SDV-4C+-TLN4F issues with Intel RSTe Raid Apr 23, 2017
Processors and Motherboards X10SDV 8-core for iperf/bw testing. Apr 19, 2017

Share This Page