Intel Xeon D-1500 Series Discussion

Discussion in 'Processors and Motherboards' started by Patrick, Mar 9, 2015.

  1. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    Dear Patrick

    From my 5028d using ubuntu 14.04.3 kernel 3.19-0.31

    root@pdc:~# lspci -vnn -s 03:00.0
    03:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:15ad]
    Subsystem: Super Micro Computer Inc Device [15d9:15ad]
    Physical Slot: 0-1
    Flags: bus master, fast devsel, latency 0, IRQ 11
    Memory at fbc00000 (64-bit, prefetchable) [size=2M]
    Memory at fbe04000 (64-bit, prefetchable) [size=16K]
    Expansion ROM at 90100000 [disabled] [size=512K]
    Capabilities: [40] Power Management version 3
    Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
    Capabilities: [70] MSI-X: Enable- Count=64 Masked-
    Capabilities: [a0] Express Endpoint, MSI 00
    Capabilities: [e0] Vital Product Data
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [140] Device Serial Number 00-00-c9-ff-ff-00-00-00
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
    Capabilities: [1b0] Access Control Services
    Capabilities: [1c0] Latency Tolerance Reporting

    root@pdc:~# lspci -vnn -s 05:00.0
    05:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
    Subsystem: Super Micro Computer Inc Device [15d9:1521]
    Flags: bus master, fast devsel, latency 0, IRQ 16
    Memory at fb420000 (32-bit, non-prefetchable) [size=128K]
    I/O ports at e020
    Memory at fb444000 (32-bit, non-prefetchable) [size=16K]
    Capabilities: [40] Power Management version 3
    Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
    Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
    Capabilities: [a0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [140] Device Serial Number 0c-c4-7a-ff-ff-74-07-88
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
    Capabilities: [1a0] Transaction Processing Hints
    Capabilities: [1c0] Latency Tolerance Reporting
    Capabilities: [1d0] Access Control Services
    Kernel driver in use: igb

    What is preventing it from enable SR-IOV? the card and also the cpu is definitely capable of vt-d.... its sad thing to hear it cant!
     
    #601
  2. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,947
    Likes Received:
    858
    I think they are talking about the omboard controllers. The i350 is a normal PCIe device. The onboard NICs aren't over PCIe which is why you can't pass them through with VT-d.
     
    #602
  3. OBasel

    OBasel Active Member

    Joined:
    Dec 28, 2010
    Messages:
    494
    Likes Received:
    62
    Does that mean the O model is Optical? I think I'd want SFP+ Optical and 10Gbase-T.
     
    #603
  4. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    tks miniknight for clear that up. But they have pci sid that's why I am assuming they are connected via PCIe Lane. is this function disabled in bios/firmware or it is physically impossible?
     
    #604
  5. EffrafaxOfWug

    EffrafaxOfWug Radioactive Member

    Joined:
    Feb 12, 2015
    Messages:
    1,073
    Likes Received:
    355
    Do they...? I know Patrick hasn't been able to get his hands on any ASRock xeon-D's yet but their E3 and E5 units have always been in the same ballpark from what I've seen.
     
    #605
  6. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,557
    Likes Received:
    4,485
    Here is the official response I got from Intel:
     
    #606
  7. Dom

    Dom New Member

    Joined:
    Oct 22, 2015
    Messages:
    5
    Likes Received:
    0
    It is, using a Cirrus Logic controller I believe.

    Here's the 2015 Catalog:
    http://www.asrockrack.com/general/Catalog2015.pdf

    I apologize if i'm unable to post external links, i'll remove if so.
     
    #607
    Last edited: Oct 26, 2015
  8. Dom

    Dom New Member

    Joined:
    Oct 22, 2015
    Messages:
    5
    Likes Received:
    0
    Good to see these in the wild, thank you for that. I'd wish there was some more communication via Facebook or announcements regarding the availability from Intel and release time of the MB.

    Do you know if you can Passthrough the LSI controller? I'm under the assumption you could!
     
    #608
  9. Dom

    Dom New Member

    Joined:
    Oct 22, 2015
    Messages:
    5
    Likes Received:
    0
    What are we talking about here, a few watts??
     
    #609
  10. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    o_O http://www.supermicro.com/manuals/superserver/mid-tower/MNL-5028D-TN4T.pdf manual page 95 has option to enable sr_iov (I guess that is only for the pci-e slot device not those on integrated SOC)

    So this Xeon D SOC platform in a sense is purposely crippled by Intel so that it cannot run many VM guests due to the networking part will be its bottle neck (it has 10 gbe connections but cannot use it in by VM effectively as
    openvswitch/bridge max throughput is about 6gbps only)

    So for SM board, 10gbe is only good for serving as NAS, and the VM can only utilize the dual i350 eth via OpenvSwitch (which also use more cpu resources).

    It should be or otherwise what is the point making the Xeon D has VT-D functionality at all? right now just hope Asrock has not put the LSA controller behind any pci-e switch (which shouldn't)
     
    #610
    Last edited: Oct 26, 2015
  11. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,775
    Likes Received:
    1,116
    Does anyone know if the SoC based NICs can be supported via DPDK? I think many folks would tell you that running a DPDK-based v-Switch is more "directional" for Intel than reliance on SR-IOV for network visualization.
     
    #611
  12. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    #612
    Last edited: Oct 26, 2015
  13. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    I will do some testing on Open vSwitch over DPDK on Ubuntu It is interesting to see if dpdk acceleration on OVS network performance.
     
    #613
  14. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,775
    Likes Received:
    1,116
    I have test results (that I unfortunately cannot share) with a highly optimized OVS-compatible vSwitch based on DPDK. The results are phenomenal - though it does suck up a core-per-NIC due to DPDK polling activity.
     
    #614
    EluRex and Patrick like this.
  15. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,557
    Likes Received:
    4,485
    Not sure offhand. I am 99% sure that the i354 on the Avoton/ Rangeley is supported by DPDK.

    The fact that the X550 is listed next to ixbe here - DPDK doc makes me hopeful that it is supported.
     
    #615
  16. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    So it's a trade off CPU for Network performance, but core per NIC is a hefty price.
     
    #616
  17. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,775
    Likes Received:
    1,116
    Yup. DPDK vSwitch with two 10Gbe NICs pretty much requires two cores dedicated to the vSwitch.
     
    #617
  18. acmcool

    acmcool Banned

    Joined:
    Jun 23, 2015
    Messages:
    611
    Likes Received:
    76
    Its good I gave up on Xeon D platform ..:)
    Better off with V3 xeon and Chelsio 10g nic..
     
    #618
  19. PigLover

    PigLover Moderator

    Joined:
    Jan 26, 2011
    Messages:
    2,775
    Likes Received:
    1,116
    Also, remember that the trade off is more complex. When comparing SR-IOV vs DPDK you do have to consider the CPU impacts of DPDKs polling. But remember that with SR-IOV you have to expose DMA+memory management to the NIC, which disables a number of very useful vitalization features.

    So to real trade off between two methods of getting high performance network IO under visualization: SR-IOV at the cost of other virtualization features (Migrations, memory ballooning, etc) vs DPDK which keeps those features in tact at the cost of CPU polling overhead.

    Note this trade off is not unique to the D-1500 CPU series. It comes up here because one option (SR-IOV) appears to be disabled on the on-SoC NICs.
     
    #619
    Last edited: Oct 26, 2015
    EluRex likes this.
  20. Remko

    Remko New Member

    Joined:
    Oct 26, 2015
    Messages:
    19
    Likes Received:
    5
    Muh... absolutely not amused where this discussion is going. Especially:


    :mad: The fact is I bought a couple of boards specifically for ... you already guessed it ... semi-heavy virtualization. Furthermore, even Supermicro itself still advertises with SR-IOV on their press site @ Supermicro | News | Supermicro® Launches New Line of Low Power, High Density Server Solutions Supporting Intel® Xeon® Processor D-1500. See quote; I highlighted SR-IOV.


    Is it a hardware firmware issue or a driver issue? Why are a lot of sites touting this as the 'ultimate' virtualization or 'must get' platform (Tinkertry, ServeTheHome, etc)... :confused: What is heavy virtualization btw?


    PS. Offtopic -> Intel Xeon D-1540 10GbE X552/X557 driver/VIB for VMware ESXi 6 on Supermicro 5028D-TN4T system's X10SDV-TLN4F mobo expected by early Nov. 2015. See post @ Tinkertry
     
    #620
    Last edited: Oct 26, 2015
Similar Threads: Intel Xeon
Forum Title Date
Processors and Motherboards Intel XEON QS vs ES Nov 29, 2019
Processors and Motherboards Intel NUC - Xeon with graphics Aug 9, 2019
Processors and Motherboards Intel s2600cp2: Does it support all Ivy Bridge Xeons? Jul 16, 2019
Processors and Motherboards What is the cheapest server/barebone/motherboard that will support a Intel Xeon 6138F CPU? May 22, 2019
Processors and Motherboards 2nd Generation Intel Xeon Scalable and Optane DCPMM Launch Apr 2, 2019

Share This Page