Intel Xeon D-1500 Series Discussion

Discussion in 'Processors and Motherboards' started by Patrick, Mar 9, 2015.

  1. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
  2. jonaspaulo

    jonaspaulo Member

    Joined:
    Jan 26, 2015
    Messages:
    44
    Likes Received:
    2
    @ggg thanks for pointing that out, in fact I also have found that thread but I believed more on the release notes from the ixgbe drivers which mention:

    "This is the most current release of the ixgbe driver for Linux, which supports kernel versions 2.6.18 up through 4.2.1."

    In regards to features/bugs/fixes the most useful for me in each ixgbe release are (Intel Ethernet Drivers and Utilities - Browse /ixgbe stable at SourceForge.net):

    -4.0.3:
    * Support for new X550 hardware (new features not enabled yet)
    * VxLan RX checksump offload support

    -4.1.1
    * Support for new X550 hardware

    -4.1.5
    the support for kernel versions 2.6.18 up through 4.2.1. (at least from what they mention)


    So to be clear no one has successfully updated the ixgbe driver version to other than the default on linux kernels 4.x ?

    @ggg can you tell me your current versions for the kernel and ixgbe? Thanks

    Thanks again
     
    #682
  3. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    @jonaspaulo

    So it doesn't seem you need 4.1.5, just what is upstream in kernel 4.1+.

    I think most have upgraded their kernel instead of compiling ixgbe for their kernel.

    If you do build your own kernel on Ubuntu 14.04 from git rather then the Ubuntu sources then you might be more successful building ixgbe.

    If you get a more recent kernel from Ubuntu that would work as well. Perhaps you can find a PPA or try this: medigeek/kmp-downloader ยท GitHub

    One thing is clear, ixgbe 4.1.5 is not going to compile against your current Ubuntu kernel without modifications to the module's source.
     
    #683
  4. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    But not the Ubuntu distributions of those versions :)
     
    #684
  5. jonaspaulo

    jonaspaulo Member

    Joined:
    Jan 26, 2015
    Messages:
    44
    Likes Received:
    2
    Yes that could be the reason. They also mention the distributions under which it is supported:
    - RHEL 6.7 - RHEL 7.1 - SLES 11SP4 - SLES 12 - SLES 12SP1 Beta 3


    Thanks a lot for all the help!

    @EluRex can you just comment on my issue with your current setup?
    You have Ubuntu with kernel 4.2.3 and a Xeon d-1540 chip with everything working right?
    Did you upgrade the ixgbe driver or just let the default for 4.2.3?
     
    #685
    Last edited: Nov 3, 2015
  6. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,697
    Likes Received:
    500
    I don't think @EluRex has been able to get a successful ping from his SR-IOV interfaces yet. So this may or may not work.
     
    #686
  7. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    Sorry all, its been a terrible week for me so far.... some older pbx failing... and some pdc RAID1 failing... I just do not have to time to do the test.... I am actually more eager to do the testing then anyone else... lets someone can carry on the test https://forums.servethehome.com/index.php?threads/sm-5028d-tn4t-x10sdv-tln4f-x557-sr-iov-issue.7309/ where I left off..
     
    #687
  8. jonaspaulo

    jonaspaulo Member

    Joined:
    Jan 26, 2015
    Messages:
    44
    Likes Received:
    2
    Yes he just answered me in a PM.

    So to recap and based on the experiments I made and all the people here in the thread:

    - (@EluRex test) "Ubuntu 14.04.3 with Kernel 4.2.3 with success X557 VF and pci-passthrough but not yet test for a successful ping test. The other NIC Intel i350 confirmed with SR-IOV/pci passthrough/successful ping (also on Proxmox 4)" - with original ixgbe 4.0.1.k

    - Ubuntu 14.0.4.3 with Kernel 4.2.3 doesn't allow to install ixgbe 4.1.5 so we stay at the default ixgbe 4.0.1.k - missing ping test on X557 SR-IOV
    - Ubuntu 14.0.4.3 with Kernel 3.19.0-31-generic (default) allowed me to install ixgbe 4.1.5 - but still hadn't the time to search for how to properly activate SR-IOV and do a ping test on the X557 VF

    So we either stay at the default kernel 3.19.0-31 with ixgbe 4.0.1.k or upgrade to kernel 4.2.3 with ixgbe 4.0.1.k. One must test which has the best results. For now it seems that the second option is on the lead.

    Thanks all
     
    #688
  9. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,697
    Likes Received:
    500
    if you can't successfully ping what do you have?
     
    #689
  10. jonaspaulo

    jonaspaulo Member

    Joined:
    Jan 26, 2015
    Messages:
    44
    Likes Received:
    2
    Right now not much. I have been busy.

    a@ubuntukvm:~$ uname -r

    3.19.0-31-generic



    a@ubuntukvm:~$ lspci | grep Ethernet

    03:00.0 Ethernet controller: Intel Corporation Device 15ad

    03:00.1 Ethernet controller: Intel Corporation Device 15ad

    05:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

    05:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)



    a@ubuntukvm:~$ uname -r

    3.19.0-31-generic



    a@ubuntukvm:~$ dmesg | grep Gigabit

    [ 3.965229] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.2.15-k

    [ 4.166681] igb 0000:05:00.0: Intel(R) Gigabit Ethernet Network Connection

    [ 4.334709] igb 0000:05:00.1: Intel(R) Gigabit Ethernet Network Connection

    [ 7.784969] Intel(R) 10 Gigabit PCI Express Network Driver - version 4.1.5

    [ 9.348577] ixgbe 0000:03:00.0 eth0: Intel(R) 10 Gigabit Network Connection

    [ 11.291505] ixgbe 0000:03:00.1 eth1: Intel(R) 10 Gigabit Network Connection





    a@ubuntukvm:~$ echo '7' > sudo cat /sys/class/net/em1/device/sriov_numvfs

    a@ubuntukvm:~$ cat /sys/class/net/em1/device/sriov_numvfs

    0


    It doesn't seem to allow me to write the sriov_numvfs value :s


    [edit] @EluRex which bios version do you have on your supermicro board?

    Thanks
     
    #690
    Last edited: Nov 4, 2015
  11. Remko

    Remko New Member

    Joined:
    Oct 26, 2015
    Messages:
    19
    Likes Received:
    5
    Kind of off-topic but ran into major issue. Received all the hardware last week (see hardware spec beneath). System is hanging on "System Initializing w/ post code 00".

    I do suspect the memory modules. Question: the LRDIMM modules are not supported on the Supermicro X10SDV-TLN4F ?

    Glad to see the fork regarding SR-IOV!

    Hardware specifications
    - Supermicro X10SDV-TLN4F
    - 4x Samsung Dram 32gb ddr4-2133 cl15 lrdimm qr (m386a4g40dm0-cpb)
    - ChassisSupermicro SC-505-203b w/ additional fans
    - Intel 750 1.2 TB
     
    #691
  12. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    2015-11-05_13-23-03.jpg
     
    #692
  13. jonaspaulo

    jonaspaulo Member

    Joined:
    Jan 26, 2015
    Messages:
    44
    Likes Received:
    2
    Thanks @EluRex

    So a short update on this SR-IOV topic, and no I still didn't have the time to test it fully.

    Requirements for Enablement of SR-IOV on both i350 and X520 eths:
    -Linux Kernel 4.2.+ (as EluRex already proved)
    -DO NOT UPGRADE BIOS to 1.0b because there is some kind of bug/error where in this version the SR-IOV option (or ASPM as Supermicro references it on their newest board BIOS) vanishes. I have been in talks with SM about this and already received a custom 1.0b BIOS which has the option reenabled. I am waiting for their response on the GA for this.

    Meanwhile I have also reported the warning some of you already noticed on the PCIe bandwidth an waiting for their response.

    [ 14.825404] ixgbe 0000:03:00.1: PCI Express bandwidth of 2GT/s available
    [ 14.869386] ixgbe 0000:03:00.1: (Speed:2.5GT/s, Width: x1, Encoding Loss:20%)
    [ 14.912905] ixgbe 0000:03:00.1: This is not sufficient for optimal performance of this card.
    [ 14.998712] ixgbe 0000:03:00.1: For optimal performance, at least 20GT/s of bandwidth is required.
    [ 15.087831] ixgbe 0000:03:00.1: A slot with more lanes and/or higher speed is suggested.


    I will try soon to attach VFs (from different PFs, since there seems to be a bug even on the latest ixgbe/ixgbevf drivers that doesn't allow communication between VFs on the same PF) to different VMs and perform a simple ping test.

    Thanks
    BR
     
    #693
    gigatexal likes this.
  14. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,697
    Likes Received:
    500
    so you have it enabled, just haven't tested a ping and got warnings about bandwidth?
     
    #694
  15. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    It was too late!
    https://forums.servethehome.com/ind...0sdv-tln4f-x557-sr-iov-issue.7309/#post-65781

    I updated one of the server which is ready to send to production rack! I am soooooo mad now! is there the original bios 1.0a available for downgrade?!
     
    #695
  16. Davewolfs

    Davewolfs Active Member

    Joined:
    Aug 6, 2015
    Messages:
    329
    Likes Received:
    31
    Please let us know how things go with SM ripping out SR-IOV from bios. 1.0a was not stable for me.

    FYI, 10G is now enabled on ESXi.

    2015-11-05_2308.png
     
    #696
  17. TheHaughom

    TheHaughom New Member

    Joined:
    Oct 28, 2015
    Messages:
    11
    Likes Received:
    0
    How quiet are these 1540-boards?
    I'm contemplating buying one for a new home-lab build, but it will run in a Node 304 which sits in my office.

    If they are noisy I'd rather wait for the E3 v5 CPU's, and put on a liquid cooler or something.
     
    #697
  18. Davewolfs

    Davewolfs Active Member

    Joined:
    Aug 6, 2015
    Messages:
    329
    Likes Received:
    31
    I have the 5028 case. Quiet to me is basically hear nothing. That is how my office PC is. This thing definitely makes some noise but is not crazy loud. I keep mine upstairs behind the TV and the dogs still manage to sleep at night :)
     
    #698
    Last edited: Nov 6, 2015
  19. TheHaughom

    TheHaughom New Member

    Joined:
    Oct 28, 2015
    Messages:
    11
    Likes Received:
    0
    Sounds good. Shouldn't be a problem swapping out the fan with a larger one either, which would be more quiet.
    Or maybe do some home-modding to get a liquid cooler on it.
     
    #699
  20. hjfr

    hjfr Member

    Joined:
    Nov 21, 2013
    Messages:
    73
    Likes Received:
    19
    For those who have a xeon D (and linux with kernel > 3.9), does it support intel_pstate ?
    It's a new power scaling driver for modern Intel CPUs (normally since Intel SandyBridge processors)

    You can verify this with the command (it must return "intel_pstate" if support is ok):
    cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver​
     
    #700
Similar Threads: Intel Xeon
Forum Title Date
Processors and Motherboards Intel XEON QS vs ES Nov 29, 2019
Processors and Motherboards Intel NUC - Xeon with graphics Aug 9, 2019
Processors and Motherboards Intel s2600cp2: Does it support all Ivy Bridge Xeons? Jul 16, 2019
Processors and Motherboards What is the cheapest server/barebone/motherboard that will support a Intel Xeon 6138F CPU? May 22, 2019
Processors and Motherboards 2nd Generation Intel Xeon Scalable and Optane DCPMM Launch Apr 2, 2019

Share This Page