Intel Xeon D-1500 Series Discussion

Discussion in 'Processors and Motherboards' started by Patrick, Mar 9, 2015.

  1. FQB

    FQB New Member

    Joined:
    Oct 26, 2015
    Messages:
    12
    Likes Received:
    1
    I'm in the same boat - just haven't bought it just yet.

    - According to Anandtech's previous review:
    "Speaking of networking, the SoC will have bandwidth for two 10GbE connections direct, which will work both in 1G and 2.5G modes. These are optimized for virtualization, allowing 128 Tx and Rx queues per port as well as SR-IOV and VMDq enhancements. With the integration on board, driver support should also be easier to manage rather than external controller solutions."
     
    #621
  2. Jerry Chen

    Jerry Chen New Member

    Joined:
    Oct 26, 2015
    Messages:
    6
    Likes Received:
    1
    Actually, the onboard 1Gb NIC is SR-IOV capable. I had a blog post involving SR-IOV while doing GPU passthrough: GPU Passthrough via vfio-pci with KVM on Ubuntu 15.04
     
    #622
    FQB likes this.
  3. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,475
    Likes Received:
    220
    I know Paul is on here but he sells those Supermicro machines or has a deal to sell them that he pushes a lot. I don't consider him an independent party with Tinkertry like I would AT, Engadget STH. I'm not saying he doesn't have good stuff, but I consider him as independent as a Supermicro salesperson at this point.

    I'm also thinking there's a difference between support for PCIe devices and on die devices in this area. I'm guessing you'll be fine and @PigLover from what I have read is a real world networking guru. I reread everything he renders opinion on.
     
    #623
    FQB likes this.
  4. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70

    Neither do I, I have purchased two of these servers and sr-iov is critical for 10 gbe port due to openvswitch network performance limitations or like what @PigLover suggested trading CPU core for network performance.

    Like what Patrick said, its not a show stopper, the 10 gbe can be dedicated using for file server (zfs) purpose, used one cpu core as openvswitch + dpdk acceleration for the other 10 gbe for vm to use (maybe reduce max vm from 16 to 14?, just use more containers than vm then). The two 1gbe ports can be passthrough to Pfsense vm as WAN and LAN.
     
    #624
  5. jonaspaulo

    jonaspaulo Member

    Joined:
    Jan 26, 2015
    Messages:
    44
    Likes Received:
    2
    I have also bought one of these and needed SR-IOV. I checked the press release for the boards at the time on SM site and they said they supported it as well , as you mentioned, many other sites.

    This is really not a good move by them. I am a little upset..
     
    #625
  6. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    funny thing happened with kernel 4.2.3..... all sr-iov show up.... how?![emoji7] [​IMG]
     
    #626
  7. FQB

    FQB New Member

    Joined:
    Oct 26, 2015
    Messages:
    12
    Likes Received:
    1
    as Jerry Chen pointed out before:

    " the onboard 1Gb NIC is SR-IOV capable." but not for 10GB
     
    #627
  8. Davewolfs

    Davewolfs Active Member

    Joined:
    Aug 6, 2015
    Messages:
    337
    Likes Received:
    31
    If future revisions support it and these boards don't I'd be mad too.
     
    #628
    Last edited: Oct 27, 2015
  9. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    the lspci shows even the 10gbe card with VF... 03.xx.x are all intel x557 10gbe card's VF device
     
    #629
    FQB likes this.
  10. Jerry Chen

    Jerry Chen New Member

    Joined:
    Oct 26, 2015
    Messages:
    6
    Likes Received:
    1
    On my Ubuntu 15.04 kernel 3.9 environment, I'm unable to enable VFs for X552/557. What are your steps to reproduce such result?
     
    #630
  11. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    I am on ubuntu 14.04.3 and manually upgrade the kernel to 4.2.3 ( I am not able to use 4.2.4 or 4.2.5 due to zfs dkms compile failed, I am on zfs root file system)

    root@pdc:~# uname -r
    4.2.3-040203-generic

    lspci -vnn | grep Ethernet
    03:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:15ad]
    03:00.1 Ethernet controller [0200]: Intel Corporation Device [8086:15ad]
    03:10.0 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.1 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.2 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.3 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.4 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.5 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.6 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:10.7 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.0 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.1 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.2 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.3 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.4 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.5 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.6 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    03:11.7 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]

    root@pdc:~# cat /etc/modprobe.d/ixgbe.conf
    options ixgbe max_vfs=8

    The iommu group is seperated and ready to be use for kvm vfio pci passthrough

    /sys/kernel/iommu_groups/45/devices/0000:03:10.0
    /sys/kernel/iommu_groups/46/devices/0000:03:10.2
    /sys/kernel/iommu_groups/47/devices/0000:03:10.4
    /sys/kernel/iommu_groups/48/devices/0000:03:10.6
    /sys/kernel/iommu_groups/49/devices/0000:03:11.0
    /sys/kernel/iommu_groups/50/devices/0000:03:11.2
    /sys/kernel/iommu_groups/51/devices/0000:03:11.4
    /sys/kernel/iommu_groups/52/devices/0000:03:11.6
    /sys/kernel/iommu_groups/53/devices/0000:03:10.1
    /sys/kernel/iommu_groups/54/devices/0000:03:10.3
    /sys/kernel/iommu_groups/55/devices/0000:03:10.5
    /sys/kernel/iommu_groups/56/devices/0000:03:10.7
    /sys/kernel/iommu_groups/57/devices/0000:03:11.1
    /sys/kernel/iommu_groups/58/devices/0000:03:11.3
    /sys/kernel/iommu_groups/59/devices/0000:03:11.5
    /sys/kernel/iommu_groups/60/devices/0000:03:11.7
     
    #631
  12. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    @Jerry Chen

    Use kernel 4.2+ ...

    ixgbe: add define for X557 PHY ID · torvalds/linux@c2c78d5 · GitHub

    ... or build the ixgbe module for your earlier kernel from the code on sourceforge.

    Thanks @EluRex and @Jerry Chen for confirming SR-IOV works for you.

    I had thought it should from the PCI info provided by @wago.

    BTW, the num_vfs kernel module parameter is deprecated per the warning.

    This is one way to do it with systemd:

    cat /etc/tmpfiles.d/igb.conf
    w /sys/class/net/eth[0-1]/device/sriov_numvfs - - - - 7
    w /sys/class/net/eth[2-3]/device/sriov_numvfs - - - - 64

    The first is for 7 VFs for an I350. The second for an X557 - look at the 64!!! :)

    A better way then with tmpfiles.d would probably be with something specific to udev.
     
    #632
    Last edited: Oct 27, 2015
  13. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    @EluRex Thanks!

    Finally, the IOMMU groups!

    Matching the ZFS for Linux while staying with more recent kernels can be frustrating :cool:

    I've been considering using this platform and it's nice to know that what I expected would work does indeed work!
     
    #633
  14. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    @EluRex

    BTW it took me too long to compose that post with link and example from my config.

    I posted some of the same info right after you. Sorry!
     
    #634
  15. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    tell me about it... my configuration also adds mellanox infiniband ofed driver and scst srpt into the mix.... I have to choose my kernel version very very very carefully.
     
    #635
  16. Jerry Chen

    Jerry Chen New Member

    Joined:
    Oct 26, 2015
    Messages:
    6
    Likes Received:
    1
    Code:
    root@xeon-kvm-host:/tmp/ixgbe-4.1.5/src# modinfo ixgbe
    
    filename:       /lib/modules/4.0.0-040000-generic/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
    
    version:        4.1.5
    
    license:        GPL
    
    description:    Intel(R) 10 Gigabit PCI Express Network Driver
    
    author:         Intel Corporation, <linux.nics@intel.com>
    ...
    
    Code:
    [  213.798236] ixgbe: I/O Virtualization (IOV) set to 4
    
    [  213.798240] ixgbe: 0000:03:00.0: ixgbe_check_options: IOV is not supported on this hardware.  Disabling IOV.
    
     
    #636
  17. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    Code:
    root@pdc:~# modinfo ixgbe
    filename:       /lib/modules/4.2.3-040203-generic/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko
    version:        4.0.1-k
    license:        GPL
    description:    Intel(R) 10 Gigabit PCI Express Network Driver
    author:         Intel Corporation, <linux.nics@intel.com>
    srcversion:     E08010C0D72963989E9A05E
    alias:          pci:v00008086d000015ADsv*sd*bc*sc*i*
    ....
    alias:          pci:v00008086d000010B6sv*sd*bc*sc*i*
    depends:        mdio,ptp,dca,vxlan
    intree:         Y
    vermagic:       4.2.3-040203-generic SMP mod_unload modversions
    signer:         Build time autogenerated kernel key
    sig_key:        CE:E6:5B:C1:E3:07:7F:6F:F5:58:77:52:AC:20:3F:9D:65:F7:05:DA
    sig_hashalgo:   sha512
    parm:           max_vfs:Maximum number of virtual functions to allocate per physical function - default is zero and maximum value is 63. (Deprecated) (uint)
    parm:           allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599-based adapters (uint)
    parm:           debug:Debug level (0=none,...,16=all) (int)
     
    #637
  18. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    Code:
    root@pdc:~# dmesg|grep ixgbe
    [    0.000000] Command line: BOOT_IMAGE=/ROOT/ubuntu@/boot/vmlinuz-4.2.3-040203-generic root=ZFS=rpool/ROOT/ubuntu ro boot=zfs rpool=rpool bootfs=rpool/ROOT/ubuntu intel_iommu=on ixgbe.max_vfs=8
    [    0.000000] Kernel command line: BOOT_IMAGE=/ROOT/ubuntu@/boot/vmlinuz-4.2.3-040203-generic root=ZFS=rpool/ROOT/ubuntu ro boot=zfs rpool=rpool bootfs=rpool/ROOT/ubuntu intel_iommu=on ixgbe.max_vfs=8
    [    7.208439] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.0.1-k
    [    7.208439] ixgbe: Copyright (c) 1999-2014 Intel Corporation.
    [    7.208607] ixgbe 0000:03:00.0: Enabling SR-IOV VFs using the max_vfs module parameter is deprecated - please use the pci sysfs interface instead.
    [    9.240064] ixgbe 0000:03:00.0 0000:03:00.0 (uninitialized): SR-IOV enabled with 8 VFs
    [    9.512691] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 2.12.1-k
    [    9.512692] ixgbevf: Copyright (c) 2009 - 2012 Intel Corporation.
    [    9.611689] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 4, Tx Queue count = 4
    [    9.678012] ixgbe 0000:03:00.0: PCI Express bandwidth of 2GT/s available
    [    9.678013] ixgbe 0000:03:00.0: (Speed:2.5GT/s, Width: x1, Encoding Loss:20%)
    [    9.678014] ixgbe 0000:03:00.0: This is not sufficient for optimal performance of this card.
    [    9.678015] ixgbe 0000:03:00.0: For optimal performance, at least 20GT/s of bandwidth is required.
    [    9.678015] ixgbe 0000:03:00.0: A slot with more lanes and/or higher speed is suggested.
    [    9.722230] ixgbe 0000:03:00.0: MAC: 5, PHY: 6, PBA No: 020000-000
    [    9.722231] ixgbe 0000:03:00.0: 0c:c4:7a:74:09:18
    [   15.215889] ixgbe 0000:03:00.0 eth1: IOV is enabled with 8 VFs
    [   15.238000] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection
    [   15.238201] ixgbe 0000:03:00.1: Enabling SR-IOV VFs using the max_vfs module parameter is deprecated - please use the pci sysfs interface instead.
    [   16.787177] ixgbevf 0000:03:10.1: enabling device (0000 -> 0002)
    [   16.788293] ixgbevf 0000:03:10.1: PF still in reset state.  Is the PF interface up?
    [   16.788293] ixgbevf 0000:03:10.1: Assigning random MAC address
    [   18.431419] ixgbevf 0000:03:10.1: 86:c7:2f:a5:60:0c
    [   18.431420] ixgbevf 0000:03:10.1: MAC: 4
    [   18.431420] ixgbevf 0000:03:10.1: Intel(R) 82599 Virtual Function
    [   18.431577] ixgbevf 0000:03:10.3: enabling device (0000 -> 0002)
    [   18.432709] ixgbevf 0000:03:10.3: PF still in reset state.  Is the PF interface up?
    [   18.432709] ixgbevf 0000:03:10.3: Assigning random MAC address
    [   19.363463] ixgbevf 0000:03:10.3: 62:5a:23:07:bd:59
    [   19.363464] ixgbevf 0000:03:10.3: MAC: 4
    [   19.363465] ixgbevf 0000:03:10.3: Intel(R) 82599 Virtual Function
    [   19.363609] ixgbevf 0000:03:10.5: enabling device (0000 -> 0002)
    [   19.364718] ixgbevf 0000:03:10.5: PF still in reset state.  Is the PF interface up?
    [   19.364719] ixgbevf 0000:03:10.5: Assigning random MAC address
    [   19.364849] ixgbevf 0000:03:10.5: 92:65:d9:a2:cd:48
    [   19.364849] ixgbevf 0000:03:10.5: MAC: 4
    [   19.364850] ixgbevf 0000:03:10.5: Intel(R) 82599 Virtual Function
    [   19.364978] ixgbevf 0000:03:10.7: enabling device (0000 -> 0002)
    [   19.366081] ixgbevf 0000:03:10.7: PF still in reset state.  Is the PF interface up?
    [   19.366082] ixgbevf 0000:03:10.7: Assigning random MAC address
    [   19.366202] ixgbevf 0000:03:10.7: de:0c:6b:25:63:08
    [   19.366203] ixgbevf 0000:03:10.7: MAC: 4
    [   19.366203] ixgbevf 0000:03:10.7: Intel(R) 82599 Virtual Function
    [   19.366329] ixgbevf 0000:03:11.1: enabling device (0000 -> 0002)
    [   19.367439] ixgbevf 0000:03:11.1: PF still in reset state.  Is the PF interface up?
    [   19.367440] ixgbevf 0000:03:11.1: Assigning random MAC address
    [   19.367568] ixgbevf 0000:03:11.1: 4a:fe:8f:ee:a9:1b
    [   19.367569] ixgbevf 0000:03:11.1: MAC: 4
    [   19.367569] ixgbevf 0000:03:11.1: Intel(R) 82599 Virtual Function
    [   19.367697] ixgbevf 0000:03:11.3: enabling device (0000 -> 0002)
    [   19.368804] ixgbevf 0000:03:11.3: PF still in reset state.  Is the PF interface up?
    [   19.368804] ixgbevf 0000:03:11.3: Assigning random MAC address
    [   19.368925] ixgbevf 0000:03:11.3: ce:85:10:5b:70:dc
    [   19.368925] ixgbevf 0000:03:11.3: MAC: 4
    [   19.368926] ixgbevf 0000:03:11.3: Intel(R) 82599 Virtual Function
    [   19.369050] ixgbevf 0000:03:11.5: enabling device (0000 -> 0002)
    [   19.370171] ixgbevf 0000:03:11.5: PF still in reset state.  Is the PF interface up?
    [   19.370172] ixgbevf 0000:03:11.5: Assigning random MAC address
    [   19.370308] ixgbevf 0000:03:11.5: a6:60:df:d0:d1:34
    [   19.370309] ixgbevf 0000:03:11.5: MAC: 4
    [   19.370310] ixgbevf 0000:03:11.5: Intel(R) 82599 Virtual Function
    [   19.370433] ixgbevf 0000:03:11.7: enabling device (0000 -> 0002)
    [   19.371535] ixgbevf 0000:03:11.7: PF still in reset state.  Is the PF interface up?
    [   19.371536] ixgbevf 0000:03:11.7: Assigning random MAC address
    [   19.371658] ixgbevf 0000:03:11.7: d2:bc:8a:08:3a:8a
    [   19.371659] ixgbevf 0000:03:11.7: MAC: 4
    [   19.371660] ixgbevf 0000:03:11.7: Intel(R) 82599 Virtual Function
    [   19.371673] ixgbe 0000:03:00.1 0000:03:00.1 (uninitialized): SR-IOV enabled with 8 VFs
    [   19.748219] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 4, Tx Queue count = 4
    [   19.814547] ixgbe 0000:03:00.1: PCI Express bandwidth of 2GT/s available
    [   19.814549] ixgbe 0000:03:00.1: (Speed:2.5GT/s, Width: x1, Encoding Loss:20%)
    [   19.814549] ixgbe 0000:03:00.1: This is not sufficient for optimal performance of this card.
    [   19.814550] ixgbe 0000:03:00.1: For optimal performance, at least 20GT/s of bandwidth is required.
    [   19.814551] ixgbe 0000:03:00.1: A slot with more lanes and/or higher speed is suggested.
    [   19.858764] ixgbe 0000:03:00.1: MAC: 5, PHY: 6, PBA No: 020000-000
    [   19.858765] ixgbe 0000:03:00.1: 0c:c4:7a:74:09:19
    [   21.708071] ixgbe 0000:03:00.1 eth6: IOV is enabled with 8 VFs
    [   21.727074] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection
    [   21.727153] ixgbevf 0000:03:10.0: enabling device (0000 -> 0002)
    [   21.728283] ixgbevf 0000:03:10.0: PF still in reset state.  Is the PF interface up?
    [   21.728284] ixgbevf 0000:03:10.0: Assigning random MAC address
    [   22.123348] ixgbevf 0000:03:10.0: 8e:0c:94:c5:aa:f0
    [   22.123349] ixgbevf 0000:03:10.0: MAC: 4
    [   22.123349] ixgbevf 0000:03:10.0: Intel(R) 82599 Virtual Function
    [   22.123360] ixgbevf 0000:03:10.2: enabling device (0000 -> 0002)
    [   22.124467] ixgbevf 0000:03:10.2: PF still in reset state.  Is the PF interface up?
    [   22.124468] ixgbevf 0000:03:10.2: Assigning random MAC address
    [   22.124597] ixgbevf 0000:03:10.2: 9e:a9:c2:aa:1e:18
    [   22.124597] ixgbevf 0000:03:10.2: MAC: 4
    [   22.124598] ixgbevf 0000:03:10.2: Intel(R) 82599 Virtual Function
    [   22.124605] ixgbevf 0000:03:10.4: enabling device (0000 -> 0002)
    [   22.125706] ixgbevf 0000:03:10.4: PF still in reset state.  Is the PF interface up?
    [   22.125706] ixgbevf 0000:03:10.4: Assigning random MAC address
    [   22.125832] ixgbevf 0000:03:10.4: 6e:c3:69:e7:ff:4e
    [   22.125832] ixgbevf 0000:03:10.4: MAC: 4
    [   22.125833] ixgbevf 0000:03:10.4: Intel(R) 82599 Virtual Function
    [   22.125840] ixgbevf 0000:03:10.6: enabling device (0000 -> 0002)
    [   22.126941] ixgbevf 0000:03:10.6: PF still in reset state.  Is the PF interface up?
    [   22.126942] ixgbevf 0000:03:10.6: Assigning random MAC address
    [   22.943349] ixgbevf 0000:03:10.6: 86:df:7b:b6:81:e4
    [   22.943350] ixgbevf 0000:03:10.6: MAC: 4
    [   22.943351] ixgbevf 0000:03:10.6: Intel(R) 82599 Virtual Function
    [   22.943362] ixgbevf 0000:03:11.0: enabling device (0000 -> 0002)
    [   22.944516] ixgbevf 0000:03:11.0: PF still in reset state.  Is the PF interface up?
    [   22.944517] ixgbevf 0000:03:11.0: Assigning random MAC address
    [   22.944643] ixgbevf 0000:03:11.0: a6:41:27:17:7b:f5
    [   22.944643] ixgbevf 0000:03:11.0: MAC: 4
    [   22.944644] ixgbevf 0000:03:11.0: Intel(R) 82599 Virtual Function
    [   22.944649] ixgbevf 0000:03:11.2: enabling device (0000 -> 0002)
    [   22.945749] ixgbevf 0000:03:11.2: PF still in reset state.  Is the PF interface up?
    [   22.945750] ixgbevf 0000:03:11.2: Assigning random MAC address
    [   22.945863] ixgbevf 0000:03:11.2: d6:c7:c2:9f:a3:97
    [   22.945864] ixgbevf 0000:03:11.2: MAC: 4
    [   22.945865] ixgbevf 0000:03:11.2: Intel(R) 82599 Virtual Function
    [   22.945871] ixgbevf 0000:03:11.4: enabling device (0000 -> 0002)
    [   22.946978] ixgbevf 0000:03:11.4: PF still in reset state.  Is the PF interface up?
    [   22.946979] ixgbevf 0000:03:11.4: Assigning random MAC address
    [   22.947093] ixgbevf 0000:03:11.4: 42:3f:49:4f:8f:98
    [   22.947094] ixgbevf 0000:03:11.4: MAC: 4
    [   22.947094] ixgbevf 0000:03:11.4: Intel(R) 82599 Virtual Function
    [   22.947100] ixgbevf 0000:03:11.6: enabling device (0000 -> 0002)
    [   22.948245] ixgbevf 0000:03:11.6: PF still in reset state.  Is the PF interface up?
    [   22.948246] ixgbevf 0000:03:11.6: Assigning random MAC address
    [   22.948354] ixgbevf 0000:03:11.6: 72:02:ed:13:7f:05
    [   22.948355] ixgbevf 0000:03:11.6: MAC: 4
    [   22.948355] ixgbevf 0000:03:11.6: Intel(R) 82599 Virtual Function
    [   23.162283] ixgbevf 0000:03:11.7 em2_7: renamed from eth10
    [   23.979141] ixgbevf 0000:03:11.6 em1_7: renamed from eth17
    [   34.531118] ixgbe 0000:03:00.1 rename27: renamed from eth6
    [   35.502452] ixgbevf: Unable to start - perhaps the PF Driver isn't up yet
    the only thing concerns me is
    ixgbe 0000:03:00.0: PCI Express bandwidth of 2GT/s available
    ixgbe 0000:03:00.0: (Speed:2.5GT/s, Width: x1, Encoding Loss:20%)
    ixgbe 0000:03:00.0: For optimal performance, at least 20GT/s of bandwidth is required.

    your modinfo shows
    I believe kernel version 4.2.x is the key


    ixgbe 0000:03:00.0: Enabling SR-IOV VFs using the max_vfs module parameter is deprecated - please use the pci sysfs interface instead.

    I need to find out how to use pci sysfs way to enable it
     
    #638
    Last edited: Oct 27, 2015
  19. EluRex

    EluRex Active Member

    Joined:
    Apr 28, 2015
    Messages:
    209
    Likes Received:
    70
    Code:
    03:10.1 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    Subsystem: Super Micro Computer Inc Device [15d9:15ad]
    Flags: fast devsel
    [virtual] Memory at fb300000 (64-bit, non-prefetchable) [size=16K]
    [virtual] Memory at fb200000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [70] MSI-X: Enable- Count=3 Masked-
    Capabilities: [a0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [1b0] Access Control Services Kernel driver in use: vfio-pci
    
    03:10.2 Ethernet controller [0200]: Intel Corporation Device [8086:15a8]
    Subsystem: Super Micro Computer Inc Device [15d9:15ad]
    Flags: fast devsel
    [virtual] Memory at fb504000 (64-bit, non-prefetchable) [size=16K]
    [virtual] Memory at fb404000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [70] MSI-X: Enable- Count=3 Masked-
    Capabilities: [a0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [1b0] Access Control Services Kernel driver in use: vfio-pci
    Confirm pci-passthrough and freebsd able to see the card
    Xshot-0041.png

    pfsense VM sees 10gbe card.
     
    #639
    AmshTemp likes this.
  20. ggg

    ggg Member

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    @EluRex

    export NUM_VFS=7
    export ETH_NUM=eth0
    echo $NUM_VFS > /sys/class/net/$ETH_NUM/device/sriov_numvfs

    Change ETH_NUM to match the interface you want to do that for. You should have 4 (2 from I350 and 2 from X557).

    Adjust the NUM_VFS to match the number of VFs you want per interface up to the max for the PF.

    A way to have this done at boot with tmpfiles.d is in my earlier post.
     
    #640
    Last edited: Oct 27, 2015
    EluRex likes this.
Similar Threads: Intel Xeon
Forum Title Date
Processors and Motherboards Recommended RAM for Intel Xeon W-22xx Feb 29, 2020
Processors and Motherboards Intel roadmap for Xeon-E ? Feb 26, 2020
Processors and Motherboards Intel XEON QS vs ES Nov 29, 2019
Processors and Motherboards Intel NUC - Xeon with graphics Aug 9, 2019
Processors and Motherboards Intel s2600cp2: Does it support all Ivy Bridge Xeons? Jul 16, 2019

Share This Page