Any recommendation on motherboard that supports 4 x PCIe 3.0 x16/x16/x16/x16?

Discussion in 'Machine Learning, Deep Learning, and AI' started by frozen, Aug 27, 2017.

  1. frozen

    frozen Member

    Joined:
    Jul 29, 2017
    Messages:
    36
    Likes Received:
    0
    Hello, I am considering to use threadripper as the cpu for my machine. I am having a hard time in finding a motherboard that supports the cpu and also allows four Nvidia GPU running at full speed at the same time. Any recommendation on the motherboard (brand and model number please)? Thanks
     
    #1
  2. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,178
    Likes Received:
    267
    Some mainboards for threadripper have 4* x16 slots, but at best 2 of them are x16 electrically. The other slots are running at x8 speeds.
     
    #2
    frozen likes this.
  3. frozen

    frozen Member

    Joined:
    Jul 29, 2017
    Messages:
    36
    Likes Received:
    0
    Yes, I know. Why don't they make x16 for all four slots? Is this a technical problem or such board should be out in the near future?
     
    #3
  4. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,178
    Likes Received:
    267
    It's a "technical problem": you need some pcie lanes for a chipset which connects sound cards, usb, sata and other devices to the cpu.
     
    #4
    frozen likes this.
  5. frozen

    frozen Member

    Joined:
    Jul 29, 2017
    Messages:
    36
    Likes Received:
    0
    Thanks. So, if I want to have x16 for all four slots for the GPUs, I will need to use Intel CPU? I read that the i7920x and the i7900K are comparable with the threadripper 1900x.
     
    #5
  6. Netwerkz101

    Netwerkz101 Active Member

    Joined:
    Dec 27, 2015
    Messages:
    185
    Likes Received:
    40
  7. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,178
    Likes Received:
    267
    Like @Netwerkz101 said you will need a dual cpu setup.
    Epyc could be an alternative, but so far no mainboards with more than 2 x16 slots were released.
     
    #7
    frozen likes this.
  8. frozen

    frozen Member

    Joined:
    Jul 29, 2017
    Messages:
    36
    Likes Received:
    0
    Thanks.

    For such dual CPU motherboard, can I try one CPU with 2-4 GPU first (16x16 or 16x16x8x8 respectively). Then, add one more CPU later?


    If I don't know how to write code to perform parallelism, is't 2 PCs each with 2x16 slots better and faster to get work done? i.e. One PC with 2x 1080Ti and another PC with also 2x 1080Ti. Then, I can perform simulation with four different sets of parameters.

    I suppose the total price would be just a little bit more expensive.
     
    #8
  9. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,178
    Likes Received:
    267
    That depends on the mainboard and how the pcie slots are connected

    This is an example from the supermicro x10dai mainboard:
    X10dai.JPG

    About performance: I have no experience with machine/deep learning, can't say what's better (Maybe @Patrick can tell more). From the posts that I saw you will need an interconnect with low latency and high bandwidth like infiniband.
     
    #9
  10. frozen

    frozen Member

    Joined:
    Jul 29, 2017
    Messages:
    36
    Likes Received:
    0
    Somebody did a test and found that there is no noticeable difference between PCIe3.0 16x16x16x16 and 8x8x8x8. Probably they were talking about frame rates. Does the conclusion apply to deep learning as well?
     
    #10
  11. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    10,784
    Likes Received:
    3,766
    Gaming and deep learning are different. Dual CPU is not a bad way to go. I would not spend more than $1500 on CPUs as someone noted in your other thread.
     
    #11

Share This Page