Any recommendation on motherboard that supports 4 x PCIe 3.0 x16/x16/x16/x16?

frozen

Member
Jul 29, 2017
36
0
6
30
Hello, I am considering to use threadripper as the cpu for my machine. I am having a hard time in finding a motherboard that supports the cpu and also allows four Nvidia GPU running at full speed at the same time. Any recommendation on the motherboard (brand and model number please)? Thanks
 

i386

Well-Known Member
Mar 18, 2016
1,936
505
113
31
Germany
Some mainboards for threadripper have 4* x16 slots, but at best 2 of them are x16 electrically. The other slots are running at x8 speeds.
 
  • Like
Reactions: frozen

frozen

Member
Jul 29, 2017
36
0
6
30
Some mainboards for threadripper have 4* x16 slots, but at best 2 of them are x16 electrically. The other slots are running at x8 speeds.
Yes, I know. Why don't they make x16 for all four slots? Is this a technical problem or such board should be out in the near future?
 

i386

Well-Known Member
Mar 18, 2016
1,936
505
113
31
Germany
It's a "technical problem": you need some pcie lanes for a chipset which connects sound cards, usb, sata and other devices to the cpu.
 
  • Like
Reactions: frozen

frozen

Member
Jul 29, 2017
36
0
6
30
Thanks. So, if I want to have x16 for all four slots for the GPUs, I will need to use Intel CPU? I read that the i7920x and the i7900K are comparable with the threadripper 1900x.
 

frozen

Member
Jul 29, 2017
36
0
6
30
Thanks.

For such dual CPU motherboard, can I try one CPU with 2-4 GPU first (16x16 or 16x16x8x8 respectively). Then, add one more CPU later?


If I don't know how to write code to perform parallelism, is't 2 PCs each with 2x16 slots better and faster to get work done? i.e. One PC with 2x 1080Ti and another PC with also 2x 1080Ti. Then, I can perform simulation with four different sets of parameters.

I suppose the total price would be just a little bit more expensive.
 

i386

Well-Known Member
Mar 18, 2016
1,936
505
113
31
Germany
For such dual CPU motherboard, can I try one CPU with 2-4 GPU first (16x16 or 16x16x8x8 respectively). Then, add one more CPU later?
That depends on the mainboard and how the pcie slots are connected

This is an example from the supermicro x10dai mainboard:
X10dai.JPG

About performance: I have no experience with machine/deep learning, can't say what's better (Maybe @Patrick can tell more). From the posts that I saw you will need an interconnect with low latency and high bandwidth like infiniband.
 

frozen

Member
Jul 29, 2017
36
0
6
30
Somebody did a test and found that there is no noticeable difference between PCIe3.0 16x16x16x16 and 8x8x8x8. Probably they were talking about frame rates. Does the conclusion apply to deep learning as well?
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,908
4,871
113
Somebody did a test and found that there is no noticeable difference between PCIe3.0 16x16x16x16 and 8x8x8x8. Probably they were talking about frame rates. Does the conclusion apply to deep learning as well?
Gaming and deep learning are different. Dual CPU is not a bad way to go. I would not spend more than $1500 on CPUs as someone noted in your other thread.