DeepLearning11: 10x NVIDIA GTX 1080 Ti Single Root Deep Learning Server (Part 1)

[Patrick Kennedy]

We built a 10 GPU deep learning machine with 10x NVIDIA GTX 1080 Ti Founders Edition cards and break down what you need to know to do the same

The post DeepLearning11: 10x NVIDIA GTX 1080 Ti Single Root Deep Learning Server (Part 1) appeared first on ServeTheHome.

Continue reading...

 

[gigatexal]

i probably missed it but what does single root give you? is it something based on SR-IOV?

 

[Patrick]

i probably missed it but what does single root give you? is it something based on SR-IOV?

Basically, all of the GPUs sit off CPU and on PLX switch(es). You also will typically put your Mellanox Infiniband NIC on there. With single root you get huge bandwidth and low latency GPU to GPU transfers. You also bypass saturating the QPI bus since there is much less going to CPU2.

 

[T_Minus]

You may have touched this already @Patrick but what's going on with QPI on newer gen E5s?

 

[gigatexal]

Basically, all of the GPUs sit off CPU and on PLX switch(es). You also will typically put your Mellanox Infiniband NIC on there. With single root you get huge bandwidth and low latency GPU to GPU transfers. You also bypass saturating the QPI bus since there is much less going to CPU2.

Does it effectively make a dual cpu box a single cpu box? Oh wait! The cards sit on an expander with a switch that then interfaces with cpu0?

 

[PigLover]

Why bother with the 2nd CPU at all in this configuration? You aren't likely to be doing too much CPU intensive work beyond managing jobs onto the GPUs. You've already forced all of PCIe onto one CPU using the PLX switches. And anything that ends up on CPU2 or the memory associated with CPU2 will be subject to cross-CPU latency issues.

Seems like this is a perfect candidate for a single-socket design, perhaps one where the extra IO flexibility of EPYC might give it an edge.

 

[Patrick]

Why bother with the 2nd CPU at all in this configuration? You aren't likely to be doing too much CPU intensive work beyond managing jobs onto the GPUs. You've already forced all of PCIe onto one CPU using the PLX switches. And anything that ends up on CPU2 or the memory associated with CPU2 will be subject to cross-CPU latency issues.

Seems like this is a perfect candidate for a single-socket design, perhaps one where the extra IO flexibility of EPYC might give it an edge.

My initial thoughts but everyone is doing 2P for more RAM capacity it seems.

And the EPYC point, AMD EPYC Infinity Fabric v. Intel Broadwell-EP QPI Architecture Explained

You would end up using PCIe switches as well. Single socket EPYC does not have enough PCIe lanes to play in this space without switches. You need 160x for the GPUs and 8-16x for a NIC.

 

[DWSimmons]

@Patrick What 1080ti with radial/blower fan did you settle on? My shopping for the 11GOC produced almost all axial/into-case fans. I saw an EVGA and one other, MSI?, all the rest were founders edition iirc.