Deeplearning011 - 10x GTX 1080 Ti Build (Deep learning and crypto mining)

[Patrick]

Although DeepLearning010 was great, there are a few items that need to change. First, it was still using both CPUs for PCIe lanes and was not all off a single root. I ended up upgrading the second box to the 4028GR-TR2. That two has two main bonuses. First, it is single root. Second, it supports up to 10x GPUs instead of 8.

BTW - here are the goodies that went in the lab today. Two of the GPUs and some of the RAM went in the DeepLearning011 machine:



I will save someone from counting. 100x DDR4-2400 RDIMMs. 2x HPE Omni-Path adapters (x16 editions), 2x E5-2698 V4's, 2x S3700 400GB's, 2x GTX 1080 Ti's and Optane!

That is all, obviously, not going into the DeepLearning011 box. Actually for an ISC demo, the DeepLearning011 box and then AMD EPYC.

Still waiting on all of the 1080 Ti's to arrive.

 

[MiniKnight]

That is droolworthy for sure.

 

[msvirtualguy]

It's crazy the amount of computing power you can get into a few RU these days. It's like everyone can have their own supercomputer.

 

[Klee]

Now I know who won the powerball......

 

[maze]

Now I know who won the powerball......

And power Bill

Looking good. Massive setup for sure

 

[Patrick]

@maze This is certainly an investment area.

The 8x GPU box can use $1/ hour in Northern CA electricity. The 10x GPU box is certainly going to go over that. A bit scary here is that each machine only has 2 of 24 drive bays filled. 4x 2kW Titanium power supplies.

 

[maze]

Well, with the current crypto spike im sure its going to be a decent investment - unless the marked crash again.

But yes, pool mining auto converting to btc should be good as it is now.

What kind of hashing power are you getting from the box?

 

[Patrick]

What kind of hashing power are you getting from the box?

It is about 6400 Sol/s for ZEC mining.
Ethereum Mining is around 330MH/s

 

[MiniKnight]

#beatsmode

 

[Patrick]

A bit scary, but this put up a pretty impressive Peak Power number today.

 

[trumee]

What library are you using for deep learning?

 

[Patrick]

@trumee - I am spending most of my time using Keras + TF. We have other folks on these machines using other libraries.

This machine is getting features on the main site today.

 

[Jim Dowling]

Would the GTX 1080Ti blower edition cards work on this setup (like this - ZOTAC GeForce® GTX 1080 Ti Blower | ZOTAC). They are retailing at about 100 dollars cheaper than the FE cards you included in the writeup. Great writeup!

 

[Patrick]

@Jim Dowling not sure of the type of cooler they are using. They also do not have back plates with that version. They may work, just not verified.