Since we already did the NVIDIA Tesla P100 16GB SXM2 Monero mining speed, I figured I would burn-in the 8x Tesla V100 system for 15 minutes with this before benchmarking. Using the same Docker container for both. Here are the results.
What is really interesting here is that the Tesla P100 16GB SXM2 setup was essentially the same speed, but slightly faster. The SXM2 modules have higher TDP. Perhaps that helps. Power consumption of this server is 1.397kW versus 1.33kW on the Tesla P100 box, but the ~60W can be accounted for from the differences in configuration. Both systems are using 4x Delta brand 2.2kW power supplies that are 80Plus Platinum rated.
You can see that the containers are using more memory, but that is not really yielding more performance.
Looks like it is running this stable, so time for Resnet50 training.
Code:
[2019-01-06 02:11:22] accepted (153/0) diff 100001 (65 ms)
| THREAD | GPU | 10s H/s | 60s H/s | 15m H/s | NAME
| 0 | 0 | n/a | 1527.0 | 1527.0 | Tesla V100-PCIE-32GB
| 1 | 1 | n/a | 1526.8 | 1526.5 | Tesla V100-PCIE-32GB
| 2 | 2 | n/a | 1526.7 | 1526.7 | Tesla V100-PCIE-32GB
| 3 | 3 | n/a | 1524.1 | 1524.7 | Tesla V100-PCIE-32GB
| 4 | 4 | n/a | 1526.1 | 1526.0 | Tesla V100-PCIE-32GB
| 5 | 5 | n/a | 1526.2 | 1526.4 | Tesla V100-PCIE-32GB
| 6 | 6 | n/a | 1525.8 | 1526.4 | Tesla V100-PCIE-32GB
| 7 | 7 | n/a | 1526.9 | 1526.8 | Tesla V100-PCIE-32GB
[2019-01-06 02:11:34] speed 10s/60s/15m n/a 12210.0 12211.0 H/s max n/a H/s
Code:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.48 Driver Version: 410.48 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla V100-PCIE... Off | 00000000:3D:00.0 Off | 0 |
| N/A 59C P0 119W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla V100-PCIE... Off | 00000000:3E:00.0 Off | 0 |
| N/A 59C P0 120W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 2 Tesla V100-PCIE... Off | 00000000:40:00.0 Off | 0 |
| N/A 62C P0 122W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 3 Tesla V100-PCIE... Off | 00000000:41:00.0 Off | 0 |
| N/A 57C P0 119W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 4 Tesla V100-PCIE... Off | 00000000:B1:00.0 Off | 0 |
| N/A 58C P0 116W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 5 Tesla V100-PCIE... Off | 00000000:B2:00.0 Off | 0 |
| N/A 58C P0 123W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 6 Tesla V100-PCIE... Off | 00000000:B4:00.0 Off | 0 |
| N/A 61C P0 121W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 7 Tesla V100-PCIE... Off | 00000000:B5:00.0 Off | 0 |
| N/A 58C P0 119W / 250W | 31160MiB / 32480MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
Looks like it is running this stable, so time for Resnet50 training.