@Patriot you were right
Swapped to 512/256 GB
Here is the Supermicro SuperBlade with 2x of these cards:
sthroot@dualgridm40:~$ sudo nvidia-smi | tee nvidia-smi.txt
Mon Apr 11 13:17:30 2016
+------------------------------------------------------+
| NVIDIA-SMI 352.63 Driver Version: 352.63 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GRID M40 Off | 0000:05:00.0 Off | N/A |
| 38% 26C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GRID M40 Off | 0000:06:00.0 Off | N/A |
| 38% 27C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 GRID M40 Off | 0000:07:00.0 Off | N/A |
| 38% 28C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 3 GRID M40 Off | 0000:08:00.0 Off | N/A |
| 38% 28C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 4 GRID M40 Off | 0000:83:00.0 Off | N/A |
| 38% 27C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 5 GRID M40 Off | 0000:84:00.0 Off | N/A |
| 38% 27C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 6 GRID M40 Off | 0000:85:00.0 Off | N/A |
| 38% 28C P0 16W / 53W | 9MiB / 4092MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 7 GRID M40 Off | 0000:86:00.0 Off | N/A |
| 0% 28C P0 15W / 53W | 9MiB / 4092MiB | 1% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
ASRock Rack unit stuck on initializing with 4 cards, I then changed for the other 4 and that did not help. Trying 2 now and still stuck at System Initializing...