44 cores 512gb X10DRG-Q 4-way SLI

maxermaxer

Active Member
Oct 28, 2016
224
26
28
45
Thank you very much Venturi. One more question. Why do you need to put the MB on a tray instead of drill holes and stick it onto the chassis board? Do you want to rise the motherboard because of not enough expansion slot holes?


Sent from my iPhone using Tapatalk
 

Venturi

Active Member
Apr 22, 2016
123
165
43
Thank you very much Venturi. One more question. Why do you need to put the MB on a tray instead of drill holes and stick it onto the chassis board? Do you want to rise the motherboard because of not enough expansion slot holes?


Sent from my iPhone using Tapatalk

I did that so that I had more room got the video card riser and positioning.

After I built it, I figured out I could have just drilled the stand-Offs right into the board.


however, I also needed it for the support tube that holds the video card power cables up, so that they cables are not hanging on the cards pulling them down
 
  • Like
Reactions: maxermaxer

psylenced

New Member
Aug 21, 2016
19
5
3
41
I'm working to convert server 2016 to desktop, but running into a few issues so far (but I'll get it figured out eventually)
1. Update engine is NOT suitable to a server, not enough control on selecting/deselecting updates
2. Telemitry - yes, you can turn it off, but not completely
3. Multiple telemitry and recursive services including
additional task scheduler services that re-enable telemetry even if they are deleted.
I seem to remember reading about a very out of the box fix for the task scheduler issue. (This is from memory).

The task scheduler would create tasks by creating a file in %systemroot%\System32\Tasks

What they tried to do was fiddle around with permissions and creating dummy files to stop that task getting recreated in which none of that worked, windows always won.

The fix that did work however, was to create a directory with the same name of the deleted task.

While windows can do whatever it wants to recreate the task file, it cant create it if the new file will create a duplicate directory entry.
 

Venturi

Active Member
Apr 22, 2016
123
165
43
I seem to remember reading about a very out of the box fix for the task scheduler issue. (This is from memory).

The task scheduler would create tasks by creating a file in %systemroot%\System32\Tasks

What they tried to do was fiddle around with permissions and creating dummy files to stop that task getting recreated in which none of that worked, windows always won.

The fix that did work however, was to create a directory with the same name of the deleted task.

While windows can do whatever it wants to recreate the task file, it cant create it if the new file will create a duplicate directory entry.

ok, I got that to work, slightly more complicated than I was anticipating but yes.

I still don't have a solution to the windows update service, that is a different problem that cannot be solved locally ;)
 
  • Like
Reactions: psylenced

Venturi

Active Member
Apr 22, 2016
123
165
43
Small update,

the pc/server is now using (4) four Titan Xp cards....
...blistering cuda performance
 

Venturi

Active Member
Apr 22, 2016
123
165
43
...I don't think you know the headaches that come with it.

As far as posting pics, the Titan XP are visually IDENTICAL to Titan X Pascal, so... a bit of a letdown, same cooling solution.

but I may post some scores/benchmarks in the next week or so
 
  • Like
Reactions: gigatexal

maxermaxer

Active Member
Oct 28, 2016
224
26
28
45
Small update,

the pc/server is now using (4) four Titan Xp cards....
...blistering cuda performance
Hey Venturi, look forward to that! I have a similar build with same motherboard same chassis and 4x 1070 GTX. I use it for Octane rendering (not sure if you are familiar). I found that the C-state CPU setting in the BIOS makes the OctaneBench score much worse than disabling it. I am talking about 30% performance gap indeed.

Would you mind sharing with me your BIOS setting and particularly for CPU advance setting part? Thanks!!


Sent from my iPhone using Tapatalk
 

Venturi

Active Member
Apr 22, 2016
123
165
43
I'll see if I have handy those screen shots and upload them for the bios settings.
generally speaking the C states used are 3 and 6, I'll see what I ended up with when I get home from work...
(this whole working for a living thing is highly overrated )
 

Venturi

Active Member
Apr 22, 2016
123
165
43
some of that got optimized a bios version back. My read and write speeds are around 13GB/s to 31GB/s, ram keeps up well, there are a few tricks to get the most out of it

1 enable above 4G decoding
2, Set Cache on Die rather than Home snoop, early snoop or late snoop
3. Assign storage to the pch channels and leave the normal channels for the 4 video cards, alternating from cpu 0 and cpu 1

my OS array is on twin Samsung 960 pro in 8x
my D and E array are raid 0 across 10 950pro ssd

full macrium backup of all OS and apps takes about 2minutes and 24 seconds
 
Last edited:
  • Like
Reactions: gigatexal

mlaznar

New Member
I'm new here, but I've done a few builds from time to time...


Short summary

Here are the specs:

Supermicro X10DRG-Q
2x E5-2699 v4 (44 cores/ 88HT) (3.7ghz turbo)
512GB ram DDR4 2400mhz ecc reg
(QUAD SLI) 4 Titan Xp
2x Samsung NVMe 961Pro pcie 3.0 (os drives)
10x Samsung 850 Pro SSD RAID (apps drive)
LG 31MU97z 4096x2160 true 4K rev C
modded P5 case, Noctua heatsinks
Digital power supply 1600w
Windows Server 2012 R2 Data Center, Ubuntu 15


Fully functional
Board allows use of all 7 slots while using 4 sli cards

Using 2 NVMe 950 Pro

Will be putting one pane of glass on the front

Runs quiet and cold

Build pains, tweaks and optimization are to come

Case was heavily modified to accommodate HUGE motherboard, power supply, and the equivalent of 11 pci slots

Had to come up with a way to hold and keep cool 10 ssd 850 pro drives


Nothing, absolutely nothing was easy about any of this


Anyhow enjoy some pics


Opinions?
View attachment 2952 View attachment 2946 View attachment 2947 View attachment 2948 View attachment 2949 View attachment 2950 View attachment 2951 View attachment 2946 View attachment 2948 View attachment 2949


Ok cinebench score


Ok read and write score

the board has no audio jack on the back, but does have the standard HD header

The board has audio, a'97, and HD7
Audio is alive and well, and gaming

I used the HD out and mic in

I made a custom output to the connector on the board, I put audio in and out on the back of the case spine, green and pink, see picture

Hello there Venturi,

I have been looking at this amazing post for quite a long time, amazing work. Congratulations.

I already own 2 Xeon E5620 v4 and 2 32 GB RAM DDR4-2400 and my plan is to buy the big monster motherboard X10DRG-Q. I do not know if you can give me some advice on what you have to do in order to accomodate the motherboard in the Thermaltake Core P5. The reason to get this motheboard is that it fits 4 GPUs in 4 PCI-E x16 mode. For my system I will only be using that four slots, the others are not so important at this moment.

Lets begin with my first doubt:

Did you drill the case in order to attach the motherboard to it?¿? Or the factory holes were fine to attacth it

Thanks for all in advance
I will be waiting for your response
Manuel Luis Aznar
 

gigatexal

I'm here to learn
Nov 25, 2012
2,766
529
113
Portland, Oregon
alexandarnarayan.com
some of that got optimized a bios version back. My read and write speeds are around 13GB/s to 31GB/s, ram keeps up well, there are a few tricks to get the most out of it

1 enable above 4G decoding
2, Set Cache on Die rather than Home snoop, early snoop or late snoop
3. Assign storage to the pch channels and leave the normal channels for the 4 video cards, alternating from cpu 0 and cpu 1

my OS array is on twin Samsung 960 pro in 8x
my D and E array are raid 0 across 10 950pro ssd

full macrium backup of all OS and apps takes about 2minutes and 24 seconds
A 2TB p4800X would be a nice upgrade in terms of latency I think. And you could use the other SATA channels for large spinning rust for backups of things.
 

Venturi

Active Member
Apr 22, 2016
123
165
43
Hello there Venturi,

I have been looking at this amazing post for quite a long time, amazing work. Congratulations.

I already own 2 Xeon E5620 v4 and 2 32 GB RAM DDR4-2400 and my plan is to buy the big monster motherboard X10DRG-Q. I do not know if you can give me some advice on what you have to do in order to accomodate the motherboard in the Thermaltake Core P5. The reason to get this motheboard is that it fits 4 GPUs in 4 PCI-E x16 mode. For my system I will only be using that four slots, the others are not so important at this moment.

Lets begin with my first doubt:

Did you drill the case in order to attach the motherboard to it?¿? Or the factory holes were fine to attacth it

Thanks for all in advance
I will be waiting for your response
Manuel Luis Aznar

Well, for starters I am on a converted P3 case with the thermal glass panel.

I drilled an aluminum plate onto the P3 and had pre drilled that for the stand offs of the motherboard

I then drilled out the back of the P3 to accommodate 10 SSD and the 1600w psu.

Converting the case, fabricating an ssd holder, cutting the aluminum plate, cutting the back plate, drilling for the audio jacks.... about a day of trial and error.


I am now on a small update
I'm using 1TB of Ram and the Titan V ceo 32GB edition
 

Attachments

  • Like
Reactions: William

Venturi

Active Member
Apr 22, 2016
123
165
43
Hello there Venturi,

I have been looking at this amazing post for quite a long time, amazing work. Congratulations.

I already own 2 Xeon E5620 v4 and 2 32 GB RAM DDR4-2400 and my plan is to buy the big monster motherboard X10DRG-Q. I do not know if you can give me some advice on what you have to do in order to accomodate the motherboard in the Thermaltake Core P5. The reason to get this motheboard is that it fits 4 GPUs in 4 PCI-E x16 mode. For my system I will only be using that four slots, the others are not so important at this moment.

Lets begin with my first doubt:

Did you drill the case in order to attach the motherboard to it?¿? Or the factory holes were fine to attacth it

Thanks for all in advance
I will be waiting for your response
Manuel Luis Aznar
Here are some closeup pics of the plate and tooling:
 

Attachments

  • Like
Reactions: William

mlaznar

New Member
Here are some closeup pics of the plate and tooling:
Oh thank you very for the promptly replies...

You made a good work, but, for me it seams to be too much much...I will be looking at another kind of solution...I think I will be looking for a more simple motherboard, where the board fix with no problem at all...

Thanks for all the replies again...because you have save my money and time.

Thanks,
Manuel
 

mlaznar

New Member
Here are some closeup pics of the plate and tooling:
In the end I found a motheboard and I think I am going to try the building of the system...

Do you know @Venturi if the space between GPU slots are standard spacing (4 centimeters)?¿?

I have attached two pictures to this post:

One shows distances between the holes of the board
The other one is the tray I have to fix the board. That tray has 11 slots for cards...

With the one that have the distances and looking at my tray it seams that I can fix the board to the tray in at least four holes...

What do you think about that @Venturi

Thanks again
Manuel
 

Attachments