44 cores 512gb X10DRG-Q 4-way SLI

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

maxermaxer

Active Member
Oct 28, 2016
304
56
28
49
Small update,

the pc/server is now using (4) four Titan Xp cards....
...blistering cuda performance
Hey Venturi, look forward to that! I have a similar build with same motherboard same chassis and 4x 1070 GTX. I use it for Octane rendering (not sure if you are familiar). I found that the C-state CPU setting in the BIOS makes the OctaneBench score much worse than disabling it. I am talking about 30% performance gap indeed.

Would you mind sharing with me your BIOS setting and particularly for CPU advance setting part? Thanks!!


Sent from my iPhone using Tapatalk
 

Venturi

Active Member
Apr 22, 2016
177
187
43
I'll see if I have handy those screen shots and upload them for the bios settings.
generally speaking the C states used are 3 and 6, I'll see what I ended up with when I get home from work...
(this whole working for a living thing is highly overrated )
 

Venturi

Active Member
Apr 22, 2016
177
187
43
some of that got optimized a bios version back. My read and write speeds are around 13GB/s to 31GB/s, ram keeps up well, there are a few tricks to get the most out of it

1 enable above 4G decoding
2, Set Cache on Die rather than Home snoop, early snoop or late snoop
3. Assign storage to the pch channels and leave the normal channels for the 4 video cards, alternating from cpu 0 and cpu 1

my OS array is on twin Samsung 960 pro in 8x
my D and E array are raid 0 across 10 950pro ssd

full macrium backup of all OS and apps takes about 2minutes and 24 seconds
 
Last edited:
  • Like
Reactions: gigatexal

mlaznar

New Member
I'm new here, but I've done a few builds from time to time...


Short summary

Here are the specs:

Supermicro X10DRG-Q
2x E5-2699 v4 (44 cores/ 88HT) (3.7ghz turbo)
512GB ram DDR4 2400mhz ecc reg
(QUAD SLI) 4 Titan Xp
2x Samsung NVMe 961Pro pcie 3.0 (os drives)
10x Samsung 850 Pro SSD RAID (apps drive)
LG 31MU97z 4096x2160 true 4K rev C
modded P5 case, Noctua heatsinks
Digital power supply 1600w
Windows Server 2012 R2 Data Center, Ubuntu 15


Fully functional
Board allows use of all 7 slots while using 4 sli cards

Using 2 NVMe 950 Pro

Will be putting one pane of glass on the front

Runs quiet and cold

Build pains, tweaks and optimization are to come

Case was heavily modified to accommodate HUGE motherboard, power supply, and the equivalent of 11 pci slots

Had to come up with a way to hold and keep cool 10 ssd 850 pro drives


Nothing, absolutely nothing was easy about any of this


Anyhow enjoy some pics


Opinions?
View attachment 2952 View attachment 2946 View attachment 2947 View attachment 2948 View attachment 2949 View attachment 2950 View attachment 2951 View attachment 2946 View attachment 2948 View attachment 2949


Ok cinebench score


Ok read and write score

the board has no audio jack on the back, but does have the standard HD header

The board has audio, a'97, and HD7
Audio is alive and well, and gaming

I used the HD out and mic in

I made a custom output to the connector on the board, I put audio in and out on the back of the case spine, green and pink, see picture

Hello there Venturi,

I have been looking at this amazing post for quite a long time, amazing work. Congratulations.

I already own 2 Xeon E5620 v4 and 2 32 GB RAM DDR4-2400 and my plan is to buy the big monster motherboard X10DRG-Q. I do not know if you can give me some advice on what you have to do in order to accomodate the motherboard in the Thermaltake Core P5. The reason to get this motheboard is that it fits 4 GPUs in 4 PCI-E x16 mode. For my system I will only be using that four slots, the others are not so important at this moment.

Lets begin with my first doubt:

Did you drill the case in order to attach the motherboard to it?¿? Or the factory holes were fine to attacth it

Thanks for all in advance
I will be waiting for your response
Manuel Luis Aznar
 

gigatexal

I'm here to learn
Nov 25, 2012
2,922
611
113
Portland, Oregon
alexandarnarayan.com
some of that got optimized a bios version back. My read and write speeds are around 13GB/s to 31GB/s, ram keeps up well, there are a few tricks to get the most out of it

1 enable above 4G decoding
2, Set Cache on Die rather than Home snoop, early snoop or late snoop
3. Assign storage to the pch channels and leave the normal channels for the 4 video cards, alternating from cpu 0 and cpu 1

my OS array is on twin Samsung 960 pro in 8x
my D and E array are raid 0 across 10 950pro ssd

full macrium backup of all OS and apps takes about 2minutes and 24 seconds
A 2TB p4800X would be a nice upgrade in terms of latency I think. And you could use the other SATA channels for large spinning rust for backups of things.
 

Venturi

Active Member
Apr 22, 2016
177
187
43
Hello there Venturi,

I have been looking at this amazing post for quite a long time, amazing work. Congratulations.

I already own 2 Xeon E5620 v4 and 2 32 GB RAM DDR4-2400 and my plan is to buy the big monster motherboard X10DRG-Q. I do not know if you can give me some advice on what you have to do in order to accomodate the motherboard in the Thermaltake Core P5. The reason to get this motheboard is that it fits 4 GPUs in 4 PCI-E x16 mode. For my system I will only be using that four slots, the others are not so important at this moment.

Lets begin with my first doubt:

Did you drill the case in order to attach the motherboard to it?¿? Or the factory holes were fine to attacth it

Thanks for all in advance
I will be waiting for your response
Manuel Luis Aznar

Well, for starters I am on a converted P3 case with the thermal glass panel.

I drilled an aluminum plate onto the P3 and had pre drilled that for the stand offs of the motherboard

I then drilled out the back of the P3 to accommodate 10 SSD and the 1600w psu.

Converting the case, fabricating an ssd holder, cutting the aluminum plate, cutting the back plate, drilling for the audio jacks.... about a day of trial and error.


I am now on a small update
I'm using 1TB of Ram and the Titan V ceo 32GB edition
 

Attachments

  • Like
Reactions: William

Venturi

Active Member
Apr 22, 2016
177
187
43
Hello there Venturi,

I have been looking at this amazing post for quite a long time, amazing work. Congratulations.

I already own 2 Xeon E5620 v4 and 2 32 GB RAM DDR4-2400 and my plan is to buy the big monster motherboard X10DRG-Q. I do not know if you can give me some advice on what you have to do in order to accomodate the motherboard in the Thermaltake Core P5. The reason to get this motheboard is that it fits 4 GPUs in 4 PCI-E x16 mode. For my system I will only be using that four slots, the others are not so important at this moment.

Lets begin with my first doubt:

Did you drill the case in order to attach the motherboard to it?¿? Or the factory holes were fine to attacth it

Thanks for all in advance
I will be waiting for your response
Manuel Luis Aznar
Here are some closeup pics of the plate and tooling:
 

Attachments

  • Like
Reactions: William

mlaznar

New Member
Here are some closeup pics of the plate and tooling:
Oh thank you very for the promptly replies...

You made a good work, but, for me it seams to be too much much...I will be looking at another kind of solution...I think I will be looking for a more simple motherboard, where the board fix with no problem at all...

Thanks for all the replies again...because you have save my money and time.

Thanks,
Manuel
 

mlaznar

New Member
Here are some closeup pics of the plate and tooling:
In the end I found a motheboard and I think I am going to try the building of the system...

Do you know @Venturi if the space between GPU slots are standard spacing (4 centimeters)?¿?

I have attached two pictures to this post:

One shows distances between the holes of the board
The other one is the tray I have to fix the board. That tray has 11 slots for cards...

With the one that have the distances and looking at my tray it seams that I can fix the board to the tray in at least four holes...

What do you think about that @Venturi

Thanks again
Manuel
 

Attachments

Venturi

Active Member
Apr 22, 2016
177
187
43
In the end I found a motheboard and I think I am going to try the building of the system...

Do you know @Venturi if the space between GPU slots are standard spacing (4 centimeters)?¿?

I have attached two pictures to this post:

One shows distances between the holes of the board
The other one is the tray I have to fix the board. That tray has 11 slots for cards...

With the one that have the distances and looking at my tray it seams that I can fix the board to the tray in at least four holes...

What do you think about that @Venturi

Thanks again
Manuel

It seems that the holes on that side of the board (gpu) are the standard distance, except for the bottom one in the corner, at least from where I am looking. You may have to post one separately in the corner. At least from my view
 

mlaznar

New Member
It seems that the holes on that side of the board (gpu) are the standard distance, except for the bottom one in the corner, at least from where I am looking. You may have to post one separately in the corner. At least from my view
I do not know whether you can confirm or not if the distance between each PCI express x16 slot is the standard one (that is 4 centimeters). In the picture with the holes distances there are 4 black lines that represent the distance between each GPU slot. I want to know if that distance is FOUR centimeters...It would be great if you can confirm that point...
 

Venturi

Active Member
Apr 22, 2016
177
187
43
The distance between each pci slot is the standard distance down to the .100 of a millimiter
 

mlaznar

New Member
I don’t know, I didn’t use the io shield, I don’t even know where it might be ;)
Hello again venturi,

A guy from the UK was selling this motherboard (SuperMicro X10DRG-Q) and have the board on hand and as you know he gave me the picture with the distances. In the end he is going to keep the motherboard because he found very cheap CPUs...

I ask him a few more questions about the distance because he has the board cleared to look at distances in between the holes and PCI express...and he said the following about it:

"""The distance between the slots is standard on all motherboards. 2 PCIE brackets width, it's universal.
The distance on the blue line you drew is the same distance as on the one I did on the other side, 15.5
The little pink distance is about 1.5-2.5 cm.

Hope this helps...

Any case that can take extended atx will be compatible with it. Not all your holes will align but you don't need all of them anyway..."""


What do think venturi?¿?
 

Venturi

Active Member
Apr 22, 2016
177
187
43
I'm in possession of two spare boards (x10drg-q) if you are interested.

I am a big fan of silent, so the less fans the better. I use the P3 series from thermal-take but reqires modification.

There are of course some huge cases that can do it if you don't mind oversize (I am a fan of smallest footprint possible x power and Silence)

some of the cases from CaseLabs will fit it in the HPTX category
 
  • Like
Reactions: gigatexal