44 cores 512gb X10DRG-Q 4-way SLI

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Venturi

Active Member
Apr 22, 2016
177
187
43
Thoughts on the t5 case? Any issues keeping it free from dust or fear of static shock?
So far, not really. I have the side clear panel on so that mostly protects the innards. The room where it resides (will) has a tile floor and dust has not been an issue. (..and I keep a can of compressed air handy)

Obviously, in the sandwich design the innards have exposure on the outer slit, however, I picked this case because I could get adequate ventilation for all that processing power.

I could have gone with a regular case, but wanted to avoid fan noise. So this was my best compromise.

Hardest mod on this case was accommodating 12 slots in the back. Required some effort and redesign. Obviously also the fact that the board is as large as a football field since it accommodates 4 full double wide video cards and 3 usable slots at the same time ( not to mention 44 cores and 512GB of ram, 2 nvme drives and 10 ssd drives, as well as a 1650w power supply...)


So, it was some engineering and some compromises. The case felt compromised when I was done with it.
 
  • Like
Reactions: Wlad and gigatexal

vrod

Active Member
Jan 18, 2015
241
43
28
32
Awesome build, just a pity you wasted money on 4 titans when the new 1080's perform better. :)
 

Venturi

Active Member
Apr 22, 2016
177
187
43
Well sir, that is your opinion based on your understanding, thank you for your valuable input

The truth, unfortunately is more complicated for my simple understanding so there's a probability you're right



My humble opinion is that at a resolution at or above 4096x2160, with max candy, avoiding blurr engines above Fxaa, and running quad sli, in a correct configuration, that the Titan X performance is ahead in the scenarios I need. I looked at the purchase long and hard but since u have had my Titan X SC since they day they were released, more than a year ago

It also helps that I can configure them in a real quad sli mode with the appropriate scaling, something that as of now, the 1080 can't do.


For the record, you cannot get proportional quad sli scaling unless you have a Multi-socket board

Without busting all those out there with misinformation.

One must distribute and configure the QUAD SLI, it does not scale well/at all on one cpu or a single allocation of pcie lanes, end each card needs to be fed by the cpu (s)

Quad sli, correctly configured down to the custom sli profile, and by custom I mean not the simple nv control panel choices, will scale 3.5x over a single in QUAD AFR2 and 3.2x QUAD AFR1 over single

important:

So so very very important

Really important:

ONLY IF:

You have multicores in 2 SOCKETS, each socket feeding the ALTERNATE gpu per pcie express socket, WITHOUT GOING THROUGH THE PCH, and memory regions to feed it, and set up custom profile derivatives respective if primary and alternate cards.



SET EACH SOCKET, LANED TO THE ALTERNATE CPU SOCKET, PER 1,3 AND 2,4 position video cards
Then, only then will you get 3.5x scaling in Quad sli

And only with specific titles, os, api and only with all the resources to feed it

AND
ONLY IF YOU MAKE THE PROFILES
The Nv control panel does not give you access to that world

And set up gets more complicated
QUAD AFR2 success depends on which card is mapped as primary, and when to select QUAD AFR1
and you can only use hardware/driver Fxaa when playing higher or at 3840x2160, I play at 4096x2160
Then
Only then --Will quad sli show the gain

Example
Single socket Novice dual sli or quad sli at 87FPS Max candy, physics, various AA implementation (which really only add blurr)

In a pro, multi socket config
268FPS Fxaa, max candy and physics

There's more, you also have to understand how to apply fast synch, triple buffering, v synch and also know why in sli the buffering is a different animal, how it works in OpenGL, and the differences in DirectX, the need for memory beyond 8Gb and so forth.





Is it worth it? To me it's glorious when it all comes together. A complicated sculpture / engine.

That's why Nvidia doesn't officially support it, ---too demanding and complicated for the median prosumer
So they have the silly subscribe for a download key to enable 3 or 4 gpu sli, either way, they have to balance the four tiers of development and game / app to use it or make a profile to do it outside of the developer inter agreements


The 1080 is ahead of the Titan X SC about 11-16% in common scenarios, it is not ahead in the resolutions I like or at least not by enough margin to worry about it. However, in quad sli, real, configured correctly, for now the Titan X sc is what I need and use. Plus my titan SC has a custom bios with higher clocks than the regular version and I've had them more than a year and know the setup well


Anyhow, that's my humble opinion.
I hope to contribute more than I take in this forum.



Truth is complicated
Im posting this here and not the normal forums to avoid the trolls, haters, misinformed and those that rather destroy than create
 
Last edited:

vrod

Active Member
Jan 18, 2015
241
43
28
32
Well if you had the Titans for a longer time I of course understand. I was under the impression that you just purchased them. I wouldn't buy 1080's either if I were you. :)

Would you be able to test out the quad sli scaling? It sounds pretty interesting that a single cpu wouldn't be able to run that as well as 2 cpu's since 8 lanes per graphics card is just fine. From what I've heard most games don't scale well because they are not optimized. I run sli myself, just with 970's.

If you have a program that can use all 8 gpu's then it obviously makes a lot of sense for you to keep the titans. :)
 
  • Like
Reactions: maxermaxer

Venturi

Active Member
Apr 22, 2016
177
187
43
Well if you had the Titans for a longer time I of course understand. I was under the impression that you just purchased them. I wouldn't buy 1080's either if I were you. :)

Would you be able to test out the quad sli scaling? It sounds pretty interesting that a single cpu wouldn't be able to run that as well as 2 cpu's since 8 lanes per graphics card is just fine. From what I've heard most games don't scale well because they are not optimized. I run sli myself, just with 970's.

If you have a program that can use all 8 gpu's then it obviously makes a lot of sense for you to keep the titans. :)

It's unfortunate that Nvidia has started playing complicated marketing games and there're is a fair amount of FUD surrounding the 1080.

Honestly, if they come out with s Titan equivalent upgrade is probably buy it :) I just loathe having to interpret the truth from the information they provide
 
Last edited:
  • Like
Reactions: Wlad

RobertFontaine

Active Member
Dec 17, 2015
663
148
43
58
Winterpeg, Canuckistan
You need to have deep pockets to buy Nvidia's halo products 4 at a time. If you are willing to pay the upgrade tax you can chase the bleeding edge but usually if you need that much compute aws or something similar is going to make more sense than trying to turn a workstation into a full blown supercompute.

Personally if my workstation can emulate a smaller version of a full blown compute environment I am more than happy with my dungeon lab.
 

RobertFontaine

Active Member
Dec 17, 2015
663
148
43
58
Winterpeg, Canuckistan
As an aside the x9drg-qf and X10drg-qf are physically designed to split the pcie lanes with 2 cards per cpu in order to support compute.

It is my understanding that most dual workstation boards will put the 4 cards on CPU0 so that transfer running on 1 cpu is practical and I believe that this also improves gaming performance through reduced latency where the cpu is not heavily used.

Size wise the drg is incredibly awkward. Mine is supposed to be coming back from RMA this week and I will be mounting on plywood for the interim. Dual 2670's, 128gb, 2 xeon phis (still haven't picked up two more) and a gtx580 while I have empty slots

This is soon going to be my machine learning, data mining, kaggle box.

It would be nice to see the price of compute drop dramatically but I don't see it happening soon. This version of Pascal has essentially crippled fp16. Maxwell gimped fp64... NVidia has taken to unrealistic hype on it's consumer lines while at the same time gutting prosumer functionality to get there.

They are intentionally segmenting general purpose compute. I really hope Intel or the phone manufacturers reach into Nvidia's graphics market and give them a push.

AMD has taken the fiscally responsible path this year and is going straight to the price point where most cards actually sell rather than playing the halo game.

Knight's Landing on the Intel side is very exciting but I can't afford to play. Between the exchange rate and the price of entry is about $10k Canadian. My wife would kill me in my sleep.
 

Venturi

Active Member
Apr 22, 2016
177
187
43
I don't know that yet
However the pascal has less cores and can do less computational

I personally have not benched them
Just reading up a lot
 
  • Like
Reactions: Wlad and gigatexal

lpallard

Member
Aug 17, 2013
276
11
18
Damn! I love this! For shelling out nearly $15k on a computer, I hope

1. Work is the first and foremost reason you spent that many coins!
2. Its company sponsored or you got a grant for research
3. You will be performing CFD, FEA, or something for humanity and science...

Thats almost as I spent on my brand new car!! (but probably a lot more exciting to use ;)
Post some benchmarks and show us how you use this to "work"
 

Venturi

Active Member
Apr 22, 2016
177
187
43
Damn! I love this! For shelling out nearly $15k on a computer, I hope

1. Work is the first and foremost reason you spent that many coins!
2. Its company sponsored or you got a grant for research
3. You will be performing CFD, FEA, or something for humanity and science...

Thats almost as I spent on my brand new car!! (but probably a lot more exciting to use ;)
Post some benchmarks and show us how you use this to "work"
1. I'm not going to discuss how much I spent ;)
2. It's paid for out of my own pocket and was 80% from the sale of my prior rig, which was funded from 80% of the sale of the prior rig, going back 6 generations to my first rig
3. Work wise - It's used for medical imaging, nuclear medicine, isotopes, and db troubleshooting

But I didn't build it just for work

Anyhow, every rig, helped pay its successor:

here are the prior rigs, including the first water-cooled dual cpu rig,the first dual cpu SLI, and the first dual CPU Quad SLI

Anyhow, here are some of my first PCs from 15+ years ago
















My "Blue Period" and the first dual CPU quad SLI ever!

(NVidia had not even made the quad Sli sound yet, but beta driver support was there in developer drivers)
and always multi cpu and raid ;)








"Cable Zen" was a pain then too



















Ending with the current rig







V
 
Last edited:

lpallard

Member
Aug 17, 2013
276
11
18
Well this is even more impressive than I hoped for! So you've been into ultra high end rigs for a while now eh? Nice! I wish I could have been able to resell my previous rigs and expense up to 80% of a new one with the proceedings but things like they are with computers, they lose about as much of their value before need to upgrade so...

Nuclear imagery and isotopes? Similar to what I contemplated (nuclear engineering for power production) before becoming a mechanical engineer. Do you have a business of your own and work from home? For finite element analysis you may not need that much horsepower but for scientific applications and numerical simulations, I see the need for more processing power. But 512GB ram????? Holy BATMAN ON !?!??!?!

We see so many forum posts on other forums and on youtube about kids spending several thousands of $$$ on gaming rigs with wrong (read enterprise) hardware and all kind of overly exaggerated specs they simply dont need (dual xeon 10 core with 64GB ram to "transcode videos"?? C'mon, gimme a break ;) With that being said, I like to hear about the rationale and overall usage of these monster rigs.

Like I said, we always talk about the internals and technical stuff here but rarely do we get to know the "behind the scene".

Anyways, congrats and I'd be personally interested to see what you can accomplish with such a powerful system!!
 
  • Like
Reactions: Centauri68

gbeirn

Member
Jun 23, 2016
70
17
8
123
Amazing setup. I had no idea so many 2cpu.com forum members are now here. I was a poster on there long ago as well. Welcome and thanks for sharing!
 

Venturi

Active Member
Apr 22, 2016
177
187
43
image.jpeg Plenty of airflow
Ran thermal camera just make sure
Motherboard and components run several degrees colder than a case with a fan blowing in it

And the video cards pull a lot of air through and push out the back
Processors flow in the same direction
I put a lot of thought into it, but don't take my word for it, check the picture for light-medium load temps
Room temp was low 70's
 
Last edited:
  • Like
Reactions: Wlad