Yet another CFD build

alex_stief

Active Member
May 31, 2016
625
189
43
35
Build’s Name: stupid RAM prices :mad:
Operating System/ Storage Platform: Opensuse Leap
CPU: 2x Intel Xeon E5-2667v2
Motherboard: ASRock EP2C602-4L/D16
Chassis: Phanteks Enthoo Pro
Drives: Samsung 850 Evo 250GB, 3x WD red 4TB software RAID5
RAM: 16x16GB Micron 2Rx4 DDR3L-1333 reg ECC, running at DDR3-1866@1.5V
Add-in Cards: Nvidia GTX 1050TI 4GB, passively cooled from Palit
Power Supply: Seasonic Focus Platinum 750W
Other Bits: 2x Noctua NH-U14S coolers, 6x Arctic F14 PWM PST, side panels lined with noise insulation from BeQuiet, a PCIe wifi adaper and an optical drive.

Usage Profile: Mostly fluid simulation stuff and code development. Needed a lot of memory and did not want to spend 3000$ on RAM alone. So I used Micron DDR3-1333 which I had bought dirt-cheap a while ago. This dictated the platform choice.

We considered using AMDs Epyc platform for this build as well because it is just awesome for the kind of work we do. But in order to keep the costs significantly lower we chose the latest DDR3 platform. The CPUs may not have the best performance/$, but I wanted to get the best possible performance out of the old platform for codes that may not scale perfectly on a large number of cores.
Overclocking memory was pretty painless. Although a full run of memtest86+ completed without errors using 1.4V memory voltage, the system still hung occasionally when putting heavy load on the memory subsystem. Now with 1.5V it is rock-stable and still within the specifications of DDR3L. I am pretty happy with that, application benchmarks indicate up to 26% higher performance compared to DDR3-1333. The motherboard chose CL12 for latency which is surprisingly good.
A note on the motherboards built-in fan controls: only the two CPU fan headers can be controlled based on system temperatures. However there is only one preset, no "silent" or "balanced" mode. The other 6 PWM fan headers only have fixed fan speed settings between 1-9 and "full-on".

While one of those decommissioned HP or DELL workstations might have been cheaper for now, their lack of upgradeability through proprietary connectors and form factors would have been more expensive in the long run. Plus parts like power supplies do not get better over time. The plan is to upgrade to a DDR4 platform (probably AMD Epyc or its successor) as soon as memory prices normalize. And the motherboards in these systems usually do not allow a memory overclock as ASRock does.

Pictures today or tomorrow. Done

Power draw: Idle power consumption at the wall is somewhere between 120W-130W.
Under "normal" load that the workstation will usually encounter during daily usage power draw sits around 310W.
However, using synthetic load ("stress") I was able to get the power draw up to 430W stressing CPU and memory subsystem alone. Quite a bit more than I would have expected, but perfect for stability testing.
 
Last edited:

anoother

Member
Dec 2, 2016
122
18
18
31
Interesting; how did you overclock the RAM? Just set the speed in BIOS, or did you have to burn SPD?
 

alex_stief

Active Member
May 31, 2016
625
189
43
35
Altering SPD values is what I was trying to avoid. Many DIMMs are locked anyway in this regard, i.e. write-protected SPD. Your only chance would be to solder a non-protected SPD chip to each DIMM which is beyond my capabilities. How to unlock a locked memory SPD.
This ASRock board is one of the few dual-socket boards that simply let you choose the memory speed (in the Range of DDR3-800 to DDR3-1866) and even adjust some voltages. Memory up to 1.6 or 1.65V IIRC.
With the system at hand, I tried some of the other DIMMs I had lying around. Samsung DDR3L-1600 reg ECC run at DDR3-1866 with no issues. Elpida DDR3L-1333 reg ECC only posted with up to DDR3-1600.
My conclusion is: If you want to pull off memory overclocking in a similar way, choose memory from a reputable brand. And more importantly: DDR3L. This allows you to increase stability by bumping up the voltage without excessively high voltages. DDR3L has a minimum voltage of 1.35V, but 1.5V is still within the specifications. Oh and don't use quad-rank memory.
In my opinion the research that went into this was worth it since DDR3-1866 is significantly more expensive even when buying used. I never found any cheaper than ~80€/16GB. Not to mention the satisfaction of getting "free" performance
 
Last edited:
  • Like
Reactions: anoother

alex_stief

Active Member
May 31, 2016
625
189
43
35
Not showing the mess I created with the cables behind the motherboard tray might lead to this impression. The side panel just barely went back on :D
I have no idea why some people want glass side panels on both sides. But jokes aside, I am pretty pleased with how it turned out. Expansions for those 24pin and 8pin ATX cables definitely make things easier.
 

Evan

Well-Known Member
Jan 6, 2016
3,071
512
113
I have no idea why people really want glass sides at all or in some cases even vents on side or top of case, makes positioning cases so much harder :(
(eg stacked tight beside each other, it stacked on top of each other)

Anyway that’s just a personal thing, guess that’s why I like Mac’s and very clean lines etc, certianly no rgb coolers,ram,cases, etc for me :)
 

alex_stief

Active Member
May 31, 2016
625
189
43
35
I would also avoid top vents if possible, but there is just no way to get quiet cooling for such a build with only one exhaust fan at the rear. At least not quiet by my standards. Of course I could just run the rear fan at 1200rpm, but that would be above my personal annoyance threshold PAT®
 

i386

Well-Known Member
Mar 18, 2016
1,936
505
113
31
Germany
I have a chassis with a window. At the time the normal chassis (lian li pc-a75) was not available and the version with a window ((lian li pc-a75x) was 30€ cheaper AND available :D