Newbie needs to buy a high-end server - advice please!

Discussion in 'DIY Server and Workstation Builds' started by Tagaravabha, Feb 10, 2018.

  1. Tagaravabha

    Tagaravabha New Member

    Joined:
    Feb 10, 2018
    Messages:
    5
    Likes Received:
    1
    Dear server experts! I am a researcher looking into buying a high-end server for some memory-intensive simulations (mostly finite-element method for those that are interested). I know that I'll need a minimum of 1TB of RAM, as many physical cores as I can get (for parallel computing), and an array of 15K rpm hard drives, but I have no idea of what else might be needed. I came across the following specs for a *refurbished* server and wanted some advice on anything I might be missing that will be necessary for its smooth operation, or anything I might need to watch out for. I have some comments in-line with the specs and some additional questions that I've included at the end of this message, and would really appreciate your advice! Thank you in advance:


    • Dell PowerEdge R920 server 4U rack, supports up to 6TB of memory in 96 slots (I know this is discontinued and replaced with the R940)
    • 4 x Intel Xeon 6-core E7-8893 v2 3.4GHz 37.5MB cache/processor (are there faster/cheaper newer processors, e.g. i5, out there? Do they offer the same amount of cache?)
    • 64 x 16GB registered server RAM (does the brand of RAM really matter, e.g. between Hynix, Samsung, Kingston?)
    • 10 x DELL 600GB 15K rpm 2.5" SAS Hard Drives (is there any need for a RAID configuration if my priority is speed of writing and reading? Should some of the drives, e.g. for the OS, be excluded from the RAID setup? What kind of RAID would you recommend?)
    • 24 x 2.5" SFF SAS/SATA/SSD BAYS
    • PERC H730P 2GB NV cache RAID controller
    • IDRAC7 express (Is there a big difference between IDRAC7 and IDRAC8? Would the latter even be supported on the R920?)
    • No optical drive (is there ever a need for an optical drive?)
    • Redundant 1100W power supply (does "redundant" here mean two supplies with a total power of 1100W, or 1100W each? Is this enough for what I want to put in here?)
    • Broadcom 5720 1GB quad port daughter card
    • Up to 10 PCI-E slots: 8 x PCI-E 3.0 + 1 RAID slot + 1 NDC slot + 2 optional PCI-E slots (what are these, and what can I do with them?)
    • E-VGA NVIDIA GEForce GTX 980 TI 6GB (Will this card even work on this system? Can the power supply handle it? If not, what would be a good supported alternative? I don't need GPUs for computation, but will it help when accessing the server remotely?)
    • Rail Kit
    • No bezel


    Some additional questions:


    1. What would be a decent price for the above system (please note, it is refurbished)?
    2. Are there any components that are not included, but which will definitely be needed? If so, what are they?
    3. I heard that Intel will stop supporting/selling these processors after 2019. Is this a big problem?
    4. Any suggestions on a good-quality reliable UPS for this server?


    Thank you!

    TRB
     
    #1
  2. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    419
    Likes Received:
    108
    As someone from a similar field (CFD) with a little experience in FEM here are my two cents: what matters the most here is memory bandwidth, not CPU cores.
    So you will certainly need a few cores for performance, but more importantly: enough memory bandwidth to feed the cores. Otherwise scaling will be bad and money on "faster" CPUs will be wasted.
    Since you say you need 1TB of RAM, the only cost-effective option is older hardware that uses cheap DDR3 reg ECC. Speed matters here, not the brand. Cheap refurbished servers are often equipped with DDR3-1066 or even slower memory although the CPUs support up to DDR3-1600. For your applications, memory speed will affect performance almost 1:1. So make sure to check which memory type you get for your money. The CPUs themselves are a pretty decent pick for this type of applications: medium core count, high clock speed. More cores probably won't help much due to the lack of memory bandwidth and increasing parallelization overhead.
     
    #2
    Patrick likes this.
  3. Tagaravabha

    Tagaravabha New Member

    Joined:
    Feb 10, 2018
    Messages:
    5
    Likes Received:
    1
    Thank you! I believe the memory is 1600MHz. I'd be very interested to know your responses to my other questions - thanks again.
     
    #3
  4. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,569
    Likes Received:
    4,507
    Any chance you can go SSD instead of those 600GB 15K rpm drives? For example: HGST WD s840 Enterprise Commercial SAS SED 2TB 6GB/s SSD 2.5" Z16IZF2E-2TBUCZ | eBay --- different seller but potentially a thread worth reading https://forums.servethehome.com/index.php?threads/2tb-hgst-s840-enterprise-sas-sed-ssd.18245/

    You do not need a GPU if you just need 2D output. These servers have them built-in.

    Agreed with @alex_stief that if you are looking for 1TB inexpensively, DDR3 is the answer.

    You could also do 32x32GB DDR4 in a dual AMD EPYC server to hit 1TB, but you will have lower memory speeds and be buying more expensive DDR4.

    I usually buy RAM in an order that goes something like: Samsung->Micron or SK.Hynix-> Others (only if I have to)
     
    #4
  5. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    419
    Likes Received:
    108
    I can only weigh in on stuff I know about. Questions left out indicate that I have no idea.
    There are lots cheaper and faster processors. I5 and the likes are not an option since you probably need a shared memory system and these processors do not support more than one CPU per motherboard. Xeon E5-2xxx are only dual-CPU capable, E5-4xxx have less QPI links, i.e. slower interconnect for shared memory. The Xeon E7 v2 are about as good as it gets for what you need and possibly want to spend.
    Might be ok if you can't find a cheaper offer with less HDDs included. I strongly recommend SSD-based storage for pre- and post-processing. And depending on the FEM software, also for solving. Some commercial FEM packages are notorious for heavy disk-I/O during solving even if the model fits into RAM.
    1100W each. The power output should be sufficient, Don't know if it has the PCIe connectors you need for a graphics card.
    AFAIK, usually not.
    Are you absolutely sure that you need one? Are power failures a regular occurrence in your area?
     
    #5
  6. Tagaravabha

    Tagaravabha New Member

    Joined:
    Feb 10, 2018
    Messages:
    5
    Likes Received:
    1
    Thanks all for your help! In the end, I did purchase this server with a few modifications -- no HDDs (I bought 3 x 2TB Micron SSDs instead) and no graphics card. I also ensured that the RAM was Samsung and 1600MHz. It cost me CAD $14K.
     
    #6
    T_Minus likes this.
  7. alex_stief

    alex_stief Active Member

    Joined:
    May 31, 2016
    Messages:
    419
    Likes Received:
    108
    That sounds like a lot of money. 11k USD?
     
    #7
  8. BobbyB

    BobbyB Member

    Joined:
    Dec 26, 2016
    Messages:
    32
    Likes Received:
    9
    Check the configured, actual speed of the memory once you get it.
    The DIMMs themselves might be 1600, likely they will only run at 1066 or 1333 in 2DPC (dimms per channel). IIRC LRDIMMs (32GB and up in these R920s) can do 1333 always, 2-3DCP with ordinary RDIMMs are often limited to 1066 even with 1.5V in these R920s. Dell owners manual has the info.
    Again, check what speed you can set.
    Had a quick look on some R9x0's in work - R920 with 32GB LRDIMM 1600 runs at 1333. R930 E7v4 with 32GB LRDIMM DDR4 2400 runs at 1600 and R940 with Skylake-SP has 64GB LRDIMM 2666 runs at 2666. Each box with all slots populated.
     
    #8
  9. Tagaravabha

    Tagaravabha New Member

    Joined:
    Feb 10, 2018
    Messages:
    5
    Likes Received:
    1
    Thanks - the RAM modules I've ordered are 16GB each (64 x 16GB); are you saying these are not supported by the R920?? If they are, will I still see the same slowdown? I was told the model of the memory would probably be Samsung M393B2G70QH0-YK0...
     
    #9
  10. BobbyB

    BobbyB Member

    Joined:
    Dec 26, 2016
    Messages:
    32
    Likes Received:
    9
    They should work, but likely the speed will be set to either 1333 or 1066. Check the R920 owner's manual, it should list the memory speed settings with the combinations or RDIMM/DPC/voltage/mode. I honestly don't know if it will let you override it to 1600, maybe it does.
     
    #10
  11. Tagaravabha

    Tagaravabha New Member

    Joined:
    Feb 10, 2018
    Messages:
    5
    Likes Received:
    1
    Thanks - I checked the manual and it looks like 16GB RDIMM/DPC=3/1.5V will run at 1066MHz. It says that 1.35V is "not applicable" -- does that mean it wont work in this configuration?
     
    #11
Similar Threads: Newbie needs
Forum Title Date
DIY Server and Workstation Builds ~~Help~~: ~~~Newbie~~~First~~~Server~~~ Oct 25, 2018
DIY Server and Workstation Builds Need advice on Personal workstation build(newbies) Oct 4, 2017
DIY Server and Workstation Builds Newbie advice: Dell T330 vs Synology vs DIY for file server and Plex May 3, 2017
DIY Server and Workstation Builds Newbie Build, need input Sep 6, 2016
DIY Server and Workstation Builds newbie here, please review this config Nov 21, 2012

Share This Page