Building infrastructure from zero.

Discussion in 'DIY Server and Workstation Builds' started by GCM, Nov 13, 2015.

  1. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    So, I'm building out some new server infrastructure for one of my clients. They pretty much have nothing, and had all 220ish users on Dropbox or Google Drive... The engineering team will be sharing the rack space once they move into the office come December.

    Here is what I came up with:

    Hyper-visor OS - Ubuntu Server
    VM - KVM
    Directory Services - OpenDJ
    Storage - ZFS from Ixsystems
    Network - Core network will be 10GbE, branching off to 1GbE. Video editors *may* get 10GbE to the desktop.
    Rack - It's completely self contained, fully sealed. Very interesting to work with.

    Parts list:

    SWITCH x2 : Juniper ex3300-24t
    STORAGE: TrueNAS Z30 - Usable 18TB - 128GB RAM - 2x SSD l2arc - SSD ZIL - 2x 10GbE
    SERVER x2: RAX XS4-1130V3-SH (E3 CPU, 32GB RAM, Raid 10 with HGST 4TB Drives)
    UPS x2 : EATON 5P UPS 2200VA
    RACK: CC42U8000M6-R2

    So, here are a few pictures of what I've got going so far:

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
     
    #1
    Patrick, Moothusala and Chuntzu like this.
  2. Moothusala

    Moothusala New Member

    Joined:
    Oct 21, 2015
    Messages:
    6
    Likes Received:
    2
    Is that a Neat Patch I spy in the rack shot? I love those things.

    I'd be interested to hear more of what you think of that sealed rack system.
     
    #2
  3. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    It is! I literally put them in EVERYWHERE I touch the network. I've lost count of how many I've purchased...

    And we'll see in a week how the sealed rack is working out. I can tell you one thing, that rack is the heaviest rack I've dealt with, and that AC unit was incredibly hard to get on by myself due to the weight. The rack itself is very nice though, completely sealed, swinging doors both front and back, adjustable depth.
     
    #3
  4. Martin Jørgensen

    Martin Jørgensen New Member

    Joined:
    Jun 3, 2015
    Messages:
    28
    Likes Received:
    6
    Looks great :;)

    I can recomend Freeipa instead of OpenDJ
     
    #4
    whitey likes this.
  5. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    What's your reasoning behind that? I admit, I've probably spent 20+ hours sifting through all the open source directory services.
     
    #5
  6. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,837
    Likes Received:
    1,491
    Sweet rack! It gives me some ideas... :)
     
    #6
  7. alex1002

    alex1002 Member

    Joined:
    Apr 9, 2013
    Messages:
    518
    Likes Received:
    17
    This is a really nice project you got.
     
    #7
  8. Martin Jørgensen

    Martin Jørgensen New Member

    Joined:
    Jun 3, 2015
    Messages:
    28
    Likes Received:
    6
    Redhat is behind FreeIPA
    See this video,
     
    #8
  9. whitey

    whitey Moderator

    Joined:
    Jun 30, 2014
    Messages:
    2,764
    Likes Received:
    858
    Another vote for Freeipa :-D
     
    #9
  10. Lost-Benji

    Lost-Benji Member

    Joined:
    Jan 21, 2013
    Messages:
    368
    Likes Received:
    18
    I see that either yourself or the client is trying to do things cheap possibly.

    That being said, the server you are looking at is already screaming alarms. You will need a smaller and separate array to place the Host OS and VM's on. Do not even try installing the lot to a 8TB volume. A pair of Mirrored SSD's to hold Host OS and VM's, create another VHD and attach from the big array that will NOT handle 10GbE very well as you will only see around 220-230MB/sec out of the array with limited I/O depending on the RAID controller.
     
    #10
  11. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    Cool, I'll check it out! The main things I need are excellent compatibility with integrating into Google Apps,

    Noted :)
     
    #11
  12. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    Sort of both.

    Honestly, the services running on the VM will be minuscule compared to what most people are running. I did crunch the numbers of what would be required. There is zero reason for the VM's to utilize the 10GbE array except when backing up nightly. Really, I mean it when I say those two servers are going to be hosting bare bones VM's.

    What you do not see yet is an actual proper speced KVM infrastructure that is currently hosted elsewhere, and will be brought into the rack come December. This build was literally meant to hold them over until then, and then utilize the two servers for other purposes.

    That being said, the ZFS box will stay. I'll be doing a separate post with a full write up on it, since it's a TrueNAS system. I'll post the price, full specs, and detail shots.
     
    #12
  13. Lost-Benji

    Lost-Benji Member

    Joined:
    Jan 21, 2013
    Messages:
    368
    Likes Received:
    18
    Not sure if there is bad terms being used here, are you referring to PCoIP in place of the KVM you mention? I ask because I actually use a Dell R5400 as my daily rig and very familiar with these setups and love them, just not used enough.

    If there is to be any more gear, the UPS will be way under speced for other workstations/servers.
     
    #13
  14. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    UPS's will also be brought over too! What you see now is one set up, with a second being transplanted soon.
     
    #14
  15. Dajinn

    Dajinn Active Member

    Joined:
    Jun 2, 2015
    Messages:
    512
    Likes Received:
    78
    Were they opposed to using Microsoft software/os for virtualization and active directory?
     
    #15
  16. GCM

    GCM Active Member

    Joined:
    Aug 24, 2015
    Messages:
    137
    Likes Received:
    43
    Very opposed.

    Their entire core infrastructure for the site / app is on AWS, and averages about 35 million uniques a month.
     
    #16
Similar Threads: Building infrastructure
Forum Title Date
DIY Server and Workstation Builds Help! I’m building a small FreeNAS server Nov 6, 2019
DIY Server and Workstation Builds Need Advice Building a 16+ ZFS NAS Oct 17, 2019
DIY Server and Workstation Builds Building a tiny (Sub 3.9 Litres) workstation Sep 8, 2019
DIY Server and Workstation Builds In process of building server, EPYC7351P, Asrock EPYCD8-T2 Aug 11, 2019
DIY Server and Workstation Builds Building a mini fanless Server Akasa Euler M / Supermirco A2SDI-2C-HLN4F Jun 28, 2019

Share This Page