FINAL Bachelor Build - Xeon D vSAN Cluster

Discussion in 'DIY Server and Workstation Builds' started by IamSpartacus, Mar 28, 2016.

  1. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    I'm getting married next year and thus this is my last opportunity to spend a good chunk of money on my "toys" before I need wifey approval for everything (queue the tears and boos). I've been saving up for a network overhaul for some time now.

    My plan is to build a 3-node VMware vSAN cluster for the purpose of having full redundancy of all my VMs. I primarily want to be able to take a single node offline while keeping the Linux VM running my media dockers online as well as the rest of my VMs (Windows AD, DNS, etc.). I also just want to generally play around with VMware clustering, HA, and vMotion for my own personal knowledge as I'd like to implement some of this at work at some point.

    So without further ado, this is the hardware that will be going into a new server rack that will sit in my home office (I will build out a full network diagram once I have everything in place and will update you all with that in a subsequent post).


    Rack: 22U Linier Server Rack
    [​IMG]

    I will be drilling custom holes in the back door of this rack to install 3 x 120mm exhaust fans. I also plan to (at least attempt to) install some sound dampening materials on the back, side, and top panels of the rack.


    Equipment Going in the Rack (from bottom to top)

    UPS: CyberPower 2U 900w rack mountable UPS - [2U]

    ESXi Node #1: vSAN Storage Only Node + UnRAID VM - [4U]

    ESXi Node #2 & #3 (Identical Nodes): Hyperconverged Computing/Storage Nodes - [2U each]

    10GbE Switch: Dell X1052 Smart Managed Switch (48 x 1Gb, 4 x SFP+ ports) - [1U]

    pfSense Box: Router and Firewall - [1U]

    24-Port Patch Panel: Cable Matters Rackmount or Wallmount 24-Port Cat6 RJ45 Patch Panel - [1U]


    Just a note about the iStarUSA cases. I know a lot of people around here are down on them because they are for the most part overpriced compared to what you can get for higher quality used chassis' from say SuperMicro. However, the main reason I decided to go with them is because of the chassis depth since the rack I'm using only supports up to 24.5" of mounting space. And while the 4U trayless case is a hefty pricetag, I love their trayless offerings as I've used them previously and they are super convenient.

    Equipment has already started to arrive so I will start posting pics soon :D.


    ** Final Build Post Here **


    [​IMG]
     
    #1
    Last edited: Jan 15, 2017
    ridney, Ramos, dba and 2 others like this.
  2. sth

    sth Active Member

    Joined:
    Oct 29, 2015
    Messages:
    254
    Likes Received:
    38
    its not just the money post marriage you need to worry about (hey, you'll have two incomes right!) but the time it takes to faff and maintain this gear! :)

    congrats on the new toys and the upcoming wedding.
     
    #2
  3. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Haha I know. I've got about 2-3 years (barring any "mistakes") before the kids come :).
     
    #3
  4. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,557
    Likes Received:
    4,485
    We just need more pictures. And @IamSpartacus I got married and this stuff just moved to a datacenter (and expanded by 14x the rack space in the unseen datacenters.) :)
     
    #4
    Ramos, rnavarro, Evan and 2 others like this.
  5. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    @Patrick pictures will be forthcoming tonight and over the next 1-2 weeks as parts arrive. And yea, I can't make any promises on what will become of my network in the future. Whenever gigabit fiber becomes a feasible option for me I could certainly see myself moving my setup to a datacenter.
     
    #5
  6. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,696
    Likes Received:
    499
    wow - how much is the rack?
     
    #6
  7. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Got it for $700 new. Not cheap but was the only one that fit my size requirements.
     
    #7
  8. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,696
    Likes Received:
    499
    i am new to enclosures like that but it looks like a really good one, 700 seems reasonable. I too am looking forward to your build man also congrats on getting married
     
    #8
  9. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Yea from what I've read it's a really solid rack so I'm anxious to get working on it. Thanks for the kind words :).
     
    #9
    gigatexal likes this.
  10. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Does anyone know how I could attach intake fans to this bottom cutout of my rack somehow?

    [​IMG]
    EDIT: I just spoke to Kendall Howard and apparently the rack ships with a cover plate to close up that large rectangular cable cutout. I might be able to cut 2 120mm fan cutouts into that plate to use for intake.
     
    #10
    Last edited: Mar 28, 2016
  11. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,557
    Likes Received:
    4,485
    Another option is to just get some sheet metal and a drill or plexi glass. The bottom side is very easy to do this with as gravity helps.
     
    #11
  12. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    You're saying sheet metal (and obviously plexi) would be easier to drill hole cutouts then through the provided cover right?

    Also, is there type of shelf/vent I can install in the bottom 1U to direct the intake air from the bottom to the front of the rack?
     
    #12
    Last edited: Mar 28, 2016
  13. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    2,109
    Likes Received:
    960
    Yes, fans in a 1u tray blowing cool air upward.
     
    #13
  14. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,557
    Likes Received:
    4,485
    Just saying it is another option that is fairly easy to do if you do not want to use the stock cover.
     
    #14
  15. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Fan just blowing str8 up will be directed to the front even if under a UPS?
     
    #15
  16. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    2,109
    Likes Received:
    960
    Mount the fan tray on top to draw cool air in and exhausted out to top
     
    #16
  17. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Pardon my ignorance here but I'm not following what you're suggesting.

    My rack has fan cut outs on the top (you can see them through the bottom rectangular cut out I posted a few posts ago) so if I wanted to put fans at the top I could. I'm looking to have some sort of intake fans at the bottom of the rack, direct that air to the front (so the servers use it for intake) and then have either top/rear exhaust fans remove the heat from the rack.
     
    #17
  18. pc-tecky

    pc-tecky Member

    Joined:
    May 1, 2013
    Messages:
    196
    Likes Received:
    24
    The racks at the co-lo datacenters that I've been at had locking mesh front and rear doors so air could flow front to back, or had a closed loop enclosure (kind of like a meat locker at Sam's or Costco). The idea is to control and direct heat away from the servers. Optional 1U, 2U, 3U, etc. filler panels could be installed to prevent cross flow of cold air into the hot air aisle and vise versa. Some like the idea while others find faults with it. Panels or a shroud could be installed to block air from escaping to the rear and sides therefore directing air to the front. Fans at the rear and/or the top of the enclosure will work in a similar fashion as a forced air furnace/air conditioner does - they both will have an intake (pulled from the room, negative pressure) and an output (heated/cooled air forced back into the room, positive pressure). You can have a push setup (fans in front/bottom), a pull setup (fans at rear/top), and a push-pull (fans front/top and fans rear/bottom). Fans and vents from the bottom might not be necessary. Use observations and laws of physics to your advantage - heat rises. Servers tend to "pump" air front to back (in direction of air flow from system fans), so heat would logically be found towards the rear and top of the enclosure. Lastly, you can have an upward draft (air rises) or downward draft (air sinks) based on direction of fans that create the flow of air. Some places go as far as to have liquid cooling heat exchangers (radiators) found either in the front of the rack to supply cooler air or at the rear of the rack as a means to move heat away from the servers.
     
    #18
  19. miraculix

    miraculix Member

    Joined:
    Mar 6, 2015
    Messages:
    116
    Likes Received:
    24
    Hey I have a somewhat similar 3 node VSAN setup with 3 X10SDVs, along with other stuff in the cabinet (general FreeNAS boxes and such). I'm using a Navepoint 15U 600mm deep wall-mount enclosure, re-purposed as a free-standing enclosure with casters bolted on.

    The shorter depth on mine is definitely limiting and I agree the 700mm on yours is a great "sweet spot". I may swap mine with a deeper enclosure and mount this on the garage wall for 100% network gear (it's where all my Cat6A cabling comes in anyway), but it was dirt cheap and I love having something right here in my office that is so small, powerful, and relatively quiet with modification.

    Some suggestions:

    • Before adding any acoustic foam panels, consider putting in an "underlay" of butyl deadener sheets... pretty much the same stuff used to quiet down car door panels, etc. but look online for cheaper bulk sources instead of going to your local automotive store (way too expensive there). I have both foam panels and butyl sheets, and I think the butyl reduced noise the most. It'll add considerable weight though.
    • AC Infinity makes quiet and reasonably priced 120mm fans and thermostat controllers. I'm using 2x 120mm USB fans on the top (blowing out) and two on the bottom (blowing in), controlled with their thermostat.... basically a bottom-to-top air channel.
    • Exhausting air out the back instead of the top is fine, but do it as high as possible.
    • Try to leave room at the bottom and piece together some form of ducting to bring cool bottom air to the front, where it goes through and out the back of each individual server chassis, and up and exhausted out of the cabinet. Use rack shelves and blank rack unit panels to help with the ducting/channeling of air... but even cardboard, etc. is doable!
    • From what I understand, positive air pressure is better than negative. With the latter you'll get dust sucked in through any and all the air gaps throughout the cabinet. However, "equal pressure" the way I'm doing might be OK as long as you seal up as many gaps/cracks as possible. I plan to add a box with replaceable air filter on the bottom.
     
    #19
  20. IamSpartacus

    IamSpartacus Well-Known Member

    Joined:
    Mar 14, 2016
    Messages:
    1,930
    Likes Received:
    421
    Thank you for your suggestions @miraculix. The butyl deadener sheets is a great idea I hadn't thought of that. It's not cheap though like you said. I'm going to have to see how loud this rack is with all my gear in it first and then adjust from there accordingly.

    I did pickup a pair of the AC Inifinity fans to use as exhaust on the top of my rack for starters. I'm going to monitor that for a few days and see how the temps are and again adjust from there.

    Do you have any suggestions as to how to direct pulled air from the bottom to the front of the rack? I've been trying to determine the best way to do that but I'm still searching for a good option.
     
    #20
Similar Threads: FINAL Bachelor
Forum Title Date
DIY Server and Workstation Builds Threadripper Rosewill L4500 build in progress, help final steps Oct 23, 2019
DIY Server and Workstation Builds Need some advice on finalizing my massive NAS upgrade Oct 9, 2018
DIY Server and Workstation Builds ESXI 5.5 GPU passthrough - Finally. Mar 18, 2017
DIY Server and Workstation Builds Final planning stage for me plex server May 16, 2016
DIY Server and Workstation Builds Finalizing new ESXi build - looking for last touches/suggestions Mar 30, 2016

Share This Page