FINAL Bachelor Build - Xeon D vSAN Cluster

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
I'm getting married next year and thus this is my last opportunity to spend a good chunk of money on my "toys" before I need wifey approval for everything (queue the tears and boos). I've been saving up for a network overhaul for some time now.

My plan is to build a 3-node VMware vSAN cluster for the purpose of having full redundancy of all my VMs. I primarily want to be able to take a single node offline while keeping the Linux VM running my media dockers online as well as the rest of my VMs (Windows AD, DNS, etc.). I also just want to generally play around with VMware clustering, HA, and vMotion for my own personal knowledge as I'd like to implement some of this at work at some point.

So without further ado, this is the hardware that will be going into a new server rack that will sit in my home office (I will build out a full network diagram once I have everything in place and will update you all with that in a subsequent post).


Rack: 22U Linier Server Rack


I will be drilling custom holes in the back door of this rack to install 3 x 120mm exhaust fans. I also plan to (at least attempt to) install some sound dampening materials on the back, side, and top panels of the rack.


Equipment Going in the Rack (from bottom to top)

UPS: CyberPower 2U 900w rack mountable UPS - [2U]

ESXi Node #1: vSAN Storage Only Node + UnRAID VM - [4U]

ESXi Node #2 & #3 (Identical Nodes): Hyperconverged Computing/Storage Nodes - [2U each]

10GbE Switch: Dell X1052 Smart Managed Switch (48 x 1Gb, 4 x SFP+ ports) - [1U]

pfSense Box: Router and Firewall - [1U]

24-Port Patch Panel: Cable Matters Rackmount or Wallmount 24-Port Cat6 RJ45 Patch Panel - [1U]


Just a note about the iStarUSA cases. I know a lot of people around here are down on them because they are for the most part overpriced compared to what you can get for higher quality used chassis' from say SuperMicro. However, the main reason I decided to go with them is because of the chassis depth since the rack I'm using only supports up to 24.5" of mounting space. And while the 4U trayless case is a hefty pricetag, I love their trayless offerings as I've used them previously and they are super convenient.

Equipment has already started to arrive so I will start posting pics soon :D.


** Final Build Post Here **


 
Last edited:
  • Like
Reactions: ridney, Ramos, dba and 2 others
S

sth

Active Member
Oct 29, 2015
403
100
43
its not just the money post marriage you need to worry about (hey, you'll have two incomes right!) but the time it takes to faff and maintain this gear! :)

congrats on the new toys and the upcoming wedding.
 
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
sth said:
its not just the money post marriage you need to worry about (hey, you'll have two incomes right!) but the time it takes to faff and maintain this gear! :)

congrats on the new toys and the upcoming wedding.
Click to expand...
Haha I know. I've got about 2-3 years (barring any "mistakes") before the kids come :).
 
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
Patrick said:
We just need more pictures. And @IamSpartacus I got married and this stuff just moved to a datacenter (and expanded by 14x the rack space in the unseen datacenters.) :)
Click to expand...
@Patrick pictures will be forthcoming tonight and over the next 1-2 weeks as parts arrive. And yea, I can't make any promises on what will become of my network in the future. Whenever gigabit fiber becomes a feasible option for me I could certainly see myself moving my setup to a datacenter.
 
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
gigatexal said:
i am new to enclosures like that but it looks like a really good one, 700 seems reasonable. I too am looking forward to your build man also congrats on getting married
Click to expand...
Yea from what I've read it's a really solid rack so I'm anxious to get working on it. Thanks for the kind words :).
 
  • Like
Reactions: gigatexal
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
Does anyone know how I could attach intake fans to this bottom cutout of my rack somehow?


EDIT: I just spoke to Kendall Howard and apparently the rack ships with a cover plate to close up that large rectangular cable cutout. I might be able to cut 2 120mm fan cutouts into that plate to use for intake.
 
Last edited:
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,545
5,874
113
Another option is to just get some sheet metal and a drill or plexi glass. The bottom side is very easy to do this with as gravity helps.
 
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
Patrick said:
Another option is to just get some sheet metal and a drill or plexi glass. The bottom side is very easy to do this with as gravity helps.
Click to expand...
You're saying sheet metal (and obviously plexi) would be easier to drill hole cutouts then through the provided cover right?

Also, is there type of shelf/vent I can install in the bottom 1U to direct the intake air from the bottom to the front of the rack?
 
Last edited:
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,545
5,874
113
Just saying it is another option that is fairly easy to do if you do not want to use the stock cover.
 
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
Marsh said:
Mount the fan tray on top to draw cool air in and exhausted out to top
Click to expand...
Pardon my ignorance here but I'm not following what you're suggesting.

My rack has fan cut outs on the top (you can see them through the bottom rectangular cut out I posted a few posts ago) so if I wanted to put fans at the top I could. I'm looking to have some sort of intake fans at the bottom of the rack, direct that air to the front (so the servers use it for intake) and then have either top/rear exhaust fans remove the heat from the rack.
 
P

pc-tecky

Active Member
May 1, 2013
202
26
28
The racks at the co-lo datacenters that I've been at had locking mesh front and rear doors so air could flow front to back, or had a closed loop enclosure (kind of like a meat locker at Sam's or Costco). The idea is to control and direct heat away from the servers. Optional 1U, 2U, 3U, etc. filler panels could be installed to prevent cross flow of cold air into the hot air aisle and vise versa. Some like the idea while others find faults with it. Panels or a shroud could be installed to block air from escaping to the rear and sides therefore directing air to the front. Fans at the rear and/or the top of the enclosure will work in a similar fashion as a forced air furnace/air conditioner does - they both will have an intake (pulled from the room, negative pressure) and an output (heated/cooled air forced back into the room, positive pressure). You can have a push setup (fans in front/bottom), a pull setup (fans at rear/top), and a push-pull (fans front/top and fans rear/bottom). Fans and vents from the bottom might not be necessary. Use observations and laws of physics to your advantage - heat rises. Servers tend to "pump" air front to back (in direction of air flow from system fans), so heat would logically be found towards the rear and top of the enclosure. Lastly, you can have an upward draft (air rises) or downward draft (air sinks) based on direction of fans that create the flow of air. Some places go as far as to have liquid cooling heat exchangers (radiators) found either in the front of the rack to supply cooler air or at the rear of the rack as a means to move heat away from the servers.
 
M

miraculix

Active Member
Mar 6, 2015
151
51
28
Hey I have a somewhat similar 3 node VSAN setup with 3 X10SDVs, along with other stuff in the cabinet (general FreeNAS boxes and such). I'm using a Navepoint 15U 600mm deep wall-mount enclosure, re-purposed as a free-standing enclosure with casters bolted on.

The shorter depth on mine is definitely limiting and I agree the 700mm on yours is a great "sweet spot". I may swap mine with a deeper enclosure and mount this on the garage wall for 100% network gear (it's where all my Cat6A cabling comes in anyway), but it was dirt cheap and I love having something right here in my office that is so small, powerful, and relatively quiet with modification.

Some suggestions:

  • Before adding any acoustic foam panels, consider putting in an "underlay" of butyl deadener sheets... pretty much the same stuff used to quiet down car door panels, etc. but look online for cheaper bulk sources instead of going to your local automotive store (way too expensive there). I have both foam panels and butyl sheets, and I think the butyl reduced noise the most. It'll add considerable weight though.
  • AC Infinity makes quiet and reasonably priced 120mm fans and thermostat controllers. I'm using 2x 120mm USB fans on the top (blowing out) and two on the bottom (blowing in), controlled with their thermostat.... basically a bottom-to-top air channel.
  • Exhausting air out the back instead of the top is fine, but do it as high as possible.
  • Try to leave room at the bottom and piece together some form of ducting to bring cool bottom air to the front, where it goes through and out the back of each individual server chassis, and up and exhausted out of the cabinet. Use rack shelves and blank rack unit panels to help with the ducting/channeling of air... but even cardboard, etc. is doable!
  • From what I understand, positive air pressure is better than negative. With the latter you'll get dust sucked in through any and all the air gaps throughout the cabinet. However, "equal pressure" the way I'm doing might be OK as long as you seal up as many gaps/cracks as possible. I plan to add a box with replaceable air filter on the bottom.
 
IamSpartacus

IamSpartacus

Well-Known Member
Mar 14, 2016
2,520
652
113
miraculix said:
Hey I have a somewhat similar 3 node VSAN setup with 3 X10SDVs, along with other stuff in the cabinet (general FreeNAS boxes and such). I'm using a Navepoint 15U 600mm deep wall-mount enclosure, re-purposed as a free-standing enclosure with casters bolted on.

The shorter depth on mine is definitely limiting and I agree the 700mm on yours is a great "sweet spot". I may swap mine with a deeper enclosure and mount this on the garage wall for 100% network gear (it's where all my Cat6A cabling comes in anyway), but it was dirt cheap and I love having something right here in my office that is so small, powerful, and relatively quiet with modification.

Some suggestions:

  • Before adding any acoustic foam panels, consider putting in an "underlay" of butyl deadener sheets... pretty much the same stuff used to quiet down car door panels, etc. but look online for cheaper bulk sources instead of going to your local automotive store (way too expensive there). I have both foam panels and butyl sheets, and I think the butyl reduced noise the most. It'll add considerable weight though.
  • AC Infinity makes quiet and reasonably priced 120mm fans and thermostat controllers. I'm using 2x 120mm USB fans on the top (blowing out) and two on the bottom (blowing in), controlled with their thermostat.... basically a bottom-to-top air channel.
  • Exhausting air out the back instead of the top is fine, but do it as high as possible.
  • Try to leave room at the bottom and piece together some form of ducting to bring cool bottom air to the front, where it goes through and out the back of each individual server chassis, and up and exhausted out of the cabinet. Use rack shelves and blank rack unit panels to help with the ducting/channeling of air... but even cardboard, etc. is doable!
  • From what I understand, positive air pressure is better than negative. With the latter you'll get dust sucked in through any and all the air gaps throughout the cabinet. However, "equal pressure" the way I'm doing might be OK as long as you seal up as many gaps/cracks as possible. I plan to add a box with replaceable air filter on the bottom.
Click to expand...
Thank you for your suggestions @miraculix. The butyl deadener sheets is a great idea I hadn't thought of that. It's not cheap though like you said. I'm going to have to see how loud this rack is with all my gear in it first and then adjust from there accordingly.

I did pickup a pair of the AC Inifinity fans to use as exhaust on the top of my rack for starters. I'm going to monitor that for a few days and see how the temps are and again adjust from there.

Do you have any suggestions as to how to direct pulled air from the bottom to the front of the rack? I've been trying to determine the best way to do that but I'm still searching for a good option.
 
You must log in or register to reply here.
Top Bottom