Build my own or buy Supermicro?

[JohnnyBeGood]

Hey all,

Currently in my homelab I have Lenovo ThinkServer TS140 Tower Server System that needs to be moved from office to the garage. It is limited to 32GB of RAM and my love for VMs on XCP-ng is growing
I will most likely end up with this Wall Mount Server Cabinet. What I'm not sure is if I should buy used Supermicro with CPU and memory on eBay or build my own?
I really like the idea of IPMI remote management and reliability of those boards, that's why I'm leaning toward Supermicro route. For 3+ years I have TrueNAS running in a VM with 4 x 3TB HDs and IT mode flashed M1015 and that is the reason for larger case. In the future I would like to upgrade/add more drives.
My question to you guys is what route would you go, build from scratch or buy used? So far Intel Xeon served me well and I'm open to the suggestions.

Thanks in advance for your time/replies!

 

[i386]

Wall mounted racks/cabinets are usually not made for servers but for "small" network devices. There are some short server chassis, but they are usually limtied to 4 or less hdds.
Witz zfs adding storage can be a pain in the ass: you can't add single hdds to the vdev (there is a alpha version of that feature which was announced in 2017? but it's still not there yet). You either have to add in mirrors (2 hdds) or in vdevs (if the 4x 3tb are in a raidz you would need 4x hdds for expansion)

My servers & workstation are build from scratch with a mix of new and used parts to keep costs down and to use what I have available. I like the diy route for stuff at home because I can replace parts easily, with bought stuff/prebuilt server/computers you might end up with mainboards, psus or fans that have proprietary form factors and won't work in standard chassis.
Giving advice for cpus and mainboards is difficult with the description in your posts. There are many (server and "consumer") cpus that support now >32GB ram.

 

[RolloZ170]

What I'm not sure is if I should buy used Supermicro with CPU and memory on eBay or build my own?

what do you mean "build my own" ? you want to build motherboard & CPU DIY ? good luck then !!!
buy used or buy new with warranty, hard to decide.

 

[JohnnyBeGood]

Wall mounted racks/cabinets are usually not made for servers but for "small" network devices. There are some short server chassis, but they are usually limtied to 4 or less hdds.
Witz zfs adding storage can be a pain in the ass: you can't add single hdds to the vdev (there is a alpha version of that feature which was announced in 2017? but it's still not there yet). You either have to add in mirrors (2 hdds) or in vdevs (if the 4x 3tb are in a raidz you would need 4x hdds for expansion)

My servers & workstation are build from scratch with a mix of new and used parts to keep costs down and to use what I have available. I like the diy route for stuff at home because I can replace parts easily, with bought stuff/prebuilt server/computers you might end up with mainboards, psus or fans that have proprietary form factors and won't work in standard chassis.
Giving advice for cpus and mainboards is difficult with the description in your posts. There are many (server and "consumer") cpus that support now >32GB ram.

Thanks for taking time to reply!

The one I linked has the following:
Dimensions: 21.65" Length x 17.72" Depth x 24.53" Height
Is that not enough?

I will have to look into replacing/adding new drivers.

What was your base, did you started with the case first and worked your way up ie. MB, CPU etc. or?

What really got me is that I had a spare EVGA high efficiency power supply and once I wanted to replace the original Lenovo, I did not had right 14pin connector (I think) for the MB power. Which means all the stuff inside is preoperatory. If the power would die on me, the server would be offline until I found correct power supply.

I guess some recommendations of the popular setups with low power consumption would the best.

 

[JohnnyBeGood]

what do you mean "build my own" ? you want to build motherboard & CPU DIY ? good luck then !!!
buy used or buy new with warranty, hard to decide.

Thanks for taking time to reply!

With "build my own" I mean instead of buying pre-built ie. Lenovo ThinkServer TS140 Tower Server System I would start with buying cabinet, case, motherboard, CPU/RAM, power supply etc.

 

[i386]

Is that not enough?

That depends on the server chassis you want to use. Usually the depth is the problem: depth of server + some space for cables/usb ports etc = minimum depth of server rack

What was your base, did you started with the case first and worked your way up ie. MB, CPU etc. or?

For my fileserver yes. my requirements were at least 16 hdd bays + support for "dirt as cheap" network cards (;D). After that I chose a mainboard + cpu combination (I went for the first iteration with a dual core xeon d system with onboard 10GBE sfp+ nics).

I would recomment to look for a mainboard that has some room for expansion. 1 slot each for: hba/controller, nic and nvme "hba" (either 2x m.2 carrier or retimer/redriver aic for 2x u.2&/u.3 ssds)

What really got me is that I had a spare EVGA high efficiency power supply and once I wanted to replace the original Lenovo, I did not had right 14pin connector (I think) for the MB power. Which means all the stuff inside is preoperatory. If the power would die on me, the server would be offline until I found correct power supply.

yeah, proprietary connecors and form factors suck.

 

[mattventura]

If you can find one with good components, buying a complete used server can be viable. But often, the cheapest option somehow ends up being to buy a whole server, sell the parts you don't need (typically CPU/RAM/MB since many of them are still ancient DDR3 stuff), and upgrade from there. I say as someone who's currently stuck on said "ancient DDR3 stuff", your best bet is probably a first or second gen Epyc.

If you do want U.2 as the poster above mentions, just be aware it gets a little expensive. At that point, you're likely better off looking for a motherboard that has NVMe ports builtin so that you aren't paying for a redriver/retimer/HBA. But that's harder to find on older boards (you're most likely looking at X11/H11 or newer). Then there's also the issue of the backplane itself - you either buy a chassis that comes with an NVMe-compatible BP, or buy it after the fact (~$200 for the 12-drive 2U backplane with 4 NVMe slots). A simple M.2 or U.2 carrier card is much cheaper, you just lose out on the ability to access them from the front of the chassis, as well as (probably) hotswap.

 

[audiophonicz]

Build your own, it can be done with short depth wall mount racks. These "servers" are 10.5" deep.

 

[mattventura]

Build your own, it can be done with short depth wall mount racks. These "servers" are 10.5" deep.

What are those chassis you've got there? I've been looking for something like those.

 

[RolloZ170]

similar one ?

https://www.amazon.com/DMC-Taiwan-Rackmount-MicroATX-Motherboard/dp/B07VFVLDB3

https://www.amazon.com/Industrial-Chassis-19-Pc-ATX/dp/B00O9O2PV0

 

[audiophonicz]

What are those chassis you've got there? I've been looking for something like those.

So, the 2 shiney ones are OnLogic 1.5U cases, brackets flipped
1.5U Rackmount Case

The next one down is a modified Plink 2U case that I flipped, trimmed 3" off the back of the case, and added a DC barrel jack and IcyDock 2.5" cage into the front (picoPSU). 3D printed parts mount the drive cage inside.
2U Front Access (USB3.0 / 2.0) ( 2 x 3.5" + 2.5" ) or ( 5 x 2.5") HDD Bay (14.17" Deep) (Micro-ATX / Mini ITX) Rackmount Chassis IPC-2360F (PLinkUSA.com / RackBuy.com)

similar one ?
https://www.amazon.com/DMC-Taiwan-Rackmount-MicroATX-Motherboard/dp/B07VFVLDB3

Those are the ones I wanted, but unfortunately are 4 times the price of the Plinks, and come with fans and PSUs that I didnt want.

 

[Markess]

What I'm not sure is if I should buy used Supermicro with CPU and memory on eBay or build my own?

I assume when you say this you mean a used Supermicro server complete, vs building one from used Supermicro parts?

The one I linked has the following:
Dimensions: 21.65" Length x 17.72" Depth x 24.53" Height
Is that not enough?

There are chassis availale to fit in shorter depth racks. There are even some that can hold a lot of disks by "stacking' the disks above/below the motherboard and power supplies. The issue is that these aren't the "common" way to do things, and anything uncommon tends to cost more. If you have issues with depth and/or you want/need to mount the server on the wall, you can consider wall mounting a standard depth server in a wall mount rack that orients them vertically instead of horizontally. Simple bracket types as well as fully enclosed ones are available. Since your original rack link was for Amazon, a search there for "Vertical Server Rack" will return a lot of options and give you a visual of what they'd be like

What really got me is that I had a spare EVGA high efficiency power supply and once I wanted to replace the original Lenovo, I did not had right 14pin connector (I think) for the MB power. Which means all the stuff inside is preoperatory. If the power would die on me, the server would be offline until I found correct power supply.

If you want to use a more standard ATX Power supply with your TS140, you can use a conversion cable. Try searching "14 to 24 pin lenovo adapter" on Ebay. Be aware that there's some shoddy as well as high quality gear for these. Probably want to pick a seller with decent feedback for this item specifically.

Cheers!

 

[JohnnyBeGood]

I assume when you say this you mean a used Supermicro server complete, vs building one from used Supermicro parts?

Thank you all for the replies!

That is correct.
In my research I'm coming to find out that it would be cheaper to build my own vs. buying Supermicro server complete system, I came across this Asus X99-WS/IPMI It has IPMI and paired with ie. Intel Xeon E5-2699 V4 it would be a great system with 64GB of RAM. Thoughts?

 

[mattventura]

Thoughts?

How much are you looking to spend?

 

[JohnnyBeGood]

How much are you looking to spend?

Since I already have power supply and a spare ATX case (before I buy a new 2U case and cabinet) I will only need MB, CPU, and RAM so $400-500.

 

[mattventura]

Since I already have power supply and a spare ATX case (before I buy a new 2U case and cabinet) I will only need MB, CPU, and RAM so $400-500.

At that price range, LGA2011-3 is probably a good bet. A first gen Epyc is a little above that range (600-700 including 8x8GB RAM) but would be a pretty big bump in performance (Haswell is 8 years old at this point).

 

[audiophonicz]

You mentioned wall mount racks. which are for smaller stuff, and thats a big board.. especially for maxing at 128GB.
How many nodes of this size you planning to put in this server cabinet?
Seems like youre building a large all-in-one box to replace a smaller all-in-one box?

 

[JohnnyBeGood]

You mentioned wall mount racks. which are for smaller stuff, and thats a big board.. especially for maxing at 128GB.
How many nodes of this size you planning to put in this server cabinet?
Seems like youre building a large all-in-one box to replace a smaller all-in-one box?

At this point, TS140 is in my office along with 2 x consumer-grade switches. All my network cables are routed to the office. At some point in the future, I would like to get a bigger switch and eliminate those two Netgear. Other than that, I'm not planning to put anything else into that cabinet. But, who knows few years back I said the same for 32GB TS140

 

[JohnnyBeGood]

At that price range, LGA2011-3 is probably a good bet. A first gen Epyc is a little above that range (600-700 including 8x8GB RAM) but would be a pretty big bump in performance (Haswell is 8 years old at this point).

True. It is old. Any recommendations for upgrade to LGA2011-3 socket and any specific first gen Epyc?