the plan started like: the microserver is getting old and i would like to do something really nice with it...
also i have some spare HDDs that would like to put to good use before EOL...
- 4*2Tb 3.5" sata wd red
- 3*1Tb 3.5" sata (various)
- 2*1Tb 2.5" sata
- 4*146Gb 2.5" sas 15K
- 4*146GB 2.5" sas 10K
i'm aware it's a quite mix of drives, and right now only the 2Tb are in use...
Considering that right now i just have a room, the system should be reasonably silent to used in an open room and not make your ears bleed.
alternative solutions are welcome
*ugh*.
Okay, I know that it's pretty alluring to buy a bunch of EOLed servers off the rack and put them in your house, but seriously, don't. Well, do it if you have a "walled garden" in your place that you can throw servers in (a sound isolated, climate controlled room preferably with a rack and a dedicated 15+ Amp circuit). Some of us can afford to do so (say, a dedicated room or an underused garage), some are still "making their bones", so to speak. That's okay. I went through the same stages in the past.
If it's only a rented room, think of something that you can put up on a shelf, run 24/7 and still give you a decent night of sleep - you also have to think about the consequences of hanging a bunch of power hungry 1/2U servers on your power circuit. In my past residences I can't even microwave a burrito and keep the room air conditioning running at the same time in the living room, lest I trip the circuit breakers. You don't want to get into a situation similar to me where an ex- significant other messed up a RAID array because she was drying her hair and the circuit trip bought the servers down (it's partially my fault - the UPS battery was giving faulty status). And no, I didn't dump her because she messed up the array. I dumped her because she's...crazy.
So, what to do? First, ditch those SAS 146GB spinners - there's nothing more useless than small hard drives - even if they are SAS. In terms of latency, throughput and ability to deal with abuses, even the old Intel X25 SSDs will trample all over them, and on the newer stuff like the Crucial BX500s you don't even have a prayer - the NVMes are even faster. You shouldn't think re-use as much as replacement as they are a liability - like the old IT guy joke goes: hey, I have 1 flaky drive. How do I make it reliable? Add another one to make a slightly less flakey RAID array. And how do you make that array slightly less flakey? Add another 2 drives to make it even less flakey (RAID 10 it, or RAID5 it). Oh, and you will need an extra drive as a hot backup. Don't forget that you can only tolerate one or 2 drive failures, and any failures will require a long and I/O heavy rebuild period for the entire array (better pray to the elder gods during this rebuild period and hope that nothing else fails during this time). Hooh boy - before you know it you are looking at an 8 disk array eyeing for a 16...which is ridiculous.
At some point you need to recognize the signs of data/drive hoarding and admit that it's a problem. Then deal with it. You deal with it by deduping data, deleting/backing up stuff (into offline media or into the cloud) that is aged out, and getting rid of old low capacity drives and consolidating whenever possible. I have 4 6TB drives inside my N40L and I am now struggling to fill them with data - I parked a 40GbE Mellanox in my N40L and turned it into a FreeNAS storage box serving out NFS and iSCSI.
Now, what about computing? You could consider NUCs, but I consider them to be a little bit limiting in terms of expansion (no PCIe x8 or x16, only M.2 Slot E for Wifi cards), and most have limited CPU power to bring to the party (typically the U-series Cores). Some of them are also noisy when under load (my Gigabyte BRIX Pro 5775R certainly fit in that category - fast as hell (runs as well as a Xeon E3-1285v4) but if you push it under load, it spins up the fans and turns into a hair dryer - I ran Skanect (3D photogrammetry app) on my Brix until it was replaced with a used iMac (CUDA cores, baby!). It had a Realtek NIC and kinda sucked for Proxmox (mostly because of the limited networking - you can't park a Solarflare/Mellanox/Intel i540 in there and enable SRIOV). The other NUCs are like that - they might run Intel i217 NICs (which is a more stable animal, but no SRIOV VF support), but they still aren't great. You are still left wanting for I/O. The same also apply for Mac Minis (even though the 2011/2012s are good devices - I should know, I am a proud owner of a 2011 Mac Mini). You can in theory wire up multiple Mac Minis using their lightning ports, but if you need it to point to a NAS, that's an expensive TB2 to SFP+ adapter (500 bucks each).
What about repurposing the "big" thin clients? A few HP thin clients have PCIe slots (the gt7725/t620 Plus/t730/t740s), some have decent computing power (the gt7725s have the same family of CPUs as the N40L, the t620 plus runs the same family of chips as the PS3, the t730s are similar to a Broadwell-U, and the t740 is a Ryzen embedded device which should outrun the base model 2018 Mac Mini), good RAM expansion options (the t620 Plus/t730/t740 can all handle 32GB of RAM), and can be bought relatively inexpensively (all except the t740 can be had for less than 250 USD/piece if you hustle on eBay). Their problem? SRIOV doesn't work as intended, so if you want to do some fancy virtual networking in ESXi or Proxmox, either you can't do it, or it needs to be hacked to semi-work. I actually have a t730 connected to my N40L, and it does nothing but run my VMs. It works well enough (I did light up the onboard+fiber NICs for bonded networking, and a Mellanox 40GBit for iSCSI to the N40L NAS), but the big payoff is in the quietness/low impact to my electric bill (my cats sleep next to the machnes all the time). I don't need SRIOV (yet), but we shall see how well the BIOS support is on the t740 thin clients. If it is ACSCtl kosher/can do full SR-IOV, I'll most likely delegate part of the card as a VF and use it to do clustering in ESXi or Proxmox.
So what does that leave us? Well you can consider grabbing a few recent off-lease Corporate PC workstations - the Dell Optiplex 7050s, the Lenovo ThinkCentres, or the HP EliteDesks. Their Mini/Micro models are essentially corporate NUCs, their SFFs should have at least 2 RAM slots and one PCIe x16 slot, and their minitowers should have 4 RAM slots and 2 PCIe slots each. They should all be relatively stable and well documented, should have SRIOV enabled in the BIOS, they have decent CPUs (i7-6700Ts are nothing to laugh at), and best of all, they are designed to be QUIET. If you visit your local bank and see the PC parked on the corner, that should be one of those. They have limited resale values on eBay, and whenever vendors like HP or Dell announce new models or whenever a large corporate refresh happens, the inventory gets dumped and the pricing goes down (last I checked an Optiplex 7050 goes for about 500 USD). Consider buying one or 2, throw a few SSDs in there (boot drives don't need to be big) wire them up using 10/40 GbE (Twinax cables work well here), and then repurpose the Microserver as a NAS. The outlay is around the same ballpark as a 1 or 2U, you'll see less impact on your electric bill, and your room won't sound like a data center (which it should not be, anyways)
