2nd hand server or tinyminimicro cluster

[samellis]

Hi, i have hit a bit of a crossroads in my homelab on what way to go forward

At the moment I have the following
-Server Asrock NUC with a 4core AMD embedded CPU, 16GB RAM and 2 1GB NICS
-Storage 12bay QNAP with 4 1GB NICS and two SFP+ - used as an iSCSI target for the server and for my desktop computer
-Networking All 1GB unifi
-Virtualisation software Proxmox
-backup - Proxmox backup server running on HP Microserver in a seperate building

Workload (current)
Proxmox containers (Fileserver, jellyfin, piHole, owncloud)
Proxmox VM's Linux Mint VM, Windows 10VM


At the moment it is all working but i am approaching the limit of the CPU on the VM host and I want to look into doing more with it i have two options that look good on paper but i want to confirm what people think

Option1 - one powerful server
I have found a Dell R530 2U rack server with 64GB RAM, 4 1gig NIC ports and two E5-2630 v3 - The idea i have with this would be to remove one of the Xeons to bring power consumption down and install a 10Gig SFP+card and connect it to the QNAP with a DAC cable leaving me with the bandwidth of all 4 1gig ports for clients/vm's
This option can be done for under £200 once the card is bought and two SSD's to act as mirrored boot devices in the server and a NIC but it would be going near where I work so heat and noise and power usage is a worry, hence the removing of one CPU and looking at Dells as i have seen you have better fan control on them

Option2 - Tinyminimicro cluster
Purchase a pair of Lenovo M720's with 16GB RAM and a i5 8500t, install a 10G SFP+ nic in both of them, connect them via DAC cables to a 10G SFP+ switch that connects to the QNAP and make a cluster (i would be spinning up a pi or something like that to act as a 3rd tie breaker box for the Quorum).


From what i can see the pros of the one powerful server is that it is cheap, even with one CPU has plenty of cores but gives me the option of doubling the power by dropping the removed CPU back in - My worry would be noise, heat, power usage and it being a single point of failure. Looking at the CPU it is a 85w part so not awful but still quite high

The benefits of the tinyminimicro cluster that i can see are redundancy (This could be ideal as it means if one dies i dont need to panic), less power usage, less heat and less noise so would potentially be more pleasant to work around. Also as it is a desktop chip i can use its built in GPU to take some transcoding strain from the main CPU cores. However apart from the SFP card for the iSCSI storage i would only have 1Gig on each box for networking unless i use USB to Ethernet adapters to give me more 1gig connections, This will be the more expensive option as one Lenovo M720 currently goes 2nd hand for the same price as the Dell R530 and that is before i have bought a switch and NICS etc. I know i can spread this out by buying one Lenovo now and getting another later in the year/next year

So to people who have either run one big host or multiple small ones what way would you go?

 

[Patriot]

Check hotdeals T740s for $53 are a neat option.

 

[6c352e2e-f287-445a-9cd3-d]

I wouldn't want to use a dual socket machine and have only one CPU in it. If you want a single CPU, better go for something with a single slot mainboard.

Perhaps you can still find a Dell R320 (if you do, expect that the backup battery of the RAID controller may need to be replaced for about EUR 15 due to age). You can put a 10G NIC into an R320. You can go lower with an i210 if you want to; power consumption is about half with the i210 (about 80 vs 40 Watts, with 16 vs 24GB RAM).

I'm saying this because I was disappointed with dual socket Proliants with a single X5690 in them. They seemed rather lame and it just doesn't feel right. Having all the redundancy of a 2nd CPU slot, a 2nd set of memory banks, an actual NUMA board and whatever else two sockets require isn't the way to go when you don't need all that.

I wouldn't mess with the fan control. R320, i210 and r7920 (wich is almost identical to a R730) are pretty quiet, and there's no advantage to somehow limiting the fans and risk overheating. For the Proliants with the X5690, they came with some of the fans missing since they weren't needed, but we added them later because the server room gets unreasonably hot and we happened to get a bunch of extra spare fans, and it'll at least improve cooling the disks. I doubt you can just remove fans from a R530 without the remaining ones spinning up when a CPU is removed, but I don't know for sure.

If you go for a R530 with 2 CPUs, I'd estimate I'll be around 120--130W. With only one CPU I'd think maybe more like 100W. IIRC you can have up to 14 cores in a R320, but you can figure that out.

 

[samellis]

One of the reason i was looking at the 530 was due to it being a 2u unit. I was thinking that a bigger unit would have bigger fan so potentially run quieter - ive not played with the 1u Dell server before, what are they like volume wise?

 

[louie1961]

I am sure you are budget constrained, but honestly I would mess with either of those. I have an E5-2690 v3 in my testing server, and it was relegated to the testing environment because of the amount of heat it generates. I try to not turn it on unless I am actually working with it. It is an HP Z640 workstation. My main Proxmox host is a HP Elitedesk Mini that I picked up a while back with an i5-12500T processor. It has a 10GBE NIC in it and I love it. Six cores and 12 threads is more than enough horsepower for my three Wordpress sites, an instance of Nextcloud (all 4 in their own VMs), two more VMs running docker images (12 total) plus 2 LXC containers running other apps. I could probably put a lot more work on this machine if I had more RAM. My third proxmox node is an N100 machine that runs Ansible and two instances of OpenMediaVault as back up targets for my NAS machines (separate hardware-Synology and Terramaster). All of that is to say I would find something more recent than an i5-8500T

I would also probably upgrade your network at some point too. No sense in putting in 10gbe NICs without a 10gbe capable switch. My network is a mix of 2.5g and 10g, and my managed switch can handle both.