Home Server Build - Advice / Input Needed!

[Keir]

Hi,

I hope I have posted this in the right section!

I am currently looking at getting a home server setup.
I have spent some time thinking about it and doing research but would like some advice from some of you who have experience with this stuff.

What I am looking at getting.
Dell PowerEdge T130
32GB DDR4 ECC RAM (Input welcome on specifics)
Samsung NVMe 250GB SSD (Is this compatible with the motherboard in the T130? I think I'd need a PCIe card to plug it into, if it is compatible with the motherboard)
Western Digital Mechanical HDD's (Input on the range would help, bit confused about those!)

Network Overview
I would use my existing router (ASUS RT-AC87U) purely as a modem.
This would be connected to one network port on the T130.
The other network port would then connect to a switch which will handle all wired connections, one of which would be to a wireless access point to handle the WiFi on the network.

What I would like to do with it.
Part of the idea of getting this home server is to educate myself on configuring and managing this thus I have plans to use three VM's running different OS'.

I would be running the VM's from the NVMe drive using VMWare ESXi.

VM #1: Sophos UTM
This would be used to handle pretty much everything my router currently does as well as some security bits.

VM #2: Windows Server
This would be used to run a Plex server as well as a general file server.

VM #3: Ubuntu (or other *nix variant)
This would be used to run a web server and possibly an FTP server.

Questions
- Would this server be able to run the three VM's at the same time without any issues?
- With regards to the Plex server, would this server have any issues transcoding (specifically H265 as I read that this can be quite intensive)
- Are there any issues with what I have outlined above that anybody can see?
- Would I benefit from putting a GPU in the server? I know some tasks can be offloaded onto the GPU but would any of what I am planning to do make use of one?

If anyone has any feedback or advice, I would really appreciate it, and if you need any questions answered, just ask!

Thanks!

Keir

 

[Jeff Robertson]

Hi, some of your questions depend on which CPU you purchase. 32GB RAM should be plenty for more than 3 VMs so you are good there. Plex depends heavily on which CPU you put in it, stick with a quad core with a decent clock and you will be fine. Do you want to play with GPU passthrough? If so throw in a GPU, my guess is that it won't benefit you much. Sounds like a fun project!

 

[pricklypunter]

I find it's always best to break down what you need to achieve with each VM. Run only the absolute minimums to achieve that reliably, and you'll be very surprised just how low you can go these days. That means if you want a media server, like Plex, have that in it's own VM, File Server in another etc. Other benefits would be smaller troubleshooting domains, smaller attack surfaces should there be an intrusion or virus/ malware outbreak of some kind etc, you get the idea. It always easier on end users to take down a misbehaving VM, if you have to, and lose maybe one or two resources while you fix the issue, than it is to lose a whole bunch of things at the same time

The benefits of adding a GPU to the mix would really depend on what you are planning to do with it. If it's just for bragging rights, forget it, modern CPU's do a bang up job these days on most tasks, but if you are wanting to meddle with something like VDI and virtual desktops, then by all means a middle of the road GPU might be of benefit to you. If you plan on later running graphic intensive rendering or some such you might also get into using high end nvidia grid and vGPU stuff, but it's not cheap and you would also be wanting to up your game regarding the rest of the hardware

On the magnetic disk front, I would say WD RE4's if you are looking for some performance, are a decent choice. WD Red's, if you are looking for a lower power route, without too much of a penalty. Personally, I am partial and pretty biased towards HGST disks and if you can find a reasonable deal on some Ultrastars, they would not be a bad choice either for some performance, with the Deskstar NAS being the lower power, less performant variety.

Oh and just from my experience, Plex runs really sweet on Debian, as does ZoL if you want to dabble with ZFS for your storage, OK, I like Debian too

Sophos UTM is a good choice, but you might also want to give others a test drive to see what fits best for you, pfSense is also fairly popular for example.

Good luck with the build

 

[Keir]

Hi, some of your questions depend on which CPU you purchase. 32GB RAM should be plenty for more than 3 VMs so you are good there. Plex depends heavily on which CPU you put in it, stick with a quad core with a decent clock and you will be fine. Do you want to play with GPU passthrough? If so throw in a GPU, my guess is that it won't benefit you much. Sounds like a fun project!

I find it's always best to break down what you need to achieve with each VM. Run only the absolute minimums to achieve that reliably, and you'll be very surprised just how low you can go these days. That means if you want a media server, like Plex, have that in it's own VM, File Server in another etc. Other benefits would be smaller troubleshooting domains, smaller attack surfaces should there be an intrusion or virus/ malware outbreak of some kind etc, you get the idea. It always easier on end users to take down a misbehaving VM, if you have to, and lose maybe one or two resources while you fix the issue, than it is to lose a whole bunch of things at the same time

The benefits of adding a GPU to the mix would really depend on what you are planning to do with it. If it's just for bragging rights, forget it, modern CPU's do a bang up job these days on most tasks, but if you are wanting to meddle with something like VDI and virtual desktops, then by all means a middle of the road GPU might be of benefit to you. If you plan on later running graphic intensive rendering or some such you might also get into using high end nvidia grid and vGPU stuff, but it's not cheap and you would also be wanting to up your game regarding the rest of the hardware

On the magnetic disk front, I would say WD RE4's if you are looking for some performance, are a decent choice. WD Red's, if you are looking for a lower power route, without too much of a penalty. Personally, I am partial and pretty biased towards HGST disks and if you can find a reasonable deal on some Ultrastars, they would not be a bad choice either for some performance, with the Deskstar NAS being the lower power, less performant variety.

Oh and just from my experience, Plex runs really sweet on Debian, as does ZoL if you want to dabble with ZFS for your storage, OK, I like Debian too

Sophos UTM is a good choice, but you might also want to give others a test drive to see what fits best for you, pfSense is also fairly popular for example.

Good luck with the build

Thank you for the feedback!

I totally forgot to add the CPU I was thinking of going with.
By default, the machine comes with a Xeon E3-1220 v5 3.0GHz, but I am considering bumping that up to the Xeon E3-1240 v5 3.5GHz - what are your thoughts?

My initial thought was the Western Digital Red's as I have used them before and they seem good and reliable and reasonably priced. As this thing will be on 24/7 as well, I am wary of power efficiency with the whole thing too.

I do have a spare nVidia GeForce 290 which I might put in it just because it's currently sitting in a cupboard gathering dust but after doing some research, it doesn't appear Plex makes use of the GPU for transcoding (unfortunately) and that's the only thing I initially thought it might be useful for.

I did consider running the Plex server within a *nix VM but just from an educational standpoint, I thought having everything in a different OS might be pretty good but I'm open to playing around with that a bit.

I'll check out ZoL (honestly, I haven't even heard of it!)

 

[ttabbal]

My preference would be to run the file server separate, and not on Windows. If you go with ESXI, FreeNAS or napp-it are great easy to use file server solutions with ZFS support. For Plex, it doesn't really matter, but why burn resources for Windows when a simple Linux will work fine? If you need Windows for something else, sure, but if not, it's that much CPU/RAM you don't spend on the OS overhead. Not to mention licensing.

The other nice thing with separating things out is that if something breaks, you only have to worry about that one service. For example, I have a container running Plex. If that breaks, everything else works fine. Worst case, I reinstall the whole Plex container. There's not a lot of custom configuration, so it doesn't take long. And there's nothing else in there to interfere with it. And with ZFS as the filesystem, I can just take snapshots once in a while to make sure I can always roll back to a known good configuration.

There are pros and cons to everything, of course, so check out the options and let us know if you have questions. I think this forum has experienced users with most things tech/server related.

 

[vl1969]

ttabbal >> have a question for you,
I am in process of building out the home VM server.
my goal is Proxmox VE 4.4 as Host.

VM for File server
VM for Downloads like Deluge, Coach Potato, Sonar etc.
VM ??? for not sure yet what else I will have there, if I can I also will try to setup a VM for SAS Development training (it's a free VM appliance based on CentOS?? but it only comes for Virtual Box so will try to port that to Proxmox KVM )
I only have a single server with a bunch of disks in it. ZFS is only an option for OS drive and VM local storage set. (projected setup 2x120 SSD in ZFS raid-1 for OS and 2x1TB HDD ZFS raid-1 for local store)
all other drives are mix and match of 1TB, 2TB ,3TB and I read that it is difficult to build and maintain ZFS on such mish-mush setup.

so do you suggest to build out a ZFS sets on all this on HOST and run file server with virtual disk on that?
or build the host as planed and do a pass-through of other disks/controller into VM like OpenMediaVault or FreeNas and manage that there?
the other option is to build out BTRFS raid-10 pool (it is perfectly fine working with different disks like that)
and do NFS share into all VMs as needed?

would love to hear from a more experienced people on this one.

 

[ttabbal]

My main server is Proxmox. What I did was run a ZFS mirror as root for the main install. Then set up containers and VMs using that same pool as the main storage. The big storage array is 10 mirror pairs, various sizes. I do the sharing from the main host, because I have a lot of different filesystems (ZFS is a pool, then filesystems on top of that). And passing them all into a VM/Container for sharing was annoying. At the time Proxmox bind mounts were limited in number. I'm not sure that's still the case.

I mostly use containers, as they are a bit more efficient. Containers share the host kernel and such, so you don't have to have a full boot up inside them like a VM does. It's a bit like BSD jails, chroot, Solaris Zones, etc.. Storage can be bind mounted into a container so that it looks like a local disk to the applications in the container. This is nice as you don't have to deal with NFS or other network based sharing for strictly local access. If the application you want to run is not Linux based, you need to use a VM to run whatever OS it needs. And there are some things that don't play well in containers. The only one I've run into is LTSP though.

I would bundle the downloader apps into a single container, but some people prefer to split them up. It depends on how intrusive they are to the system to me. Most of them are somewhat self contained, so I don't mind them sharing. Containers with bind mounts are nice here as these tend to do a lot of large file manipulation, so removing small amounts of overhead in file access can make a big difference you might not see with basic file server duty.

ZFS mirrors will use the smallest disk size in the mirror as the total available space. So if you put a 4TB with a 1TB, you get a 1TB mirror. It works fine, but has the obvious downside that some of the space on the larger drive is not usable. If you can pair up similar sized drives, you can create a pool from a 1TB mirror, and a 4TB mirror just fine. It's not ideal, but it's fine. I'm not sure how BTRFS gets around that. Perhaps just trying to mirror onto whatever space happens to be available rather than by device. I don't personally consider BTRFS stable enough for my data, but it's your data so only you can make that call.

While you can do HBA passthrough on Proxmox, it's more annoying than on ESXI. There is a lot of manual setup. I decided it wasn't worth the bother and used the native ZFS on the Proxmox host. There are some possible downsides, it puts more configuration on the host rather than in a VM, but ZFS stores the important bits on the array itself. So if I had to completely re-setup Proxmox from scratch I only really have to re-setup Samba and NFS settings. And if you back up the config files for those and the containers/VMs, the restore isn't that big of a deal.

I don't know about OpenMediaVault, but FreeNAS is ZFS based, so it has the same limitations ZFS would have on Proxmox.

For my needs, my highest priority is data safety. So if that means I'm inconvenienced by having to buy pairs of drives, I'm ok with that. It helped significantly that I got in on a deal for 2TB drives for $33/ea. Used enterprise drives, but after through testing I only had one to return. I had a similar return rate with brand new drives, so I'm satisfied.