Home NAS Rebuild -- Supermicro X8 Based

Hi, to everyone I'm rebuilding for the 11-th time my NAS, and VM host, but I have some serious HW limits and want some help!?

Goals:
NAS for SMB, NFS, FTP file sharing across a bunch of OS.
UPnP/iTunes Media Server/ala Plex
Torrent server
VM host for (MusicBrainz VM Server, pfSense or other router OS)
Multi-OS Computer Backup (1 Mac OS 9, 1 Mac OS X 10.4, 2 Win10 Machines)

Hardware at my disposal:
"NAS"
Supermicro SC836 Dual 1300W PSU, TQ backplane
Supermicro X8DTH-iF
1x Intel XEON E5540 2.53GHz
2x Kingston 1600MHz DDR3 4GB DIMM
1x LSI 1064 SAS 3Gb/s 4 port card
2x Gigabit ethernet Dual port cards
1x Sony slim-line BluRay/DVD
1x 4GB SATA SSD (Unused)
1x 500GB SATA OS-Drive
Storage-Spaces 4.5TB JBOD (1x WD Green 2.5TB + 2x WD Green 1TB)
Storage-Spaces 2TB JBOD (1x Seagate Constellation 2TB)

ElCheapo combo case, Thermaltake 650W PSU
Biostar G41

1x Intel Dual Core E5800 2.8GHz
1x Kingston 1333MHz DDR3 2GB DIMM
1x LG DVD-RW
2x Gigabit ethernet Dual port cards
1x 320GB SATA OS-Drive
1x 750GB SATA data drive

Finally the clients machines are:
Lenovo Y50-70 Intel i7, 16GB RAM, 2TB HDD -- Work & Game Laptop
Microsoft Surface Pro3 Intel i5, 8GB RAM, 256GB SSD -- Media Consumption
Amazon FireTV Stick, connected to TV & audio system -- YouTube Player
Sony Xperia XZ1 64GB -- Mobile Media Consumption​

 

Well it depends on how much you want to change but I have a similar system to the NAS and use FreeNAS to do a lot of the same things.

My system is:

X8dt6-F
Dual Xeon E6540
48GB of DDR3
7 x 4tb HGST In a ZFS raidZ3 as I will eventually be expanding to multiple vDev's
Chenbro NR40700 case Chenbro - Products
Mellanox connect-X2


I am using the system to run Transmission with OpenVPN, syncthing, nextcloud, and plex as well as for SMB services to my home network.

You could easily do much the same with ESXI as a hypervisor but you will need to increase the ram and change the LSI 1064 out to something like the LSI SAS 9211-8i https://www.servethehome.com/lsi-sas-2008-raid-controller-hba-information/ and then setup your pool. Running ESXI you could virtualize whatever you wanted and probably run everything from one box. And DDR3 ram is cheap on ebay so max out the board with 4GB modules.

I would however suggest using the e5800 for your router exclusively. IMHO it's better to keep the router on it's own hardware so you run less risk of goofing things up and exposing your nas to the world. You can pull the 750 GB drive and use it elsewhere. I use OPNsense on a L5630 with 4GB of ram and a laptop HDD that is around 300 GB and it has worked just fine.

Using FreeNAS you should be able to run pretty much everything else using jails which is much lower overhead than setting up multiple VM's as well. This will allow older hardware to run for much longer with minimal upgrades. My system has been up and running for almost 5 years now and my only changes since I built it have been the case and adding in a 10GB fiber card for faster transfers with my desktop.

BTW musicbrainz will run just fine in a jail on FreeNAS since it is FreeBSD based. User:Rootwyrm/FreeBSD Server - MusicBrainz Wiki though if you are serving media to clients Plex will be able to handle that quite well. I currently have around 2200 movies, 11985 music tracks, and 4290 tv epiosodes running on mine. I also have some movies that are 3D and 4K which are kept separate from the regular movie files but amount to much less space. The only thing I would suggest is spring for the plexpass if you have multiple users especially children so you can control what they have access to.

I also have some large files stored on my system that I use for gaming. One of the largest is FlightGear which has about 10GB of scenery stored on the NAS. You could probably store all your game files on the NAS and run them directly if you wanted to. And running SyncThing allows me to backup tablets and phones directly to the NAS and I am able to also keep files that I want synchronized across multiple devices like the dog's rabies certificate backup copies of ID's and other important data I want at my fingertips everywhere I go. I have gotten to the point where all my downloads as well as other files are just saved on the NAS and local computers just have the OS and some installations that don't play nice with being put in a place other than a local drive.

The only thing you may end up having an issue with is if you start transcoding to multiple clients but you can easily pick up some faster 5600 series cpus and increase the amount of ram you can work with as well as the power you have on your NAS Intel Xeon X5650 2.66GHz Six Core (BX80614X5650) Processor | eBay The other option is to make sure almost everything is in a format that plays nice with the devices. I have found MP4's with AAC audio to be the best for everything I have come across, MKV's with x264 and AC3 audio works with some stuff but has to be slightly modified by the server and direct streamed vs direct played https://support.plex.tv/articles/20...play-direct-stream-and-transcoding-different/

I should also mention that if you go my route you should pull that kingston 1600 mhz ram and use it elsewhere. The cpu's only support up to ddr3 1333 for the x5600 series and the 5540 you have only supports up to ddr3 1060 Go onto ebay and buy some ECC server pull ram and use it.

 

Good night, Night Shade

Sorry for answering so late...

So the main constraints are HW based and I really mean that, its was really troublesome getting the server down here to Ecuador, South America, it cost me about $350 in shipping and handling just to get it imported. (And that was when I worked in a company that imports computer parts, with a really, really good price "employee discounted").
Aside from the HDD's everything has been imported or does come from my tiny imported parts bin.
I was going to get 128GB of server ram and dual XEON's X5690, a SAS2 8 port + IBM SAS expander & required cables, but again if you think of a pair of XEON cpu's, that cost maybe $200 on US eBay here locally they will be not available or unobtanium ($800+ each one), or the price if imported would be around $300-350.

Here in Ecuador old PC's have crazy prices… don't get me started on server parts and pieces………………
​

It's a long running project I was supposedly going to upgrade this year most components...
What I am running right now is this, and I believe I didn't write well the first post...

• OS is Win 2016 Essentials (this provides SMB, NFS and FTP)
• UPnP/iTunes Media Server is running with "Ampache", I want something like XBMC or Kodi, but I don't like they overwrite and move the files, data and/or metadata. I really like the idea of firing my FireTV and playing whatever I have on the server. (I haven't enough time to configure Ampache very well and the Apache, PHP install is very flakey)
• Torrent Server is being run by uTorrent but for 2500+ torrents it is becoming a chore and really inadequate, to seed keep organized, etc.
• VM Host is Hyper-V, (it doesn't auto start the services, is giving me a big headache)
• Multi-OS Computer Backup MacOS Side is not done, Win is done by Win 2016 Essentials.

So getting out of WinServer, will mean getting all the data out of the array, and rebuilding it on FreeNAS, but I will loose the easy storage expansion....

Really nice piece of equipment you got...!!!

 

Importation sucks unless you have channels to send it through. I have been really lucky with my build and looking at some X5600 series cpu's for around 80 bucks as an upgrade to my E5640's. Then it would be either 8GB or 16GB sticks of ram to swap into the beast and kick the twelve 4GB sticks I currently have to my step daughters system. Not long after I built mine I ended up building a 1U that is similar for my father that he bought drives for and a 1U for my step daughter to use but she is responsible for getting drives for but still hasn't. That is another point where Syncthing comes in handy I might add.

ZFS can easily expand but it's just different. If you are storing a bunch of VM's you would want those on a pool of mirrors. The more vDev's that you have the faster the access so ideally you would have eight or ten drives in four or five vDev's. ZFS Raidz Performance, Capacity and Integrity Comparison @ Calomel.org is a great page for reading up on zfs and some of the information about how things work. The one downside to a ZFS pool is that if you have a pool setup in a raidZ3 with ten drives on a single vDev to expand it you have two choices. Add another ten drives in a second vDev or replace and resilver the individual drives and once all drives are replaced you can expand the pool. So if you want easy expansion you can run multiple small vDev's at a penalty of space or large vDev's and buy a lot more drives to expand the size. You CAN also create multiple pools and you should already know that sharing the same data via SMB and NFS can wreak havoc so two pools would not be an issue. I have a single drive in a separate pool from my main storage for basically unimportant data that I have to process every couple of days and transfer on to the main pool where I want it.

The disadvantage of multiple pools is each one will end up being a separate mount point on the clients which can be a pain at times compared to just having a super sized pool that is easy to access.

The best thing I can suggest is throw together a test box and take it for a spin and see how you like it though. There is a bit of a learning curve but it sounds like you will take to it with minimal problems.

And read up on the pool's and raidZ levels as well. Stripes, mirrors, RaidZ1, RaidZ2 and RaidZ3 are the different types you may work with. Aaron Toponce : ZFS Administration, Part I- VDEVs I should also mention that you can also do some fancy tricks with ZFS in certain use cases that I seriously doubt windows server can handle like intent logs and arc. One of the most interesting things I can do with my FreeNAS is read a file once from the pool and if it is not stored in ARC it will read at the maximum pool throughput. Once that has been done once and it is in the ARC it transfers from the servers RAM. If you have a lot of stuff happening and want to give the spinning discs even more of a breather and have maxed the RAM you can look at L2ARC and add SSD's into the mix as an extra place to store files. Aaron Toponce : ZFS Administration, Part IV- The Adjustable Replacement Cache

And there is also ZIL which can be useful for lots of big writes Aaron Toponce : ZFS Administration, Part III- The ZFS Intent Log

IMHO FreeNAS has so much more going for it storage wise it's hard to even imagine being back on a windows box again. And you can use FreeNAS to run VM's as well so if you must have something that can not be provided by FreeNAS or ran in a jail you can virtualize on the same box though using it does come with some overhead for the virtualization it is a perfectly viable option.

There is also a forum with a lot of the FreeNAS people out there. I was active there but decided it was no longer a place to be for me but there is a lot of write ups and information that can be useful.