unRAID Behemoth Build (Stage 2)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
Howdy guys, lets start with a couple of teaser pictures (ignore the clutter and mess I have 2 kids under 5 :)):




I've been floating around the forum for a few months now offering and seeking advice when I can.
I even had the opportunity to meet @Patrick at the HPE conference in June ~ really smart, solid dude, and is as awesome as he seems.
I've finally found a groove for my NAS and part selection for my build for my first 'real' STH post.

I've been researching and hunting for a solution to replace my QNAP NAS. While the QNAP is great it was really pricey and I'm not getting the value out of it I was hoping for. While trying out various OS'es and collecting parts the past few months I finally settled on unRAID. I like the drive spindown, ease of use, single drive expand-ability, and dataloss handling in catastrophic scenarios ~ particularly since a large portion of the usage is for Plex so performance is back burner to efficiency.

So lets go with the parts list next... and yes, that is 15 hard drive slots plus the 3x 5.25" bays :cool::
Supermicro X10SRH-CLN4F (3008 flashed to IT-mode)
Xeon E5-2660 v3
Crucial 8x 16GB DDR4-R 2133
4x HUSML4040ASS600 Hitachi 400GB SAS SSD (Cache/VM Storage)
(The above parts were purchased from other members :), shout out to @Esxi Novice, @sfbayzfs, @Spotswood, and @uppie1414)

Corsair 750D (w/ 3 extra HDD enclosure bays) ~ 20 data drives 4 cache drives
1x CSE-M35T-B1 (5x 3.5")
Dell H310 flashed to 9211-8i IT-mode ~ Snagged from work
Dell 9206-16e
Mellanox ConnectX-3 CX322A Dual port 10gb SFP+
Nvidia Quadro P600 ~ Passed through to Plex VM ~ (Upgrade to a P2000?)
4x x500GB Samsung Evo SSD (raid 10 brtfs)
2x 8TB WD Ultrastar SAS DC HC510 (parity)

18x 8TB WD White Shucks (have 15 currently)
PSU - RM 850x

Custom build DAS case made by @Spotswood ~ 10 data drives
1x CSE-M14TQC (4x 2.5")
2x CSE-M35T-B (5x 3.5")
4x 400GB Hitachi SAS SSD
10x 8TB WD White Shucks
PSU - Corsair CX600M

Only missing a few pieces bolded above (I'm waiting for the BF/xmas deals to pick up some more easystores).
All said and done should be just over 200TB usable :D.

I'll be posting more pictures and updates as I start moving the parts and pieces into the case.

Comments, suggestions, and feedback is welcome ~especially around PSU/parity drive selection
(the shucks are kinda set in stone as they're hard to beat on price).

Cheers!~
 
Last edited:

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
First PSU maybe look at the EVGA's too? They've got some good units and it'd let you price shop more.

Maybe do CX3 Pro? I'd get dual 40G and downgrade to 10G using an adapter. That'll let you upgrade in the future very easily

I'd just want to know how loud this is gonna be.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Nvidia Quadro P600 ~ Passed through to Plex VM ~ (Upgrade to a P2000?)
If you'e running unRAID why not run Plex in a docker container instead of a VM? As for the P600, is it the 2GB of 4GB version? How many (and what kind) of transcodes are you looking to support?
 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
First PSU maybe look at the EVGA's too? They've got some good units and it'd let you price shop more.

Maybe do CX3 Pro? I'd get dual 40G and downgrade to 10G using an adapter. That'll let you upgrade in the future very easily

I'd just want to know how loud this is gonna be.
I’ll lookout at EVGA also, I like corsair because I can buy spare molex lines easily for the psu, and Seasonic for the extra 2y of warranty.

I’m barely using 1/4 of 10g when doing a transfer, 40g is a long way off, I’ll keep it in mind though.

Aiming for sub 35db :), that’s why I went the consumer case instead of a hot swap 4u chassis.

If you'e running unRAID why not run Plex in a docker container instead of a VM? As for the P600, is it the 2GB of 4GB version? How many (and what kind) of transcodes are you looking to support?
Ease of migration, I was running it in esxi previously and so my vm direct cloned easily. I’m also lazy and its familiar :p. I have a dozen docker containers running other things though, including my unifi controller and NVR.

P600 is a 2gb, but I’m not even using the mod, only once or twice a year 2 transcodes are going simultaneously (1080p or less), the P2000 upgrade is more epeen related :p.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
I’ll lookout at EVGA also, I like corsair because I can buy spare molex lines easily for the psu, and Seasonic for the extra 2y of warranty.

I’m barely using 1/4 of 10g when doing a transfer, 40g is a long way off, I’ll keep it in mind though.

Aiming for sub 35db :), that’s why I went the consumer case instead of a hot swap 4u chassis.


Ease of migration, I was running it in esxi previously and so my vm direct cloned easily. I’m also lazy and its familiar :p. I have a dozen docker containers running other things though, including my unifi controller and NVR.

P600 is a 2gb, but I’m not even using the mod, only once or twice a year 2 transcodes are going simultaneously (1080p or less), the P2000 upgrade is more epeen related :p.
Yea stick with the P600. You've still got some headroom. Is your VM Windows or Linux? I only ask because I also migrated from Vmware (4 node vSAN cluster) to a single Unraid server and moving my Plex server from Ubuntu Server to Unraid docker was simple and quick. Managing Plex via docker vs a VM opens up a lot of options in Unraid and I couldn't imagine migrating my services back to VMs now.
 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
Yea stick with the P600. You've still got some headroom. Is your VM Windows or Linux? I only ask because I also migrated from Vmware (4 node vSAN cluster) to a single Unraid server and moving my Plex server from Ubuntu Server to Unraid docker was simple and quick. Managing Plex via docker vs a VM opens up a lot of options in Unraid and I couldn't imagine migrating my services back to VMs now.
Ubuntu 18.04 headless, I’ve played with it in docker before and don't mind it, but I’m also trying to learn linux better, I can get by but I bumble through alot of things. I plan on making a backup plex that I’ll like be tossing in docker though. Big thing I didn't want to lose was the previously played and on deck info etc.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Ubuntu 18.04 headless, I’ve played with it in docker before and don't mind it, but I’m also trying to learn linux better, I can get by but I bumble through alot of things. I plan on making a backup plex that I’ll like be tossing in docker though. Big thing I didn't want to lose was the previously played and on deck info etc.
I hear you. Watch status and everything comes over fine btw. I have a backup sever I can run Plex on during downtime and all I do is rsync the appdata over in a manner of minutes and I'm up and running with no one the wiser.
 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
I hear you. Watch status and everything comes over fine btw. I have a backup sever I can run Plex on during downtime and all I do is rsync the appdata over in a manner of minutes and I'm up and running with no one the wiser.
Long term my plan is to have a second unraid box to act as a backup of my main (with less space or at least alot less but larger drives) and I’ll run my backup plex on that. My system is currently in a rosewill nighthawk 117 (both of the cases came from craigslist for $40/ea :D) and that nighthawk will become the second unraid box likely.

Thats a good idea on the rsync to the backup do you just take the whole folder and overwrite it? Or wahts your process? (I want mine to run a second instance so if I can’t get to it, the backup can be used without action)
 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
Stage 2


Chunk it all in the case! Edited the parts list a bit in blue in the original post.
So part of the changes I opted for a pair of DC HC510 SAS drive I picked up from @redeamon that I'm running preclear on now, I'm hoping this will speed up the parity check a bit so I can justify running it more than once a month, 19 hours is just too long.
Gonna use the current parity disks for data.

I'll be bending the 5.25 cage slats to fit the 5x 3.5' enclosure next and installing the SSD on the back side of the case (though I'm not sure I wanna take pictures of that, its a rats nest of cables).
 
  • Like
Reactions: redeamon

redeamon

Active Member
Jun 10, 2018
291
207
43
I went down this route about a 2 years ago (unRaid). Eventually I went esxi with a Ubuntu ZFS data server and I couldn't be happier. The performance is rock solid and it's just so easy to backup/restore with ZFS.
 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
I'm going the opposite direction, I was on esxi with a R420 and a qnap NAS previously (moved the vms to unraid but still on the qnap for data and transitioning to the unraid box).
Got tired of the heat, noise and power generated by the rack servers.
The QNAP is still nice but definitely overly pricey and I think I could resell it and build a better, faster, and larger unraid box as a backup for less cost.

My issue with ZFS/freenas when evaluating was having to add X drives at a time (likely 6 in a vdev). I much prefer unraid/qnap that allow you to add one disk at a time so I don't have to drop a grand on hard drives or replace with larger drives one by one (which I can still technically do with unraid).
That comboed with the multiple drive failure reduced risk, where I only lose the data on one disk if multiple drives failed (beyond the two parity) and the lack of need for high performance for plex/media streaming.
The highest performance need is the VMs and having them sit on a raid 10 ssd cache is performance enough.
 

redeamon

Active Member
Jun 10, 2018
291
207
43
That's true, unRaid does have a sweet spot when used in the way you described.

To avoid the ZFS vdev issue as described above, you'd have to go with Raid10 which is expensive. ZFS is an all or nothing approach.
 
Last edited:
  • Like
Reactions: nikalai

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
Then I'm wasting about 80% more space was my issue with that.
I may still go with a freenas build down the road just to play with it and try it out more since its free, but overall just wasn't the best solution for my use case.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
ZFS IMO makes zero sense for a media storage server. Not sure if that's the point of this build or not as I don't recall the initial build thread.
 

redeamon

Active Member
Jun 10, 2018
291
207
43
Let me know if you do. I've played around with unRaid and ZFS quite a bit. You can go with Ubuntu ZFS and patch it up, it doesn't have to be FreeNas if that's not your thing.
 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
ZFS IMO makes zero sense for a media storage server. Not sure if that's the point of this uild or not as I don't recall the initial build thread.
Correct 90% use case is media storage, secondary is running VMs that use said media storage, remainder is general backup/archive storage.

Let me know if you do. I've played around with unRaid and ZFS quite a bit. You can go with Ubuntu ZFS and patch it up, it doesn't have to be FreeNas if that's not your thing.
I'll keep that in mind thanks, I'm a fan of appliance/AIO that apply to the OS like freenas.
If I do ZFS on ubuntu I'm going to be inclined to keep it up to date and have to restart my storage like once a week :p.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
Correct 90% use case is media storage, secondary is running VMs that use said media storage, remainder is general backup/archive storage.
Yea that's pretty much my exact use case. Unraid is fantastic for that. What containers are you running/planning to run on this bad boy? This is what I've got running currently.

 

Spartacus

Well-Known Member
May 27, 2019
788
328
63
Austin, TX
:eek: like only 2/3 of those those pretty much, I'm trying to get rid of unifi-video, only have 1 camera left for that (just going ring all around so I dont have to do local storage).
I migrated all of mine from esxi, so my containers are in a Ubuntu VM still and I've been lazy so far on migrating them into direct containers.