[Advice] DIY NAS with 8-bay for home streaming

[jfromeo]

I am planning to build a DIY NAS with 8 hotswap 3,5" bay for home streaming with the following needs:

• Use: 24/7 Homeserver for streaming music (flac), videos (mkv) and games (7z)
• ISP Connection: 300MB/300MB FTTH
• Budget: <1500€ (HDDs not included)
• Hard disks: 8 x Seagate Archive HDD 8TB SATA3
• RAID: None
• SO: FreeNAS 9.3 x64

The server must be capable of transcoding 4K H.265, Flac and decompress 7Z on-the-go. It must be enough to serve to at least 2-3 LAN devices simultaneously.

The main objective is to build the most compact and efficient possible 8-bay system with decent computing capabilities, as well as to be upgradeable and last at least +5 years.

I have almost decided the components:

CASE: U-NAS NSC-800
8-bay hot-swappable. 2x120mm fan holders. 1U PSU. PCI-e riser. Front USB 2.0 port.

PSU: Seasonic SS-500L1U
500W. 80+Gold. "Skylake" Ready.

MB: AsRock Rack C236 WSI
C236 chipset. Mini-itx. Xeon E3 v5 compatible. Dual channel DDR4. 8 SATA 6GB/s. No IPMI.

CPU: Intel Xeon E3-1260L v5
45W TDP. No iGPU (less TDW)

RAM: Crucial 2x16GB DDR4 ECC
Dual channel. ECC. 1.20v

HDD: Seagate Archive 8TB
128MB cache. 3 year warranty. <1.0W idle. Good ratio €/TB. 6 platters.

USB: Sandisk Ultra Fit 32GB
32GB. USB3.0

COOLER: Thermalright AXP-100
Passive. <47mm height

And I have the following doubts.

1. Freenas, Nas4free or Openmediavault? For 24/7 homestreaming with no RAID.
2. Would you make a bigger investment in order to get a mini-itx board with support for Xeon E5 v3 (like this one AsRock Rack EPC612D4), Quad channel So-DIMM ECC DDR4 memory, IPMI, and an additional SAS/SATA3 expander card to get 8 SATA ports?
3. Would it be a 128GB USB3.0 flash drive overkill for this system? Is 32GB future-proof enough? Difference in price is from 10€ to 40€.

Thanks in advance.

 

[Chuckleb]

How were you going to assemble the drives and what does the read/write usage look like? The archive drives are great for streaming data to it and from it, not as great for rewrites. I am testing a few and use them to dump data to (once) and retrieve from. In an array of any sort and you end up with lots of rewrites of data.

When I placed my order (60 drives), the Seagate rep wanted to be sure we understood the implications of SHM.

 

[nephri]

I would not use "Seagate Archive" except for "cold backup archive". Prefer 4 or 6To like HGST, WD Red NAS or other at your convenience.
You can go to the "enterprise class" route as well for the HDD.

 

[nk215]

If you want to transcode 4K video to 2-3 devices at the same time (you didn't say what bps you want), you'll need more than an E3-1260L. Transcode 4K at 30mbps needs an i7-5930k for 2 devices.

 

[CreoleLakerFan]

1. Freenas, Nas4free or Openmediavault? For 24/7 homestreaming with no RAID.
2. Would you make a bigger investment in order to get a mini-itx board with support for Xeon E5 v3 (like this one AsRock Rack EPC612D4), Quad channel So-DIMM ECC DDR4 memory, IPMI, and an additional SAS/SATA3 expander card to get 8 SATA ports?
3. Would it be a 128GB USB3.0 flash drive overkill for this system? Is 32GB future-proof enough? Difference in price is from 10€ to 40€.

Thanks in advance.

I have a NAS built on the NSC-800.

The PSU you have selected is overkill. I have a Xeon E3-1220L in my NAS, and with 8 4TB green drives with "medium" APM settings it idles at ~69W. My drives pull 4-5W each in idle, if your drives truly idle at 1W you are going to use even less power. I have a 300W SS Gold 1U PSU in my build, and even it is overkill. Also, you will need a 1U/FlextATX bracket to mount the PSU into the chassis. They can be found on ebay for a pittance.

The drive cages are pre-wired with two 4-1 reverse breakout cables, so unless you want to take the thing apart and run individual SATA cables to the motherboard you will need a HBA/board with two SFF-8087 connectors. And I do mean "take it apart;" to access the drive cage connections the rear plate must be completely detached from the rest of the chassis. If you run 8 individual SATA cables you are going to restrict drive airflow - the cooling fans are behind the drive cage and the only place to run the cables is between the cage and the fans.

Speaking of which, the cooler you have selected is not going to fit. The height of the cooler is 47mm but that does not account for the height of the fan. The board mounts vertically on the left side of the chassis and it is TIGHT in there. I use a Silverstone NT07-115X. It's 23mm, and the fan rests beneath the cooler fins. It's rated for up to 65W - your CPU is 45 ... should not present a problem.

If you do go the HBA route, consider selecting a CPU with an iGPU or a board with IPMI or a serial port. My E3-1220L doesn't have an iGPU, and it was kind of a pain to build the system, then swap out GPU for HBA and finish the configuration via web console. It's even more of a pain when something goes wrong and I need local console.

32GB is plenty for an embedded OS like the ones you are considering. FreeNAS and NAS4Free are built on FreeBSD, OMV is built on Debian Linux - if that sort of thing makes a difference to you. FreeNAS has a notoriously poor support community, which is a minor annoyance but doesn't detract from the functionality and stability of the product. All three are solid choices - I would recommend you look at available plugins and see what appeals most to you.

 

[CreoleLakerFan]

Why 32GB of RAM? My system has 8GB of RAM with 8 x 4TB in RAIDZ2 and it can saturate dual 1Gbps links without breaking a sweat. For home use, 32GB is probably overkill, unless you are planning on doing more with the system than you have stated.

 

[EffrafaxOfWug]

Ditto nephri's comment on the archive drives, or any other SMR drives for that matter - they're really only intended as WORM media and I don't think they'll really work with any degree of performance in a RAID array. Or are you planning on having them presented to freenas et al just as JBODs...? Personally I'd stick to bog-standard platter-based drives. Storage Review had a good rundown of how the drives behave; there's software in development to mitigate these deficiencies but it's a long way from production yet.
Seagate Archive HDD Review (8TB) | StorageReview.com - Storage Reviews

Without an integrated GPU and no IPMI presumably you're not going to be using an HBA and will be putting in a (temporary?) GFX card in the PCIe slot to do the installation...?

PSU - I think you could get away with a considerably lower power one and I don't think the 500W will fit in the NSC 800; I'm using the 300W in this enclosure with eight 3TB drives, HBA, two SSDs and an E3 v2 and even at boot/spin-up time or during a rebuild the server never draws more than 150W.

Edit: ninja'd by CLF

 

[EffrafaxOfWug]

Longer post now I'm not shooting out of the door at work, although now it looks like I'm replying to myself.

ASRock have a "server" version of the C236 board in the pipeline, product page here. Main difference is that it has the IPMI chip which is *well* worth the wait and the money IMHO... but of course it only comes with six SATA ports.
ASRock Rack > E3C236D2I

Had completely forgotten the NSC-800 comes with 2x8087 ready wired for the backplane - and yup, rewiring that thing with SATA -> SATA cables will be a major exercise in cable management in a case that size. When I did my builds in the NSC-800 I rewired the backplane like this and as you can see from this there's very little room for "fat" SATA cables. Personally I'd go the HBA route with something like the trusty M1015, but it'll mean an extra 7-10W on your power budget.

Cooler-wise the Noctua LH-9i is basically perfect for this case if quietness is more important than cheapness. My other didn't-care-about-the-noise build used this HSF, it's quiet enough and small enough but still uses those bloody push-pins, the one CLF posted looks much better in that regard. Regardless of whether the Thermaltake would fit or not, I don't think there's enough airflow in this chassis to trust to a passive cooler - especially if you plan on populating the first drive slot.

CPU-wise I don't have any experience of streaming transcodes of x265... but given that I struggle to get realtime encoding on my 6-core xeon E5 1650v3 for 1080p content I think you're going to be hard pressed to achieve anything like realtime transcoding of 4k H.265 (unless you mean transcoding from H.265 into H.264...?). Streaming FLAC or anything to do with 7Z shouldn't cause it to break out in a sweat (although of course it's pretty easy to configure 7zip to bring a server to its knees if you're doing very heavy-duty compression or you need >100MB/s compression speeds). xz does the same thing for me on linux and compressing big files like disc images can take forever.

32GB is an interesting amount... it's overkill for 90% of all things a NAS would conceivably be used for, but a bit too little to be used for things like virtualising lots of other servers (disclaimer: I run 3-4VMs on my NAS in 16GB of RAM but those are minimal linux builds that'll run perfectly happily in 256MB of RAM). By all means buy it if you can afford it but I don't think you'll need-need it.

 

[Lost-Benji]

I can see the use of the RAM for the FreeNAS ZFS side. I don't do transcoding as I just get the correct version in the first place and direct stream it. However, I haven't seen any mention of using a GPU to do the work?????

I personally would be on a Windows environment but then look at a Media Server app that uses GPU rather than CPU.

 

[EffrafaxOfWug]

It's probably very much the topic for the HTPC thread, but all the GPU-assisted encoders I've seen have given pretty poor quality encodes for a not-that-great improvement in encoding speed (15-20% was a figure I seem to remember from a year or so back)... combined with almost no servers having GPU resources of any shape or form I don't think it's the right path to go down.

Intel QuickSync however... that's another kettle of fish as it gives very good performance at rather good quality (it's not as good as x264 by and large but I would hazard a guess most people who stream video from their servers at home don't mind about that too much) but IIRC it'll only encode H264 and VP8 and I've no idea if any media servers support it (never really found any point in them). It is however supported on the E3 v5's I think.

 

[gigatexal]

transcoding on the fly h.265 is a tall order no? what chips can do this these days?

 

[RichardDickJohnson]

Where did you find that NSC-800 case for sale? Don't see anyone that actually sells it.

 

[CreoleLakerFan]

Where did you find that NSC-800 case for sale? Don't see anyone that actually sells it.

You can only source the chassis direct from the manufacturer:

U-NAS NSC-800 NAS Server Chassis

 

[jfromeo]

Wow! Many thanks for the answers. I see I was completely wrong about my build, not only for technical reasons but for space limitations too.
I have read throuhfully all the pieces of advise, and I have reformulated the build:

CASE: Silverstone DS380
8-bay hot-swappable. 3x120mm fans. SFX-L PSU. Front USB 3.0 port.

PSU: Silverstone Strider SX500-LG
500W. 80+Gold. 120mm fan. SFX-L

MB: AsRock Rack EPC613D4I
C613 chipset. Mini-itx. Xeon E5 v3 compatible. Quad channel DDR4 ECC. 4 SATA 6GB/s. IPMI. Dual LAN.

CPU: Intel Xeon E5-2620 v3
85W TDP. 6 cores (12 with HT). No iGPU (less TDW)

RAM: Crucial 4x8GB DDR4 ECC SO-DIMM
Quad channel. ECC. 1.20v

HDD: 8 x Seagate Archive 8TB
128MB cache. 3 year warranty. <1.0W idle. Good ratio €/TB. 6 platters.

SAS: LSI 9300-8i
PCI 3.0. 8 x 12Gb/s SAS/SATA ports

SSD: ???

COOLER: ???

Main doubts are now arond the system disk and the CPU cooler. I have "upgraded" the chipset/platform (LGA2011-3) and processor (Xeon E5 v3), as long with the SAS controller for a cleaner installation (and because the m-ITX board only comes with 4 SATA ports). I have lowered the requirements; only 4K H.265 transcoding at a time.

 

[gigatexal]

here's some info on decoding h.265 Preparing for HEVC, the next great video codec

 

[EffrafaxOfWug]

CASE: Silverstone DS380
8-bay hot-swappable. 3x120mm fans. SFX-L PSU. Front USB 3.0 port.

Not a big fan of this case personally - bought one for a secondary server that didn't have the space constraints my main one does and it didn't really feel half as solid as the NSC-800, was noisier and not as well cooled as it uses a weird fans-blowing-into-the-side-of-the-drive-cage-but-not-really air flow method and the drive trays were rattly. Sent it back and bought another NSC-800. I should note that my space and noise limitations are probably much tighter than yours.

PSU: Silverstone Strider SX500-LG
500W. 80+Gold. 120mm fan. SFX-L

500W still massive, massive overkill. IMHO if you stick with the silverstone case you should see if you can find someone selling the Seasonic 300W SFX Gold. Haven't sampled it myself but I've yet to run into a Seasonic product that wasn't better than the competition.

MB: AsRock Rack EPC613D4I
C613 chipset. Mini-itx. Xeon E5 v3 compatible. Quad channel DDR4 ECC. 4 SATA 6GB/s. IPMI. Dual LAN.

You're paying a lot here for a one-of-a-kind motherboard, E5 xeon and ECC SO-DIMMs... have you totted up what an 8-core Xeon-D platform might set you back in comparison?

HDD: 8 x Seagate Archive 8TB
128MB cache. 3 year warranty. <1.0W idle. Good ratio €/TB. 6 platters.

Still don't think you should touch these drives with a 10ft finglonger for a general purpose RAID array, at least not for a few years when the SMR dust will have settled... whislt with some software tweaks they should perform well enough in occasional-burst sequential RW operations, for things like an array rebuild they're going to be glacially slow as far as I can work out (especially for parity RAID which is much more randommy [technical term] in its write/rebuild patterns than RAID1 or RAID10).

If you want 8TB drives without SMR your choices are:
Ultrastar He8 | HGST
Seagate 8TB Enterprise NAS... once they're actually released.

It's at this point someone more knowledgeable than me might jump in with a quick word on various drive-pooling technologies that might be more compatible with SMR drives...

What constraints are you living with storage-wise? What's your minimum-size array as of day 1 (e.g. your current dataset plus 50%) and your expected rate of increase over the projected lifespan? Personally I would be judging my storage needs based on 6TB drives for at least the next 18 months since they're the largest commonly available size that doesn't touch SMR... I think if you project yourself needing 30TB or more (approx. size of 8x6TB in RAID6) in the next 5 years I would look at enclosures with more than eight drive slots so that you can expand your array with additional drives (much cheaper than replacing eight drives with slightly biggerer ones).

SAS: LSI 9300-8i
PCI 3.0. 8 x 12Gb/s SAS/SATA ports

Better IMHO to get an M1015 from fleabay or similar and reflash it to the 9211-8i IT if you're feeling slightly adventurous... much cheaper than buying a branded LSI and if you're just using platter drives there's no need to go PCIe 3. Last time I bought stuff it was cheaper to buy three M1015's than it was to buy a single branded 9211-8i...

I have lowered the requirements; only 4K H.265 transcoding at a time.

Key point here isn't how many streams you're transcoding to - it's what direction you're transcoding in I assume you've got 4k H.265 video that you wanted transcoded into 4k H.264 video in realtime...? Not bothered doing any myself yet but I'll try some 4k x264 benches at some point, but based on my back-of-a-fag-packet calculation (and ignoring quick sync) I think that a 4k 1080p realtime transcode done at non-insane settings should be theoretically doable. Realtime x265 is at least several years away, at least as far as not using a 2P-server or bonkers many-cored Xeons.

My two groats...

 

[EffrafaxOfWug]

Not offtopic, but an aside into the transcoding aspect of your proposed build.

As an experiment just pulled down a 4k render of Big Buck Bunny and tried encoding it on my E5-1650v3, that's six SMT-enabled cores at 3.5GHz plus change from turboingisinations. Used MeGUI for the encoder wrapper, x265 1.8+2 (LigH) and x264 v2638 [current MeGUI unstable versions], no crop and resize, CQ 21.5 and medium preset. CPU hovered in between 60-90% utilisation for both encodes (i.e. SMT was doing it's thing pretty well)... some of that eaten by the 32bit AVS > 64bit x265 shim (MeGUI still not got a 64bit version of avisynth) and we get;

4k H265 30fps > 4k H265 30fps: 5.94fps, 6617kB/s
4k H265 30fps > 4k H264 30fps: 12.92fps, 11386kB/s

I'm sure, like wot 'appened with x264, that x265 will get substantially faster over time (first time I tried out x265 I think my 2600K managed about 5fps on SD content) but I don't think all the tweaking in the world is going to 4k transcoding performing at anything like realtime for the foreseeable except on substantially more powerful hardware.

 

[Lost-Benji]

I do concur with what Wug's has said, the mainboard is a little exotic and I personally don't like the ASRock stuff as every one of their products has been an issue of some kind. The choice of HBA, much better and future proofed. The drives, I too a wary of Seagates, I feel a WD offering is better suited.

Now, for the last questions, Cooler will be anything that will fit in the chassis. There are some good little ITX friendly ones but for now, sticking with stockie until you get it going and have more time for R&D. The OS drive, an Intel 535 series SSD will do a sweet job. No need for RAIDing it, just load OS, set it all up and make a backup config. If shit goes bad, replace and push config back.

 

[pLu]

As others have pointed out; don't buy SMR drives for a NAS. The WDC Red drives are probably a good pick but I'd buy HGST for faster drives.

ECC RAM is a must in a serious NAS. IPMI is really nice to have. An overpowered PSU is inefficient.

 

[jfromeo]

Thanks again.

I'll have a look at the 8-core Xeon-D option, wait for some 8TB NAS drives from WD (Red) and HGST. And I will stick to 4K streaming, given the computing power needed.

Just another question about the case. I don't manage to find any mini-itx hotswappable case like the U-NAS NSC-800 or DS380 but for 12 drives, instead of 8.

The Synology DS 24XX is perfect. Anyone knows the source? Or is it exclusive from Synology?

Regards.