Client NAS Build

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

BoredSysadmin

Not affiliated with Maxell
Mar 2, 2019
1,050
437
83
I have raidz2 with a single 12 disks vdev - plenty fast for streaming video. Sync Writes sucked, so I took an educated risk and disabled sync writes. Not too concerned over power loss as I have a UPS and a 20kVA standby generator with auto switch. As async writes are straight to memory - it always fully saturates my network. RaidZ1x2 is basically Raid50, yes it's faster than raid6, but with 16Tb rust, it would be way too risky to go that route.
 
  • Like
Reactions: sboesch

sboesch

Active Member
Aug 3, 2012
467
95
28
Columbus, OH
What's wrong with striping two raidz datasets? I've been doing it for years, but not with anything larger than 4tb disks, what's the caveat with the 16tb disks? I found raidzx2 stripes to be fast as hell, and efficient with the space. Mirrors don't even have as good as performance as raidzx2.
I always use UPS's, I wouldn't dare do otherwise.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
What's wrong with striping two raidz datasets? I've been doing it for years, but not with anything larger than 4tb disks, what's the caveat with the 16tb disks? I found raidzx2 stripes to be fast as hell, and efficient with the space. Mirrors don't even have as good as performance as raidzx2.
You answered you own question. Disks that large are the risk. I can't even fathom trying to do a resliver with 16TB disks. The risk of another drive failing in the vdev during that process is way too high for my liking based on how long it would take.
 
  • Like
Reactions: itronin and sboesch

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
What's wrong with striping two raidz datasets? I've been doing it for years, but not with anything larger than 4tb disks, what's the caveat with the 16tb disks? I found raidzx2 stripes to be fast as hell, and efficient with the space. Mirrors don't even have as good as performance as raidzx2.
I always use UPS's, I wouldn't dare do otherwise.
RaidZ2 nothing
 

itronin

Well-Known Member
Nov 24, 2018
1,233
793
113
Denver, Colorado
Ignoring the ZFS points (which are all well made, well articulated and I agree with them esp don't use Z1).

Some questions:

Is hot/swap and/or blinky drive lights something the client wants or is requiring hence the chassis selection?

From the client's perspective I am sure they feel that they are dropping a large chunk of change today but if they are as much of a DH as you describe what's the growth plan? Can the client describe how fast they grew their data usage?
Possible solutions? Second server? add a JBOD chassis? If the latter plan today for an HBA later - reserve a slot for it.

What's the estimated life-time of this server? What was the lifetime of the current synology solution?
Were you able to talk to them about the cost with a view to the total lifetime of the solution? I have to wonder if that would help with their budget.

My mantra with DH (and exchange users) "If you build it, they will fill it" - and faster than you or they think.

I may have missed it but I don't recall seeing anything about the power protection for this system? What's the plan? Based on the client's usage what is the likelihood of a power supply failure or street power failure impacting data in flight?

Some comments on your design.
I see little value burning the 2 m.2 slot's for boot as you may find a better use for them - even as itinerant high(er) speed storage. ARC isn't likely to help you - In your use case I do not expect a performance improvement from a dedicated zil/slog but I have to wonder if the inflight protection for writes may be desirable? the m.2 slots (2) support 110's on that motherboard so nvme with PLP could be used if you wanted that.
I'd make sure though that those m.2 slots will NOT rob x4 lanes from other slots.

you don't appear to be using the onboard SATA ports. Since this is an mATX motherboard in a large chassis you have TONS of room and with the NORCO you can get the Norco two drive shelf (or 4 x 7mm/9.5mm using velcro) that is designed to mount inside the chassis. Consider the onboard SATA ports for your boot drives and maybe even an expander to go to a single HBA for internal storage.

With the NORCO consider right angle SFF-8087's for the 6 backplane connections you need to make. IMO and in that case the cable management using straight SFF-8087's may impede your air flow *and may* put strain on the backplane boards with the cable bends required, the right angle connectors solve that problem for you. the correct SFF-8087 are pretty easy to find. SFF-8643 to right angle SFF-8087 maybe not so much.

(opinion) I don't see SAS3 buying you anything today or the mid future and I believe you said used enterprise HBA's are okay? Perc H310's and the like are $20.00, replace the thermal paste, bump the firmware and call it a day for the HBA's. Buy an extra and have it ready as part of the package.

I get the "new" part - what about new old stock? For example (and just an example) Tyan S5510GM3NR is about 100.00 shipped off the bay. You could buy 2 for the price of the ASROCK. CPU's will be a lot cheaper too. I say CPU's because for the price of the ASROCK you could buy two motherboards with CPUs for a little over half the cost of the ASROCK+CPU. Downside is memory - that board is limited to 32GB udimm but that may well be enough for your use case.

If size, blinkies, and hot swap are not that big a deal have you considered the 48 bay SM/Chenbro? Yes, you'll have to deal with the drive trays (3d printer and/or get someone to fab them) but it can be found NEW and in budget and redundant power supplies and growth in one chassis...

With the NORCO do purchase a new box of band-aids and have that handy during the build process. We all probably leave a little bit of ourselves in any build but with the NORCO I have found it is a bit more than typical.

Don't get me wrong, I think the NORCO is a fine solution for some scenarios. I feel it breaks down when you try and push it to its max capabilities (ie. a fully populated chassis, lack of redundant power supplies, trying to use it for production 24x7x365).
 
Last edited:
  • Like
Reactions: Discountsocks

SRussell

Active Member
Oct 7, 2019
327
152
43
US
What I don't understand is why the clients wants someone else to build something if he knows enough about FreeNAS.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
What I don't understand is why the clients wants someone else to build something if he knows enough about FreeNAS.
Maybe he's too busy or doesn't enjoy the building part and only the "using" part.

I came to this realization myself for almost all my hobby's there's a certain point of "building" vs "enjoying" and if I Cross that ratio it can ruin the fun, or the hobby entirely.

My 02 \ guess :D
 

msvirtualguy

Active Member
Jan 23, 2013
494
244
43
msvirtualguy.com
For some of these very reasons I went with a beefy QNAP and haven't looked back. Expansion of ZFS is whyI shy away, unraid is ok but still limited from a perf perspective and I figured why do I want to manage the software stack when someone else can do that for me and I have great expansion. My unit is about a year and a half old now but it's really not showing it's age and performance is great. Does Caching w/dual NVMe, Flash Tiering and expansion has been simple and painless.

When my friends or family ask me to "build" i'm like get a purpose built NAS, synology, QNAP and while I agree, it's not as fun, etc, it's been so much better for quality of life.

I've had them all, FreeNAS, Unraid, etc and the QNAP TS-1685 I bought has been by far the most elegant and simplified solution. There is a lot to be said for simplification from a implementation and day 2 perspective, support..etc.
 

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
For some of these very reasons I went with a beefy QNAP and haven't looked back. Expansion of ZFS is whyI shy away, unraid is ok but still limited from a perf perspective and I figured why do I want to manage the software stack when someone else can do that for me and I have great expansion. My unit is about a year and a half old now but it's really not showing it's age and performance is great. Does Caching w/dual NVMe, Flash Tiering and expansion has been simple and painless.

When my friends or family ask me to "build" i'm like get a purpose built NAS, synology, QNAP and while I agree, it's not as fun, etc, it's been so much better for quality of life.

I've had them all, FreeNAS, Unraid, etc and the QNAP TS-1685 I bought has been by far the most elegant and simplified solution. There is a lot to be said for simplification from a implementation and day 2 perspective, support..etc.
The problem with prebuilt is little control over CPU/GPU. A Xeon D would be no where near enough for my and many others' needs. If the purpose is for such large media library chances are it needs to support a lot of simultaneous transcoding.
 

msvirtualguy

Active Member
Jan 23, 2013
494
244
43
msvirtualguy.com
My QNAP can handle a vast array of GPU's. You are correct, most don't. but there are some that do but I get it, i'm sure there are reasons where these types of solutions don't fit. It's all about he requirements and working back from that.
 
Last edited:

IamSpartacus

Well-Known Member
Mar 14, 2016
2,515
650
113
My QNAP can handle a vast array of GPU's. You are correct, most don't. but there are some that do but I get it, i'm sure there are reasons where these types of solutions don't fit. It's all about he requirements and working back from that.
Being able to handle GPUs is huge. I was not aware of that. That definitely makes it a very flexible option. That cost though...
 

BeTeP

Well-Known Member
Mar 23, 2019
653
429
63
The Ryzen 3 1200/4Gb model in the same chassis starts from $2500. Still overpriced for a cheaply made consumer gear.