Need Server Build Advice

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
K

Kneelbeforezod

Active Member
Sep 4, 2015
529
122
43
46
I have been thinking about building a server/NAS box for some time now - just never got around to actually doing but i want to do it right hence this post. I have certain requirements/needs to be filled so i'll spell those out so hopefully then you all can tell me how best to go about it.

As hobby I edit and upload news clips from Fox News, CNN, MSNBC, Fox Business Etc and i do may some money at it. Right now I have about 100 Terabytes of video but the drives are just house in external enclosures. What I need is a Server/NAS that will be on 24/7/365 which will be the box to store everything. So i want it to be low poer but also decently powerful I plan on running W7 or 10 and it will also do some MPG to MP4 conversions with handbrake since it takes longer to do handbrake conversions over the network -better to do it locally. I wa thinking of running something like FlexRAID, SNAPRaid or Disparity but this all new to me so there may be better software options. I know nothing abut ZFS.

I need to a way to really never run out of storage since i am recording 24/7. I knmow this will mean i will need to add future box down the line.

Hardware wise i was thinking getting a bare 24 bay or maybe a 36 bay SM server as long as it has the drive trays. Question is what's a good choice for a low power CPU but yet decently powerful for handbrake work. Or should it a dually? Given that it will have a lot of drives, what about the controller - something like a M1015?

And what's the best choice for software?

Any an all tips or ideas are welcome.
 
T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,640
2,058
113
Low power and 100TB+ and recording 24/7 is going to be rather hard to accomplish, especially if you add some redundancy to that storage or entire system in your case since you need it 24/7.

At 100TB and growing I would only consider 8TB drives or larger to help you grow and keep power as low as possible.

Not that it will be cheap or low power... but do you have a need for a backup copy of your data?
 
K

Kneelbeforezod

Active Member
Sep 4, 2015
529
122
43
46
T_Minus said:
Low power and 100TB+ and recording 24/7 is going to be rather hard to accomplish, especially if you add some redundancy to that storage or entire system in your case since you need it 24/7.

At 100TB and growing I would only consider 8TB drives or larger to help you grow and keep power as low as possible.

Not that it will be cheap or low power... but do you have a need for a backup copy of your data?
Click to expand...
A backup would be nice but it also depends on cost and power too. Are you thinking a Hardware raid setup?
 
S

scp

Member
Aug 5, 2015
67
12
8
46
Funny, I was just about to ask a similar question. Do you think the Debian/Merger/SnapRaid is a good solution to finally free me of my Drobo NAS collection? This solution looks really slick as far as being able to add on drives as you go.
 
T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,640
2,058
113
Kneelbeforezod said:
A backup would be nice but it also depends on cost and power too. Are you thinking a Hardware raid setup?
Click to expand...
RAID isn't a backup.

But if you could afford a 2nd system that worked like the first you could mirror your data to it, and if something goes wrong with the 1st utilize the 2nd until you fix the 1st.

I'm not sure how important that existing 100TB or going forward how important your old/archive videos actually are but if you need them then you should think about a backup/archive strategy too.

Depending on uptime importance is what i'd determine if you want to use some sort of RAID in your primary system. It's going to add cost, and power for sure especially with 100TB+ already needing a home.

You also don't want to be at or near capacity with most file systems either so that means having room to grow without hitting your head against that margin too... and with 24/7 and 100TB already that's going to bump you up again.

Do you need your 100TB+ and old videos 'online' ? Accessible "now" ?

You could build a backup/archive system that migrates data from your live system and doesn't run 24/7 to save power, and keep say 50TB online all the time.. maybe the last 7 days? I don't know your needs but I likely would try not to keep all data online :) to save power$$$ and cost to build, etc...

Depending on your system you can use jBOD and power them on/off as needed too if you didn't want to go with a complete seperate system
 
K

Kneelbeforezod

Active Member
Sep 4, 2015
529
122
43
46
T_Minus said:
RAID isn't a backup.
Click to expand...
Right, but i wasn't sure what kinda idea you had so i took a guess.

T_Minus said:
But if you could afford a 2nd system that worked like the first you could mirror your data to it, and if something goes wrong with the 1st utilize the 2nd until you fix the 1st.

I'm not sure how important that existing 100TB or going forward how important your old/archive videos actually are but if you need them then you should think about a backup/archive strategy too.

Depending on uptime importance is what i'd determine if you want to use some sort of RAID in your primary system. It's going to add cost, and power for sure especially with 100TB+ already needing a home.

You also don't want to be at or near capacity with most file systems either so that means having room to grow without hitting your head against that margin too... and with 24/7 and 100TB already that's going to bump you up again.

Do you need your 100TB+ and old videos 'online' ? Accessible "now" ?

You could build a backup/archive system that migrates data from your live system and doesn't run 24/7 to save power, and keep say 50TB online all the time.. maybe the last 7 days? I don't know your needs but I likely would try not to keep all data online :) to save power$$$ and cost to build, etc...

Depending on your system you can use jBOD and power them on/off as needed too if you didn't want to go with a complete seperate system
Click to expand...
I generally am working with video from the last one of 2 days so I don't necessarily need everything available all the time. So 7 days would be plenty. I'd also like to be able access the files remotely.

Any thoughts on W10 JBOD +Parity vs SnapRAID or similar?
 
T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,640
2,058
113
Sorry I'm confused your first post said a box to store everything, and then mentions windows... I thought you wanted to virtualize windows and storage systems now it sounds like you want to run them all under 1 OS is that correct, or are you still considering the options?
 
K

Kneelbeforezod

Active Member
Sep 4, 2015
529
122
43
46
main reason for Windows is so i can run handbrake on the same machine to mpg to mp4 conversions and I figured it would nice with the rest of my network which is all windows. Also i am most familiar with Windows, Windows Server, Visual Studio, stuff like that.

But i am open to other options if there are distinct advantages
 
T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,640
2,058
113
Well, if you run a virtualized environment you can do a lot more with a beefier server :)
IE: 2, E5-2670 V1 with 64GB RAM can you multiple windows and your single storage instance. It won't be "low low" power though and I'm not sure you have a need for making it like that. Just throwing out some more ideas for ya to think over.

HGST or WD RED :) should be great choice.
 
keybored

keybored

Active Member
May 28, 2016
280
66
28
Those new 10TB IronWolf Seagate drives look good but they probably won't come down from their MSRP ($535) for a while. :(
 
wiretap

wiretap

Active Member
Jul 14, 2015
128
88
28
Michigan
With the case of constantly changing large video files, I don't think I would recommend a snapshot RAID style system for the whole thing. That would just be very taxing on the CPU/RAM/disks in order to sync and scrub them with so many large snapshot updates. It would take up quite a bit of time running the task.

In order to avoid the high costs of a quality hardware RAID controller with a boatload of ports, the system could be split up. The massive archive video portion could go on a separate snapshot RAID environment hooked up to a simple HBA (like a few IBM m1015's or similar) Then the latest ~7 days of captured video could go to a hardware RAID smaller set of disks, say just a few TB's (or whatever your volume size needs to be).

The hardware RAID would give greater disk speeds for faster editing and multi-stream HD capture capability as well. If your 7 days of video don't take up a _ton_ of space, you could even use SSD's for this volume to make editing and rendering times much faster. With a snapshot RAID system, you're usually limited to single disk speed. You can however use a tiered system in conjunction with SSD's for increased speed (as cache) until the data is written to the hard disks.

The files that need to be archived could be put into a monitored folder that auto-transfers them to the snapshot RAID portion of the system, ran as a daily or weekly scheduled task.
 
You must log in or register to reply here.
Top Bottom