Slloooooooow progress. I'm not even planning on buying hardware until summer 2017, want to have all my ducks in a row for what hardware to buy first. (also wont have time to configure/test/etc)
This is a place for public feedback on "the whole system". I'll have a "topic or two to solve first" before wanting to jump too far ahead, but if i'm designing myself into a corner here's where to give me a warning/correct gross misunderstandings I may have.
For instance until i've hammered down the SAS Expanders and PSU's I wont even be shopping for motherboards yet. I have separate topics for that/discussing specific topics elsewhere, but i'll be compiling/updating critical parts to this thread when they get narrowed down.
Build’s Name: The Poor Man's Drobo (named because Drobo-like progressive upgradeability will be designed on the lowest possible budget while actually exceeding Drobo performance, capacity, protection levels, and data recoverability/no proprietary formats. Designed for disks to be added, upgraded in size in place, and primary hardware to be upgraded over time as well. Zero concern for cosmetics, willing to tolerate some increased hassle at the beginning to save money for drives like lacking hot swap bays and server rated hardware.)
Operating System/ Storage Platform: TBD, probably virtualized Linux with SnapRAID as primary storage array
CPU: TBD at least quad core to start
Motherboard: TBD probably consumer level at first
Chassis: 2-4 common inexpensive probably even used ATX cases to start (with 8-11 effective bays each, so 3x11 or 4x8) literally bolted together to keep cost overhead per drive down, as it is acceptable to power down server to change drives for now, option to upgrade to hot swap trays later. (when I have the $ for a fancy 36 bay hot swap and can put motherboards in rackmounts i'll be happy - first I have to get enough data online though)
Drives: minimum expect 16 hard drives to start (common 3.5 SATA some decased from USB I already have) of 3-8TB sizes, expecting 19-21 hard drives + 3 opticals as a likely configuration, max configuration allowed for on this build about 29-30 hard drives+3 opticals. External Ultrium LTO6 drive. One hot swap tray to ease loading data into or out of the array. (ie copying a failing drive to its physical replacement) 1-2 drives treated as hot spares/any indication of failure while i'm out of town results in a drive copy command to reconstruct data and if successful pointing the SnapRAID array to use the hot spare until I can come home and physically replace it. (failure to copy means a SnapRAID recovery has to run of course to rebuild it thats what its for)
RAM: TBD prob 24-32gb to start mostly for virtualization needs
Add-in Cards: still deciding which SAS HBA and SAS Expander cards to use but will probably be using one 24port to 36 port Expander card for drives, though triple HBA isn't ruled out. Eventually 10gb Ethernet HBA when I get a 10g switch.
Power Supply: Multiple PSU's (2-4x) each powering separate drivesets, and able to power off a stripe of drives that wont be needed for awhile (under server virtualization/consolidation some are only intended to be accessed by one session) and force a form of staggered spinup whether the drives want to or not. (since SAS and SATA both tolerate hot swapping shouldn't bother them at all long as I don't do it before data is done writing) Also reduces cost-per-reliable-watts as 1kw and above PSU's seem to climb rapidly in price.
Other Bits:
GOAL:"minimum overhead cost per drive" maximized to what a single common 120vac 15-20amp house circuit allows. This seems to be about 32-33 drives. I'm aware more is possible but at increasing cost-per-drive or attention to detail (choosing specific drives over others/cant use "any") so if I need that much data i'll be splitting into two fulltime servers possibly even before I reach the max configuration. Though i'm hoping 32x8tb 256tb raw is enough for anybody. Wanting a MAID like configuration. (array of idle disks to save power and because it's no different than powering up manually the drives I want to use to serve data - some will sleep more than others and spent all night off anyways) Has to use drives I already have til they die despite higher power use/hope to put smaller ones to sleep when not in use so it don't matter anyway.
GOAL: Basic server virtualization though not to the extreme, what was originally going to be like 2-3 systems i'm trying to force into one for power and cost savings. When I start needing much more than four cores a second server might happen instead of a dual socket board as the cost-overhead-per-chip is less for two mobos instead of one dual socket. This will just be what I call a financially optimized build where second or even third servers could be powered up as needed. Intended to run things like firewall, routing, home media/data server, and some basic virtualized sessions from thin clients. "Learning server/home lab" until I upgrade to something fancier/first learn and see how things work, what bottlenecks in the real world usage, etc.
GOAL: Optimized for progressive upgradeability, that's one reason for using SnapRAID. The Budget Drobo without the downsides. I can literally get a new server board, plug in the same drives, and it shouldn't matter at all. If my hardware becomes deficient (ie expanding virtualization needs) it will later get upgrades, don't worry.
Usage Profile:
Home media/file/backup server - the eventual plan is to use this for serious video production (my original post about a 300gb storage server) but I dont want to trust it with irreplaceable footage until i've tested it, so until then it will just serve unimportant blurays and such for at least a few months. Also a single always on server that drive backups on house workstations will mirror to, and a general NAS files sharepoint for groupware/collectively accessed projects from three workstations (to start).
House firewall - undecided which/still learning but probably pfSense
House QoS router/switch - plan to have this prioritize the right packets to VOIP and certain uses over others (console gaming latency cuz i'm not the only one in the house and i'm not currently paying for the ISP
)
Virtualized sessions - via Xen or VMware ESXi and VNC or similar, replacing a few computers in the house with thin clients, fairly light use to start and for awhile ("learning lab")
Tape preparation/writing - compiling up data to be written to Ultrium tapes in a RAIT array. (redundant array of inexpensive tape - each set of 10 tapes or so will have 1-2 parity tapes providing an alternative restoration method if the mirror set fails) This was originally meant to be on a physically separate server (and may be again) but this avoids some data redundancy of data that is both to be served and written. Talks with the SnapRAID author show that tape writeable parity sets should be viable to restore lost volumes from though it will be the first thing to test.
Home automation experiments/security footage motion flagging/recording type stuff (mild use/"learning lab", this is being played with for a separate side business project with another house member) - just like the tape prep server this can also break away into a separate box if it ends up being a problem/too much virtualized sessions going on
v1, v2, v3?
I have to get my feet wet somewhere so this is just v1, as I learn from any bottlenecks or mistakes eventually drives at least will be migrated into a v2 solution likely with different newer hardware. I'm also trying to be careful enough that I don't hit "limits" too fast/that if I just keep slapping drives in or upgrading existing drives it will continue to serve as a reliable Drobo-esque bitbucket for years.
This wont be the only server in the house, just the only "always on" 24/7 multipurpose one for now. I plan to separately be learning FreeNAS ZFS but that is less suited for the "easy Drobo-like upgrades" and maximal space utilization SnapRAID is great for. Heard too many stories of total datapool loss to want to start with ZFS as a learner. At some point if loads are too high, the single overutilized server becomes two always on servers or even three. If more than 32 drives are needed one file server becomes two on separate house circuits.
This is a place for public feedback on "the whole system". I'll have a "topic or two to solve first" before wanting to jump too far ahead, but if i'm designing myself into a corner here's where to give me a warning/correct gross misunderstandings I may have.
For instance until i've hammered down the SAS Expanders and PSU's I wont even be shopping for motherboards yet. I have separate topics for that/discussing specific topics elsewhere, but i'll be compiling/updating critical parts to this thread when they get narrowed down.
Build’s Name: The Poor Man's Drobo (named because Drobo-like progressive upgradeability will be designed on the lowest possible budget while actually exceeding Drobo performance, capacity, protection levels, and data recoverability/no proprietary formats. Designed for disks to be added, upgraded in size in place, and primary hardware to be upgraded over time as well. Zero concern for cosmetics, willing to tolerate some increased hassle at the beginning to save money for drives like lacking hot swap bays and server rated hardware.)
Operating System/ Storage Platform: TBD, probably virtualized Linux with SnapRAID as primary storage array
CPU: TBD at least quad core to start
Motherboard: TBD probably consumer level at first
Chassis: 2-4 common inexpensive probably even used ATX cases to start (with 8-11 effective bays each, so 3x11 or 4x8) literally bolted together to keep cost overhead per drive down, as it is acceptable to power down server to change drives for now, option to upgrade to hot swap trays later. (when I have the $ for a fancy 36 bay hot swap and can put motherboards in rackmounts i'll be happy - first I have to get enough data online though)
Drives: minimum expect 16 hard drives to start (common 3.5 SATA some decased from USB I already have) of 3-8TB sizes, expecting 19-21 hard drives + 3 opticals as a likely configuration, max configuration allowed for on this build about 29-30 hard drives+3 opticals. External Ultrium LTO6 drive. One hot swap tray to ease loading data into or out of the array. (ie copying a failing drive to its physical replacement) 1-2 drives treated as hot spares/any indication of failure while i'm out of town results in a drive copy command to reconstruct data and if successful pointing the SnapRAID array to use the hot spare until I can come home and physically replace it. (failure to copy means a SnapRAID recovery has to run of course to rebuild it thats what its for)
RAM: TBD prob 24-32gb to start mostly for virtualization needs
Add-in Cards: still deciding which SAS HBA and SAS Expander cards to use but will probably be using one 24port to 36 port Expander card for drives, though triple HBA isn't ruled out. Eventually 10gb Ethernet HBA when I get a 10g switch.
Power Supply: Multiple PSU's (2-4x) each powering separate drivesets, and able to power off a stripe of drives that wont be needed for awhile (under server virtualization/consolidation some are only intended to be accessed by one session) and force a form of staggered spinup whether the drives want to or not. (since SAS and SATA both tolerate hot swapping shouldn't bother them at all long as I don't do it before data is done writing) Also reduces cost-per-reliable-watts as 1kw and above PSU's seem to climb rapidly in price.
Other Bits:
GOAL:"minimum overhead cost per drive" maximized to what a single common 120vac 15-20amp house circuit allows. This seems to be about 32-33 drives. I'm aware more is possible but at increasing cost-per-drive or attention to detail (choosing specific drives over others/cant use "any") so if I need that much data i'll be splitting into two fulltime servers possibly even before I reach the max configuration. Though i'm hoping 32x8tb 256tb raw is enough for anybody. Wanting a MAID like configuration. (array of idle disks to save power and because it's no different than powering up manually the drives I want to use to serve data - some will sleep more than others and spent all night off anyways) Has to use drives I already have til they die despite higher power use/hope to put smaller ones to sleep when not in use so it don't matter anyway.
GOAL: Basic server virtualization though not to the extreme, what was originally going to be like 2-3 systems i'm trying to force into one for power and cost savings. When I start needing much more than four cores a second server might happen instead of a dual socket board as the cost-overhead-per-chip is less for two mobos instead of one dual socket. This will just be what I call a financially optimized build where second or even third servers could be powered up as needed. Intended to run things like firewall, routing, home media/data server, and some basic virtualized sessions from thin clients. "Learning server/home lab" until I upgrade to something fancier/first learn and see how things work, what bottlenecks in the real world usage, etc.
GOAL: Optimized for progressive upgradeability, that's one reason for using SnapRAID. The Budget Drobo without the downsides. I can literally get a new server board, plug in the same drives, and it shouldn't matter at all. If my hardware becomes deficient (ie expanding virtualization needs) it will later get upgrades, don't worry.
Usage Profile:
Home media/file/backup server - the eventual plan is to use this for serious video production (my original post about a 300gb storage server) but I dont want to trust it with irreplaceable footage until i've tested it, so until then it will just serve unimportant blurays and such for at least a few months. Also a single always on server that drive backups on house workstations will mirror to, and a general NAS files sharepoint for groupware/collectively accessed projects from three workstations (to start).
House firewall - undecided which/still learning but probably pfSense
House QoS router/switch - plan to have this prioritize the right packets to VOIP and certain uses over others (console gaming latency cuz i'm not the only one in the house and i'm not currently paying for the ISP
)Virtualized sessions - via Xen or VMware ESXi and VNC or similar, replacing a few computers in the house with thin clients, fairly light use to start and for awhile ("learning lab")
Tape preparation/writing - compiling up data to be written to Ultrium tapes in a RAIT array. (redundant array of inexpensive tape - each set of 10 tapes or so will have 1-2 parity tapes providing an alternative restoration method if the mirror set fails) This was originally meant to be on a physically separate server (and may be again) but this avoids some data redundancy of data that is both to be served and written. Talks with the SnapRAID author show that tape writeable parity sets should be viable to restore lost volumes from though it will be the first thing to test.
Home automation experiments/security footage motion flagging/recording type stuff (mild use/"learning lab", this is being played with for a separate side business project with another house member) - just like the tape prep server this can also break away into a separate box if it ends up being a problem/too much virtualized sessions going on
v1, v2, v3?
I have to get my feet wet somewhere so this is just v1, as I learn from any bottlenecks or mistakes eventually drives at least will be migrated into a v2 solution likely with different newer hardware. I'm also trying to be careful enough that I don't hit "limits" too fast/that if I just keep slapping drives in or upgrading existing drives it will continue to serve as a reliable Drobo-esque bitbucket for years.
This wont be the only server in the house, just the only "always on" 24/7 multipurpose one for now. I plan to separately be learning FreeNAS ZFS but that is less suited for the "easy Drobo-like upgrades" and maximal space utilization SnapRAID is great for. Heard too many stories of total datapool loss to want to start with ZFS as a learner. At some point if loads are too high, the single overutilized server becomes two always on servers or even three. If more than 32 drives are needed one file server becomes two on separate house circuits.
Last edited:
