ZFS mass storage server (no virtualization) recommendations

lpallard

Member
Aug 17, 2013
265
11
18
lets start with what you want and what you NEED.
What I want:

2U or 3U enclosure with 12 to 16 hotswap bays
renown brand name with proven reliability and compatibility
Capability to add PCIE cards such as HBA's and infiniband
No need for RAID as the OS (FreeBSD) will be installed on a standard 200GB SATA HDD. The P410 controllers are interesting but do I really need them for a simple freebsd install for a storage server? I dont think so..
Capability to deal with SSD's for ZFS ZIL and L2ARC
At least 16GB RAM with possibility to increase to at least 48GB (assuming what I've read on FreeNAS forums or other ZFS forums with at least 1GB of RAM per TB of data)
=> 12x hotswap with current 2TB drives = 24TB raw (not usable in RaidZ2 or Z3) but when I replace failing drives with larger ones (3TB or even 4TB?) then the overall capacity of that storage server will most likely exceed 36TB raw...
(in all honesty I will most likely use heavy redundancy and parity to maintain reliable storage for years to come...)

What I need:

the simplest and most reliable server capable of running a 24 to 36TB zpool (raw).

I currently have a supermicro server.. I love it! just SIMPLE and effective. I've never owned a Proliant or Dell server but Im sure they are just as good, its just that with supermicro I feel the compatibility is just +++++
 

lpallard

Member
Aug 17, 2013
265
11
18
Not looking to swap... just looking to complement my VM server with a reliable ZFS storage server for backups of my VM's, possible strorage of them via iscsi, backup of 5TB+ of personal data, SQL databases, 2TB+ of family / friends' stuff, etc..
My current server is tailored more toward virtualization with proxmox and 15k rom SAS drives and a M5016 RAID controller.. Installing FreeNAS in a VM and have it work with a hardware RAID seems really not to be the best thing at all..

ZFS is better on a baremetal with just JBOD.
 

Diavuno

Active Member
I personally avoid X7 hardware because of the high power costs in CA, the age is also unsettling because Im usually buying for production work...
'
but this might be a good fit for you.

Supermicro 2U Server X7DBE 2X Xeon E5430 Quad Core 2 66GHz 32GB 2 x 160GB | eBay


It's actually what I use for my test environment, and home server. (though that ebay listing has a better motherboard)

to finish that server into a ZFS monster you'll need any 8 port SATA controller (or sas HBA) and some disks!!
 

lpallard

Member
Aug 17, 2013
265
11
18
I personally avoid X7 hardware because of the high power costs in CA, the age is also unsettling because Im usually buying for production work...
'
but this might be a good fit for you.

Supermicro 2U Server X7DBE 2X Xeon E5430 Quad Core 2 66GHz 32GB 2 x 160GB | eBay
...
to finish that server into a ZFS monster you'll need any 8 port SATA controller (or sas HBA) and some disks!!
Only problem is, the X7DBE has a ESB2 Intel drive controller chipset which only takes 2TB drives... I wonder if its limited to the oboard SATA ports or if I install a M1015 to connect the backplane I can use larger than 2TB?

If someone can clarify I'd appreciate. I may be able to put my hands on a Superchassis 16 bays with dual L5420 and 48GB RAM (with 8 dimms free for future expansion) for less than $550
 

lpallard

Member
Aug 17, 2013
265
11
18
The backplane that comes with it is the Supermicro SAS-836TQ and I **believe** it is a simple backplane but there is not much in the manual... Not much that I understands anyways! ;)

The chassis manual talks about expander backplanes SAS-836-EL1 & SAS-836-EL2 but with the fact that the chassis comes with the SAS-836TQ makes me believe it has an "upgraded" backplane??
 

lpallard

Member
Aug 17, 2013
265
11
18
so the m1015 will connect to 8 ports and those will support larger drives.

any connected to the mainboard will be limited
Or I just install a second M1015 for 8 more 2tb+ drives :D

I think I will order that beast now. Will post back when I get it!
 

lpallard

Member
Aug 17, 2013
265
11
18
Hmmm.. ok if someone has more experience with ebay than me please let me know what you think of that. A bit out of topic but..

Have you guys dealt with "MrRackables" from ebay before??

I think this seller also does business under the unixsurplus.com website. Are they thrustworthy? Asking because something out of the ordinary is happening with a listing I am interested in and before I buy I want to make sure I wont get riped off...
 

bds1904

Active Member
Aug 30, 2013
271
76
28
I have gotten serval systems from them in the past, as well as a rackable se3016 expander. I have never had an issue at all. Their shipping charges are well worth the cost, they use that expanding foam packing stuff most of the time. Never gotten a damaged product from them so I can't comment on a return service.
 

bwillcox

Member
Jan 20, 2013
32
0
6
Tejas
I've also done business with Unix Surplus. They accidentally sent me the wrong chassis on an order and made it right with little grief. The server they sent me works great. Will likely do business with them again down the road.
 

lpallard

Member
Aug 17, 2013
265
11
18
A bit of a follow up!

1. MrRackables & UnixSurplus are indeed the same seller... I bought from MrRackacbles, and got a invoice from UnixSurplus.

2. So far so good with the purchase from UnixSurplus. If this machine goes as expected, I may buy again from them.

3. They however did not supply the necessary screws for the Supermicro drive caddies. A bit of research showed that I need 6-32 screw, 1/4" long with a VERY small head. I found this Supermicro kit on the Egg, but I wonder if ita gonna work... Standard 6-32 hard drive screws have a round head (not a countersunk one) and are not fitting inthe 16 bay chassis...

SUPERMICRO MCP-410-00005-0N Screw Bag 100 Pieces for 3.5" Tray - Newegg.com

Can anyone recommend a reliable source for these little buggers??

Thanks!
 

Nindustries

New Member
Nov 11, 2013
12
0
1
If you want to build something yourself;
Ri-Vier RVS206A ; 150 euro
Seasonic OEM 400W SS-400L2U 80+ Gold - Modular ; 119 euro
AsRock C2750D4I ; 343 euro
Kingston KVR16N11K2/16 ; 169 euro

And you got yourself a 8-core 2U kick-asss server right there that sips power. Replace the fans to make it quiet too.
It's what I have at home. You can later switch out the memory for 16GB DIMMS to get to a whopping 64GB memory. I'm running embedded FreeNAS on it from a usb memory stick, saves me a drive slot!

If you live outside of Europe (for the case/PSU), try contacting support. They're a Norco reseller after all.
 

MikeC

Member
Apr 27, 2013
59
11
8
UK
A bit of a follow up!

1. MrRackables & UnixSurplus are indeed the same seller... I bought from MrRackacbles, and got a invoice from UnixSurplus.

2. So far so good with the purchase from UnixSurplus. If this machine goes as expected, I may buy again from them.

3. They however did not supply the necessary screws for the Supermicro drive caddies. A bit of research showed that I need 6-32 screw, 1/4" long with a VERY small head. I found this Supermicro kit on the Egg, but I wonder if ita gonna work... Standard 6-32 hard drive screws have a round head (not a countersunk one) and are not fitting inthe 16 bay chassis...

SUPERMICRO MCP-410-00005-0N Screw Bag 100 Pieces for 3.5" Tray - Newegg.com

Can anyone recommend a reliable source for these little buggers??

Thanks!
Here you go...

Supermicro MCP-410-00005-0N Screw Bag & Label f/ 24x Hotswap 3.5" HDD Tray | eBay