MichaelD's FreeNAS Server

[michaeld]

Here's my FreeNAS server built with a mix of new and old parts. The old parts are the case and the HDs; 500GB Samsung SATAII drives I've had for at least 6 years. I orginally bought 10 of them; I have 7 left. They are hardy drives for sure as they've been running basically 24/7 in one system or another the whole time. The new parts are everything else. I chose FreeNAS for three different reasons:

#1. It is free.
#2. It has a GUI
#3. I have been able to figure it out so far

I'm a Windows guy thru and thru and maintain a Windows Server 2008/Windows 7 environment at work. But my command line knowledge is pretty limited; you can stop snickering now, Linux people. I never knew about FreeNAS until about a month ago when I stumbled upon it while searching for "free NAS operating systems." I run Ubuntu 12xx on my netbook, so I'm not totally foreign to *nix OSs, but again, I need a GUI.

I'm running the latest FreeNAS distro; 8.3.0 Release P1 X64, on a 16GB USB 2.0 stick.

Hardware List
Case: 4U, 30" deep rackmount, modded with grill-less 120mm fan wall running Antec 3-speed fans. (set on Low, they are almost silent)
PS: SeaSonic M12II 850w SS-850AM
Motherboard: Supermicro X9SCA-F
Memory: 16GB (2x8GB) DDR3-1333 ECC
CPU: Xeon E3-1230V2 3.3GHz
Boot Drive: Sandisk 16GB USB 2.0 stick
Storage Drives: 6x500GB SATAII (Samsung FJ501 (or something like that)) Drives are configured RAIDZ2 for 1.7TB of space. I realize this is a pittance for many here, but it suits my needs at home.
HBA: IBM M1015, flashed to LSI 9211-IT, per the fantastic walkthru posted on this site
DVD/RW: Most excellent Samsung something-something-model SATA drive

I'd like to say I purchased these parts w/this build in mind, but truth be told it was a lucky coincidence. This MB/CPU/RAM was a short-lived HD video editing rig. This MB has no onboard sound, so really didn't serve too well as a stand-alone system. But hey, it all worked out! I have an Internal SATA-to-External SATA bracket in the mail. My sole external HD is E-SATA and it should make backups a lot faster. I do plan on purchasing another large (3TB) external HD for backups as well.

Non-professional, cell phone pics!!!

The rack. The FreeNAS server is the top box. My main/everyday/gaming box is in the middle, and the bottom box is my testing/flavor of the day box. Underneath the rack is an APC 1500-watt UPS. Mounted behind the rack is a HP V1910 16-port Gigabit switch.

One, 4-disk HD cage and 2 disks mounted to the big drive cage. You can just see the two HDs mounted behind the face plates to the right of the cage.

Rear view. Note the hi-tech 5.25-to-3.5" adapter brackets on the two HDs!

Long shot of the whole thing w/o the drive cage installed.

Closeup of the fan wall. Typically in this type of install there would be fan grills on the fronts of those fans. I've had about 10 different systems in this case and have never had a problem of cables hitting the fans. I've been very careful in cabling/zip typing/routing. Underneath the middle and right fans you can see a small PCB. That is a fan control board that came with the case. It has a 150-decibel buzzer on it that goes off if you're using less than 5000RPM hovercraft fans. Needless to say, I don't use that fan control board. The X9SCA motherboard has five onboard fan headers (including for the CPU) so it fits my needs in this case, perfectly. I really love this MB.

Various pics of the whole system w/the M1015 installed/cables run.

Finally, an ugly pic of the HP V1910 switch.

I'm currently copying over about 1.3TB of data from an external HD....getting roughly 15MB/s WRITE speeds. I'm expecting reads to be much faster...my testing over the past week was getting me roughly 80MB/s reads on the CIFS shares. Thanks for looking.

 

[Jeggs101]

Do you know what kind of case that is? Performance does seem low even for freenas.

 

[michaeld]

I've had the case for a long time, but I believe it's an HEC brand 4000 series...but many of these cases from different brands look identical and probably have the same OEM. What attracted me to the case 6+ years ago was the totally open cage up front. Back then backplane-equipped cases were 4x the price they are now. It was cheaper to buy the case and a bunch of 5-drive cages...which incidentally all developed dead ports and I junked them. So far, this single i-Star BPN series, sledless enclosure has been great.

I agree that so far, my writes speeds are horrible. But in my testing last week, I copied about 10GB of data to a 3-drive RAIDZ array and the write speeds were a bit faster. My READ speeds from that 3-drive array were in the 75MB/s range, which I was happy with. This current array is a 6-drive RAIDZ2, which may explain the lousy writes. I'm expecting the reads to be a lot faster. We'll see how it works out. One thing that might be killing my current writes are the fact that I'm copying over 1.4-ish TB of small, random files. MP3, jpegs, office docs, etc. I've got some large .iso files that are in the 3GB+ range, but the majority of the data is small files.

I'm most concerned with data integrity and availability. Any reads over 40MB/s will make me happy. Faster is always better, of course. Also, I should note that I messed around with trying to get a LAGG group going but couldn't. My switch supports LAGG and LACP and I tried various combos of creating the group on the switch first, then using it on the Freeenas server...ran into all sorts of problems. So I'm copying my data first so it's safe, then I'll snapshot the whole thing, then I'll mess around with getting LAGG running.

 

[PigLover]

Setting up a LAG won't make it go faster...

 

[michaeld]

Once the box is fully up and running (all data copied over), several different clients will be accessing it at once. I have Norton Ghost loaded on all my Windows boxes and it backs up to the NAS, and the box is a fileserver as well.

 

[michaeld]

Just finished copying the data over and I ran a read test. Over my wired gigabit network, I got 81MB/s average copying a 3.2GB iso file. Very happy! Don't really care about the write performance all that much.

 

[michaeld]

Just discovered something. I normally use a program called Directory Sync Pro to copy my backups back and forth. It does file comparision, overwrites older with newer files, deduplication, etc. It also avoids any weird Windows file permission errors. Well, I have about half a TB of raw digitial video from all my family movies. I decided to just do a copy/paste. I'm getting 75-85MB/s WRITES. Now that is some smokin' performance for just 6, old 7200rpm SATA drives. Loving FreeNAS!

*edit*
I just had to do a screen grab. This is copying large (1GB-ish) .avi files from an E-SATA connected HD over the gigabit network to the FreeNAS box. You just can't do much better on GbE than this.

 

[gigatexal]

in CIFS/SAMBA on FBSD it's important to load the aio module

kldload aio

then update the smb.conf to reflect teh changes

socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY
min receivefile size=16384
use sendfile=true
aio read size = 16384
aio write size = 16384
aio write behind = true
dns proxy = no

 

[michaeld]

Thanks for the tip, gigatexal. I would like to know for sure what those commands do before applying them. From my limited command line knowledge I'm going to guess:

Sets my Receive Buffer to 128Kb
Sets my AIO Read Size to 16Kb
Says I don't have a DNS proxy

The last entry is true; I don't run a proxy server at home. My cable modem is my gateway and my DNS server. I don't know what an "AIO Read Size" is and I also don't know why I would want to set my Recieve Buffer to 128KB. Why not smaller or larger? I'm trying to learn and appreciate your help.

 

[gigatexal]

I haven't done too much in the way of reading about it. I just tried it, those lines placed in my SMB.conf and saw my writes go from 17MB/s to 65-75MB/s

 

[michaeld]

Thanks again. Did you see a couple of posts up that I mentioned I was using a copy program when I was getting those slow write speeds? However, when using that program, I was also copying thousands of very small files. So I'm not sure what works for sure and what does not. My main box has Symantec Endpoint Protection loaded on it. Even disabled, if I'm trying to copy over any files it THINKS are harmful, I get a popup and the operation stops. Very inconvenient when the copy operation takes 10-14 HOURS and runs overnight. That's why I use that program; it avoids those nasty popups. It's something I've learned to live with.

Being that I know for sure my Raw DV and Rendered DV folders have no "questionable" files, I just used Windows to copy the data over. I'm not sure if adding those config lines to my FreeNAS server would fix anything. Now that all my data is copied over, read speeds is really all I'm caring about. Though possibly replication/snapshot tasks will require me to fiddle around in there...I really don't have much of an idea of what I'm doing in FreeNAS. I'm winging it with the User Manual in hand and my Windows Server/Client OS knowledge to help.