FreeBSD vault

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

kfoda

New Member
Jan 5, 2011
11
0
0
Build’s Name: vault
Operating System/ Storage Platform: FreeBSD 8 STABLE/ZFS
CPU: Xeon X3430
Motherboard: ASUS P7F-E
Chassis: Lian Li PC-343B cube
Drives: 7 x WDC WD15EADS + 3 x SAMSUNG HD154UI (raidz2), 10 x SAMSUNG HD204UI (raidz2), 2 x Hitachi HTE722020K9A300 (mirrored rpool)
RAM: 16G ECC Kingston
Add-in Cards: ASUS PIKE 1068E, Intel SASMF8I
Power Supply: Seasonic X-400 Fanless 400W 80plus Gold
Other Bits: 5 x Lian Li EX-H34 (hot swap 4 into 3 bays), Scythe Kaze Server Fan Controller, 3 x Arctic Cooling 120mm F12 TC Fan, 2 x Arctic Cooling 80mm F8 Pro TC Fan.

Usage Profile: Firewall, File Server for home LAN.

Other information…

Primarily used as a fileserver for a Popcorn Hour c200 media box, plus providing internet access via PF firewall. Runs 24x7. If zpools are not in use the disks get spun down to save power.

Power when idle is 72W i.e. main zpools in standyby/powered down

Arctic Cooling TC fans only start to ramp up at 32DegC. The fan controller runs the fans on the 5 x disk canisters at low speed if < 32degC.

Was running on Supermicro Atom X7SPA-H with the same 10 x 1.5TB Drives and a PicoPSU 150. That system was using about 60W. The PicoPSU was "iffy" supplying the start current for the 10 drives.
 

kfoda

New Member
Jan 5, 2011
11
0
0
How did you configure the disks to spin down when zpool not in use?
I just have a cron job that runs once an hour. It checks if there has been any IO activity on the pool, or if a smb mount is alive or if the pool is scrubbing or re-silvering, otherwise it shuts them down.
 

odditory

Moderator
Dec 23, 2010
381
59
28
Interesting, so does the cron job issue the sleep command to each drive separately (and thus you'd have to modify the job each time you add or change drive configuration), or does it issue it to the pool somehow? And are you using just plain FreeBSD 8 or are you using a GUI like zfsGURU?
 

kfoda

New Member
Jan 5, 2011
11
0
0
Interesting, so does the cron job issue the sleep command to each drive separately (and thus you'd have to modify the job each time you add or change drive configuration), or does it issue it to the pool somehow? And are you using just plain FreeBSD 8 or are you using a GUI like zfsGURU?
Just plain FreeBSD STABLE branch.

The job is just shell script. It queries the pool for drive ids, so no need to change:

drives=`zpool status $ZPOOL | egrep "da[0123456789]" | awk '{print $1}' | tr '\n' ' '`

then has a while loop to do a "camcontrol stop" or "camcontrol standby" to each drive.

I can post it here or send it via email if you want. It's a bit crude.
 

kfoda

New Member
Jan 5, 2011
11
0
0
I can post it here or send it via email if you want. It's a bit crude.
OK, here it is in case anyone wants it, modify as you see fit.

usage:

zpool-spindown.sh poolname

==============================
#!/usr/local/bin/bash

#
# zpool-spindown.sh
#

ZPOOL="$1"

if [ -z "$ZPOOL" ]
then
echo "zpool name required"
exit 2
fi

PATH=/usr/local/bin:/bin:/usr/bin:/usr/sbin:/usr/local/sbin:/bin:/sbin
export PATH

# Cleanup any tmp file if present
if [ -f /tmp/zpool.iostat ]
then
rm -f /tmp/zpool.iostat
fi

# Name of samba share to check if mounted
SMBSHARE="media"

# Get drives for pool
drives=`zpool status $ZPOOL | egrep "da[0123456789]" | awk '{print $1}' | tr '\n' ' '`
firstdrive=`echo "$drives" | awk '{print $1}'`

# Activity checks
smbactive=`smbstatus -S | grep -A 6 "Connected at" | grep $SMBSHARE | wc -l | awk '{print $NF}'`
scrubrunning=`zpool status $ZPOOL | egrep "scrub in progress|resilver in progress" | wc -l | awk '{print $NF}'`
spundown=`smartctl -n standby -H /dev/$firstdrive | tail -1 | grep "STANDBY" | wc -l | awk '{print $NF}'`

if [ -f /tmp/locate.running ]
then
echo "Locate running...Aborting spindown!"
exit 3
elif [ $smbactive -gt 0 ]
then
echo "Samba share is mounted...Aborting spindown"
exit 3
elif [ $scrubrunning -eq 1 ]
then
echo "Scrub/resilver is running...Aborting spindown"
exit 3
elif [ $spundown -eq 1 ]
then
echo "Spundown already...Aborting spindown"
exit 3
fi

# Longer IO Activity check - only perform if got past above
zpool iostat $ZPOOL 30 2 | tail -1 > /tmp/zpool.iostat
reading=`cat /tmp/zpool.iostat | awk '{print $(NF-1)}' | awk -F\. '{print $1}' | sed -e 's/K//g' | sed -e 's/M//g'`
writing=`cat /tmp/zpool.iostat | awk '{print $NF}' | awk -F\. '{print $1}' | sed -e 's/K//g' | sed -e 's/M//g'`
rm -f /tmp/zpool.iostat

if [ $reading -gt 0 ]
then
echo "Pool shows IO activity...Aborting spindown"
exit 3
elif [ $writing -gt 0 ]
then
echo "Pool shows IO activity...Aborting spindown"
exit 3
fi

drives=($drives)
type=""

driveop () {
drive=$1
# Need to issue differnt command to ada vs da devices!!!
type=`echo $drive | cut -c 1`
if [ $type = "d" ]
then
camcontrol stop $drive
elif [ $type = "a" ]
then
camcontrol standby $drive
fi
return
}

drives_count=${#drives[@]}
index=0

while [ "$index" -lt "$drives_count" ]
do
driveop ${drives[$index]}
printf "Spindown Drive %s\n" ${drives[$index]}
let "index = $index + 1"
done
===============================
 

odditory

Moderator
Dec 23, 2010
381
59
28
Impressive. Did you create that from scratch yourself? Maybe I've been living in a cave but my understanding is ZFS pool spindown has been a "challenge" and not really workable for various reasons.
 
Last edited:

kfoda

New Member
Jan 5, 2011
11
0
0
Impressive. Did you create that from scratch yourself? Maybe I've been living in a cave but my understanding is ZFS pool spindown has been a "challenge" and not really workable for various reasons.
The loop in the script was from someone else's script to get drives temperature using smartmontools.

When I had a look at it, I thought it would be easy enough to adjust for spinning down the pool. It works very effectively. I have another similar script that starts the drives. I run that before the cron scheduled weekly scrubs, although the pool's drives will start ok themselves if accessed. I then just thought about on what conditions not to perform the spindown to add to the script.

After that, I went looking for things that appeared to be waking the pool up.

Weekly /etc/periodic/weekly/310.locate script does. Modified it to create a flag file. Check for that in spindown script and don't spindown if present.

/etc/periodic/security/100.chksetuid wakes them up as well. I chose to stop running this for now.

smartmontools "smartctl" commands in config and script should be modified to add "-n standby" parameter to command. This prevents it waking up the drives on queries. I use this in the spindown script to see if the first drive in any pool is already in standby.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
I kinda think that those scripts would be a good addition to the FreeBSD, ZFSguru, and FreeNAS forums. kfoda mind posting them also as a reference there?

Also, on the P7F-E did you get the ASMB4-iKVM? I was bummed that those are still being sold as add-in modules.
 

odditory

Moderator
Dec 23, 2010
381
59
28
maybe kfoda and sub.mesa could collaborate to see if its something that could be implemented in zfsGURU.
 

kfoda

New Member
Jan 5, 2011
11
0
0
I kinda think that those scripts would be a good addition to the FreeBSD, ZFSguru, and FreeNAS forums. kfoda mind posting them also as a reference there?.
Ok, will do.

ASMB4-iKVM[/URL]? I was bummed that those are still being sold as add-in modules.
No, I didn't bother for just a home machine, but still wanted server grade plus ECC for stability. Savings on the PIKE vs Intel or LSI card meant that it ended up being reasonable total price for main components.