ZFS disk activity concerns with OpenIndiana

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Mr. F

Active Member
Sep 5, 2011
172
30
28
I just got things up and running with OI and napp-it.

Everything is going great, and with my raidz of (4) 7K3000's I am getting anywhere from 350-400+ MB/sec write speed. I'm very happy with that.

The thing that has me concerned, however, is that the behavior the array exhibits during writes. I have the 4 drives on an LSI2008 controller, and when I have a sustained copy running over the network the drives seem to all continually make a quick (less than 1s) write (which is audible with these drives, I'm used to that), then go quiet for about 5 seconds, then another quick write, and so on as long as the copy is sustained.

Initially I chalked this up to the fact that the incoming transfers are limited to around 100MB/sec by the GigE interface, and that the array was capable of writing so much faster (4x or more) it was just servicing it's write cache that much faster than the data was coming in. This seems like a perfectly reasonable explanation of what's going on, but I'm hopeful that someone can offer some confirmation of this, or offer an alternative explanation.

As I said, I haven't seen any detrimental performance issues; quite the opposite in fact--the performance is excellent. I just want to make sure there isn't something I'm missing that's putting unnecessary wear on my disks that could cause a premature failure.

Setup: oi-151 under ESXi / 8GB VRAM / (4)7K3000 / LSISAS2008(passthrough) / Intel 1000 NIC (passthrough)

Any insight from anyone on this?
 

PigLover

Moderator
Jan 26, 2011
3,184
1,545
113
Perfectly expected behavior. ZFS writes caches writes until it has a "stable" write to put onto disk. Once it has a stable write it will write all of the disks in the pool together. It does this in order to ensure that the content of the disks is always self-consistent, which is why you don't need a tool like chkdsk or fsck for ZFS.

Since your writes are bottlenecked by the LAN then you see this "strange" behavior.

Its even more interesting to see it on a large array - 20 or 24 drive lights flashing in unison...
 

Mr. F

Active Member
Sep 5, 2011
172
30
28
Thanks for the reply! I figured there was nothing to worry about, but since it's going to be such a critical system for me it didn't hurt to get some input from the community.

...I can certainly imagine the peeps a large array would make, though I'd most certainly keep it in a soundproof room to prevent it driving me mad. 4 drives is already enough to relegate my storage server to the basement.