Performance drop after adding another vdev.

[hagak]

I had a pool consisting of 6 mirrors (12 drives total). I added two more drives ie one more mirrored vdev to the pool. After doing that I ran bonnie and noticed that both my reads and writes were about half what they were prior to adding the drives.

Is this speed drop due to ZFS putting a preference to the drives with the lower capacity? if so any way to balance this short of copying everything from one pool to another and back?

 

[brutalizer]

if so any way to balance this short of copying everything from one pool to another and back?

I suspect you can just move the data to a new directory. Everything on the same zpool. Something like this:
# mkdir BalancedData
# mv -R UnbalancedData BalancedData

I dont think you need to copy the data to another server. I guess you can check the disk activity with "zpool iostat". Start a movie and monitor which disks are used with the command. Copy the movie to another directory and monitor the disks again, and see if all disks are used.

Does my suggesion work?

 

[gea]

I suspect you can just move the data to a new directory. Everything on the same zpool. Something like this:
# mkdir BalancedData
# mv -R UnbalancedData BalancedData

I dont think you need to copy the data to another server. I guess you can check the disk activity with "zpool iostat". Start a movie and monitor which disks are used with the command. Copy the movie to another directory and monitor the disks again, and see if all disks are used.

Does my suggesion work?

I suppose this cannot work because you need a real "write whole data not only directory references" -
so you need a move (=copy + delete) between two ZFS filesystems.

If you move within a filesystem, data it not rewritten.

 

[dswartz]

Or you can copy within the same FS and delete the original?

 

[gea]

Or you can copy within the same FS and delete the original?

All newly written datablocks spread over all disks so a edit or copy is the key factor.

 

[brutalizer]

If you move within a filesystem, data it not rewritten.

Yeah, I wondered about this too. I was hesitant to suggest you create a new filesystem, and move all data to the filesystem. But that should work, yes?

The correct way should be to create a new fileystem and move all data to the new filesystem, then all data would be balanced between the old and new vdev. Right?

 

[hagak]

I created a new FS and copied the data from the one FS to the new one. This FS was just one of many but contained a larger amount of data. After the copy and deleting the older copy of the FS, I reran bonnie and saw a marked improvement of the speed, still not quite up to before adding the new drives but good enough.