Nice to find some writing on mdadm RAID5/6. I will for sure look into your extensive work and use it as a reference for double checking my own configurations.
There are many reasons to prefer mdadm over HW RAID. Just the power consumption of a SAS controller is enough to dump it!. We all have a responsibility to lower our power consumption! It seems those LSI controllers, even in non-RAID mode, seem to use 8-15 watt in idle! How much does mdadm consume? One could claim close to nothing.
I only have one issues with mdadm: KVM and Windows guests.
It's my experience that good performance of RAID5(6) is only achievable by heavy interaction between mdadm and the host page cache.
Active use of the page cache I see several hundreds MB/s, with no use it drops to 50MB/s or below (talking HDDs).
This might not seem as a problem until you start using MS Windows guests on KVM, if the guest-storage is located on a host mdadm RAID5 device. I suspect the problem is precisely caused by issues with madam effective use of the page cache in combination with the Virtio-blk and Virtio-SCSI driver. No matter how my configuration is the result is not really good.
Yes, you can get initial good performance in some combinations, bot not persistent.
Any knowledge on this specific matter out there?
In my tweaking I've already looked at and tested QEMU iothread, IO, host and guest cache settings, QCOW2 and block device and of course both Virtio-blk and Virtio-SCSI drivers.
