Hello everyone!
We are running ZFS on OmniOS with 8xM500 in a raid-z2 setup. We have 8 (soon 16) xenserver machines accessing the SAN over NFS. We have sync=always thus all our writes to the VM datastores are synchronous.
We now need to improve our sync-write speed (we only get about 130 MB/s or so currently). Settings sync=disabled gives us better write speeds (1.2GB/s).
I have tried to read up on what to use for a ZIL. I have found the following alternatives,
The ZIL should sustain our writes for 10 seconds (have not checked our zfs_txg_timeout since im on the road , source Nex7's Blog: ZFS Intent Log)
We hope to buy 40Gbit/s infiniband for storage traffic soon to. But currently its 10Gbit ethernet. Ideal this means we need to have a ZIL of 25-50 GB or so in the future. I doubt we will have that much heavy writes but in theory that is our wanted ZIL size…..
Now the s840z specification indicate a higher write latency, 39μs, than the zeusRAM (>23μs). But the s840z is twice the size and half the price. Have anyone tested it?
It seems to be a better alternative than the intel SSDs (65μs).
What do you guys think?
We are running ZFS on OmniOS with 8xM500 in a raid-z2 setup. We have 8 (soon 16) xenserver machines accessing the SAN over NFS. We have sync=always thus all our writes to the VM datastores are synchronous.
We now need to improve our sync-write speed (we only get about 130 MB/s or so currently). Settings sync=disabled gives us better write speeds (1.2GB/s).
I have tried to read up on what to use for a ZIL. I have found the following alternatives,
- ZeusRAM, seems to be a great device for it, http://www.napp-it.org/doc/manuals/benchmarks.pdf.
- s840z, a 2.5’’ zil alternative s840Z 2.5″ ZIL SAS SSD | HGST Storage
- DDRdrive?
- Marvell WAM module?
- Intel S3700 or S3500
The ZIL should sustain our writes for 10 seconds (have not checked our zfs_txg_timeout since im on the road , source Nex7's Blog: ZFS Intent Log)
We hope to buy 40Gbit/s infiniband for storage traffic soon to. But currently its 10Gbit ethernet. Ideal this means we need to have a ZIL of 25-50 GB or so in the future. I doubt we will have that much heavy writes but in theory that is our wanted ZIL size…..
Now the s840z specification indicate a higher write latency, 39μs, than the zeusRAM (>23μs). But the s840z is twice the size and half the price. Have anyone tested it?
It seems to be a better alternative than the intel SSDs (65μs).
What do you guys think?