I have a NetApp DS4246 with Dell compellent controller and 24 4tb 7.2k rpm SAS drives. I connected it to a dell r730xd with LSI SAS9207-8E and installed TrueNAS Scale.
After creating a pool with mirrored vdevs (6 mirrored vdevs with 3 drives each), I started testing read and write speeds using dd and file copy over smb. I see a weird issue, where if I have only a single file read from pool and no other operations being performed on the pool (scrub, data copy etc), I only get ~50 MB/s read speeds. However, if I simultaneously start another dd/smb file copy essentially doing multiple reads from the pool, then the read speeds bumps up to 700 MB/s aggregate (i.e. the other file transfer speeds up as well).
The write speeds are unaffected and even with one write a time, I get speeds around 700 MB/s which is good. It's only the reads that are weird. I tried changing HBA to Dell H200e (flashed to IT mode) and also switching to Netapp IOM6 instead of dell compellent and same problem.
If I have 12 or less drives in a pool, then read speeds are fine even for single read operation, i.e. I get around ~400 MB/s (good for 4 vdevs).
Does anyone have any ideas what's going on???
After creating a pool with mirrored vdevs (6 mirrored vdevs with 3 drives each), I started testing read and write speeds using dd and file copy over smb. I see a weird issue, where if I have only a single file read from pool and no other operations being performed on the pool (scrub, data copy etc), I only get ~50 MB/s read speeds. However, if I simultaneously start another dd/smb file copy essentially doing multiple reads from the pool, then the read speeds bumps up to 700 MB/s aggregate (i.e. the other file transfer speeds up as well).
The write speeds are unaffected and even with one write a time, I get speeds around 700 MB/s which is good. It's only the reads that are weird. I tried changing HBA to Dell H200e (flashed to IT mode) and also switching to Netapp IOM6 instead of dell compellent and same problem.
If I have 12 or less drives in a pool, then read speeds are fine even for single read operation, i.e. I get around ~400 MB/s (good for 4 vdevs).
Does anyone have any ideas what's going on???