I swapped out 4 of my older HGST 4TB drives for new Toshiba 5TBs in my FreeNAS server, which is serving backend storage for my 2 ESXi servers over iSCSI. After the drive swaps completed, I noticed a problem with the performance. I ran a quick CrystalDiskMark to see if there were any improvements and I saw no only it did not improve, it dropped the 4K results significantly. From 250MB/s 4kQD32 read down to 10MB/s for instance. After some Googling and asking around on FreeNAS forum and Reddit, it turns out my pool was misaligned.
I am not entirely sure if this had been there all along (likely) but now I am looking for some suggestions.
First, my current pool consists of
It sounds like I need to recreate the pool in order to align them drives correctly. My question is, just how much of a performance hit is this currently costing me? And is there any less invasive way to fix it?
I am not entirely sure if this had been there all along (likely) but now I am looking for some suggestions.
First, my current pool consists of
Code:
da0 at mps0 bus 0 scbus0 target 8 lun 0
da0: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da0: Serial Number 74D3K4ICFS9A
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da1 at mps0 bus 0 scbus0 target 9 lun 0
da1: <ATA TOSHIBA MD04ACA5 FP1A> Fixed Direct Access SCSI-6 device
da1: Serial Number 54O2K1XQFS9A
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da2 at mps0 bus 0 scbus0 target 11 lun 0
da2: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da2: Serial Number 84P1K4MVFS9A
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da3 at mps0 bus 0 scbus0 target 12 lun 0
da3: <ATA TOSHIBA MD04ACA5 FP1A> Fixed Direct Access SCSI-6 device
da3: Serial Number 649FK2WCFS9A
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da4 at mps0 bus 0 scbus0 target 14 lun 0
da4: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da4: Serial Number 74D3K4IGFS9A
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da5 at mps0 bus 0 scbus0 target 17 lun 0
da5: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da5: Serial Number 84P1K4MOFS9A
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da6 at mps0 bus 0 scbus0 target 21 lun 0
da6: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da6: Serial Number 955AK57IFS9A
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da11 at mps0 bus 0 scbus0 target 26 lun 0
da11: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da11: Serial Number Y5N9K731FS9A
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da7 at mps0 bus 0 scbus0 target 22 lun 0
da7: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da7: Serial Number 95GAK5LRFS9A
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da8 at mps0 bus 0 scbus0 target 23 lun 0
da8: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da8: Serial Number Y5NAK6R9FS9A
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da9 at mps0 bus 0 scbus0 target 24 lun 0
da9: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da9: Serial Number Y5REKBRZFS9A
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
da10 at mps0 bus 0 scbus0 target 25 lun 0
da10: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SCSI-6 device
da10: Serial Number Y5RDKB6JFS9A
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 4769307MB (9767541168 512 byte sectors: 255H 63S/T 608001C)
Code:
[root@freenas] ~# zdb | grep ashift
ashift: 12
ashift: 12
ashift: 9
ashift: 12
ashift: 12
ashift: 12
ashift: 12