badblocks settings throughput tests

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

cactus

Moderator
Jan 25, 2011
830
75
28
CA
TL;DR Version
When using badblocks in write-mode, I found a block size of 4096 and the number of blocks tested of 4096 or greater to give me the best throughput using an HP Micro Sevrer.
Warning: This will destroy anything that is currently on your HDD
Code:
root@server:~# badblocks /dev/sdX -wsb 4096 -c 4096
In-Depth Version
I have started using badblocks to break in and test new HDDs and test and wipe drives I am repurposing. With 2TB+ drives this process can take quite a while for each pass of the destructive write-mode test (-w) which writes and verifies a pattern on each block of the drive. I did some testing to see what block count to test at a time gives the highest throughput at a constant block size. All these tests were done using the first 4GiB of a Seagate ST3000DM001 to limit testing time, but this means the results are a ceiling for whole drive throughput. The drive is in an HP N40L with 4GB of ram and Ubuntu server 12.04.1 LTS.

Throughput measurement was taking using iotop in batch mode at 1s intervals.
Code:
root@server:~# iotop -bou root | grep -e /dev/sdX > test_value.out
The output of iotop, even in batch mode, is not great for analysis, but a little vim-foo and opening the output as a CSV gets the data into Excel.

I settled on a 100% arbitrary fixed number for block size (-b) of 4096 bytes, default is 1024 bytes. I started testing with the number of blocks tested at a time (-c) at double the default of 64.
Code:
root@server:~# time badblocks /dev/sdX -swb 4096 -c test_value 1048576 0
The data are average instantaneous throughputs. I used the first three passes (0xaa, 0x55, 0xff) for each test_value and calculated an average instantaneous throughput for each pass. I then averaged the throughput of the three passes to get the values in the table. I should apologize to any statisticians reading this and screaming at your monitors...

badblocks Results
Test ValueAvg. Read (MB/s)Avg. Write (MB/s)Combined Avg. (MB/s)Test Duration Wall Time MM:SS.SS
128174.2044.98109.597:43.77
256178.0371.95124.995:19.96
512179.08102.72140.904:11.29
1024177.53130.48154.003:37.72
2048152.62151.14151.883:36.10
4096157.48164.17160.823:23.97
8192159.10171.00165.053:19.82
16384159.57174.64167.103:16.81
32768160.98176.41168.703:15.88

I wanted something to compare my throughput results to, so I ran read and write tests using DD using direct IO, a block size equal to 4096*test_value, and a count to limit me to 4GiB/4.3GB. I used direct IO because badblocks has a flag to force buffered IO. If anyone knows for sure if badblocks uses direct IO, let me know. Otherwise, I will look at badblocks code later to verify.
DD Results
Test ValueDD Read (MB/s)DD Write (MB/s)
12818847.0
25618875.1
512188107
1024188136
2048187157
4096188160
8192188167
16384187170
32768187172

The DD tests give me some wildly varying read values and write values that increase. After comparing both data sets, I find badblocks decrease in read throughput as block count increases interesting. Badblocks first writes to the disk and when done goes back and reads from the entire disk verifying the data. I am guessing it is a limitation of how they are checking the blocks, I will have to look at the code to see if I can find anything.

So, if you use badblocks -wb 4096 use at least -c 4096 to get highest throughput with the HP micro server.
 
Last edited:

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
writes look close while reads are off. very strange that dd read speeds are almost exactly the same across your table. Maybe because you are only using 4GB so that's why so consistent.
 

john4200

New Member
Jan 1, 2011
152
0
0
I suspect there is something wrong with your system or your HDD. Maybe driver problems or some block device setting poorly configured. I suppose the Seagate could just be terrible at moderate size sequential transfers, but that seems unlikely. My badblocks read and write speed for a 2TB Samsung HDD are saturated at -b 4096 -c 4 , which is nowhere near your -c 4096.

Code:
% iostat -m /dev/sdu 1
Code:
% badblocks -sv -b 4096 -c 1 /dev/sdu
     reads
 -c   MB/s
===========
   1  84
   2 124
   4 138
   8 138
  16 138
  32 138
  64 138
 128 138
 256 138
 512 138
1024 138
2048 138
4096 138
Code:
% badblocks -svw -b 4096 -c 1 -t 0x55 /dev/sdu
     writes
 -c   MB/s
===========
   1  83
   2 120
   4 136
   8 136
  16 136
  32 137
  64 136
 128 136
 256 136
 512 136
1024 137
2048 136
4096 136
Code:
% dd if=/dev/zero of=/dev/sdu bs=4k oflag=direct

 bs  writes
 k   MB/s
===========
  4   83
  8  120
 16  136
 32  136
 64  136
Code:
% cat /sys/block/sdu/queue/read_ahead_kb
128

% cat /sys/block/sdu/queue/nr_requests
128

% cat /sys/block/sdu/queue/scheduler
noop deadline [cfq]

% dmesg | grep -i mpt2sas | grep -i version
[    0.920228] mpt2sas version 13.100.00.00 loaded
[    1.459218] mpt2sas0: LSISAS2008: FWVersion(12.00.00.00), ChipRevision(0x02), BiosVersion(07.23.01.00)
[    2.121578] mpt2sas1: LSISAS2008: FWVersion(12.00.00.00), ChipRevision(0x02), BiosVersion(07.23.01.00)
[    4.470156] mpt2sas2: LSISAS2008: FWVersion(12.00.00.00), ChipRevision(0x02), BiosVersion(07.23.01.00)

% dmesg
[251485.568817] scsi 8:0:40:0: Direct-Access     ATA      SAMSUNG HD204UI  0001 PQ: 0 ANSI: 6
[251485.568827] scsi 8:0:40:0: SATA: handle(0x0010), sas_addr(0x4433221102000000), phy(2), device_name(0x0000000000000000)
[251485.568829] scsi 8:0:40:0: SATA: enclosure_logical_id(0x500605b0024d5260), slot(1)
[251485.568998] scsi 8:0:40:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
[251485.569005] scsi 8:0:40:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
[251485.575350] sd 8:0:40:0: [sdu] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[251485.866835] sd 8:0:40:0: [sdu] Write Protect is off
[251485.866841] sd 8:0:40:0: [sdu] Mode Sense: 7f 00 00 08
[251485.878886] sd 8:0:40:0: [sdu] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[251486.196288]  sdu: sdu1
[251486.561966] sd 8:0:40:0: [sdu] Attached SCSI disk

Controller device @ pci0000:00/0000:00:06.0/0000:03:00.0 [mpt2sas]
  Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 02)
    host8: /dev/sds ATA Hitachi HDS5C303
    host8: /dev/sdt ATA Hitachi HDS5C303
    host8: /dev/sdv ATA SAMSUNG HD204UI
    host8: /dev/sdw ATA Hitachi HDS5C404
    host8: /dev/sdx ATA Hitachi HDS5C404
    host8: /dev/sdy ATA Hitachi HDS5C404
    host8: /dev/sdz ATA Hitachi HDS5C404
    host8: /dev/sdu ATA SAMSUNG HD204UI
 
Last edited:

cactus

Moderator
Jan 25, 2011
830
75
28
CA
john4200, all the queue settings are the same as yours. I will try a LSI2008 and some other drives.
 

cactus

Moderator
Jan 25, 2011
830
75
28
CA
Moving to a Opteron 8128 with LSI2008, I am getting similar write speeds (167.6MB/s) at -b 4096 -c 64 and a lower completion time of 3:11.9. So definitely beling limited by the micro server somehow. My first guess would be I am being CPU or memory throughput bound.

Edit: Stock settings return a similar run time as the -b 4096 -c 64 on the new setup.
 
Last edited: