Samsung 960 EVO nvme performance

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

doctor

Member
Mar 19, 2015
63
24
8
I have a pair of Samsung SSD 960 EVO 1TB's running in software RAID1 under Centos 7 (kernel 3.10) in a Dell R620. The drives are installed in PCIe adapters, and each is on a different PCIe bus (processor 1/2).

The drives seem reasonable for reads ...

# hdparm -Tt --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT cached reads: 4612 MB in 2.00 seconds = 2306.27 MB/sec
Timing O_DIRECT disk reads: 4098 MB in 3.00 seconds = 1365.24 MB/sec

# hdparm -Tt --direct /dev/nvme1n1

/dev/nvme1n1:
Timing O_DIRECT cached reads: 4130 MB in 2.01 seconds = 2058.76 MB/sec
Timing O_DIRECT disk reads: 3862 MB in 3.00 seconds = 1287.30 MB/sec

However write speed drops off very fast...

# dd if=/dev/zero of=test bs=1M count=200
200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 0.197257 s, 1.1 GB/s

# dd if=/dev/zero of=test bs=1M count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 3.75583 s, 558 MB/s

Doing a 20GB test, the server load spikes (up towards 200+) with 75% idle time, and the server log shows these:

[5145876.782688] nvme nvme1: completing aborted command with status: 0000
[5145876.793552] nvme nvme1: completing aborted command with status: 0000
[5145876.794007] nvme nvme1: completing aborted command with status: 0000
[5145876.810535] nvme nvme1: completing aborted command with status: 0000
[5145876.811053] nvme nvme1: completing aborted command with status: 0000
[5145876.811704] nvme nvme1: completing aborted command with status: 0000
[5145876.814927] nvme nvme1: I/O 800 QID 2 timeout, aborting
[5145876.814947] nvme nvme1: I/O 801 QID 2 timeout, aborting
[5145876.814951] nvme nvme1: I/O 802 QID 2 timeout, aborting
[5145876.814955] nvme nvme1: I/O 804 QID 2 timeout, aborting
[5145876.814959] nvme nvme1: I/O 805 QID 2 timeout, aborting
[5145876.814961] nvme nvme1: I/O 806 QID 2 timeout, aborting
[5145876.814964] nvme nvme1: I/O 807 QID 2 timeout, aborting
[5145876.814967] nvme nvme1: I/O 811 QID 2 timeout, aborting

# nvme smart-log /dev/nvme1
Smart Log for NVME device:nvme1 namespace-id:ffffffff
critical_warning : 0
temperature : 51 C
available_spare : 100%
available_spare_threshold : 10%
percentage_used : 3%
data_units_read : 18,907,979
data_units_written : 14,754,311
host_read_commands : 139,277,178
host_write_commands : 386,999,247
controller_busy_time : 7,947
power_cycles : 16
power_on_hours : 1,409
unsafe_shutdowns : 11
media_errors : 0
num_err_log_entries : 0
Warning Temperature Time : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1 : 51 C
Temperature Sensor 2 : 77 C
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Unfortunately that is common behavior with consumer drives ... Had the same with my 950 nvme and 850 evo...
 

CoadyTech

New Member
Nov 12, 2017
6
1
3
31
United Kingdom
I might be talking rubbish here but I think you've just discovered the different between 'consumer' and 'enterprise' NAND SSD drives.

If the NAND memory cells aren't 'clean' and pre-erased, there is a performance penalty on write operations. This is usually dealt with on enterprise drives by over-provisioning the amount of NAND memory (e.g. a 500GB drive might actually have 800GB of NAND memory). On consumer drives without as much over-provisioned NAND, we hit 'dirty' memory much faster.