Have an impact on STH - Shape review outputs

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
As many may have seen, I have been running (lots) of benchmarks on different higher-end SSDs.

Looking at IOMeter results the output has many dimensions. For example:
  • Database, Web, File Server and Workstation Profiles
  • 4-128 or 256 QD using exponential scaling

So that is essentially 4 * 6 or 7 (128 max or 256 QD max) sets of results. For each of those I have latency numbers as 21 different buckets (e.g. 0-50us, 1-2s). Even 24 sets of results with 21 buckets = 504 data points which seems like too much.

That does not include throughput, IOPS or other data points either.

Frankly, the latency bit is one of the most exciting aspects of these drives. When I looked into 1 billion records in the five latency buckets > 1s, I only found 21 operations that were in those ~1100 Drive/ Test Profile/ QD/ latency buckets.

Does anyone have thoughts around what data STH should present?

Here is Option 1 - All latency buckets
STH Distribution All Buckets Option 1.JPG

Option 2 - Consolidate 20ms and >1s
STH Distribution Consolidated Buckets Option 2.JPG

Option 3 - Consolidate further (10-20ms bucket)
STH Distribution Consolidated Buckets Option 3.JPG

Mind you, these figures are aggregated across drives, tests, and queue depths. I am also not finished going through all of the data sets yet but that shows the overall shape and this is not the final graph format. I I just wanted to get a sense of what folks thought on the bucketing. Personally, I like Option 3 but I did want to open this up for discussion.
 

iGene

Member
Jun 15, 2014
74
23
8
Taiwan
I like Option 3 more as it gives us more details.

Also there're nearly no data for latency higher than 15ms.
 
Last edited:

lmk

Member
Dec 11, 2013
128
20
18
@iGene How does option 3 give more details? The X-axis and Y-axis values have the same deltas in all 3 options. Do you mean visually/it is close up?

@Patrick The latter two are easier to 'take in'. Now for the pedantic (i.e. probably useless) comment: Option 1 will allow for the same graph (consistent format) for those times when you have outliers. I do not know how the benchmarks stack up (currently), and maybe with the newer and better drives this is a non-issue. However, it seems that every once in a while, some SSD (or firmware/controller combo) being benchmarked will show some result way out of the range of the other SSD (or said combination) being tested. Again, maybe those outliers are only going to be closer and closer (with the current and new drives) and it doesn't matter. :)
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
That is true. I think the benefit is if spindle disks get added. Here is a fun graph of what Option 1 would look like:
STH Distribution All Buckets Option 1 - Sample Data.JPG
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Here is an Option 3 view. I think it looks much better. Colors still stink though.
STH Distribution All Buckets Option 3 - Sample Data.JPG
 

Entz

Active Member
Apr 25, 2013
269
62
28
Canada Eh?
"Option 3 view" does look better, but what about adding a simple table underneath. Showing Min/Max/Avg latency? Best of both worlds?
 
Last edited:

iGene

Member
Jun 15, 2014
74
23
8
Taiwan
@Patrick The new graph does looks much better but it definitely needs better colors.

@lmk Sorry, I miss-looked the X-axis of the graph.

@Entz Showing min/avg/max latency is a good idea.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
3 does look better, but what about adding a simple table underneath. Showing Min/Max/Avg latency? Best of both worlds?
I think we will have something similar. Min and Max you could see from the graph but point taken on the break out.

And @lmk That graph took me a long time to discern so I appreciate your point. 5 shades of blue are hard to tell on the Lenovo X1 carbon gen 2 screen.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Just so folks know, I have been spending free plane time parsing iometer outputs. Expect to see more of this starting in 2015! Thanks for inputs.
 

cesmith9999

Well-Known Member
Mar 26, 2013
1,417
468
83
you have free time. must be nice... no holidays for me... annual maintenance window...
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
you have free time. must be nice... no holidays for me... annual maintenance window...
Luckily United's WiFi roll-out is not complete and the WiFi service is spotty at best. I do as little as possible on servers for that reason. On the other hand, this year has at least 11,000 more flight miles already booked so I am just batching data sets to manually go through on planes.

And to folks like @nitrobass24 I am sitting at 99,747 EQM's this year so this is not just me trying to finish up 1K.
 

nitrobass24

Moderator
Dec 26, 2010
1,087
131
63
TX
Lol nice. I didn't needs any help this year, hit 140eqp.

Do you have a database with all these benches/data sets? I was reading the Intel review and was thinking it would be nice to be able to do a side by side with other drives.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Lol nice. I didn't needs any help this year, hit 140eqp.

Do you have a database with all these benches/data sets? I was reading the Intel review and was thinking it would be nice to be able to do a side by side with other drives.
140k! Yikes.

Not yet on the database. The issue is that I am just about the worst person for data entry tasks known to man. I do have an Excel spreadsheet that I am dumping the quick results into. I am tempted to get someone on Fiverr or elance to finish keying in results. Maybe have them take all of the raw images, dump them into a spreadsheet, then I can easily average out after.

The iometer results create massive amounts of data. My extremely slimmed down spreadsheet has about 8 of the sas drives and still has ridiculous amounts of data in it still.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Do you have a database with all these benches/data sets? I was reading the Intel review and was thinking it would be nice to be able to do a side by side with other drives.
Well I had 4 done and gave up. Sent another 19 in on a Fiverr job. Very liberating!
 

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
Hopefully the colors are OK
Colors worked fine on my phone. At least you kept the drives the same colors.

I appreciate the comparisons now. Much better than just looking at 1 off raw data. I don't know why you did that in the first place.