SAS3 SSD's over SAS3 spec?

[CyklonDX]

Hello,
Just something i've found curious, as its the 3rd time i've met with SAS3 SSD's that are rated, and actually get beyond SAS3 rated speeds.

Take the dell 09JJC SAS3 SSD as example
Topping up at 1.98GB/s in reads. (I tested, and it does indeed come close to 2GB/s alone)
I was under impression sas3 single device was limited to 12Gbit/s (1.5GB/s)

 

[Rand__]

You sure its not multipathed?

What did you connect it to?

 

[i386]

How did you Test the device?

 

[CyklonDX]

As a single device, so no benefit that i could see even if it was multipathed (wasn't).

I tested both at home on lsi9300-16i with icy dock mb998ip-b using dd

and following dell servers
Dell R720, R630, R640 -> set up a vd raid0 with no cache / cache disabled
and ran crystaldiskmark

In both cases the disk topped out at around ~1980MB/s (reads)

Its based on ultrastar dc ss540

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/data-center-drives/ultrastar-ssd-sas-series/product-brief-ultrastar-dc-ss540.pdf

 

[CyklonDX]

nvm found the reason

"12G Active-Active Dual port and 12G single/dual port for enhanced reliability"
I learned something new today.

 

[mattventura]

But that dock doesn't connect to the secondary port of the drives.

It could be some kind of caching on the host side, or perhaps it's an issue of not having data written to the drive prior to reading?

 

[Sean Ho]

A straight dd wouldn't be a great test due to caching. But CDD should be reasonably useful as an approximate benchmark. Or a proper fio invocation. I wouldn't expect that dock to even connect the pins for dual-port; it looks to be an A-style backplane, directly mapping 4 bays to each 8643.

 

[i386]

and ran crystaldiskmark

You probably tested some cache with the default settings.
If you're familiar with the cli you can open the crystaldiskmark directory, open "CdmResource/DiskSpd" and run DiskSpd (an io testing tool by microsoft, succesor of sqlio that was used to test systems for sql server) directly with the correct parameters to avoid caching in os, fs etc.

 

[CyklonDX]

No it wasn't any cache being hit. Thats the disk performance.

It seems there are potential for 2 signals paths within SAS3 specification (so you can potentially have sas3 ssd pushing theoretical 3GB/s - tho i don't think its possible due to noise - likely 2ndary channel has 70% performance - but thats just my thoughts.)

 

[i386]

It seems there are potential for 2 signals paths within SAS3 specification (so you can potentially have sas3 ssd pushing theoretical 3GB/s - tho i don't think its possible due to noise - likely 2ndary channel has 70% performance - but thats just my thoughts.)

That's what @Rand__ mentioned, "multipath" or "dual link" it requires special backplanes/cables to take advantage of the second channel. That would allow it to go beyond the 12GBit/s of SAS3

 

[CyklonDX]

That's what @Rand__ mentioned, "multipath" or "dual link" it requires special backplanes/cables to take advantage of the second channel. That would allow it to go beyond the 12GBit/s of SAS3

multipath as i understood refereed to mpio on sas backplane; but i could see what he meant was right.
(any sas3 rated backplane should have support for dual port)

 

[Rand__]

Dual Link/port might have been clearer but I used what Wikipedia called it

 

[Sean Ho]

8482 allows for two SAS connections; on the backplane side these would be wired to two different expanders/IOM. Each expander in turn usually is fed by two ports (e.g., 8643). E.g., BPN-SAS3-826EL2 has a total of 8x 8643: two expanders, each with upstream (HBA) and downstream (daisy-chain DAS) links, each with two 8643 ports, each with 4 SAS3 lanes.

To utilise these redundant links, one would typically use multiple HBAs and multipathing in the OS. The manual for BPN-SAS3-826EL has diagrams toward the end describing various cabling topologies.