NVMe: 2.5" SFF drives working in a normal desktop

[Patrick]

Update 23-June-2015: Main site post with the latest - 4 solutions to add SFF 2.5" NVMe to your existing system tested is live with the latest information

We had a thread going on how to add 2.5" SFF NVMe drives to a current generation desktop system. Some of the review sites used a m.2 to SFF-8643 converter, but that is not practical if you:

• do not have an open m.2 slot, or
• want to install more NVMe drives

So we set about fixing that. Here is what you need:

• Supermicro AOC-SLG3-2E4R - $149 (Amazon or Ebay but I got from ebay) and it came with a backplane indicator cable (CBL CDAT-0674) and both full and half-height brackets - NOTE: This works in some motherboards. The Supermicro AOC-SLG3-2E4 has a PLX chip that allows bifurcation down to x4 devices. Apparently that is the version we should be looking at. It is also what is adding that cost.
• Cable from the Intel 750 series 2.5" retail kit - the label says SFF PCIe SSD Cable and there are part number markings: H73691-001 and AST 1513 - I could not find this cable for sale, but for $400 I just bought a $400 retail kit.

If you want to use 2x Intel 750 SSDs, this is an incremental $150$250 cost since the cables come in the retail kit.

Normally I would not spend $400 for a cable but alas I have much to do with it:


Just to verify how well this worked I took the Supermicro AOC and put it in a Supermicro X10SDV-TN4F motherboard and used a spare Windows Server 2012 R2 Essentials boot disk. This image I had used to get some numbers off of add-in-card based SSDs like the P3600 and P3700 so I knew it was working with other NVMe SSDs. I then took a Dell branded Samsung XS1715 and it fired right up!


Still validating performance, but the connection works on some motherboards at least. Stay tuned.

Here are the two cards Supermicro AOC-SLG3-2E4R and Supermicro AOC-SLG3-2E4 specs. I think we would want to use the Supermicro AOC-SLG3-2E4 in most cases.

and

 

[MiniKnight]

@!#!$ I just wet myself. Once I got dry I just ordered two of those cards and 4x 400GB Intel 750's. Next week will be fun for this guy!

 

[Hank C]

Sooooo.....before I wet myself as well, p3x00 doe not come with the cables?

 

[Biren78]

6x 400GB 750's later and 3 of these cards, I'm going to join in the party too. NVMe RAID 6.

 

[Hank C]

raid 0

 

[Patrick]

Sooooo.....before I wet myself as well, p3x00 doe not come with the cables?

I have ~ 6x P3600 and P3700 2.5" SFF NVMe SSDs that I purchased new. None had cables. Very frustrating.

 

[neo]

Very nice. I'd love to get a NVMe drive and try this out

 

[Hank C]

supermicro has this part number for the cable CBLSAST-0590 but can't find it anywhere...
would be great if it is 4x nvme on pcie x16

 

[cperalt1]

Supermicro CBL-SAST-0590 SAS Cables

supermicro has this part number for the cable CBLSAST-0590 but can't find it anywhere...
would be great if it is 4x nvme on pcie x16

 

[Patriot]

Just a word of warning... these are 25w drives so be sure to keep them cool. I would hate to see a, "I burnt up my NVME drive" thread.

 

[TuxDude]

Supermicro CBL-SAST-0590 SAS Cables

That cable has the wrong ends to connect directly to a 2.5" NVMe drive - it is the cable that would be used to connect from a card like the one earlier in the thread to a backplane

 

[Hank C]

Supermicro CBL-SAST-0590 SAS Cables

this would be for @Patrick

 

[neo]

@Patrick was this the cable you were looking for? Molex power connection instead. High Speed Serial Cables, Adapters and Accessories from Serial Cables

Also here is another Amphenol part number. Molex powered. Amphenol Data Center - SFF-8639 68P Straight + PWR to 36P HD Mini SAS

 

[Patrick]

Nice finds neo. That is right. That cable @ $72ea is almost $100 including tax and shipping. Might as well get the Intel 750 SSD at that point.

 

[legopc]

Are there any limitations to what motherboards you can use with that pci-e card?

 

[Patrick]

Are there any limitations to what motherboards you can use with that pci-e card?

So far I have done 2x Supermicro, 1x Gigabyte, 1x ASRock and 1x ASUS. All E5 or Xeon D based and PCIe 3.0 x8 or x16 slots.

 

[Patrick]

Downloading the Crucial p320h 350GB SLC PCIe drivers to see if it works with PCIe drives also.

 

[TangoWhiskey9]

Anyone know what the non-R version of the card does that's so great for $100 more?

 

[Chuntzu]

6x 400GB 750's later and 3 of these cards, I'm going to join in the party too. NVMe RAID 6.

This is what I like to see!!!! I was getting jazzed since I was going to buy 4 of these but six!!!2.5-3 million 4k read iops!!! I hope the CPUs don't limit the iops/max bandwidth. I can't wait to see the numbers you hit. Truth is I love Linux and all it has to offer but i want to see awindows server 2016 4node storage spaces direct set up with 2-4 nvme drives and 2 56gb mellanox cards is what I would really like too see.


If anyone doesnt know storage spaces direct is new and doesn't require the SAS fabric that server 2012 needed. Does everything over rdma. Very cool stuff!

 

[Patrick]

I am still working on getting a 10 drive chassis just for this purpose.