NVMe: 2.5" SFF drives working in a normal desktop

[Paul A. Mitchell]

That add-in card reminds me of an early SATA 6G expansion card
that ASUS was selling for a while: we have this card running well
in an older ASUS P5ND2-SLI motherboard:

ASUS Model PCIE GEN2 SATA6G Expansion Card - Newegg.com
ASUS P5ND2-SLI LGA 775 NVIDIA nForce 4 SLI Intel Edition ATX Intel Motherboard - Newegg.com

No RAID logic, however; just simple "pass thru".

What I'm hoping for is full RAID support -AND- bootable RAID
so we can install an OS on a RAID array of these high-performance
NVMe SSDs.

Remember that Intel's "Optane" (3D XPoint) memory should be
emerging this year on several different form factors e.g. 2.5".

I expect that the first company do a good job of designing
and manufacturing an NVMe RAID controller with at least
four U.2 ports will really have something.

The future is pretty bright for such technology,
because PCIe 4.0 ups the clock rate to 16G:

thus, an x16 PCIe 4.0 edge connector should realize a
raw one-way bandwidth of 16 lanes @ 2GB/s = 32 GB/s.

Add Optane to that much higher bandwidth,
and our storage subsystems should be performing
on a par with DDR2-800 or faster ramdisks!


MRFS

 

[Paul A. Mitchell]

Here's another one, with a larger heatsink,
but still appears to be a "pass-thru"
NOT a h/w RAID controller:

High Speed Serial Cables, Adapters and Accessories from Serial Cables

Very pricey at $959.00 USD.

MRFS

 

[Paul A. Mitchell]

HERE WE GO, LADIES AND GENTLEMEN!

Highpoint ... introducing industry first PCIe 3.0 x16 / NVMe RAID controller

HighPoint Global Website

[scroll down]

Upcoming RocketRAID PCIe 3.0 Generation RAID Controllers


For 2016, HighPoint has expanded our award winning, industry leading range of RocketRAID HBA’s with PCIe 3.0 host connectivity, blazing fast 12Gb/s transfer rates, and introducing industry first PCIe 3.0 x16 / NVMe RAID controller.

RocketRAID 3740A – 16x 12G SAS/SATA PCIe 3.0 x8 lane RAID Controller; supports RAID 0, 1, 5, 10, 50 and JBOD solutions for Window, Mac OS X, Linux and FreeBSD Storage platforms.

RocketRAID 3830A – 3x PCIe 3.0 x4 NVMe and 8x SAS/SATA PCIe 3.0 x16 lane controller; supports NVMe RAID solution packages for Window and Linux storage platforms.


p.s. My "wish" may come true

Want Ad: PCIe NVMe RAID controller

http://supremelaw.org/systems/nvme/4xU.2.and.SATA-E.jpg

 

[Paul A. Mitchell]

On 3/24/2016, Highpoint responded to my request for more information:

[begin quote]

Hi Paul,

Thanks for contacting HighPoint.

We are still in the early stage for RR3830x series but below is our brief information on prototype we be showing at NAB next month.

What application you seeking to use NVMe Storage, as for now the ecosystem is not mature yet and NVMe U.2 SSD still very pricy.

1. Support up to 3x PCIe 3.0 x4 SSDs
2. NVMe RAID 0, 1 and 5
3. PCIe 3.0 x16 connectivity bandwidth
4. Software package support Window platform

We are open to hear customer feedback on NVMe storage application, it will help to shape our NVMe RAID Adapter product line.

Best Regards,
May Hwang
HighPoint Technologies, Inc.
Product Marketing Department

[end quote]

My response follows:

Thank you very much, May:

We have bought about 6 RocketRAID 2720SGL
RAID controllers, and we LUV them!

http://supremelaw.org/systems/io.tests/4xSamsung.840.Pro.SSD.RR2720.P5Q.Deluxe.Direct.IO.2.bmp

But, it's time for RAID controllers to exploit
the raw bandwidth of x16 edge connectors
and PCIe 3.0 speeds.

The PCIe 4.0 clock will oscillate at 16 GHz.

The raw bandwidth of a full x16 edge connector
maps nicely into four U.2 ports @ x4 lanes each:

4 @ x4 = x16 = very elegant symmetry !

I believe HP, Dell and Kingston have a PCIe add-in card
that all use four M.2 SSDs and an x16 edge connector:

Kingston's version is described here:
http://www.liqid.com/pdf/M2_whitepaper_01042016_FINAL.pdf

HP's version is described here:
HP Reveals New Z Turbo Drive Quad Pro | StorageReview.com - Storage Reviews

Here's the Dell version:

[see above]

> Support up to 3x PCIe 3.0 x4 SSDs

We were hoping for four U.2 ports, like this:

http://supremelaw.org/systems/nvme/4xU.2.and.SATA-E.jpg

See our WANT AD here:

Want Ad: PCIe NVMe RAID controller


Further reading:

Cheap NVMe performance from HP - 9,000MB/s for $900 or so?

NVMe: 2.5" SFF drives working in a normal desktop


THANK YOU!


Cc: colleagues


MRFS

 

[R.Zoli]

Hi!

I have an ADATA SR1020 2TB SSD and try to use it in my X99 setup but it doesn't work... I didn't see it in the nvme setup and under win 10... I use with a SFF8639 cable with this cable my Micron P320h works flawlessly... Any idea?

 

[Paul A. Mitchell]

motherboard with 2 x U.2:

ASUS ROG MAXIMUS VIII HERO ALPHA LGA 1151 Intel Z170 HDMI SATA 6Gb/s USB 3.1 USB 3.0 ATX Intel Motherboard - Newegg.com

ROG MAXIMUS VIII HERO ALPHA - Specifications

 

[Paul A. Mitchell]

see also:

TURBO U.2 HOST CARD | MSI USA | Motherboard - The world leader in motherboard design

*Tested with Turbo U.2 on Z170 GAMINGM7 using 2 x Intel 750 SSD in RAID 0.

ADD-IN CARD VIA M.2 SLOT

 

[Paul A. Mitchell]

ROG MAXIMUS VIII HERO ALPHA - Overview

[begin quote]

LIGHTNING-FAST STORAGE
SPEED WITH NATIVE M.2/U.2
NVMe PCIe RAID SUPPORT

Don't want [to] be limited by the read/write speeds of a single
storage device? Then join the fast lane by building multi-
device RAID array with Maximus VIII Hero Alpha's two U.2
and one M.2 onboard connectors -- and enjoy PCIe data
transfers at lightning speed. Our lab guys put both through
their paces, achieving write speeds of 3,386 MB/s and
read operations of an incredible 3,575 MB/s.

[end quote]

Because of those numbers, we infer that the 2 x U.2 ports
are downstream of the DMI 3.0 link.

 

[Paul A. Mitchell]

found this at www.supermicro.com :

AOC-SLG3-4E4R-P

But, if I search the SuperMicro website, using their search window,
they report nothing.

I found that "AOC" mentioned here:

Supermicro | Products | SuperServers | 2U | 2028R-NR48N

search for: "Add-on Card / Module"


Also here:

Supermicro SSG-2028R-NR48N Dual Processor

search for: "(NVMe AOC by 16 - AOC-SLG3-4E4R-P)"


"by 16" implies an x16 edge connector (but can't be sure without a photograph + specs)

And here:

Supermicro Superserver 2028R-NR48N 2U DP Xeon LGA2011-3... | Acmemicro

search for "NVMe AOC by 16,HF,RoHS/REACH"

 

[Mirabis]

Anyone seen something like the FunTin PCI Adapter, but low profile ? right now there seems to be a lot of dummy space...and it might fit in low profile if differently organized.

 

[Paul A. Mitchell]

Maybe this one:

OCZ RD400/400A PCIe^® NVMe™ M.2 SSD (Solid State Drive)

 

[Mirabis]

Maybe this one:

OCZ RD400/400A PCIe^® NVMe™ M.2 SSD (Solid State Drive)

Need a U.2 connection M.2 won't fit... I contacted Funtin to ask if they have them in low profile - but they don't exist yet... offered to pay extra for a custom job but no reply. So instead I mounted the full pci and plan to cut holes in the top and add an extra housing spot lol... WOn't fit in a rack but it'll do

 

[kkisme]

Need a U.2 connection M.2 won't fit... I contacted Funtin to ask if they have them in low profile - but they don't exist yet... offered to pay extra for a custom job but no reply. So instead I mounted the full pci and plan to cut holes in the top and add an extra housing spot lol... WOn't fit in a rack but it'll do

Yes, currently 2.5 inches NVMe SDD does not fit in 1U or 2U chassis, the space does not allow, so Funtin NVMe adapter is not suitable. Unless you purchase a 90-degree turn of PCI expansion slots. Can only be replaced by wire transfer M.2, Funtin currently provide Wire and M.2 adapter. Please click on the links:
Amazon.com: Funtin PCIe NVMe SSD Adapter U.2 ( SFF-8639 ) Interface x4 Lane Slot PCI Express Card: Computers & Accessories
https://www.amazon.com/Funtin-SFF-8...srs=13967549011&ie=UTF8&qid=1465984735&sr=8-1
https://www.amazon.com/Funtin-MiniS...srs=13967549011&ie=UTF8&qid=1465984735&sr=8-4

 

[Mirabis]

Yes, currently 2.5 inches NVMe SDD does not fit in 1U or 2U chassis, the space does not allow, so Funtin NVMe adapter is not suitable. Unless you purchase a 90-degree turn of PCI expansion slots. Can only be replaced by wire transfer M.2, Funtin currently provide Wire and M.2 adapter. Please click on the links:
Amazon.com: Funtin PCIe NVMe SSD Adapter U.2 ( SFF-8639 ) Interface x4 Lane Slot PCI Express Card: Computers & Accessories
Amazon.com: Funtin U.2 (SFF-8639) to Mini-SAS (SFF-8643) Cabling for 2.5" NVMe SSD: Computers & Accessories
Amazon.com: Funtin MiniSAS HD to M.2 Card for NVMe SSD: Computers & Accessories

I saw the wire but that would require me to buy another Expander to accommodate the connection + X10DRL-i board does not have m.2 slots, so need the Mini-SAS. If you then include the fact that those require quite some air pushed against them, they have to be in the PCI slot hehe....

It'll be cheaper to get a custom chassis top made than to go that adapter route... Will inform at local cut shops and post back once it's done

 

[Mirabis]

Yes, currently 2.5 inches NVMe SDD does not fit in 1U or 2U chassis, the space does not allow, so Funtin NVMe adapter is not suitable. Unless you purchase a 90-degree turn of PCI expansion slots. Can only be replaced by wire transfer M.2, Funtin currently provide Wire and M.2 adapter. Please click on the links:
Amazon.com: Funtin PCIe NVMe SSD Adapter U.2 ( SFF-8639 ) Interface x4 Lane Slot PCI Express Card: Computers & Accessories
https://www.amazon.com/Funtin-SFF-8...srs=13967549011&ie=UTF8&qid=1465984735&sr=8-1
https://www.amazon.com/Funtin-MiniS...srs=13967549011&ie=UTF8&qid=1465984735&sr=8-4

Might be able to get it to fit:

Put it in the x16 with a Riser card split , enough lanes to do 3x NVMe + 3x Sata . this blocks 2 X8 & 1 X4
and leaves a X4 (need X8 for Raid Card).

So have to take a flexible riser to mount my raid card next to the PSU ...hmm at that point it might be better to go for the cables xD

 

[amnesia1187]

Just found these: MB516IP-B_2.5" HDD/SSD Cages_ICY DOCK manufacturer Removable enclosure, Screwless hard drive enclosure, SAS SATA Mobile Rack, DVR Surveillance Recording, Video Audio Editing, SATA portable hard drive enclosure and MB998IP-B_2.5" HDD/SSD Cages_ICY DOCK manufacturer Removable enclosure, Screwless hard drive enclosure, SAS SATA Mobile Rack, DVR Surveillance Recording, Video Audio Editing, SATA portable hard drive enclosure
Sadly only 7mm, but progress is progress . Gives me hope they will make the others.

And this: Internal Quad PCIe x4 (four SFF-8643 Connectors) to PCIe x8 Gen 3 Switch Host Card | QIP4X-PCIE8XG301

 

[Patrick]

Those Icy Dock units will not work for NVMe drives.

I do regularly ask them when we will get NVMe hot swap add-in bays.

 

[amnesia1187]

Those Icy Dock units will not work for NVMe drives.

I do regularly ask them when we will get NVMe hot swap add-in bays.

Huh? they will work with NVMe drives "16 x 2.5” NVMe / SAS / SATA SSD or HDD (Supports NVMe / SAS dual channel)"

At least based on how its described, its just too skinny to fit the chunky intel NVMe drives. My point was more that at least in theory they now have SFF-8643 ports that are wired correctly for pass-through of either type of connection note this part: "Supports 2.5” NVMe / SAS / SATA HDD/SSD (This cage does not convert drive interfaces; make sure the host supports your drive interface)."

 

[Patrick]

Here is my reasoning:

• They will not fit current NVMe 2.5" drives from HGST, Samsung, Intel and others.
• There are only 8x SFF-8643 ports, so you are going to have a maximum of 8 drives not 16
• Even if you tried using only 8x of the currently available drives, they are unlikely to fit due to the extra connectors in the rear
• If you did put 8x 25w drives in that enclosure I would be extremely worried about cooling given all of the NVMe systems we have seen from vendors to date

I could see a case where you got a m.2 NVMe to PCIe 3.0 2.5" 7mm adapter and made it work, but that is a very scary architecture.

Odds are we are going to see a transition away from SFF-8643 before we see this type of hot swap case. More systems we are seeing come out are using OCuLink for PCIe cabling.

 

[amnesia1187]

Fair enough, i dunno what possible reason they had for choosing that form factor, my point was just if they already did the engineering to modify the pinouts on one of their devices minisas hd connections, which at least in my mind means that to create more enclosures now would just be a matter of a new manufacturing run.

As to the 8 ports not 16, now that you point it out, I'm almost wondering if there is another configuration with different trays that double up for NVME and do the right connections, that would make way more sense than 7mm nvme drives