I thought you guys might like to check out an ATTO disk benchmark against Two P3700 in Software RAID 0
On A Side Note: It's Official.. I think I went overboard this time.. On the positive side I scored some great deals along the way to build my new workstation. I had a lot of the parts already from various purchases throughout 2015. This was NOT an all at once purchase...
ASUS X99-E WS/USB3.1 & ASUS Titan X GPU
Intel Xeon E5-2699 V3 (QGN1 C1 Stepping)
Corsair C70 Ammobox Case w/ AX1500i PSU
1.6TB Boot: Intel DC P3605 2.5" NVMe*
(*on MSI Turbo U.2 Host card)
4.0TB Data: Software Stiped Dual Intel DC P3700 2.5"**
(**on AOC-SLG3-2E4 PEX 8718 PCIe Switch Adapter)
256GB ECC DDR4-2133 (8x32GB Samsung M393A4K40BB0-CPB)
On a side note: I have to share this since the ATTO Disk Benchmark is right here. It's so nifty, tiny, and it's fast. It's called the MyDigitalSSD OTG. A nice accessory for the NVMe workstation in your life.. I picked up the 256GB model for $93.. I'm almost as excited about this little thing as my new NVMe based workstation.
This answers a question that has been nagging me for some time.
Start with the X99 chipset block diagram:
Intel® X99 Chipset Block Diagram
Note: "DMI 2.0 x4"
That means 4 PCIe 2.0 lanes @ 500 MB/second = 2,000 MB/second MAX HEADROOM upstream.
Your AOC-SLG3-2E4 has an x8 edge connector and
should be compatible with PCIe 3.0: 8 GHz clock /8.125 = ~1 GB/second per lane
x8 lanes @ 1 GB/second = 8 GB/second MAX HEADROOM upstream vs. 2 GB /sec above
So, with a software RAID 0, 2 x DC P3700 reach a READ speed of 5.17 GB/second.
Outstanding!
Conclusions: the PCIe expansion slot in which you installed that SM AOC
is NOT downstream of the DMI link. If it were, max READ speed
could never exceed 2.0 GB/second under any conditions.
As such, there is hope for new hardware RAID controllers
that use a full x16 edge connector, and are compatible with PCIe 3.0.
Things will get even more interesting when the PCIe 4.0 standard
gets implemented in hardware, because the clock rate is planned
to increase from 8 GHz to 16 GHz per PCIe lane!
MRFS