NVME Has arrived :D

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

neo

Well-Known Member
Mar 18, 2015
672
363
63
Neo- They're all x8 with one x16, and 0 other devices connected.

It shouldn't matter in that case?
Even though they are physically a x8 slot, they can be electrically x4 or x2. Check the manual for details if you haven't yet.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
From SuperMIcro Support

"For NVMe drives (SSD or PCIe), it need OS be install in UEFI mode. Otherwise, the NVMe drive(s) will not be detected."
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
From SuperMIcro Support:

They are checking with their "lab" as to why it's not working in "all" PCIE slots. They said that's NOT normal.

I have a single proc. board i'll try the other controller in with the 750. See if we get anything new.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Intel 750:

Seq Read Q4@64Threads - 857Mb/s Read
Seq Writes Q4@64Threads - 879Mb/s Read
4K READ 46MB/s
4K WRITE 125.1

It's rated for 900mb/s writes so I'd say I hit the write-limit of the Intel 750.

Didn't come close to the read limit at @ QD4.

Upping QD to 128 still didn't change the READ perf. in fact it was identical to QD4.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Well, I adjusted some settings in BIOS for PCIE power and changed the power management to disabled, and power to "full" and another one to "performance".

Also QD4 w/64 Threads was this test.

If I change QD to 128 like the IOMeter Intel test performance actually drops for that QD compared to 4, and the rest stay the same.

It seems I'm hitting a 900Mb/s (7.2Gbit) limit here. Which would appear to be PCIE 3.0 single lane performance. Maybe this is why my other PCIE slots aren't working, maybe the board has a problem with PCIE lanes? Or, the tests I"m doing aren't stressing this drive enough, I couldn't get IOMeter to near mimic Intel's test with it.. maybe someone has an intel-config file they can share I could use :)

The Intel 750 400gb actually performed slightly better sequential read and write than the P3700.


FWIW: The actual power usage didn't change @ idle changing PCIE settings. 165w @ Idle with ESXI 5.5 & 1 VM (Win 2012), with 1 SSD, and 1 P3700, and 128gb RAM, and 2 E5-2683 v3s.
 

Attachments

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
SuperMicro had me file an official report this AM, and tonight I got a new BIOS from them for my motherboard that's on the approved list of the "R" (Cheap Card).

I've tried the "R" Cheap NVME card in 2 other Supermicro 2011-v3 boards (not on approved list for NVME card) and while it doesn't stop booting they aren't detected at all, anywhere. I'm going to talk to SuperMicro about this once we resolve why the supported board is only running at PCIE 3.0 1x instead of 4x.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
New BIOS (or BETA I should say) didn't change anything, still only works on NVME Port #1 on the card, and still maxes out at approx 900MB/s (PCIE 3.0 1.0x w/overhead give or take).

*Motherboard is approved for this card and should be functioning 100%*
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Anyone know how to make sure the P3700 is in "FULL POWER" mode? Intel mentions it and so do some review sites but no mention on changing it or checking it...
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Well, 2 more "expensive" AOC cards are on their way, 4 cables from NEO, 2 from his other source, and a 3rd potentially with another 750 coming in July, not 100% if I'm going for a new NVME Chassis or another 750.

I should have 2-3 systems running NVME shortly. I will be playing around with them in ESXI, Win8.1, CentOS and running various "real world" simulations against existing systems. (MYSQL mainly -- and file searches, maybe some REDIS)