NVME Has arrived :D

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Ended up testing other stuff, and getting my for sale stuff going today, tomorrow it's NVME time :)
 
  • Like
Reactions: Chuntzu

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Negative on that helping.

BTW: Drive stays cool to touch even with ever so slight air flow, no air flow and it gets warm.. just sitting there plugged in of course ;) not doing anything...


Tomorrow it will be win2012 or win8.1 whichever I can find the disc for the fastest :D
 

Aluminum

Active Member
Sep 7, 2012
431
46
28
Well I couldn't get my system to boot with the card plugged in on my x99 sli+ w/5930k workstation. I'm using a M2 Samsung in there and the FusionIO drives only worked in 1 of the PCIE slots
Check the lane configurations permitted by your board, especially with SLI gpus in specific slots, its very likely the M2 pci-e lanes are exclusive with some of the physical slots. Dual gpus eating full x16 lanes each only leaves 8, and often that may only go to a single slot without muxes to split further, but you might be able to move the gpu to different slots so they get x8 each and pass another x8 to a different slot. (though this means you need a different bridge and can complicate the hell out of card spacing)

FWIW my X79 board supports configs like 16/16/8 and 16/8/8/8 from the cpu, I have a nvidia 980, the 750 SSD and a mellanox FDR card all chugging along in 3.0 mode lanes.

I can't boot to it since no UEFI version update available for X79 boards (yet, if ever) but otherwise it works fine in both W8.1 (out of box, updated to feb driver from intel) and 14.04.3 which is a fairly recent kernel.
 

Chuntzu

Active Member
Jun 30, 2013
383
98
28
Sounds like a rough go at getting this working. Sorry to hear about the struggle.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Yeah, I'm not sure why it worked fine then dissapeared and stopped working.

Logs show nothing.

This was my first CentOS 7. So it could be me, not being used to the changes as I know 7.0 was a major change. I've been using CentOS for many many years and never had so many problems with other things than I did with 7.0


Another idea I had was to use ESXI with Intel NVME drivers -- but I'm not sure how that would work? I noticed Intel had ESXI 5.5 NVME drivers, but how does that work? It obviously won't be as high of performance as baremetal but would you guys like to see me try that?

I'm going to go 8.1 or 2012 here in an hour or 2 either way :)
 

Chuntzu

Active Member
Jun 30, 2013
383
98
28
Yeah I saw your posts on the fat twin supermicro chassis (at least I think it was you) those are some awesome servers.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Yeah, I even checked on some singles and they're a couple weeks out or big $$ from someone who stocks larger quantities.

Or, maybe they're just low as a # are jumping on NVME? Who knows.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Next step was to try every PCIE slot starting at #1, not just the 2 random I chose to put it in.

I put it in slot #1, boots right into ESXI fine. No problems.

What the hell?

Something to do with the PCIE lanes?

or
Random, since it worked once before
 

markpower28

Active Member
Apr 9, 2013
413
104
43
Try esxi 6. It has Intel nvme driver out of the box. It works with pcie based p3500 without any issues
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
And the problem is solved.

With these cards apparently if you don't have the duplicator thingy mabobber on your motherboard you must use the BOTTOM plug.

And, only certain PCIE slots :)


I thought, hmm let me move cable to lower and bam reboot eSXI there it is :) storage adapters DC P3700 SFF :)

W000h000,
 
  • Like
Reactions: markpower28

markpower28

Active Member
Apr 9, 2013
413
104
43
And the problem is solved.

With these cards apparently if you don't have the duplicator thingy mabobber on your motherboard you must use the BOTTOM plug.

And, only certain PCIE slots :)


I thought, hmm let me move cable to lower and bam reboot eSXI there it is :) storage adapters DC P3700 SFF :)

W000h000,
you have only one CPU populated?
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Slight performance increase with pass-through for sequential transfer.

Rebooting now for updated NVME driver :)
 

markpower28

Active Member
Apr 9, 2013
413
104
43
if both CPU is populated then all PCI-E slot should work.

Anyway, where is the 2000 MB/s paper numbers? And 400,000 IOPS? ;)

From the P3500, I got 500 MB/s read and 800 MB/s write. Better than SATA based SSD but as as good as promised.

Same test, I got double the performance numbers on 5 years old FusionIO.
 
  • Like
Reactions: T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Intel DataCEnter CLI tool shows "new firmware" but I can't seem to locate P3700 firmware itself, trying SSD Toolbox.

Firmware updated via CLI (built in).

Rebooting VM.
 

neo

Well-Known Member
Mar 18, 2015
672
363
63
Checked the SM manual to see how many lanes the PCIe slot you plugged it into?
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,659
2,068
113
Neo- They're all x8 with one x16, and 0 other devices connected.

It shouldn't matter in that case?