NVME PM963 2.5" 960GB $300

cregan

New Member
Jul 3, 2017
6
0
1
55
I bought 9 of theses. Now I have a problem.
I tried to install them in a DL380 G7 (P410) and I the only thing I got is a solid amber light.
I didn't do my homework, it seems that the G7 only supports PCIe Gen2 and those are Gen3.

Currently, I have two G7 with Hyper-v in a cluster.
Starwind vSAN replicating over a Mellanox 10GB adapter.
My plan was to add more storage to both G7, but the plan failed miserably.


What would you suggest?
1 - Resell them on ebay
2- Buy "insert something that will make them work in a G7" $
3- Buy two servers that supports those disks.

If possible, I would like to keep my 144GB of DDR3 RAM (18 RAM slots per server)
 

sparx

Active Member
Jul 16, 2015
224
70
28
Sweden
Well. What do you mean, dont they work? Gen3 drives are backwords compatible with Gen2. You just loose some speed. The issue more likely is that your processors need a newer bios to support NVMe. Is there any for DL380 G7? Does that support nvme drives at all?
 

cregan

New Member
Jul 3, 2017
6
0
1
55
No, NVMe is not supported on the DL380 G7 as is.
The drive are not detected by the HP Smart Array P410 controller.
CPUs are X5650 and bios is P67 (latest).
 

sparx

Active Member
Jul 16, 2015
224
70
28
Sweden
Well. Thats not really strange. Since its probably a SAS/SATA controller. There are triple play controllers now. But they dont attach many NVMe drives. Also probably rather pricy. Better would probably be to direct attatch them with cable/card instead. Would probably mean some modifications to the chassie. Or to mount them internally maybe.
 

sparx

Active Member
Jul 16, 2015
224
70
28
Sweden
I of course cant say for sure. But the NVMe drives had a pretty late debute. Before that there were AHCI PCIe drives and other specialised hardware. But the most of the nvme drives came at the same time as Xeon Haswell 2011v3. So the numbers of servers with DDR3 and NVMe support. I'd wager there arent any.
 

sparx

Active Member
Jul 16, 2015
224
70
28
Sweden
How many PCIe slots do you have available? You could try something like this: http://ableconn.com/images/pimg1604075433178172.jpg
Maybe there are more specialized boards with dual U.2 or even quad? I dont know if you can set the bifurcation on your PCIe links manually. But if you can set a x16 to 4+4+4+4 in bios and find a matching board that would be doable.
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
I bought 9 of theses. Now I have a problem.
I tried to install them in a DL380 G7 (P410) and I the only thing I got is a solid amber light.
I didn't do my homework, it seems that the G7 only supports PCIe Gen2 and those are Gen3.

Currently, I have two G7 with Hyper-v in a cluster.
Starwind vSAN replicating over a Mellanox 10GB adapter.
My plan was to add more storage to both G7, but the plan failed miserably.


What would you suggest?
1 - Resell them on ebay
2- Buy "insert something that will make them work in a G7" $
3- Buy two servers that supports those disks.

If possible, I would like to keep my 144GB of DDR3 RAM (18 RAM slots per server)
If you decide to resell them, I may be an interested buyer. A couple of notes though:

I ran into a number of issues with the 10 of the 12 drives I purchased. Unfortunately, every time the power is cycled the drives go into an error state and appear to be bricked (listing "ERRORMOD" as the firmware revision from terminal and showing a total capacity equal to the drive cache size). This can be temporarily reversed via terminal, but the brick on power cycle problem is consistent. If left running without cycling the power, however, all of the drives work great for at least a few hours. The two non-affected drives list a different firmware revision than the other 10 units.

After going around with Samsung tech support (who told me to take flying leap), spending countless hours on the internet looking for firmware, and moving the drives between several different servers and workstations with absolutely no success, I may have sourced some non-official firmware that could be a solution... but I haven't had a chance to test it yet.

As a side note, when used as a single drive on a futon adapter, the one drive I tested did not exhibit the same issue.. When using multiple drives in Intel and SM purpose built NVMe servers and/or multiple drive HBA's, however, they're toast.

I'm currently traveling and about to walk into an airport, but I'll do my best to stay on top of this thread throughout the day.

-Craig
 
Last edited:
  • Like
Reactions: T_Minus

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
I'll sell them, 250$/each ?
Cregan- A quick update:

The good news is that I was able to successfully reflash the drives with the new firmware. The bad news is that the new firmware exhibits exactly the same problem and I now have 12 drives that brick on powercycle. I think this is a step in the right direction; just need to source the right file. Two of my drives originally came with firmware "CXV80W1Q" which worked fine.. Firmware "CXV83WCT" and "CXV80WT1" are a definite no go.

Here is what you get when you restart a machine with these drives in it:

 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Other than a Dell, HP or other OEM Brand Server what would be the best way to build an array using these 2.5 nvme disks in a custom build server with hot swap bays other than this intel kit reviewed a few years ago.

4 solutions tested: Add 2.5" SFF NVMe to your current system
If hot swapping is a requirement, you're basically stuck with a purpose built Supermicro, major OEM brand, or Intel NVMe server. The NVMe capable backplanes are a rarity @the moment. I recently went through this and ultimately purchased an Intel server capable of running 8x NVMe hotswap 2.5" drives (round one), and then a Supermicro server capable of utilizing 24x 2.5" NVMe drives (round two). The Intel option is somewhat reasonably priced, the Supermicro kit is not... Particularly considering that, if buying new, all Supermicro vendors force you to buy at least 2x CPU's, memory, and 4x NVMe drives to go with the "barebones" server chassis.

The Intel server is now listed in my FS thread here: Lab Cleaning

This approach uses two different Intel NVMe kits K & K2, and allows for up to 16 additional SAS3/Sata drives in the chassis (really 20 including the two mounting locations on top of the air shroud and two rear hot swap ports).

If a lack of hot swap capability is not a deal breaker, both Broadcom and Highpoint have now contacted me regarding the potential purchase of their latest NVMe capable HBA's (so they are definitely available for purchase today through the right channels). Until an NVMe backplane and/or expander is released, however, you're still limited to 4x NVMe drives/card. These cards range from $400-$600 each.

As pointed out in my previous post, until someone sources functional firmware for these PM963's, they are not usable. I'm making a little headway and hope to have a solution in the near future.

-Craig
 
  • Like
Reactions: T_Minus

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Just a quick update on this.. I've still been unable to source functional firmware and the seller has requested that I send the drives back... I would very much like to keep the drives, so if anyone has a possible source for Samsung firmware (I have a Samsung flashing utility called re-drive that seems to work great) please let me know. They're probably headed out tomorrow at some point during the afternoon.
 

Evan

Well-Known Member
Jan 6, 2016
3,149
530
113
Just a quick update on this.. I've still been unable to source functional firmware and the seller has requested that I send the drives back... I would very much like to keep the drives, so if anyone has a possible source for Samsung firmware (I have a Samsung flashing utility called re-drive that seems to work great) please let me know. They're probably headed out tomorrow at some point during the afternoon.
Does those re-drive allow flashing standard Samsung firmware onto OEM (e.g. HPE) drives by chance ?
Still getting the firmware I guess is the hard part...
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Does those re-drive allow flashing standard Samsung firmware onto OEM (e.g. HPE) drives by chance ?
Still getting the firmware I guess is the hard part...
I honestly don't have any OEM Samsung drives or associated firmware to test with.. It seems to be a powerful program though. Can take a bricked drive and completely renew it to a new firmware revision without breaking a sweat.

Like you said, however, obtaining the firmware seems to be the real trick. I actually sent a relatively hot response to Samsung tech support after a 2nd level support guy shut down all prior discussion of potentially providing firmware using a very "higher than thou" tone.

...Unfortunately, even considering my company's massive annual tech budget/purchasing power Samsung didn't come crawling back to me to me begging for my business as I had envisioned :rolleyes::rolleyes:

Guess I'm going to have to become an Intel guy.
 

Tamas

New Member
Jan 16, 2018
5
0
1
37
I ran into a number of issues with the 10 of the 12 drives I purchased. Unfortunately, every time the power is cycled the drives go into an error state and appear to be bricked (listing "ERRORMOD" as the firmware revision from terminal and showing a total capacity equal to the drive cache size). This can be temporarily reversed via terminal, but the brick on power cycle problem is consistent. If left running without cycling the power, however, all of the drives work great for at least a few hours. The two non-affected drives list a different firmware revision than the other 10 units.

-Craig
Hi,

Do you have a way of unbricking the M.2 form factor NVMe drives? Samsung Magician works for me for their 2.5" SATA kit, but not for NVMe's unfortunately...

Cheers,
Tamas
 

AJXCR

Active Member
Jan 20, 2017
565
95
28
32
Hi,

Do you have a way of unbricking the M.2 form factor NVMe drives? Samsung Magician works for me for their 2.5" SATA kit, but not for NVMe's unfortunately...

Cheers,
Tamas
Yep. What OS are you running?
 

Tamas

New Member
Jan 16, 2018
5
0
1
37
Yep. What OS are you running?
(Sorry for the delay, kinda missed the notification mail there...)

We do troubleshoot with either WinServer 2012 or Redhat, but I'm happy to boot any Linux distro if it sorts it.