PM9A3 Firmware / ERRORMOD related to fw version

rgb333

New Member
Jun 26, 2022
6
0
1
We have several PM9A3 that are coming with different firmware versions (I assume different batches)
We installed them in DELL servers R6515. Some drives went into ERRORMOD.

We noticed that drives that have the old firmware work but the newer do not:

FW GDC5302Q - WORKS
FW GDC5502Q - FAILS

It is possible to upload new firmware using nvme tools but I cannot find where to get the binary. Besides, it seems the tool allows uploading but not downloading the firmware from a working device.

We tried contacting Samsung but they do not seem to be very responsive :/

Any tips? Ideas?
 

AllenAsm

Member
Jul 30, 2018
53
3
8
I got in touch with Samsung and they said the magician software will work with them even though it says its not supported. We have the PM9A3's in a dell 7525. They are working but its a bit iffy so far.
 

rgb333

New Member
Jun 26, 2022
6
0
1
I got in touch with Samsung and they said the magician software will work with them even though it says its not supported. We have the PM9A3's in a dell 7525. They are working but its a bit iffy so far.
What fw version do you have? You can see that using 'nvme list'

As I said, with the older fw version all works..

How did you manage to talk to Samsung? we could not get a reply from them..
 

JustinG

New Member
Jan 19, 2021
2
3
3
We're facing the same issue, but with Dell R6515s. This doesn't appear to be an isolated incident with the PM9A3s and recent Dell servers, as seen here as well: R6515 and samsung PM9a3

So far, we have observed similar failures; NVMEs on the GDC5402Q & GDC5502Q firmwares appear to go offline in the iDRAC, and from the OS-perspective they drop down to 1GB available. Listing the firmware shows "ERRORMOD". GDC5302Q and older appears unaffected.

The firmware files for the PM9A3 are not readily available on the Samsung website; we were forced to reach out to our distributor - Samsung refuses to provide support directly it seems, and also requires an NDA to be signed to obtain the firmware files, which is beyond infuriating. Do you happen to have a contact from where you purchased your PM9A3s to ask for assitance in getting the firmware files from Samsung?

Regardless, here are the results so far of what we have seen:

Upgrading to version GDC5602Q (latest) appears to fix the issue, however in the Dell forum post above another poster mentions the issue still persists. We have yet to see the issue re-appear since performing the firmware upgrade.

We weren't able to use Samsung Magician to perform the upgrade, we had to utilize the "Samsung SSD Toolkit for Data center" from their download page: Samsung Magician & SSD Tools & Software Update | Samsung Semiconductor Global

Once downloaded, we ran the following to load the firmware onto the disks (again, the .bin file needed to be acquired from Samsung after signing an NDA):

First, get the disk ID (needed in the next command):
Code:
~/DCToolkit -L
================================================================================================
Samsung DC Toolkit Version 2.1.L.Q.0
Copyright (C) 2017 SAMSUNG Electronics Co. Ltd. All rights reserved.
================================================================================================
--------------------------------------------------------------------------------------------------------------------------------------------------
| Disk   | Path       | Model                              | Serial         | Firmware | Optionrom | Capacity | Drive  | Total Bytes | NVMe Driver |
| Number |            |                                    | Number         |          | Version   |          | Health | Written     |             |
--------------------------------------------------------------------------------------------------------------------------------------------------
| 3:c    | /dev/nvme3 | SAMSUNG MZQL23T8HCLS-00A07         | XXX | GDC5502Q | LNUSRG39  |    32 GB | N/A    | N/A         | Unknown     |
--------------------------------------------------------------------------------------------------------------------------------------------------
| 4:c    | /dev/nvme4 | SAMSUNG MZQL23T8HCLS-00A07         | XXX | GDC5602Q | LNUSRG39  |     6 GB | GOOD   |  0.00 TB    | Unknown     |
--------------------------------------------------------------------------------------------------------------------------------------------------
| 5:c    | /dev/nvme5 | SAMSUNG MZQL23T8HCLS-00A07         | XXX | GDC5502Q | LNUSRG39  |    32 GB | N/A    | N/A         | Unknown     |
--------------------------------------------------------------------------------------------------------------------------------------------------
Load the firmware on the disk:
Code:
~/DCToolkit --disk 3:c --nvme-firmware-download --path General_PM9A3_U.2_GDC5602Q_Noformat.bin --action 1 --slot 2 --force
"Commit" the firmware to disk:
Code:
~/DCToolkit --disk 3:c --nvme-firmware-commit --action 2 --slot 2
A reboot may be required, we don't see the firmware version change from the OS otherwise.

It also appears to be possible to rescue disks that are running GDC5502Q and entered "ERRORMOD" state by running the following format command (note data will be lost), although this has been hit or miss for us, as some drives appear to get stuck on "ERRORMOD" indefintely:
Code:
~/DCToolkit --disk 3:c --nvme-format-namespace --user-data-erase
P.S. One other helpful command I found out after wasting way too much time on this - run the following to view all the available firmware slots on the drive:

Code:
~/DCToolkit --disk 3:c --nvme-get-log-pages --firmware
Hopefully GDC5602Q really did fix the issue; so far in our lab it's been stable, I'll post back if we find any further issues. Best of luck acquiring the firmware file, it's unfortunate Samsung makes the process so difficult.
 

rgb333

New Member
Jun 26, 2022
6
0
1
Samsung support has been a DISASTER! It is hard to believe they call this a "datacenter/enterprise" product..
 

Warlock

New Member
May 1, 2018
4
0
1
26
Can anyone share more information on this?

Can't really seem to figure out how to contact Samsung about this, and have multiple drives (30+) running GDC5302Q, some of whom have had this problem, so that goes against the OP's post.

Would also be amazing if someone could PM me with latest firmware (.bin?) file so I can flash the latest firmware and see if its helpful. Contact Samsung seems to be nearly impossible as every person I have been able to reach at Samsung tells me to contact a different department than the one that are at.
 

RTM

Well-Known Member
Jan 26, 2014
899
336
63
I don't want to be a spoilsport, but I feel it needs to be mentioned (again), that Samsung normally don't support their enterprise drives directly.
You are supposed to go via the company that sold you the drives to get support (including firmware), this could be HP/Dell/etc. (if you are lucky, as they will provide firmware updates) or a company that resells them without even having a contact at Samsung (meaning, they can't ask Samsung either for firmware updates etc.)

This is why it is so difficult.

Honestly it baffles me why Samsung has such a customer unfriendly policy, as clearly a huge amount of their non 1st tier vendor (HP/Dell/etc.) branded drives end up on the open market with a complete lack of support from the vendor.

It is easy to say: but if you are buying drives on the open market, I strongly suggest buying drives from other vendors (Intel/Solidigm and Micron (ironically enough) are usually decent about this, possibly WD too).

In short: If you bought a Samsung Enterprise drive I hope you got one branded by a vendor like Dell or HP as they might actually provide you with some support (for other drives it is usually the other way around). That said I hope you find what you are looking for.
 

Warlock

New Member
May 1, 2018
4
0
1
26
Yep, got them from the "open market" and they aren't branded Dell/HPs (they seem to be Samsung OEM). I contacted the vendor I got it from but I have low hopes there. Was hoping someone who already got the firmware from Samsung could share it, or tell us how they managed to contact Samsung for it.
 

rgb333

New Member
Jun 26, 2022
6
0
1
I decided to RMA them as the distributor suggested. I will never buy Samsung DC products again. If they are intended to be sold only by dell/hp they should clarify this.
I think their policy is very misleading and could lead to legal complaints.
 

Warlock

New Member
May 1, 2018
4
0
1
26
I managed to get the latest Samsung FW and flash it, but it didn't really fix anything.

Dell provided me with Dell PowerEdge Express Flash Ent NVMe AGN Firmware Release | Driver Details | Dell UK - which is a driver they support on their own version of PM9A3, but it doesn't work on our drives because they are not branded Dells. It's interesting because the release notes mentions fixing the exact error we see.

Any ideas if there are any hacks around using this Dell FW on a Samsung (non-Dell) drive?
 

rgb333

New Member
Jun 26, 2022
6
0
1
We just returned a large amount of devices? It took us almost 6 months without a solution
We are now using Intel or Micron NVMe instead.
Samsung DC products are shockingly unsupported.. they do not even reply.
 

DarkServant

New Member
Apr 5, 2022
17
15
3
Hi

May be you also has General_PM9A3_M.2_GDC7202Q_Noformat ?
I am sorry, i have no contacts to anyone who has access to all these resources (i wish i have, to share them all...). All i have is here (at own risk): Samsung SSD Firmwares & Tool
Maybe the latest ones are not always VMware compatible and check if the drive is not vendor-locked or if its an SED/non-SED version.