Figured it out, need to grab the firmware for the dell branded version of the drive, trim out the right amount of header crap, then flash it raw using hdparm. After a successful flash, it will still report the same " GXT" version string as before, but you'll know it worked as some SMART values will be reset to zero, and if the drive was previously stuck with the ERRMOD / enumeration as a 1gb drive crap, it will be cleared and fixed
For PM863a (any size):
NOTE: If your drive has a manuf date on the label of 2019 or newer, or firmware GXT5404Q or newer, this isn't necessary, your drive has the latest fw already
File Format: Update Package for MS Windows 32-Bit
File Name: Serial-ATA_Firmware_P2YRJ_WN32_GC5B_A00_01.EXE
##when running the exe, choose extract, not install. in the extract folder, there will be a payload folder with the GC5B_NF.fwh firmware inside it
#trim dell headers etc from said file:
dd if=GC5B_NF.fwh of=PM863a-trimmed.fwh ibs=816 skip=1
#flash the trimmed file (CHECK THE DRIVE DEV ADDRESS IS RIGHT 4 U):
hdparm --fwdownload PM863a-trimmed.fwh --yes-i-know-what-i-am-doing --please-destroy-my-drive /dev/sda
##unplug drive (remove power, not just sata), replug/repower back in
##need to secure erase it for update to fully finish/clear ERRMOD
##set temp password to be able to secure erase
##the secure erase step later will remove the password/security
##again, check the drive address is correct so you don't wipe your OS drive instead
hdparm --user-master u --security-set-pass 1 /dev/sda
##secure erase the drive
##again, check the drive address is correct so you don't wipe your OS drive instead
hdparm --user-master u --security-erase 1 /dev/sda
##unplug/unpower drive, repower, you now have a clean fully working bug free/updated PM863a
I've attached a zipped up version of the already properly trimmed firmware file so you can flash it directly with hdparm as instructed above without needing to manually download or trim anything from dell