6.4TB INTEL P4608 SSD DC $250

[ano]

tryrunning solidigm firmware disc?

 

[Sacrilego]

tryrunning solidigm firmware disc?

I'll give it a shot.

*********** EDIT ***********
No good. Firmware not found for this device.

 

[UhClem]

Confirmed "broken" with Proxmox 6.2.11-1 kernel.

Confirmed working with Proxmox 5.15.107-1 kernel.

Stats for one of the units:

root@pve:~# nvme smart-log /dev/nvme4n1
Smart Log for NVME device:nvme4n1 namespace-id:ffffffff
 ...
root@pve:~# nvme smart-log /dev/nvme5n1
Smart Log for NVME device:nvme5n1 namespace-id:ffffffff

I wonder if there might be a work-around (hack-around ?) for this onerous restriction. Could (one of) you do so:

nvme id-ctrl /dev/nvmeX | grep "^nn"

(do you see where i'm going?)
Also useful to know the VendorID:\DeviceID. Please list the (P4608-relevant) line(s) from an lspci -nn command.

 

[Puppetfreek]

I have the P4618, so kinda a similar card and running Proxmox (but on the 5.15 kernel) - so far.

nvme id-ctrl /dev/nvmeX | grep "^nn" returns nn : 1 for both devices.

lspci -nn returns

da:00.0 Non-Volatile memory controller [0108]: Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller] [8086:0a54]
db:00.0 Non-Volatile memory controller [0108]: Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller] [8086:0a54]

 

[UhClem]

The nn value of 1 means that the device has (only) ONE namespace; ergo, not a candidate for the hack-around I had in mind. [It was/is a longshot anyway.]

The ID-pair[**] reported, [8086:0a54], is the same one as used by the P4600 (and most likely the P4608); I wanted to eliminate the chance that these drives were Oracle-ized and had a different ID-pair (the Dell-flavored P4600 is different). (It is the ID-pair that the kernel uses to decide IF there are any known (to the kernel) quirks for the device, and if so, just which one(s).) This particular ID-pair IS in the newer kernels, but, alas, NOT yet with the NVME_QUIRK_BOGUS_ID flag.
EDIT ; to add (for @Sacrilego, below)
[**] I.e. VendorId:\DeviceID (\ to un-smiley)
("I wrote what I meant, and I meant what I wrote ..." )
Just because Google returned you something that (seemed to (fuzzy-)match) iis immaterial unless/until YOU actually understand (technically) the underlying details, and can assess the applicability to your quest.

 

[Sacrilego]

The nn value of 1 means that the device has (only) ONE namespace; ergo, not a candidate for the hack-around I had in mind. [It was/is a longshot anyway.]

The ID-pair reported, [8086:0a54], is the same one as used by the P4600 (and most likely the P4608); I wanted to eliminate the chance that these drives were Oracle-ized and had a different ID-pair (the Dell-flavored P4600 is different). (It is the ID-pair that the kernel uses to decide IF there are any known (to the kernel) quirks for the device, and if so, just which one(s).) This particular ID-pair IS in the newer kernels, but, alas, NOT yet with the NVME_QUIRK_BOGUS_ID flag.

Yep, I believe that's the reason the flag hasn't been added to the kernel yet.

Can the sub vendor id and subsystem id be used to more closely target the device?
In the case of Oracle F640, the SVID is :108E and the SSID for these 6.4gb drives are 4871 and 4870 (The 4870 SSID was an error from Oracle).
Intel's SVID is 8086 and SSID is 4732.
Dell's SVID: 1028 and SSID: 1ff0 and so on for the other vendors...
I can only imagine that the list would become pretty big if we had to enumerate svids and ssids for every model and manufacturer.

 

[Sacrilego]

EDIT ; to add (for @Sacrilego, below)
[**] I.e. VendorId:\DeviceID (\ to un-smiley)
("I wrote what I meant, and I meant what I wrote ..." )
Just because Google returned you something that (seemed to (fuzzy-)match) iis immaterial unless/until YOU actually understand (technically) the underlying details, and can assess the applicability to your quest.

I'm assuming something got lost in translation.

What I'm suggesting is something like this example to add the flag only to the specifically affected devices and not to the entire family of P4500/P4600:

drivers/nvme/host/pci.c

{ PCI_DEVICE_SUB(0x8086, 0x0a54, 0x108e, 0x4871),    /* Oracle F640 6.4TB */
         .driver_data = NVME_QUIRK_BOGUS_NID, },

 

[Puppetfreek]

Looking at pcie.c, it seems like the Intel-branded P4500/P4600 are already existing.

    { PCI_VDEVICE(INTEL, 0x0a54),    /* Intel P4500/P4600 */
        .driver_data = NVME_QUIRK_STRIPE_SIZE |
                NVME_QUIRK_DEALLOCATE_ZEROES |
                NVME_QUIRK_IGNORE_DEV_SUBNQN, },

I wonder if the above quirks should also be applied for the Oracle-branded ones? Abit of information on these quirks can be found here: nvme.h.

 

[UhClem]

I'd really like to continue discussing this device (P46[01]8/F640/etc?) despite not actually owning any. However, in my opinion, (deeper) technical matters (of any sort) should take place in a technical forum (ie, Hdw/HDD_SSD) where more STHers can see/contribute/benefit/refine.

Since I don't have a horse in this race [cf: not owning any], I'll wait/watch for one of you ...
======
Anybody remember 1980s Usenet with cross-posting/FollowUp-To? ?).

 

[rafaelwastaken]

Got 2 of the cards and wow they run HOT! Be warned, make sure you have enough cooling especially if you're gonna be doing heavy writing! They were idle'ing at ~70C for me without any cooling.

SMART-wise, ones I got have roughly 1 and 1.6PB written to them. Not bad for $250 considering their 35PB endurance lol

 

[OstJoker]

idle'ing at ~70C for me without any cooling.

I was thinking to buy one for my home PC (mid tower case, Windows 11). Is it good idea if I have only one case fan or its better to buy something more "SOHO" grade product? +70 C doing nothing seem pretty high temperature + it will require a lot of PCI-E lanes.

 

[Propaganda]

Got 2 of the cards and wow they run HOT! Be warned, make sure you have enough cooling especially if you're gonna be doing heavy writing! They were idle'ing at ~70C for me without any cooling.

SMART-wise, ones I got have roughly 1 and 1.6PB written to them. Not bad for $250 considering their 35PB endurance lol

Interesting these run hot, the p4618s(next gen) I have run surprisingly cool. In the same open air testbench configuration/ambient I have a p4618 showing 21C and a micron 9300max is showing 41C. The 9300max might be seeing a bit more activity ATM but nothing crazy.

 

[rafaelwastaken]

I was thinking to buy one for my home PC (mid tower case, Windows 11). Is it good idea if I have only one case fan or its better to buy something more "SOHO" grade product? +70 C doing nothing seem pretty high temperature + it will require a lot of PCI-E lanes.

You should be fine as long as you have some airflow, I have a small AC Infinity sucking air over it now and it dropped the temps down to 40-50C.

Interesting these run hot, the p4618s(next gen) I have run surprisingly cool. In the same open air testbench configuration/ambient I have a p4618 showing 21C and a micron 9300max is showing 41C. The 9300max might be seeing a bit more activity ATM but nothing crazy.

Yeah I was surprised as well, especially with how big the heatsink is on it


The namespace thing is quite annoying though, but at least the bogus NID quirk does indeed fix it

 

[Damo]

Is there any update on these for the 6.2.11-1+ kernel? Now Proxmox 8 is coming soon I would like to upgrade

 

[tjk]

Anything 6.2 still see's it as a single 3.2T card, both with PVE 6.2.x and latest 6.2 for 22.04.

 

[OstJoker]

Unfortunately I've missed the deal and now the cheapest option for this drive is 370$ or more... ^-(

 

[BackupProphet]

Does not seem that the kernel developers has agreed how to fix this specific card yet. So for now you need to build your own patched kernel.

 

[friendlyguy]

Hi there!
Just thought i`d participate in this discussion: I bought 4 of them as well.
Not knowing that i got myself the oracle branded ones i am facing the same problem to get a more recent firmware.
Mine are on QDV1RF35, I`d like to get them to: QDV1CP06.
This version is officially supported with ESXi 8.0U1.
I even reached out to oracle to buy a support contract: They don't bother to sell any to private customers.
I bought them to build a lab with vmware / vsan8 and play with the new esa.
What a disappointment!
And what i find even more frustrating: NOTHING, not a single lable visible on the device itself gave away that its actually oracle and not intel.
Well maybe unless you know all firmware versions from the top of your head and recognize: ouuuu that firmware is oracle.
They work fine under windows, might even work for another S2D cluster... but without access to firmware: not for vmware.
If anybody finds a way to crossflash or stumbles across that firmware: please please please let me know.

BTW: During my research I found out that the hardware management pack is now included in newer solaris 11 releases. I setup several different machines with solaris 11.4 but all gave me "Error: Platform not supported" when running "nvmeadmin" or "fwupdate".
Not sure what i am doing wrong. One of those servers i tried is even in the official Solaris HCL from oracle listed as Solaris 11 certified.

Have a nice day everybody

 

[Sacrilego]

Hi there!
Just thought i`d participate in this discussion: I bought 4 of them as well.
Not knowing that i got myself the oracle branded ones i am facing the same problem to get a more recent firmware.
Mine are on QDV1RF35, I`d like to get them to: QDV1CP06.

I bought four as well, ended up using them in an all flash truenas scale system. Keeping them cool was a challenge.

If these are indeed oracle drives, QDV1RF35 is the latest firmware for those at this moment. You can check that on this page: Minimum Supported Oracle 6.4 TB NVMe SSD v1 Firmware Version - Oracle® 6.4 TB NVMe SSD v1 Product Notes

QDV1CP06 seems to be for the cisco version of the P4500. At least I don't see any mention of it on oracle's page.

At this point, I don't think any firmware will fix the issue with esxi. Are the compatible? According to Vmware, yes. but it doesn't mention that only one of the two name spaces will show up. Until the linux kernel maintainers agree on the best way to add the quirk for this particular model, it'll be that way. I'm afraid the same thing will happen in truenas scale once they move to a newer linux kernel.

 

[alaricljs]

There's a v2 of that product... how did you figure out how the hardware relates to that particular doc/package?

edit: ok got it, found the Oracle PN in another listing (linked one is gone)