Intel Optane Memory (m.2) and Linux

Patrick

Administrator
Staff member
Dec 21, 2010
12,181
5,257
113
Making a small test notes thread:

Test Configuration Notes
Testbed: Supermicro X10SDV-F w/ X520-da2 NIC
Location: The Intel Optane Memory 16GB m.2 card is in the m.2 slot.

Experiments
Set Intel Optane Memory as boot device in BIOS - did not work.
Install Proxmox VE 5.0 Beta on drive - installer sees the drive but does not boot.
Install Ubuntu 17.04 on drive - installer error
Install CentOS 7.3 on drive - installer error

Initial conclusion: do not use this as a boot device.

ZFS
Debian ZFS on Linux L2ARC:
Debian ZFS on Linux ZIL:

Virtualization
KVM virtualization Optane Memory pass-through: Working. Even works with Windows guests. See below.
lspci address: 01:00.0 Non-Volatile memory controller: Intel Corporation Device 2522

lsblk
Output from lsblk | grep nvme
Code:
# lsblk | grep nvme
nvme0n1            259:0    0  13.4G  0 disk
├─nvme0n1p1        259:1    0     1M  0 part
├─nvme0n1p2        259:2    0   256M  0 part
└─nvme0n1p3        259:3    0  13.2G  0 part
nvme tools
Code:
# nvme list
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     PHBT713102G7016D     INTEL MEMPEK1W016GA                      1          14.40  GB /  14.40  GB    512   B +  0 B   K3110300

# nvme smart-log /dev/nvme0
Smart Log for NVME device:nvme0 namespace-id:ffffffff
critical_warning                    : 0
temperature                         : 38 C
available_spare                     : 100%
available_spare_threshold           : 0%
percentage_used                     : 0%
data_units_read                     : 208,474
data_units_written                  : 60,916
host_read_commands                  : 4,437,501
host_write_commands                 : 2,630,720
controller_busy_time                : 0
power_cycles                        : 5
power_on_hours                      : 18
unsafe_shutdowns                    : 0
media_errors                        : 0
num_err_log_entries                 : 0
Warning Temperature Time            : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1                : 0 C
Temperature Sensor 2                : 0 C
Temperature Sensor 3                : 0 C
Temperature Sensor 4                : 0 C
Temperature Sensor 5                : 0 C
Temperature Sensor 6                : 0 C
Temperature Sensor 7                : 0 C
Temperature Sensor 8                : 0 C
 
Last edited:
  • Like
Reactions: gigatexal

PigLover

Moderator
Jan 26, 2011
3,012
1,315
113
Curious - and I'm sure you did this. But did you set the boot options in the BIOS to "UEFI Only" and do this OS installs for UEFI boot? The X10SDV (and most current SuperMicro BIOS) won't boot from m.2 NVMe unless BIOS to disable legacy boot completely (i.e., boot mode "both" won't work).
 
  • Like
Reactions: Stux

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,246
1,701
113
CA
Is this supposed to work at all?

It was my understanding the low capacity optane m2 were only for the latest gen CPU, and were only configured to be 'cache' which is why they need certain chipset to work at all.

This is why I cancelled my order until I get my latest gen system up and going to test with. Glad to see you are doing it now before me ;) ;) hehe!
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,181
5,257
113
I am pleased to announce... it works! Here is Proxmox VE 5.0 Beta Hyper-Visor with Windows Server 2012 R2 as a KVM guest, now with Intel Optane Memory using VT-d pass-through:
Intel Optane Memory Pass-through to Windows Server 2012 R2 VM KVM Debian Stretch Proxmox.JPG
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,246
1,701
113
CA
@Patrick ugh, why does intel state that then in requirements for using the m2 optane :( :( pissed I could have one on hand and using it but their "requirements" swayed me away to cancel my order....!!!
 

Evan

Well-Known Member
Jan 6, 2016
3,278
566
113
I thhave no the latest cpu's for the caching capability, if you don't use that it's simply a drive and works.
 

Biren78

Active Member
Jan 16, 2013
550
94
28
After reading all of the reviews, I thought the m.2 would only work in new Core i7's too.

That screenshot made me totally rethink Optane m.2.

wtf would I want a 16GB m.2 for? If you had a 1gbe home NAS, 100GB/day write endurance is plenty.

Now that I see Proxmox working with pcie pass thru to Windows that is hot.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,246
1,701
113
CA
@T_Minus it works just fine. Just like the bigger drives, they show up as NVMe devices.
After reading all of the reviews, I thought the m.2 would only work in new Core i7's too.
i'm a noob for asking but are you on v5 or v6 hardware that has nested virtualization?
I'm perplexed why Intel doesn't mention you CAN use this M2 without latest-gen just not for caching :(

@gigatexal it's a Xeon-D based onthe board he mentioned in 1st post, and yeah nested Windows under Proxmox based on what he said.


So @Patrick have you tried the Optane m.2 in a PCIE adapter card? Assuming any NVME m.2 card would work? I don't have a m b keyed pcie, so I need to buy a new one.
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
12,181
5,257
113
I have not used adapter cards yet. It is just on a Xeon D machine, which it should not work on. Really, these are just NVMe drives that have special caching on 7th gen desktop parts.
 
  • Like
Reactions: T_Minus

Patrick

Administrator
Staff member
Dec 21, 2010
12,181
5,257
113
BTW trying nvme format -ses 1 which is very slow.

Edit NVMe format keeps timing out even without ses.

Code:
# nvme format /dev/nvme0
format: Interrupted system call
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
12,181
5,257
113
Preview of how good performance is as a VT-d device:
Intel Optane Memory ATTO WS2012R2 Guest.JPG

Intel ARK says the 16GB model should be good for 145MB/s sequential writes / 900MB/s sequential reads. Almost spot on with the WS2012R2 guest under Linux KVM hypervisor via VT-d.
 

DWSimmons

Member
Apr 9, 2017
44
10
8
50
I'm curious/confused. If Optane is Intel hardware level that works with a Linux based kernel (proxmox), then shouldn't FreeNas theoretically be able to use Optane as an L2ARC drive.

As an aside, I've been reading reviews here and elsewhere and I can't quite get the use case for the "consumer" 16GB/32GB. I feel like the RST/SRT use case is so niche already. I've been trying to imagine what consumer value exists for this product in the first case. An L2ARC drive is literally a subset of a niche.
 

JDM

Member
Jun 25, 2016
44
22
8
30
Bummer these don't work as boot devices. Was hoping for an awesome SATA DOM Replacement, any guesses if support will come around for that later? My minimal CentOS installs are only using ~3GB of space, a 32GB would be more than plenty.