I am running a baby sister system to Bronko's system here: Napp-it SuperStorage Server 6048R-E1CR36L Performance. with the same family of NVMe SSD.
Unfortunately, I seem to be having trouble getting the L2ARC set up on the NVMe SSD. It is installed via DirectPath I/O and it is seen by OmniOS/Napp-it, but it causes instability in the system. Not sure how to resolve.
Barebone: Supermicro | Products | SuperStorage Servers | 2U | 6048R-E1CR12L (HBA: SAS3 via LSI 3008 controller; IT mode)
CPU: 2x Intel® Xeon® Haswell-EP Series Processor E5-2630 v3, 2.40 GHz, 8-Core
RAM: 64GB (4x 16GB) Hynix DDR4-2133 CL15 RDIMM
rpool: 1x 64GB SSD SATA DOM
data-pool: 4x 6TB (P/N:0F22791 HUS726060AL5210 ) 2x2 mirror
L2ARC: 1x 400GB Intel® Solid-State Drive DC P3500 Series NVMe (Use only 300GB)
SLOG: Possibly partition 15GB off of P3500
Hypervisor: ESXi 6.0 -- 2016-03-15 (Update 2)
OS: OmniOS 5.11 omnios-r151018-ae3141d April 2016
Napp-it: 16.04f
What I know:
Unfortunately, I seem to be having trouble getting the L2ARC set up on the NVMe SSD. It is installed via DirectPath I/O and it is seen by OmniOS/Napp-it, but it causes instability in the system. Not sure how to resolve.
Barebone: Supermicro | Products | SuperStorage Servers | 2U | 6048R-E1CR12L (HBA: SAS3 via LSI 3008 controller; IT mode)
CPU: 2x Intel® Xeon® Haswell-EP Series Processor E5-2630 v3, 2.40 GHz, 8-Core
RAM: 64GB (4x 16GB) Hynix DDR4-2133 CL15 RDIMM
rpool: 1x 64GB SSD SATA DOM
data-pool: 4x 6TB (P/N:0F22791 HUS726060AL5210 ) 2x2 mirror
L2ARC: 1x 400GB Intel® Solid-State Drive DC P3500 Series NVMe (Use only 300GB)
SLOG: Possibly partition 15GB off of P3500
Hypervisor: ESXi 6.0 -- 2016-03-15 (Update 2)
OS: OmniOS 5.11 omnios-r151018-ae3141d April 2016
Napp-it: 16.04f
What I know:
- I was able to attempt to partition the P3500 via Napp-it 300GB/15GB/85GB (L2ARC/SLOG/Overprovision). It choked for a long time but did come back with 296/16/88GB free.
15GB SLOG not critical at this point since all sync=disable (future use)
Overprovision due to the lower DWPD of the P3500
At this point is when I started to notice stability issues within Napp-it.
- The VMs and the file server continue to run and function without any observable issues, but anything (menus) within napp-it dealing with Disks and/or pools either takes forever to respond, or never responds (overnight).
- The OmniOS console window has a curious message shortly after boot if the P3500 is present via DirectPath. I have no physical KVM setup other than what is virtualized by ESXi?
- However, remote SSH via putty continues to work as expected even though the console is hung.
- I removed the DirectPath assignment and Napp-it again responds normally. I did this to load Intel's SSD Data Center Tool onto ESXi. However, the tools cannot even find the P3500 with only ESXi in the mix (No sharing with VMs). Although Intel notes that "On ESXi systems, the tool only works on Intel PCIe NVMe drives using the Intel ESXi NVMe driver.
- I then DirectPath'd the P3500 to a Win 7 VM I have set up, and it immediately recognized the P3500 and saw the partitioning done via Napp-it. I then tried the Window's version of the Data Center Tool and again it cannot see the P3500 (Possibly due to DirectPath?). This is all to try to check the firmware version, and run a self test.
- I was able to get Intel's SSD Toolbox (another tool for consumer drives) to see the P3500 and report some minimal information under Win 7. Intel says that is all this consumer tool can do, so no self test.
- I was able to unpartition the drive via Win 7 and set it back to an unprovisioned 400GB drive.
Attachments
-
31.1 KB Views: 94
-
15.6 KB Views: 92