Fusion-io ioDrive 2 1.2TB Reference Page

acquacow · Jul 13, 2017

BLinux said:
thanks for looking it up. i was searching the forums too, but the links to ebay were all dead. and no, thank goodness, that wasn't my seller.

They just arrived too:

View attachment 6122

Welcome to the club. Let me know if you need any support.

BLinux · Jul 14, 2017

acquacow said:
Welcome to the club. Let me know if you need any support.

thanks @acquacow !! btw, question for you... I've seen the ioDrive2 on eBay with silver heatsinks and some with black heatsinks. Is that indicative of particular revisions? And if so, what are the differences other than heatsink color? Or, was that just a change in heatsink supplier?

acquacow · Jul 14, 2017

Just a change in supplier. I think most of the Dell OEM cards are black, as well as some/most of the IBM ones. The cards are all electrically the same.

gigatexal · Jul 14, 2017

Patrick said:
Starting a thread on the Fusion-io ioDrive 2 1.2TB cards as they seem to be on the market fairly often now at under $0.50/GB. Hopefully others can chime in with experiences.

Key Specs
Capacity: 1.2TB
Formatted capacity in Windows: ~1.1TB
Random 4k read IOPS (tested via Iometer): 330K - 340K
128KB Sequential Read: 1.5GB/s
Write endurance: 16.26PB
Interface: PCIe 2.0 x8 (works in older servers)

Notes
Actual 4K Random Read IOPS using a Windows 10 Pro test machine were over 330K. That figure is higher than the 2013 rated specs.
View attachment 3413
For those wondering why that is an impressive screenshot: it shows one of these cards running over its rated IOPS. It is also pushing almost it entire 1.5GB/s rating while doing a 4K IOPS test at QD 64 (much lower than most NVMe drives can peak.)

Installation of the HP/ HPE card worked in Windows 10 Pro without issue by simply installing the driver packs from the WD/ SanDisk/ Fusion-io support site. One item to be careful about with these drives is that they are not boot drives and they do not have default drivers in Windows/ Linux. As a result, unlike NVMe drives, you will have to install the driver. You can simply register on the WD/ SanDisk/ Fusion-io site to get access to the downloads.

The performance of these drives fall off much less under heavy writes than the Intel DC P3600's. For read optimized workloads, an Intel 750 or P3600 may be a better option. If you are doing heavy data crunching or need a fast read/ write drive for video editing, the ioDrive 2 is still fairly good. It is unlikely used drives are going to have anywhere near 16.26PB of write endurance used.

Pricing
As of Sept 2016 these are now selling regularly for sub $500 for the 1.2TB card. Good pricing is in the $400 to $450 range.

ebay search: Fusion io ioDrive II 1.2TB | Lowest Price

Linux support is plug n play I’m assuming?

acquacow · Jul 14, 2017

Nothing Fusion-io is plug and play.

Linux requires a driver/kernel module that matches your kernel version, else you need to download the driver source and recompile against your current kernel via rpm-build.

It's one extra step, but you have to have kernel-headers and kernel-devel installed as well as rpmbuild packages.

If you run an unrestricted "yum update" or "yum upgrade" you will probably accidentally upgrade your kernel and then will have to rebuild your ioDrive kernel module

If you did not uninstall your ioDrive kernel module prior to a kernel upgrade, there's a high probability your old ioDrive kernel module now exists in your /boot/initrd or /boot/initramfs and will be incompatible with your new kernel even after you rebuild the drivers. To fix that you have to run a "dracut -f" to fix the boot files to include the newly compiled kernel module for the ioDrive. In the future you'll want to edit /etc/dracut.conf and include an "omit_modules+=iomemory-vsl" This will prevent the issue from happening in the first place.

BLinux · Jul 14, 2017

acquacow said:
Nothing Fusion-io is plug and play.

Linux requires a driver/kernel module that matches your kernel version, else you need to download the driver source and recompile against your current kernel via rpm-build.

It's one extra step, but you have to have kernel-headers and kernel-devel installed as well as rpmbuild packages.

If you run an unrestricted "yum update" or "yum upgrade" you will probably accidentally upgrade your kernel and then will have to rebuild your ioDrive kernel module

If you did not uninstall your ioDrive kernel module prior to a kernel upgrade, there's a high probability your old ioDrive kernel module now exists in your /boot/initrd or /boot/initramfs and will be incompatible with your new kernel even after you rebuild the drivers. To fix that you have to run a "dracut -f" to fix the boot files to include the newly compiled kernel module for the ioDrive. In the future you'll want to edit /etc/dracut.conf and include an "omit_modules+=iomemory-vsl" This will prevent the issue from happening in the first place.

There's no dkms solution?

acquacow · Jul 15, 2017

There are dkms solutions out there, we made one, but I'm not sure it ever got pushed into the finalized driver releases.

gigatexal · Jul 15, 2017

acquacow said:
There are dkms solutions out there, we made one, but I'm not sure it ever got pushed into the finalized driver releases.

The DKMS way is rad as when I go to upgrade kernels the ZFS modules automatically rebuild.

BLinux · Jul 18, 2017

ok, so reporting back on those ioDrive-II 1.2TB I got last week. one seems to work, while the other may have problems (if anyone can help, that would be great).

first, for CentOS 7, I didn't have to compile/build any driver rpms. the provided RPMs worked with the latest kernel and it was just a matter of installing RPM files. As part of that process, it ran dracut and rebuilt the initramfs images too. So, this was all very straight forward really.

Once I loaded the driver module (modprobe iomemory-vsl), the ioDrive2 showed up, but said the card was in "minimal mode" due to firmware version:

Code:

Found 1 ioMemory device in this system
Driver version: 3.2.15 build 1699

Adapter: Single Controller Adapter
    Fusion-io ioDrive2 1.205TB, Product Number:F00-001-1T20-CS-0001, SN:1241D0473, FIO SN:1241D0473
    ioDrive2 Adapter Controller, PN:PA004137009
    External Power: NOT connected
    PCIe Power limit threshold: Disabled
    PCIe slot available power: 25.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Connected ioMemory modules:
     fct0:    Product Number:F00-001-1T20-CS-0001, SN:1241D0473

fct0    Status unknown: Driver is in MINIMAL MODE:
        The firmware on this device is not compatible with the currently installed version of the driver
    ioDrive2 Adapter Controller, Product Number:F00-001-1T20-CS-0001, SN:1241D0473
!! --->    There are active errors or warnings on this device!  Read below for details.
    ioDrive2 Adapter Controller, PN:PA004137009
    SMP(AVR) Versions: App Version: 1.0.15.0, Boot Version: 0.0.3.1
    Located in slot 0 Center of ioDrive2 Adapter Controller SN:1241D0473
    Powerloss protection: not available
    PCI:04:00.0, Slot Number:2
    Vendor:1aed, Device:2001, Sub vendor:1aed, Sub device:2001
    Firmware v7.0.2, rev 108609 Public
    Geometry and capacity information not available.
    Format: not low-level formatted
    PCIe slot available power: 25.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Internal temperature: 46.76 degC, max 53.65 degC
    Internal voltage: avg 1.02V, max 1.02V
    Aux voltage: avg 2.49V, max 2.49V
    Rated PBW: 17.00 PB
    Lifetime data volumes:
      Physical bytes written: 0
      Physical bytes read   : 0
    RAM usage:
      Current: 0 bytes
      Peak   : 0 bytes

    ACTIVE WARNINGS:
       The ioMemory is currently running in a minimal state.

I know there was mention in this thread about hopping through various versions of firmware, but I guess mine was new enough I just ran fio-update-iodrive and it upgraded just fine. upon reboot, i now see this:

Code:

Found 1 ioMemory device in this system
Driver version: 3.2.15 build 1699

Adapter: Single Controller Adapter
    Fusion-io ioDrive2 1.205TB, Product Number:F00-001-1T20-CS-0001, SN:1241D0473, FIO SN:1241D0473
    ioDrive2 Adapter Controller, PN:PA004137009
    External Power: NOT connected
    PCIe Bus voltage: avg 12.05V
    PCIe Bus current: avg 0.73A
    PCIe Bus power: avg 8.61W
    PCIe Power limit threshold: 24.75W
    PCIe slot available power: 25.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Connected ioMemory modules:
     fct0:    Product Number:F00-001-1T20-CS-0001, SN:1241D0473

fct0    Detached
    ioDrive2 Adapter Controller, Product Number:F00-001-1T20-CS-0001, SN:1241D0473
!! --->    There are active errors or warnings on this device!  Read below for details.
    ioDrive2 Adapter Controller, PN:PA004137009
    SMP(AVR) Versions: App Version: 1.0.35.0, Boot Version: 0.0.9.1
    Located in slot 0 Center of ioDrive2 Adapter Controller SN:1241D0473
    Powerloss protection: protected
    PCI:04:00.0, Slot Number:2
    Vendor:1aed, Device:2001, Sub vendor:1aed, Sub device:2001
    Firmware v7.1.17, rev 116786 Public
    Geometry and capacity information not available.
    Format: not low-level formatted
    PCIe slot available power: 25.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Internal temperature: 43.31 degC, max 43.31 degC
    Internal voltage: avg 1.02V, max 1.02V
    Aux voltage: avg 2.49V, max 2.49V
    Rated PBW: 17.00 PB, 100.00% remaining
    Lifetime data volumes:
      Physical bytes written: 730,080
      Physical bytes read   : 66,733,056
    RAM usage:
      Current: 11,720,000 bytes
      Peak   : 11,720,000 bytes

    ACTIVE WARNINGS:
       The ioMemory is missing a LEB map.

The "Physical bytes written" looked really, really low. But the warning message "The ioMemory is missing a LEB map" was the beginning of my troubles. I then did a fio-detach, fio-format and it wouldn't format and gave me the same "missing LEB map" error. Looking through the system logs, I see more errors:

Code:

Jul 17 18:32:22 vm-d125 kernel: <6>fioinf VSL configuration hash: 50cc3bdba9fe52b90d1821e59d81452e4a6eac09
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf Copyright (c) 2006-2014 Fusion-io, Inc. (acquired by SanDisk Corp. 2014)
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf Copyright (c) 2014-2016 SanDisk Corp. and/or all its affiliates. All rights reserved.
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf For Terms and Conditions see the License file included
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf with this driver package.
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive driver 3.2.15.1699 pinnacles@f0f84521e1b1 loading...
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0: mapping controller on BAR 5
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0: MSI enabled
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0: using MSI interrupts
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Starting master controller
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: PMP Address: 1 1 1
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: SMP Controller Firmware APP  version 1.0.35 0
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: SMP Controller Firmware BOOT version 0.0.9 1
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Required PCIE bandwidth 2.000 GBytes per sec
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Board serial number is 1241D0473
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Adapter serial number is 1241D0473
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Default capacity        1205.000 GBytes
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Default sector size     512 bytes
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Rated endurance         17.00 PBytes
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: 100C temp range hardware found
Jul 17 18:32:22 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Maximum capacity        1294.000 GBytes
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Firmware version 7.1.17 116786 (0x700411 0x1c832)
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Platform version 16
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Firmware VCS version 116786 [0x1c832]
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Firmware VCS uid 0xaeb15671994a45642f91efbb214fa428e4245f8a
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Powercut flush: Enabled
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: PCIe power monitor enabled (master). Limit set to 24.750 watts.
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Thermal monitoring: Enabled
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0.0: Hardware temperature alarm set for 100C.
Jul 17 18:32:23 vm-d125 kernel: <6>fioinf ioDrive 0000:04:00.0: Found device fct0 (Fusion-io ioDrive2 1.205TB 0000:04:00.0) on pipeline 0
Jul 17 18:32:23 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (0:936 - 32872). EB read count = 47.
Jul 17 18:32:24 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (1:936 - 32872). EB read count = 47.
Jul 17 18:32:24 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (2:936 - 32872). EB read count = 46.
Jul 17 18:32:25 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (4:936 - 32872). EB read count = 46.
Jul 17 18:32:25 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (5:936 - 32872). EB read count = 46.
Jul 17 18:32:25 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (6:936 - 32872). EB read count = 45.
Jul 17 18:32:26 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (7:936 - 32872). EB read count = 45.
Jul 17 18:32:26 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (8:936 - 32872). EB read count = 46.
Jul 17 18:32:27 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (10:936 - 32872). EB read count = 46.
Jul 17 18:32:27 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (12:936 - 32872). EB read count = 46.
Jul 17 18:32:27 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (15:936 - 32872). EB read count = 46.
Jul 17 18:32:28 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (17:936 - 32872). EB read count = 46.
Jul 17 18:32:28 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (18:936 - 32872). EB read count = 46.
Jul 17 18:32:29 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (19:936 - 32872). EB read count = 46.
Jul 17 18:32:29 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (21:936 - 32872). EB read count = 46.
Jul 17 18:32:30 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (24:936 - 32872). EB read count = 46.
Jul 17 18:32:30 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (25:936 - 32872). EB read count = 46.
Jul 17 18:32:30 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (26:936 - 32872). EB read count = 46.
Jul 17 18:32:31 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (36:936 - 32872). EB read count = 46.
Jul 17 18:32:31 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (39:936 - 32872). EB read count = 46.
Jul 17 18:32:32 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (40:936 - 32872). EB read count = 46.
Jul 17 18:32:32 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (43:936 - 32872). EB read count = 46.
Jul 17 18:32:32 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (45:0 - 936). EB read count = 22.
Jul 17 18:32:33 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (48:936 - 32872). EB read count = 46.
Jul 17 18:32:33 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (49:936 - 32872). EB read count = 46.
Jul 17 18:32:33 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (51:936 - 32872). EB read count = 46.
Jul 17 18:32:34 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (53:936 - 32872). EB read count = 46.
Jul 17 18:32:34 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (55:936 - 32872). EB read count = 46.
Jul 17 18:32:34 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (59:936 - 32872). EB read count = 46.
Jul 17 18:32:35 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (60:936 - 32872). EB read count = 46.
Jul 17 18:32:35 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (64:936 - 32872). EB read count = 46.
Jul 17 18:32:36 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (66:936 - 32872). EB read count = 46.
Jul 17 18:32:36 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (67:936 - 32872). EB read count = 46.
Jul 17 18:32:36 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (68:936 - 32872). EB read count = 46.
Jul 17 18:32:37 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (70:936 - 32872). EB read count = 46.
Jul 17 18:32:37 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (74:936 - 32872). EB read count = 46.
Jul 17 18:32:37 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (77:936 - 32872). EB read count = 46.
Jul 17 18:32:38 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (78:936 - 32872). EB read count = 46.
Jul 17 18:32:38 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (81:936 - 32872). EB read count = 46.
Jul 17 18:32:39 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (86:936 - 32872). EB read count = 46.
Jul 17 18:32:39 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (87:936 - 32872). EB read count = 46.
Jul 17 18:32:39 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (91:936 - 32872). EB read count = 46.
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (92:936 - 32872). EB read count = 46.
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (95:936 - 32872). EB read count = 46.
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Read request ffff880073f70800 read failed (99:936 - 32872). EB read count = 46.
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr 0000:04:00.0: Error: no LEB map found.
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Unable to read LEB map.  Disabling attach.
Jul 17 18:32:40 vm-d125 kernel: <6>fioinf Fusion-io ioDrive2 1.205TB 0000:04:00.0: No leb map for NV data scan. Be patient; expect several parity substitution messages.
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: No NV data available
Jul 17 18:32:40 vm-d125 kernel: <6>fioinf Fusion-io ioDrive2 1.205TB 0000:04:00.0: Init NV data
Jul 17 18:32:40 vm-d125 kernel: <6>fioinf Fusion-io ioDrive2 1.205TB 0000:04:00.0: probed fct0
Jul 17 18:32:40 vm-d125 kernel: <6>fioinf Fusion-io ioDrive2 1.205TB 0000:04:00.0: Attaching explicitly disabled
Jul 17 18:32:40 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: auto attach failed with error EINVAL: Invalid argument
Jul 17 18:46:18 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Unhandled ioctl: 0x80081272
Jul 17 18:46:18 vm-d125 kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:04:00.0: Unhandled ioctl: 0x1260

The "read failed" has me thinking this is a defective drive??? Any suggestions if there is anything I can do here?

BLinux · Jul 18, 2017

Now, on the 2nd ioDrive-II, things looked much better. Did the same as above, by first updating the firmware and then upon reboot:

Code:

Found 1 ioMemory device in this system
Driver version: 3.2.15 build 1699

Adapter: Single Controller Adapter
    Fusion-io ioDrive2 1.205TB, Product Number:F00-001-1T20-CS-0001, SN:1241D0288, FIO SN:1241D0288
    ioDrive2 Adapter Controller, PN:PA004137009
    External Power: NOT connected
    PCIe Bus voltage: avg 11.96V
    PCIe Bus current: avg 0.74A
    PCIe Bus power: avg 8.76W
    PCIe Power limit threshold: 24.75W
    PCIe slot available power: 25.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Connected ioMemory modules:
     fct0:    Product Number:F00-001-1T20-CS-0001, SN:1241D0288

fct0    Attaching (36%)
    ioDrive2 Adapter Controller, Product Number:F00-001-1T20-CS-0001, SN:1241D0288
    ioDrive2 Adapter Controller, PN:PA004137009
    SMP(AVR) Versions: App Version: 1.0.35.0, Boot Version: 0.0.9.1
    Located in slot 0 Center of ioDrive2 Adapter Controller SN:1241D0288
    Powerloss protection: protected
    PCI:04:00.0, Slot Number:2
    Vendor:1aed, Device:2001, Sub vendor:1aed, Sub device:2001
    Firmware v7.1.17, rev 116786 Public
    1205.00 GBytes device size
    Format: v500, 2353515625 sectors of 512 bytes
    PCIe slot available power: 25.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Internal temperature: 42.33 degC, max 43.31 degC
    Internal voltage: avg 1.02V, max 1.02V
    Aux voltage: avg 2.49V, max 2.50V
    Rated PBW: 17.00 PB, 98.87% remaining
    Lifetime data volumes:
      Physical bytes written: 192,386,074,875,352
      Physical bytes read   : 77,277,084,070,728
    RAM usage:
      Current: 496,944,000 bytes
      Peak   : 496,944,000 bytes

Looks like this one got some use with about 175TB written. But at least this one did not seem to have any errors. I did a fio-detach, fio-format, fio-attach and it worked just fine and /dev/fioa showed up. I created a file system on that and mounted it just fine.

acquacow · Jul 18, 2017

For the problem drive, detach it and run "fio-sure-erase" on it, followed by a "fio-format" and see if that clears it up.

You may have to run sure-erase a few times if it errors out mid-way. Sometimes it'll correct itself after a few runs and complete.

BLinux · Jul 18, 2017

acquacow said:
For the problem drive, detach it and run "fio-sure-erase" on it, followed by a "fio-format" and see if that clears it up.

You may have to run sure-erase a few times if it errors out mid-way. Sometimes it'll correct itself after a few runs and complete.

thanks! i'm going to try that out soon... by the way, what is the LEB map? and how do I interpret those read errors? (what does the message mean and the numbers?)

BLinux · Jul 18, 2017

acquacow said:
For the problem drive, detach it and run "fio-sure-erase" on it, followed by a "fio-format" and see if that clears it up.

You may have to run sure-erase a few times if it errors out mid-way. Sometimes it'll correct itself after a few runs and complete.

so far, not having much luck on this one. i've run fio-sure-erase about 8 times now and every time it gets to about 49%, then errors out and then jumps to 100% done and aborts:

Code:

# fio-sure-erase /dev/fct0  -y
WARNING: sanitizing will destroy any existing data on the device!
Erasing blocks: [====================] (100%) |
Error: Device '/dev/fct0' had an error while sanitizing. Could not clear flash data. Make sure the device is detached first (by running fio-detach).
Failure: Fusion-io ioDrive2 1.205TB PN:F00-001-1T20-CS-0001 SN:1241D0473 FIO-SN:1241D0473 Device:/dev/fct0

That happens consistently around 49%. In the logs, I see this:

Code:

Jul 18 13:14:42 meru kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:05:00.0: Unsupported physical capacity (0), OS max 1294000000000 bytes
Jul 18 13:30:26 meru kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:05:00.0: Unsupported physical capacity (0), OS max 1294000000000 bytes
Jul 18 13:44:25 meru kernel: <3>fioerr Fusion-io ioDrive2 1.205TB 0000:05:00.0: Unsupported physical capacity (0), OS max 1294000000000 bytes

I poked around a bit to see what options i had with fio-sure-erase:

Code:

# fio-sure-erase -h
Fusion fio-sure-erase utility (3.2.15.1699 pinnacles@f0f84521e1b1)
  Copyright (c) 2006-2017 Western Digital Corporation or all its affiliates.
usage: fio-sure-erase [option] <device-node>
  Options:
   -p, --purge          Performs a write followed by an erase
   -t, --default        Do not preserve current format parameters (reset to default)
   -y, --yes            Do not prompt for confirmation
   -q, --quiet          Do not print status bar
   -v, --version        Print version information
   -h, --help           Print the help menu

So, I noticed the "-t" option and tried that too. When I did that, it errored out at the same 49% mark, but the log message above did not show. I'm guessing that log message is a result of the driver probing the ioDrive2 after the fio-sure-erase exits and not something that is logged during the sure-erase?

I also saw the "-p" option, but when I ran that, it was stuck (made no progress, but baton was still spinning) at 0% for 15 minutes so I killed off the process. Tried fio-sure-erase again with no options and still errors out at around 49%.

I'm feeling like i might be SOL on this one....

BLinux · Jul 20, 2017

just an update... contacted the seller (ebay seller: insightinvestments) and they promptly sent me a replacement and paid for the return of the defective unit. The replacement appears to work without any issues:

Code:

Found 1 ioMemory device in this system
Driver version: 3.2.15 build 1699

Adapter: Single Controller Adapter
    Fusion-io ioDrive2 1.205TB, Product Number:F00-001-1T20-CS-0001, SN:1241D0162, F
IO SN:1241D0162
    ioDrive2 Adapter Controller, PN:PA004137009
    External Power: NOT connected
    PCIe Bus voltage: avg 11.96V
    PCIe Bus current: avg 0.80A
    PCIe Bus power: avg 9.55W
    PCIe Power limit threshold: 24.75W
    PCIe slot available power: 75.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Connected ioMemory modules:
     fct0:    Product Number:F00-001-1T20-CS-0001, SN:1241D0162

fct0    Detached
    ioDrive2 Adapter Controller, Product Number:F00-001-1T20-CS-0001, SN:1241D0162
    ioDrive2 Adapter Controller, PN:PA004137009
    SMP(AVR) Versions: App Version: 1.0.35.0, Boot Version: 0.0.9.1
    Located in slot 0 Center of ioDrive2 Adapter Controller SN:1241D0162
    Powerloss protection: protected
    PCI:05:00.0, Slot Number:3
    Vendor:1aed, Device:2001, Sub vendor:1aed, Sub device:2001
    Firmware v7.1.17, rev 116786 Public
    1205.00 GBytes device size
    Format: v500, 2353515625 sectors of 512 bytes
    PCIe slot available power: 75.00W
    PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
    Internal temperature: 69.40 degC, max 69.89 degC
    Internal voltage: avg 1.01V, max 1.01V
    Aux voltage: avg 2.49V, max 2.49V
    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%
    Active media: 100.00%
    Rated PBW: 17.00 PB, 99.35% remaining
    Lifetime data volumes:
      Physical bytes written: 110,436,550,315,256
      Physical bytes read   : 39,417,863,673,696
    RAM usage:
      Current: 11,720,000 bytes
      Peak   : 502,971,840 bytes

Looks like it's had about 100TB written. Still 99.35% remaining, so not bad for $250.

acquacow · Jul 20, 2017

Oh, I was about to recommend you drop down to 3.1.1 driver/firmware then try again. Different versions of the tools have different checks/etc. LEB maps are things you can usually re-create and correct (assuming you don't need any data off the card).

BLinux · Jul 20, 2017

acquacow said:
Oh, I was about to recommend you drop down to 3.1.1 driver/firmware then try again. Different versions of the tools have different checks/etc. LEB maps are things you can usually re-create and correct (assuming you don't need any data off the card).

what is the LEB map anyway?

acquacow · Jul 20, 2017

It's the mapping of where all the erase blocks are across the drive and I believe it also keeps track of all the pointers that connect them into a contiguous data log.

Each EB also has pointer info in it, so the mapping can be rebuilt.

BLinux · Jul 20, 2017

acquacow said:
It's the mapping of where all the erase blocks are across the drive and I believe it also keeps track of all the pointers that connect them into a contiguous data log.

Each EB also has pointer info in it, so the mapping can be rebuilt.

is there a particular location where the LEB is written to? is that created during a fio-format?

considering the low bytes written count on that defective one, i wonder if perhaps they never successfully completed fio-format, and hence no LEB was ever written? or, does it come from the factory with a LEB map?

acquacow · Jul 21, 2017

It comes from the factory with a map. I believe it's on the FPGA somewhere, and also spread across the drive. It also keeps track of which blocks are bad in the NAND so it knows to skip them/etc. Completely wiping the map destroys this data, but it can be re-created over time as errors are encountered via failed writes and reads.

marcus556 · Feb 6, 2018

How does this drive compare to the new Intel Optane 900p? I was thinking about grabbing one or the other for VMs on my ESXi server. One thing that is evident is that the ioDrive2 has a better price per GB then the 900p (I say this based off eBay purchasing). It also supports older boards and I am not sure if the 900p does (to boot to the drive newer boards have to be used not sure if it will still work as a stand alone device). 900p may not be fully supported by VMWare but I have seen videos where it will run. I saw a review where the write IOPS for the ioDrive2 can reach 800k on the 1.2TB model, is this true? If so then this beats the 900p easily.

Thanks for your input!

Sent from my Pixel 2 XL using Tapatalk

Fusion-io ioDrive 2 1.2TB Reference Page

Well-Known Member

cat lover server enthusiast

Well-Known Member

I'm here to learn

Well-Known Member

cat lover server enthusiast

Well-Known Member

I'm here to learn

cat lover server enthusiast

cat lover server enthusiast

Well-Known Member

cat lover server enthusiast

cat lover server enthusiast

cat lover server enthusiast

Well-Known Member

cat lover server enthusiast

Well-Known Member

cat lover server enthusiast

Well-Known Member

New Member