Fusion-io ioDrive 2 1.2TB Reference Page

[Indecided]

Thanks @acquacow for taking a look. Here's the fio-status -a output

Found 1 VSL driver package:
   4.3.7 build 1205 Storport Driver: loaded

Found 2 ioMemory devices in this system

Adapter: ioMono  (driver 4.3.7)
        Cisco UCS 1000GB MLC Fusion ioMemory PX, Product Number:PFIO1000MP, SN:aaabbbccc
        ioMemory Adapter Controller, PN:PFIO1000MP
        Product UUID:366c7cac-0c6a-5fc4-a688-65d66ccc683a
        PCIe Bus voltage: avg 11.98V
        PCIe Bus current: avg 0.71A
        PCIe Bus power: avg 8.48W
        PCIe Power limit threshold: 24.75W
        PCIe slot available power: unavailable
        Connected ioMemory modules:
          fct0: 05:00.0,        Product Number:PFIO1000MP, SN:aaabbbccc

fct0    Attached
        ioMemory Adapter Controller, Product Number:PFIO1000MP, SN:aaabbbccc
        ioMemory Adapter Controller, PN:PFIO1000MP
        Microcode Versions: App:0.0.44.0
        Powerloss protection: protected
        PCI:05:00.0
        Vendor:1aed, Device:3002, Sub vendor:1137, Sub device:13d
        Firmware v8.9.9, rev 20200113 Public
        1000.00 GBytes device size
        Format: v501, 1953125000 sectors of 512 bytes
        PCIe slot available power: 25.00W
        PCIe negotiated link: 8 lanes at 5.0 Gt/sec each, 4000.00 MBytes/sec total
        Internal temperature: 41.34 degC, max 53.15 degC
        Internal voltage: avg 1.01V, max 1.02V
        Aux voltage: avg 1.79V, max 1.81V
        Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%
        Active media: 100.00%
        Rated PBW: 12.00 PB, 99.99% remaining
        Lifetime data volumes:
           Physical bytes written: 1,169,261,160,616
           Physical bytes read   : 974,149,019,424
        RAM usage:
           Current: 397,517,184 bytes
           Peak   : 444,698,944 bytes
        Contained Virtual Partitions:
          fct0: ID:0, UUID:629a7c7f-26e9-4fd1-af21-b8d69c4129a5

fct0    State: Online, Type: block device, Device: \\?\PhysicalDrive4
        ID:0, UUID:629a7c7f-26e9-4fd1-af21-b8d69c4129a5
        1000.00 GBytes device size
        Format: 1953125000 sectors of 512 bytes
        Sectors In Use: 234801
        Max Physical Sectors Allowed: 1953125000
        Min Physical Sectors Reserved: 1953125000

Adapter: ioMono  (driver 4.3.7)
        Cisco UCS 1000GB MLC Fusion ioMemory PX, Product Number:PFIO1000MP, SN:dddeeefff
        ioMemory Adapter Controller, PN:PFIO1000MP
        Product UUID:5e11d310-2e58-5671-891a-3a10d19a2c52
        PCIe Bus voltage: avg 11.94V
        PCIe Bus current: avg 0.68A
        PCIe Bus power: avg 8.16W
        PCIe Power limit threshold: 24.75W
        PCIe slot available power: unavailable
        Connected ioMemory modules:
          fct1: 04:00.0,        Product Number:PFIO1000MP, SN:dddeeefff

fct1    Attached
        ioMemory Adapter Controller, Product Number:PFIO1000MP, SN:dddeeefff
        ioMemory Adapter Controller, PN:PFIO1000MP
        Microcode Versions: App:0.0.44.0
        Powerloss protection: protected
        PCI:04:00.0
        Vendor:1aed, Device:3002, Sub vendor:1137, Sub device:13d
        Firmware v8.9.9, rev 20200113 Public
        1000.00 GBytes device size
        Format: v501, 1953125000 sectors of 512 bytes
        PCIe slot available power: 25.00W
        PCIe negotiated link: 8 lanes at 5.0 Gt/sec each, 4000.00 MBytes/sec total
        Internal temperature: 39.37 degC, max 49.71 degC
        Internal voltage: avg 1.01V, max 1.02V
        Aux voltage: avg 1.79V, max 1.81V
        Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%
        Active media: 100.00%
        Rated PBW: 12.00 PB, 99.99% remaining
        Lifetime data volumes:
           Physical bytes written: 867,755,546,376
           Physical bytes read   : 767,244,060,576
        RAM usage:
           Current: 448,399,104 bytes
           Peak   : 449,288,704 bytes
        Contained Virtual Partitions:
          fct1: ID:0, UUID:e62248ab-d9e7-4fa2-a03f-b3f4d667b6bd

fct1    State: Online, Type: block device, Device: \\?\PhysicalDrive3
        ID:0, UUID:e62248ab-d9e7-4fa2-a03f-b3f4d667b6bd
        1000.00 GBytes device size
        Format: 1953125000 sectors of 512 bytes
        Sectors In Use: 234801
        Max Physical Sectors Allowed: 1953125000
        Min Physical Sectors Reserved: 1953125000

Here's the output of fio-pci-check, it looks like it's a direct root port connection, no switching/bifurcation.

Root Bridge PCIe 26250 MB/sec needed max
DMI Device

      Bridge 0000:00:02.00 (04-04) (8086:3c04)
      PCI Express Root Port
          Needed 4000 MB/sec Avail 4000 MB/sec

            ioDrive3 0000:04:00.0 (3002) Firmware 20200113
            Fusion ioMemory VSL4 Device
                Slot Power limit: 25.0W (25000mw)

      Bridge 0000:00:02.02 (05-05) (8086:3c06)
      PCI Express Root Port
          Needed 4000 MB/sec Avail 4000 MB/sec

            ioDrive3 0000:05:00.0 (3002) Firmware 20200113
            Fusion ioMemory VSL4 Device
                Slot Power limit: 25.0W (25000mw)

 

[acquacow]

Hmm, all of that looks fine...

How are you testing read speeds? Do you have any output from a fio benchmark on each device you can share?

Thanks,

-- Dave

 

[drizzler]

Hello, is there any chance to rescue a duo drive section after a crashing flash process? the flashing process crashed at 15 % (computer freeze) and now 1 of 2 the two controlles/drivers is not recognized anymore.

I have a CH31 programmer to work with.

Any advices?

Thank you very much

 

[acquacow]

I don't have time to pull all the docs, but you might be able to load the VSL driver in minimal mode and re-flash it that way.

 

[tx12]

Hello, is there any chance to rescue a duo drive section after a crashing flash process? the flashing process crashed at 15 % (computer freeze) and now 1 of 2 the two controlles/drivers is not recognized anymore.

Gen2 FIO cards feature a fallback firmware - in case of programming failure, default firmware with a very early revision date must be loaded enabling you to re-start the flash process. It's possible to disrupt a programming process in a way what would hang chip on a faulty firmware, but it's not too easy to achieve.

If one chip (or both) is not present on PCIe bus anymore (eg, not listed in lspci on Linux), your only option is to reflash it via Xilinx JTAG using programming tool called iMPACT and a compatible USB-JTAG adapter (like DLC9G or DLC10, or Digilent).

 

[tx12]

I don't have time to pull all the docs, but you might be able to load the VSL driver in minimal mode and re-flash it that way.

Driver must auto-load in a minimal mode if fallback firmware was activated.

 

[tx12]

BTW, are schematics and/or board files available for Gen2 / Gen3 cards? Would be so nice to have

 

[acquacow]

BTW, are schematics and/or board files available for Gen2 / Gen3 cards? Would be so nice to have

You'd have to do some digging, but we open sourced the design of one of our cards as part of the opencompute project... so something is out there... I think it was for a gen2 ioscale card.

Edit: https://docs.huihoo.com/opencompute...o-3.2tb-io-accelerator-specification-v0.5.pdf

 

[tx12]

You'd have to do some digging, but we open sourced the design of one of our cards as part of the opencompute project... so something is out there... I think it was for a gen2 ioscale card.

Yep, indeed, it can be spotted in some news from 2013 what ioscale card was "open sourced" to OCP. All I was able to find so far is that hardware description pdf and a STEP-file with 3D model for mechanics. All that is very far from being really open source.
I think Fusion-IO was trying hard to get into Facebook so they claimed that ioscale would be open sourced to OCP. But fact, nothing but a hardware description and a step-file were contributed. That's not anywhere close to real open source hardware, to say the least.

Anyway, this hardware description is much more detailed than any usual user manual.

 

[acquacow]

Well, they were open-sourcing the hardware, there was never any plan to open source the driver/firmware.

 

[tx12]

Well, they were open-sourcing the hardware, there was never any plan to open source the driver/firmware.

Yes, but just a hardware description pdf is nowhere close to real hardware open sourcing. Open source materials for hardware are expected to include (at least) CAD source files for schematics and PCB layouts. Real OCP submissions usually include all of these.

Pretending to go open source without really open sourcing something is well expected from a company with closed source and obfuscated driver, anyway

 

[Lev]

Any help getting this working would be much appreciated. If I understand correctly, I need the v4.0 of the drivers and FW ?

c:\SWTOOLS>fio-status.exe -a

Found 1 VSL driver package:
   4.3.3 build 957 Storport Driver: loaded

Found 1 ioMemory device in this system

Adapter: ioMono  (driver 4.3.3)
        Fusion-io 1.60TB ioScale3, Product Number:F11-004-1T60-CS-0001, SN:1324G0019, FIO SN:1324G0019
        ioScale3 Adapter Controller, PN:PA005453006
        Product UUID:00000000-0000-0000-0000-000000000000
        PCIe Power limit threshold: 24.75W
        PCIe slot available power: unavailable
        Connected ioMemory modules:
          fct0: 03:00.0,        Product Number:F11-004-1T60-CS-0001, SN:1324G0019

fct0    Status unknown: Driver is in MINIMAL MODE:
                The firmware on this device is not compatible with the currently installed version of the driver | Manufacturing configuration data incomplete
        ioScale3 Adapter Controller, Product Number:F11-004-1T60-CS-0001, SN:1324G0019
!! ---> There are active errors or warnings on this device!  Read below for details.
        ioScale3 Adapter Controller, PN:PA005453006
        Microcode Versions: App:0.0.30.0
        Powerloss protection: not available
        PCI:03:00.0
        Vendor:1aed, Device:3002, Sub vendor:1aed, Sub device:3002
        Firmware v8.5.2, rev 114244 Public
        Geometry and capacity information not available.
        Format: not low-level formatted
        PCIe slot available power: unavailable
        PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
        Internal temperature: 56.11 degC, max 61.52 degC
        Internal voltage: avg 1.01V, max 1.02V
        Aux voltage: avg 1.79V, max 1.80V
        Rated PBW: 10.00 PB
        Lifetime data volumes:
           Physical bytes written: 0
           Physical bytes read   : 0
        RAM usage:
           Current: 0 bytes
           Peak   : 0 bytes

        ACTIVE WARNINGS:
            The bandwidth of the PCI slot is not optimal for the ioMemory.
            The ioMemory is currently running in a minimal state.

When I try to update the firmware, using a modified .fff INFO file by adding PA005453006 it returns this response... but I can't find this version of the firmware anywhere when googling or other search engines...

c:\SWTOOLS>fio-update-iodrive.exe fusion_4.3.3-20180423.fff
WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS
  Please wait...this could take a while

Updating: [                    ] (  0%) /
Error: Update from current firmware 8.5.2.114244 to target firmware for device 'fct0' is not allowed.
        Please install the firmware from driver v4.0.0 (firmware v8.5.24.115933)
           and also load that driver before trying this update again.

 

[warlockedyou]

Any help getting this working would be much appreciated. If I understand correctly, I need the v4.0 of the drivers and FW ?

c:\SWTOOLS>fio-status.exe -a

Found 1 VSL driver package:
   4.3.3 build 957 Storport Driver: loaded

Found 1 ioMemory device in this system

Adapter: ioMono  (driver 4.3.3)
        Fusion-io 1.60TB ioScale3, Product Number:F11-004-1T60-CS-0001, SN:1324G0019, FIO SN:1324G0019
        ioScale3 Adapter Controller, PN:PA005453006
        Product UUID:00000000-0000-0000-0000-000000000000
        PCIe Power limit threshold: 24.75W
        PCIe slot available power: unavailable
        Connected ioMemory modules:
          fct0: 03:00.0,        Product Number:F11-004-1T60-CS-0001, SN:1324G0019

fct0    Status unknown: Driver is in MINIMAL MODE:
                The firmware on this device is not compatible with the currently installed version of the driver | Manufacturing configuration data incomplete
        ioScale3 Adapter Controller, Product Number:F11-004-1T60-CS-0001, SN:1324G0019
!! ---> There are active errors or warnings on this device!  Read below for details.
        ioScale3 Adapter Controller, PN:PA005453006
        Microcode Versions: App:0.0.30.0
        Powerloss protection: not available
        PCI:03:00.0
        Vendor:1aed, Device:3002, Sub vendor:1aed, Sub device:3002
        Firmware v8.5.2, rev 114244 Public
        Geometry and capacity information not available.
        Format: not low-level formatted
        PCIe slot available power: unavailable
        PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
        Internal temperature: 56.11 degC, max 61.52 degC
        Internal voltage: avg 1.01V, max 1.02V
        Aux voltage: avg 1.79V, max 1.80V
        Rated PBW: 10.00 PB
        Lifetime data volumes:
           Physical bytes written: 0
           Physical bytes read   : 0
        RAM usage:
           Current: 0 bytes
           Peak   : 0 bytes

        ACTIVE WARNINGS:
            The bandwidth of the PCI slot is not optimal for the ioMemory.
            The ioMemory is currently running in a minimal state.

When I try to update the firmware, using a modified .fff INFO file by adding PA005453006 it returns this response... but I can't find this version of the firmware anywhere when googling or other search engines...

c:\SWTOOLS>fio-update-iodrive.exe fusion_4.3.3-20180423.fff
WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS
  Please wait...this could take a while

Updating: [                    ] (  0%) /
Error: Update from current firmware 8.5.2.114244 to target firmware for device 'fct0' is not allowed.
        Please install the firmware from driver v4.0.0 (firmware v8.5.24.115933)
           and also load that driver before trying this update again.

What version of Windows are you using? I remember some VSL 4.x.x only supported Windows Server 2012 R2 and above.

 

[acquacow]

4.3.7 is the latest driver I have downloaded for those cards, I would probably try that driver with the fusion_4.3.7-20200113.fff that goes with it.

Also:
> Format: not low-level formatted
You're going to need to fio-format the device at some point, but the fact that it's complaining about firmware and also shows this seems kinda sketchy and may make it difficult to get the card going.

-- Dave

 

[Lev]

What version of Windows are you using? I remember some VSL 4.x.x only supported Windows Server 2012 R2 and above.

Windows 10 Enterprise 21H2 build 19044.1387

I haven't encountered any version checks on the OS with any of the driver installs that I thought would of blocked me if Windows 10 was a problem. Or the error messages would indicate that.

I thought I'd just prove the card works before ultimately moving it into my linux server. If it needs a supported OS, I can move the card and try again. Is that what you'd recommend? I suspect these error messages will not change though but IDK.

 

[Lev]

4.3.7 is the latest driver I have downloaded for those cards, I would probably try that driver with the fusion_4.3.7-20200113.fff that goes with it.

Also:
> Format: not low-level formatted
You're going to need to fio-format the device at some point, but the fact that it's complaining about firmware and also shows this seems kinda sketchy and may make it difficult to get the card going.

-- Dave

I gave it a try Dave with the 4.3.7 but no luck, here's the outputs.

PS C:\> fio-status.exe -a

Found 1 VSL driver package:
   4.3.7 build 1205 Storport Driver: loaded

Found 1 ioMemory device in this system

Adapter: ioMono  (driver 4.3.7)
        Fusion-io 1.60TB ioScale3, Product Number:F11-004-1T60-CS-0001, SN:1324G0019, FIO SN:1324G0019
        ioScale3 Adapter Controller, PN:PA005453006
        Product UUID:00000000-0000-0000-0000-000000000000
        PCIe Power limit threshold: 74.75W
        PCIe slot available power: unavailable
        Connected ioMemory modules:
          fct1: 14:00.0,        Product Number:F11-004-1T60-CS-0001, SN:1324G0019

fct1    Status unknown: Driver is in MINIMAL MODE:
                The firmware on this device is not compatible with the currently installed version of the driver | Manufacturing configuration data incomplete
        ioScale3 Adapter Controller, Product Number:F11-004-1T60-CS-0001, SN:1324G0019
!! ---> There are active errors or warnings on this device!  Read below for details.
        ioScale3 Adapter Controller, PN:PA005453006
        Microcode Versions: App:0.0.30.0
        Powerloss protection: not available
        PCI:14:00.0
        Vendor:1aed, Device:3002, Sub vendor:1aed, Sub device:3002
        Firmware v8.5.2, rev 114244 Public
        Geometry and capacity information not available.
        Format: not low-level formatted
        PCIe slot available power: 75.00W
        PCIe negotiated link: 4 lanes at 5.0 Gt/sec each, 2000.00 MBytes/sec total
        Internal temperature: 48.23 degC, max 49.71 degC
        Internal voltage: avg 1.01V, max 1.01V
        Aux voltage: avg 1.79V, max 1.80V
        Rated PBW: 10.00 PB
        Lifetime data volumes:
           Physical bytes written: 0
           Physical bytes read   : 0
        RAM usage:
           Current: 0 bytes
           Peak   : 0 bytes

        ACTIVE WARNINGS:
            The bandwidth of the PCI slot is not optimal for the ioMemory.
            The ioMemory is currently running in a minimal state.

Then I tried to flash it, it failed as expected....

PS C:\> fio-update-iodrive.exe .\dell_iodrive_4.3.7-20200113.fff
WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS
  Please wait...this could take a while

Updating: [                    ] (  0%) /
Error: Firmware file does not contain firmware for device 'fct1', part 'PA005453006'.

I edited the INFO file inside the .fff file, adding PA005453006 to the top of this section

[PA005453006]
version = 8.9.9.118194
file = gen3_tangerine_fusion_8.9.9.118194.bin
format = bin
ecc = 11b
cntrpdiversion = 0.0.15
cntrpdifile = avogadro_0.0.15.pdi
cntrpdiformat = pdi
nce = 1f80000
nceversion = 1.0.9.100764
ncefile = nce_1.0.9.100764.bin
errfw = 1f80000
errfwversion = 0.0.3
errfwfile = errfw_0.0.3.bin
optrom = 3c00000
optromversion = 4.3.7.1205
optromfile = uefi-noboot-3002_4.3.7.1205.rom

Retried flashing with the modified INFO file in the .fff file

PS C:\> fio-update-iodrive.exe .\dell_iodrive_4.3.7-20200113.fff
WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS
  Please wait...this could take a while

Updating: [                    ] (  0%) /
Error: Update from current firmware 8.5.2.114244 to target firmware for device 'fct1' is not allowed.
        Please install the firmware from driver v4.0.0 (firmware v8.5.24.115933)
           and also load that driver before trying this update again.

 

[warlockedyou]

Windows 10 Enterprise 21H2 build 19044.1387

I haven't encountered any version checks on the OS with any of the driver installs that I thought would of blocked me if Windows 10 was a problem. Or the error messages would indicate that.

I thought I'd just prove the card works before ultimately moving it into my linux server. If it needs a supported OS, I can move the card and try again. Is that what you'd recommend? I suspect these error messages will not change though but IDK.

So the reason I asked was because I was looking at my notes and saw this chart for VSL Compatibility. But I am also not sure if you want official drivers or third party drivers.
I am not as knowledgeable as acqua so I might be going down the wrong rabbit hole

Attached is the VSL chart I am talking about.


@acquacow - How do you generally interpret the output of "fio-status.exe -a" that Lev posted? I am trying to understand what the actual device name is based on the output.

1. Is it an adapter type called ioMono?
2. But then it says the device is actually called ioScale3 by Fusion-io
3. Is ioScale3 different from another product say...io3 Flash Adapter?(I got this flash adapter name from a random driver file that I have saved for Windows)
4. What are these naming conventions?! I am so confused

I have a few drivers/firmware that I downloaded but I don't know what device/product they are for. So I am trying to see if I can make some sense of it and understand whether or not they can help out Lev.

 

[Lev]

So the reason I asked was because I was looking at my notes and saw this chart for VSL Compatibility. But I am also not sure if you want official drivers or third party drivers.
I am not as knowledgeable as acqua so I might be going down the wrong rabbit hole

I searched all over Google and DuckDuckGo trying to find Windows drivers v4.0.2 so I could meet the requirements of the error message output when I tried to flash. It's my understanding from reading the user guides I found for the product that it expects every firmware update to be done in sequential order. It's the expected behavior that it's not letting me flash directly to 4.3.7 and skip all the firmware versions in-between.

@warlockedyou Do you have the Windows v4.0.2 drivers or the v4.0.2 firmware?

1. Is it an adapter type called ioMono?
2. But then it says the device is actually called ioScale3 by Fusion-io
3. Is ioScale3 different from another product say...io3 Flash Adapter?(I got this flash adapter name from a random driver file that I have saved for Windows)
4. What are these naming conventions?! I am so confused

I know the feeling! the names make no sense. It kind of fits with a theory I have that perhaps this card is an engineering sample. I don't know any history on the card from before I obtained it. Even looking at the outside of the card, there's only one label.

 

[warlockedyou]

I searched all over Google and DuckDuckGo trying to find Windows drivers v4.0.2 so I could meet the requirements of the error message output when I tried to flash. It's my understanding from reading the user guides I found for the product that it expects every firmware update to be done in sequential order. It's the expected behavior that it's not letting me flash directly to 4.3.7 and skip all the firmware versions in-between.

@warlockedyou Do you have the Windows v4.0.2 drivers or the v4.0.2 firmware?



I know the feeling! the names make no sense. It kind of fits with a theory I have that perhaps this card is an engineering sample. I don't know any history on the card from before I obtained it. Even looking at the outside of the card, there's only one label.

That front picture looks really like "Fusion ioMemory SX300". Not sure if I am allowed to link the article so you can just search google images for that name and it should be in top 3 images.

If this is true and it is SX300, would it make sense to first get the driver and firmware to match before updating it? This way, you can atleast check if the drive is working.

 

[Lev]

If this is true and it is SX300, would it make sense to first get the driver and firmware to match before updating it? This way, you can atleast check if the drive is working.

Totally agree. I want to try it but I cannot seem to find v4.0.2 of the driver and firmware anywhere on the internet.