Fusion-io ioDrive 2 1.2TB Reference Page

Discussion in 'Hard Drives and Solid State Drives' started by Patrick, Sep 14, 2016.

  1. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,344
    Likes Received:
    4,294
    Starting a thread on the Fusion-io ioDrive 2 1.2TB cards as they seem to be on the market fairly often now at under $0.50/GB. Hopefully others can chime in with experiences.

    Key Specs
    Capacity: 1.2TB
    Formatted capacity in Windows: ~1.1TB
    Random 4k read IOPS (tested via Iometer): 330K - 340K
    128KB Sequential Read: 1.5GB/s
    Write endurance: 16.26PB
    Interface: PCIe 2.0 x8 (works in older servers)

    Notes
    Actual 4K Random Read IOPS using a Windows 10 Pro test machine were over 330K. That figure is higher than the 2013 rated specs.
    upload_2016-9-14_14-9-33.png
    For those wondering why that is an impressive screenshot: it shows one of these cards running over its rated IOPS. It is also pushing almost it entire 1.5GB/s rating while doing a 4K IOPS test at QD 64 (much lower than most NVMe drives can peak.)

    Installation of the HP/ HPE card worked in Windows 10 Pro without issue by simply installing the driver packs from the WD/ SanDisk/ Fusion-io support site. One item to be careful about with these drives is that they are not boot drives and they do not have default drivers in Windows/ Linux. As a result, unlike NVMe drives, you will have to install the driver. You can simply register on the WD/ SanDisk/ Fusion-io site to get access to the downloads.

    The performance of these drives fall off much less under heavy writes than the Intel DC P3600's. For read optimized workloads, an Intel 750 or P3600 may be a better option. If you are doing heavy data crunching or need a fast read/ write drive for video editing, the ioDrive 2 is still fairly good. It is unlikely used drives are going to have anywhere near 16.26PB of write endurance used.

    Pricing
    As of Sept 2016 these are now selling regularly for sub $500 for the 1.2TB card. Good pricing is in the $400 to $450 range.

    ebay search: Fusion io ioDrive II 1.2TB | Lowest Price
     
    #1
    kdotcarter and T_Minus like this.
  2. lmk

    lmk Member

    Joined:
    Dec 11, 2013
    Messages:
    128
    Likes Received:
    20
    What is the latency like?

    E.g. compared to S3500 or S3700?
     
    #2
  3. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,344
    Likes Received:
    4,294
    It is still doing a 4K random read test but showing about 0.5-0.6ms even at 330K IOPS. Generally these are going to be much lower than SATA drives.
     
    #3
  4. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,840
    Likes Received:
    813
    I'm taking away from this that these basically saturate the PCIE 2.0 bus even with 4K. Nice.
     
    #4
  5. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,533
    Likes Received:
    355
    Can the iodrive be used like a normal sata(or sas) drive in windows storage spaces?
     
    #5
  6. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,513
    Likes Received:
    1,344
    During that 'peak 4k test' any idea what power usage was vs idle on the card?
     
    #6
  7. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    1,954
    Likes Received:
    889
    No working in S2D
     
    #7
  8. lowfat

    lowfat Member

    Joined:
    Nov 25, 2016
    Messages:
    72
    Likes Received:
    57
    Will these work as a datastore in ESXi? I have an older ioXtreme that doesn't but as far as a I know most of the other drives should.

    EDIT: Found the acticle saying it does.

    EDIT2: Ordered a 1.2TB ioScale on eBay. I'm exited.
     
    #8
    Last edited: Dec 16, 2016
  9. lowfat

    lowfat Member

    Joined:
    Nov 25, 2016
    Messages:
    72
    Likes Received:
    57
    A couple of benches of the 1.2TB ioScale I bought. I believe it is nearly the same as the ioDrive 2 but higher write endurance. Paid $362USD ($486CAD after taxes/shipping).

    Had to override the PCIe slot power draw as it would only allow for 25w. Took me forever to find the command so if someone else needs it
    Code:
    fio-config -p FIO_EXTERNAL_POWER_OVERRIDE [driveserialnumber]:75000
    The ioScale

    [​IMG]

    [​IMG]

    And a 256GB Samsung 950 Pro to compare against.
    [​IMG]

    [​IMG]


    So sequential reads are slower on the ioScale. But it actually pulls ahead in the QD1 4K reads, which isn't something I expected.

    EDIT: Some low queue depth IOmeter comparisons.


    This is a 512B QD1 100% read 100% random. The 950 Pro is actually slightly quicker here. 1.3% more IOPS.

    ioScale
    [​IMG]

    950 Pro
    [​IMG]


    This is a 1K QD1 100% read 100% random. Here the ioScale pulls ahead. 12.2% more IOPS.

    ioScale
    [​IMG]

    950 Pro
    [​IMG]


    This is a 4K QD1 100% read 100% random. ioScale is only 3.3% quicker here.

    ioScale
    [​IMG]

    950 Pro
    [​IMG]
     
    #9
    Last edited: Dec 29, 2016
  10. j.battermann

    j.battermann New Member

    Joined:
    Aug 22, 2016
    Messages:
    3
    Likes Received:
    0
    I am currently thinking about getting one of these for my Workstation to store & run all my VMs on.. but was wondering how 'future-proof' these are.. as in.. are drivers for these still being maintained/updated? I had some bad experience with drivers for other 'older' devices after a fresh install of Windows 10 Anniversary Update and its requirement for signed drivers and all that...

    While certainly no one can predict the future, has anyone an idea or an educated opinion whether buying one of these for a Win10 Workstation would be advisable?

    I am currently going back and forth between one of these 1.2tb fusion-io drives and a 800gb intel dc p3600 - the later still getting firmware/driver updates and overall appearing to be more of a normal plug'n'play kind of scenario, but both would be ~ the same price (used) and I wouldn't mind having an extra few hundred gbs of storage either.

    Any advice / opinions?
     
    #10
  11. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,344
    Likes Received:
    4,294
    NVMe is much more friendly driver wise. Most modern OSes have NVMe drivers built-in. While they may not be the specialized Intel drivers, NVMe drives do work out of the box which is significantly different than Fusion-IO.
     
    #11
  12. workingnonstop

    workingnonstop Active Member

    Joined:
    Feb 24, 2016
    Messages:
    226
    Likes Received:
    54
    Speaking of special drivers... any secret to getting these to run in Windows 10? It looks like Sandisk has a page with Windows drivers on it, but wondering if you're running those or a set from somewhere else. [edit - re-read your post and saw you said "Installation of the HP/ HPE card worked in Windows 10 Pro without issue by simply installing the driver packs from the WD/ SanDisk/ Fusion-io support site." ...so guess it was that easy!]

    What is your go-to for testing these drives? I can identify Anvil, CrystalDisk, and IOMeter in the post above. Have seen comments that some of the more popular benchmarking programs are not super accurate for these faster drives.

    I had my eye on the 1.2TB versions and have seen them listed as low as $300. Missed a $300 OBO listing last night, so not sure if it sold for list price or if someone beat my offer. Picked up 2x 2.4TB versions this afternoon, though, and specs on that guy look even more nuts. Will post some test results once I get them (will make a new thread then... to not completely hijack this one!).

    Specs also list that this one may need an aux power cable, so need to figure out 1) if that's true and 2) if not included with the cards I bought, where to find them... :p
     
    #12
    Last edited: Jan 19, 2017
  13. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,840
    Likes Received:
    813
    Iometer or fio. You normally don't need an extra power cable.
     
    #13
    workingnonstop likes this.
  14. walked

    walked New Member

    Joined:
    Jan 19, 2017
    Messages:
    12
    Likes Received:
    3
    I'm seriously considering grabbing one of these to use as a Starwind VSAN cache disk. Wondering how it'd perform on a 10gb network for small-scale virtualization clustering.. I'm thinking I'm going to try it. Worst case is it ends up as my vmware workstation scratch disk..
     
    #14
    NetWise likes this.
  15. walked

    walked New Member

    Joined:
    Jan 19, 2017
    Messages:
    12
    Likes Received:
    3
    Annnd pulled the trigger.

    We'll see how this performs used as a L2 cache disk for Starwind Virtual SAN on a 10gbe network in short order..
     
    #15
  16. walked

    walked New Member

    Joined:
    Jan 19, 2017
    Messages:
    12
    Likes Received:
    3
    So I picked this up; local performance is just as expected - GREAT.

    However, no matter what, if I present the Fusion-IO on a network; performance is abysmal.

    I've tried presenting via 10gbe Copper and Fiber.
    I've tried presenting via SMB and iSCSI.

    No matter what; once network is included in the access pattern; performance goes from ~1.2gbps to ~0.2gbps. It's ugly.

    It's not a bus limitation, as I can run local iometer benchmarks agains the disk full bore, and then run a different set of network benchmarks, also full bore, and get full performance out of both.

    I've tried verifying jumbo frames; multiple host systems, the works. No matter what - as soon as I try to access via network, I'm seeing terrible performance.

    Guess I'll just use it as a local cache for performance sensitive workloads on my workstation, and move my workstation 850 evo over. Sad.
     
    #16
  17. lowfat

    lowfat Member

    Joined:
    Nov 25, 2016
    Messages:
    72
    Likes Received:
    57
    What OS on both systems?
     
    #17
  18. walked

    walked New Member

    Joined:
    Jan 19, 2017
    Messages:
    12
    Likes Received:
    3
    Server 2016; fully updated.

    NICs I've tired are Mellanox Connect-X 2 and Intel X540-T2. Both tied into 10gbe backbone that performs at full speed for all systems outside of this disk.

    Local to the bare metal host of Server 2016; everything is up to par.
    If I present the FusionIO as an SMB share; performance from any other system tanks from ~1.2gbps to ~200mbps. (and it's definitely not a network issue; because I can present my H700 array via SMB and hit ~500mbps which is what it should be)

    If I present the drive as iSCSI from Starwind VSAN; I get the same performance from remote systems.

    I can run iometer / any disk benchmark locally, maxed out, and then run full network throughput tests and have neither hit subpar numbers, which (at least to me) indicates it isnt a PCIE bottleneck.

    Kinda pulling my hair out.
     
    #18
  19. Maritime

    Maritime New Member

    Joined:
    Nov 20, 2016
    Messages:
    10
    Likes Received:
    0
    There is no way to trick OS to see two of these as one big disk? Something like RAID for 'normal' disks? Not because of speed but because of big projects that need to be on one place.
     
    #19
  20. acquacow

    acquacow Active Member

    Joined:
    Feb 15, 2017
    Messages:
    349
    Likes Received:
    154
    I should note, in your first post, the write endurance is only the original warranted write endurance. The cards can/will last a lot longer than 16PBW.

    If you have any questions or issues with the drives, let me know. I worked at Fusion-io for 6 years and have seen just about everything you might come across.

    Yes, absolutely. I have 3 in a storage space right now in Windows 10. They are great on their own, but I tried setting up SSD and HDD tiers in a pool with them and couldn't get more than disk speeds if the ioDrives were in the pool. I settled on 4 HDDs in a 2-column stripe and kept the ioDrives in a separate pool for faster stuff.

    Here's the command-line I used to build my mirrored "ioPool" of 3 ioDrive 2s in Windows 10 storage spaces:
    Code:
    #Making 3 ioDrives into mirrored storage space:
    Windows PowerShell
    Copyright (C) 2016 Microsoft Corporation. All rights reserved.
    
    PS C:\WINDOWS\system32> Get-PhysicalDisk -CanPool $True | ft FriendlyName,OperationalStatus,Size,MediaType
    
    FriendlyName          OperationalStatus          Size MediaType
    ------------          -----------------          ---- ---------
    Fusion ioCache 1200GB OK                1205000000000 SSD
    Fusion ioCache 1200GB OK                1205000000000 SSD
    Fusion ioCache 1200GB OK                1205000000000 SSD
    
    
    PS C:\WINDOWS\system32> $pd = (Get-PhysicalDisk -CanPool $True | Where FriendlyName -EQ "Fusion ioCache 1200GB")
    PS C:\WINDOWS\system32> New-StoragePool -PhysicalDisks $pd –StorageSubSystemFriendlyName “Windows Storage*” -FriendlyName “ioPool”
    
    FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
    ------------ ----------------- ------------ ------------ ----------
    ioPool       OK                Healthy      False        False
    
    
    PS C:\WINDOWS\system32> New-VirtualDisk -StoragePoolFriendlyName "ioPool" -FriendlyName ioStorage -ResiliencySettingName Mirror -UseMaximumSize
    
    FriendlyName ResiliencySettingName OperationalStatus HealthStatus IsManualAttach    Size
    ------------ --------------------- ----------------- ------------ --------------    ----
    ioStorage    Mirror                OK                Healthy      False          1.64 TB
    
    
    PS C:\WINDOWS\system32> Get-VirtualDisk ioStorage | Get-Disk | Initialize-Disk -PartitionStyle GPT
    PS C:\WINDOWS\system32> Get-VirtualDisk ioStorage | Get-Disk | New-Partition -DriveLetter “I” -UseMaximumSize
    
    
       DiskPath: \\?\storage#disk#{b254657a-f750-49c0-9a4a-79050cf3a153}#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
    
    PartitionNumber  DriveLetter Offset                                                              Size Type
    ---------------  ----------- ------                                                              ---- ----
    2                I           135266304                                                        1.64 TB Basic
    
    
    PS C:\WINDOWS\system32> Initialize-Volume -DriveLetter “I” -FileSystem REFS -Confirm:$false
    
    DriveLetter FileSystemLabel FileSystem DriveType HealthStatus OperationalStatus SizeRemaining    Size
    ----------- --------------- ---------- --------- ------------ ----------------- -------------    ----
    I                           ReFS       Fixed     Healthy      OK                      1.63 TB 1.64 TB
    
    [​IMG]

    Here's how it performs:
    [​IMG]

    We always relied on mdraid in linux and windows dynamic disks for striping ioDrives together. It's a much faster solution than anything you could do with a slow hardware ASIC.

    I have 3 in an array attached to a VM right now:
    [​IMG]

    I've got the 3 above in an array in a windows 10 VM, intel x540-T2, single link to my desktop which also has an x540-T2.

    I'm just using normal windows file sharing, jumbo frames are set at 9000 at both ends. I can nearly max out the single link just fine:

    [​IMG]

    Your drive may be power throttling if it can't detect the supported wattage from the slot. You may want to look into the Power Override options to set it to 25w so that you can get full perf.
     
    #20
    Last edited: Feb 15, 2017
    warlockedyou and Jaket like this.
Similar Threads: Fusion-io ioDrive
Forum Title Date
Hard Drives and Solid State Drives Fusion-IO ioDrive - General Channel Initiliaze Failure Apr 19, 2018
Hard Drives and Solid State Drives Fusion-io ioDrive Linux Drivers (HP branded / non-branded) discovery May 17, 2015
Hard Drives and Solid State Drives Fusion-io iodrive - This device cannot start. (Code 10) Apr 20, 2015
Hard Drives and Solid State Drives fusion-io error(-22) Dec 12, 2018
Hard Drives and Solid State Drives Fusion-IO problem: "Device is missing midprom data" Mar 4, 2018

Share This Page