Installing Fusion-io ioDrives on Proxmox VE 3.4 Debian Wheezy

Discussion in 'Software Stuff' started by Patrick, Apr 30, 2015.

  1. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    Just making a space for some public notes as I attempt this.

    Using two Fusion-io ioDrive 353GB drives in a dual Intel Xeon E5-2699 V3 box to support the existing storage (2x Toshiba 480GB PLP drives, 2x Intel S3500 800GB, 2x Intel S3700 400GB.)

    Step 1: Get software
    Since these are Fusion-io branded cards, I went to Application Acceleration – Enterprise Flash Memory Platform | Fusion-io and used my registered username/ password to browse for the right software. I just selected everything and downloaded it locally. I then uploaded it to my Proxmox VE 3.4 server

    For ease of installation, I moved the .deb files from Firmware and Software Binaries to Utilities so it was all in one place.

    Step 2: Install binaries
    There are two options, compile your own or use the .deb packages. I moved everything into one folder so i could do:
    Code:
    root@intel-e5-2699-v3:~/ioDrive/Linux_debian-wheezy/3.2.8/Software Binaries# dpkg -i iomemory-vsl-3.2.0-4-amd64_3.2.8.1350-1.0_amd64.deb fio-util_3.2.8.1350-1.0_amd64.deb libvsl_3.2.8.1350-1.0_amd64.deb fio-firmware-fusion_3.2.8.20140508-1_all.deb
    After a reboot, the driver is not loading. Troubleshooting time! Any ideas?

    Ok modprobe is not happy. I was getting nowhere
    Code:
    ln -s /lib/modules/3.2.0-4-amd64/extra/fio/iomemory-vsl.ko /lib/modules/`uname -r`
    depmod -a
    
    And now I get:

    Code:
    modprobe iomemory-vsl
    ERROR: could not insert 'iomemory_vsl': Exec format error
     
    #1
    Last edited: Apr 30, 2015
  2. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    Since that did not work, going to try the compile route:
    Code:
    nano /etc/apt/sources.list
    and add an uncommented:
    Code:
    deb http://download.proxmox.com/debian wheezy pve-no-subscription
    You need that step so you can get the pve-headers

    Update and upgrade otherwise you cannot see anything:
    Code:
    apt-get update && apt-get upgrade -y
    Need some build tools!
    Code:
    apt-get install gcc fakeroot build-essential debhelper rsync pve-headers-2.6.32-37-pve -y
    Now go to the download folder. By default, the Fusion-io/ SanDisk download now has a space in the Software Source and you cannot dpkg-buildpackage with a space in a file name (Linux is advanced). So at minimum remove the space.
    Code:
    tar xzvf iomemory*
    Code:
    Then cd into SoftwareSource\iomemory-vsl-3.2.8.1350 (replace this if a different version/path is needed)
    Now you can
    Code:
    dpkg-buildpackage
    Wait a bit and cd ..

    did
    Code:
    dpkg -i iomemory-vsl-2.6.32-37-pve_3.2.8.1350-1.0_amd64.deb
    And still nothing!
     
    #2
    Last edited: May 1, 2015
  3. tjk

    tjk Member

    Joined:
    Mar 3, 2013
    Messages:
    165
    Likes Received:
    17
    Patrick, you planning on using ceph and the FIO cards for cache? Or as a datastore?
     
    #3
  4. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,402
    Likes Received:
    197
    As much as I love Linux- this is why it sucks.
     
    #4
  5. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    I was thinking of using the 353GB ones as a primary datastore for the forums VM in Fremont.
     
    #5
  6. TuxDude

    TuxDude Well-Known Member

    Joined:
    Sep 17, 2011
    Messages:
    615
    Likes Received:
    335
    I don't suppose you have the output of the 'dmesg' command from right after the 'modprobe iomemory-vsl' command (looking at the first attempt using .debs) - if theres' anything interesting in there it'll probably be in the last 10 or 20 lines assuming 'dmesg' is run right after the error occurs.
     
    #6
  7. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    I actually just re-installed Proxmox. Going to try it again building from source. Happy I kept notes above.
     
    #7
  8. TuxDude

    TuxDude Well-Known Member

    Joined:
    Sep 17, 2011
    Messages:
    615
    Likes Received:
    335
    Well let me know if you still have issues and I'll help if I can.
     
    #8
    Patrick likes this.
  9. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    OK process this time
    1. Started by consolidating the firmware .deb into Utilities and made a folder named Source to keep things easy.
    2. Did the following:
    Code:
    nano /etc/apt/sources.list
    and add an uncommented:
    Code:
    deb http://download.proxmox.com/debian wheezy pve-no-subscription
    Code:
    apt-get update && apt-get upgrade -y
    apt-get install gcc fakeroot build-essential debhelper rsync pve-headers-2.6.32-37-pve -y
    cd Source
    tar xzvf iomemory*
    cd iomemory-vsl-3.2.8.1350
    dpkg-buildpackage
    cd ..
    dpkg -i iomemory-vsl-2.6.32-37-pve_3.2.8.1350-1.0_amd64.deb
    cd ..
    cd Utilities
    dpkg -i fio-util_3.2.8.1350-1.0_amd64.deb fio-common_3.2.8.1350-1.0_amd64.deb fio-firmware-fusion_3.2.8.20140508-1_all.deb
    reboot
    Came back with the same "Driver version: Driver not loaded"

    Tried to install remaining .deb's:
    Code:
    dpkg -i libvsl_3.2.8.1350-1.0_amd64.deb
    dpkg -i fio-sysvinit_3.2.8.1350-1.0_all.deb
    
    After the sysvinit got:
    Code:
    Selecting previously unselected package fio-sysvinit.
    (Reading database ... 48549 files and directories currently installed.)
    Unpacking fio-sysvinit (from fio-sysvinit_3.2.8.1350-1.0_all.deb) ...
    Setting up fio-sysvinit (3.2.8.1350-1.0) ...
    update-rc.d: warning: default start runlevel arguments (2 3 4 5) do not match iomemory-vsl Default-Start values (1 2 3 4 5)
    update-rc.d: warning: default stop runlevel arguments (0 1 6) do not match iomemory-vsl Default-Stop values (0 6)
    iomemory-vsl is not enabled in the init config '/etc/sysconfig/iomemory-vsl'
    
    Trying to register new drivers:
    Code:
    depmod /lib/modules/2.6.32-37-pve/extra/fio/iomemory-vsl.ko
    reboot
    
    After the reboot
    Code:
    root@intel-e5-2699v3:~# modprobe iomemory-vsl
    ERROR: could not insert 'iomemory_vsl': Exec format error
    
    hmm insmod
    Code:
    insmod iomemory-vsl.ko
    Error: could not insert module iomemory-vsl.ko: Invalid module format
    
     
    #9
    Last edited: May 1, 2015
  10. whitey

    whitey Moderator

    Joined:
    Jun 30, 2014
    Messages:
    2,734
    Likes Received:
    846
    Dunno if you saw this or if it is of any help but this guy is typically spot on. May be a bit more complicated/custom 'roll-your-own' type of setup w/ proxmox in the mix though. YMMV

    Fusion-io ioDrive use with Debian - Thomas-Krenn-Wiki

    Take care and good luck, still debating getting one of these myself.
     
    #10
    Last edited: May 1, 2015
  11. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    The process I have been using is basically that. :-/ Going to try with 3.2.10 now.
     
    #11
  12. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    Ok so the issue seems to be that there are closed source binaries that are tied to a specific kernel. The Proxmox kernel does not play well with these binaries and therefore the drivers are not getting loaded even after "compiling" the Fusion-io drivers.
     
    #12
  13. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    #13
    T_Minus likes this.
  14. canta

    canta Active Member

    Joined:
    Nov 26, 2014
    Messages:
    948
    Likes Received:
    168
    this is very common on my work on embedded linux :D.
    this is not about the closed source binary, the issue is blob file are wrapped up with specific kernel level. you can do nothing unless sandisk provide new blob file with never kernel level

    this is not proxmox kernel problem, this is sandisk problem not updating new wrapped blob with kernel level

    the easy spot is "exec mismatch" something, basically the object file is not match with kernel structure due on update on kernel that brake compatibility Object file.

    honestly, proxmox kernel is not generic debian, they modify and hack to work on openvz.

    if you need mostly as same as generic debian kernel ( without hacked for openvz), get pv-kernel.3.x.x on proxmox repro. I am sure fusion-io still not work due on obsolete wrapped blob :p.
     
    #14
    Patrick likes this.
  15. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,045
    Likes Received:
    3,996
    Yea that was the issue. They are working on 14.04.2 but I swapped to Ubuntu and Docker now
     
    #15
  16. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,358
    Likes Received:
    1,295
    So did anyone ever get Fusion IO working in Proxmox 4.4 or 5.0 beta? @Patrick @canta @TuxDude

    Swapped some SLC 160GB into a build for a buddy and went from ESXI to ProxMox and realized the article from 2015 and this thread and LOL!!
     
    #16
  17. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,358
    Likes Received:
    1,295
    No one eh?!
     
    #17
  18. h0tw1r3

    h0tw1r3 Member

    Joined:
    Oct 30, 2014
    Messages:
    44
    Likes Received:
    20
    #18
    Marsh, eva2000, Patrick and 1 other person like this.
  19. Jon

    Jon New Member

    Joined:
    Feb 28, 2016
    Messages:
    18
    Likes Received:
    0
    Hi is this a thing in 2018 or is it simpler or totally broken?
     
    #19
Similar Threads: Installing Fusion-io
Forum Title Date
Software Stuff Reinstalling Ubuntu 16.04 MAAS Jan 30, 2017

Share This Page