Installing Fusion-io ioDrives on Proxmox VE 3.4 Debian Wheezy

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
Just making a space for some public notes as I attempt this.

Using two Fusion-io ioDrive 353GB drives in a dual Intel Xeon E5-2699 V3 box to support the existing storage (2x Toshiba 480GB PLP drives, 2x Intel S3500 800GB, 2x Intel S3700 400GB.)

Step 1: Get software
Since these are Fusion-io branded cards, I went to Application Acceleration – Enterprise Flash Memory Platform | Fusion-io and used my registered username/ password to browse for the right software. I just selected everything and downloaded it locally. I then uploaded it to my Proxmox VE 3.4 server

For ease of installation, I moved the .deb files from Firmware and Software Binaries to Utilities so it was all in one place.

Step 2: Install binaries
There are two options, compile your own or use the .deb packages. I moved everything into one folder so i could do:
Code: 
root@intel-e5-2699-v3:~/ioDrive/Linux_debian-wheezy/3.2.8/Software Binaries# dpkg -i iomemory-vsl-3.2.0-4-amd64_3.2.8.1350-1.0_amd64.deb fio-util_3.2.8.1350-1.0_amd64.deb libvsl_3.2.8.1350-1.0_amd64.deb fio-firmware-fusion_3.2.8.20140508-1_all.deb
After a reboot, the driver is not loading. Troubleshooting time! Any ideas?

Ok modprobe is not happy. I was getting nowhere
Code: 
ln -s /lib/modules/3.2.0-4-amd64/extra/fio/iomemory-vsl.ko /lib/modules/`uname -r`
depmod -a
And now I get:

Code: 
modprobe iomemory-vsl
ERROR: could not insert 'iomemory_vsl': Exec format error
 
Last edited:
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
Since that did not work, going to try the compile route:
Code: 
nano /etc/apt/sources.list
and add an uncommented:
Code: 
deb http://download.proxmox.com/debian wheezy pve-no-subscription
You need that step so you can get the pve-headers

Update and upgrade otherwise you cannot see anything:
Code: 
apt-get update && apt-get upgrade -y
Need some build tools!
Code: 
apt-get install gcc fakeroot build-essential debhelper rsync pve-headers-2.6.32-37-pve -y
Now go to the download folder. By default, the Fusion-io/ SanDisk download now has a space in the Software Source and you cannot dpkg-buildpackage with a space in a file name (Linux is advanced). So at minimum remove the space.
Code: 
tar xzvf iomemory*
Code: 
Then cd into SoftwareSource\iomemory-vsl-3.2.8.1350 (replace this if a different version/path is needed)
Now you can
Code: 
dpkg-buildpackage
Wait a bit and cd ..

did
Code: 
dpkg -i iomemory-vsl-2.6.32-37-pve_3.2.8.1350-1.0_amd64.deb
And still nothing!
 
Last edited:
tjk

tjk

Active Member
Mar 3, 2013
485
201
43
Patrick, you planning on using ceph and the FIO cards for cache? Or as a datastore?
 
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
tjk said:
Patrick, you planning on using ceph and the FIO cards for cache? Or as a datastore?
Click to expand...
I was thinking of using the 353GB ones as a primary datastore for the forums VM in Fremont.
 
TuxDude

TuxDude

Well-Known Member
Sep 17, 2011
616
338
63
I don't suppose you have the output of the 'dmesg' command from right after the 'modprobe iomemory-vsl' command (looking at the first attempt using .debs) - if theres' anything interesting in there it'll probably be in the last 10 or 20 lines assuming 'dmesg' is run right after the error occurs.
 
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
TuxDude said:
I don't suppose you have the output of the 'dmesg' command from right after the 'modprobe iomemory-vsl' command (looking at the first attempt using .debs) - if theres' anything interesting in there it'll probably be in the last 10 or 20 lines assuming 'dmesg' is run right after the error occurs.
Click to expand...
I actually just re-installed Proxmox. Going to try it again building from source. Happy I kept notes above.
 
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
OK process this time
1. Started by consolidating the firmware .deb into Utilities and made a folder named Source to keep things easy.
2. Did the following:
Code: 
nano /etc/apt/sources.list
and add an uncommented:
Code: 
deb http://download.proxmox.com/debian wheezy pve-no-subscription
Code: 
apt-get update && apt-get upgrade -y
apt-get install gcc fakeroot build-essential debhelper rsync pve-headers-2.6.32-37-pve -y
cd Source
tar xzvf iomemory*
cd iomemory-vsl-3.2.8.1350
dpkg-buildpackage
cd ..
dpkg -i iomemory-vsl-2.6.32-37-pve_3.2.8.1350-1.0_amd64.deb
cd ..
cd Utilities
dpkg -i fio-util_3.2.8.1350-1.0_amd64.deb fio-common_3.2.8.1350-1.0_amd64.deb fio-firmware-fusion_3.2.8.20140508-1_all.deb
reboot
Came back with the same "Driver version: Driver not loaded"

Tried to install remaining .deb's:
Code: 
dpkg -i libvsl_3.2.8.1350-1.0_amd64.deb
dpkg -i fio-sysvinit_3.2.8.1350-1.0_all.deb
After the sysvinit got:
Code: 
Selecting previously unselected package fio-sysvinit.
(Reading database ... 48549 files and directories currently installed.)
Unpacking fio-sysvinit (from fio-sysvinit_3.2.8.1350-1.0_all.deb) ...
Setting up fio-sysvinit (3.2.8.1350-1.0) ...
update-rc.d: warning: default start runlevel arguments (2 3 4 5) do not match iomemory-vsl Default-Start values (1 2 3 4 5)
update-rc.d: warning: default stop runlevel arguments (0 1 6) do not match iomemory-vsl Default-Stop values (0 6)
iomemory-vsl is not enabled in the init config '/etc/sysconfig/iomemory-vsl'
Trying to register new drivers:
Code: 
depmod /lib/modules/2.6.32-37-pve/extra/fio/iomemory-vsl.ko
reboot
After the reboot
Code: 
root@intel-e5-2699v3:~# modprobe iomemory-vsl
ERROR: could not insert 'iomemory_vsl': Exec format error
hmm insmod
Code: 
insmod iomemory-vsl.ko
Error: could not insert module iomemory-vsl.ko: Invalid module format
 
Last edited:
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
The process I have been using is basically that. :-/ Going to try with 3.2.10 now.
 
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
Ok so the issue seems to be that there are closed source binaries that are tied to a specific kernel. The Proxmox kernel does not play well with these binaries and therefore the drivers are not getting loaded even after "compiling" the Fusion-io drivers.
 
C

canta

Well-Known Member
Nov 26, 2014
1,012
217
63
43
Patrick said:
Ok so the issue seems to be that there are closed source binaries that are tied to a specific kernel. The Proxmox kernel does not play well with these binaries and therefore the drivers are not getting loaded even after "compiling" the Fusion-io drivers.
Click to expand...
this is very common on my work on embedded linux :D.
this is not about the closed source binary, the issue is blob file are wrapped up with specific kernel level. you can do nothing unless sandisk provide new blob file with never kernel level

this is not proxmox kernel problem, this is sandisk problem not updating new wrapped blob with kernel level

the easy spot is "exec mismatch" something, basically the object file is not match with kernel structure due on update on kernel that brake compatibility Object file.

honestly, proxmox kernel is not generic debian, they modify and hack to work on openvz.

if you need mostly as same as generic debian kernel ( without hacked for openvz), get pv-kernel.3.x.x on proxmox repro. I am sure fusion-io still not work due on obsolete wrapped blob :p.
 
  • Like
Reactions: Patrick
Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,533
5,854
113
Yea that was the issue. They are working on 14.04.2 but I swapped to Ubuntu and Docker now
 
T_Minus

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,752
2,129
113
So did anyone ever get Fusion IO working in Proxmox 4.4 or 5.0 beta? @Patrick @canta @TuxDude

Swapped some SLC 160GB into a build for a buddy and went from ESXI to ProxMox and realized the article from 2015 and this thread and LOL!!
 
acquacow

acquacow

Well-Known Member
Feb 15, 2017
795
447
63
43
Just bumping this up since I have VSL 4.x working in the latest proxmox 8.1 with no issues.

I didn't take exact notes, but reconstructing some of the fiddling in my bash history, this was the rough proces:

### Fusion-io install

apt-get install git make gcc dkms fakeroot build-essential debhelper linux-headers-$(uname --r) rsync

wget fio-util_4.3.7.1205-1.0_amd64.deb

git clone GitHub - RemixVSL/iomemory-vsl4: Updated Fusion-io iomemory VSL4 Linux (version 4.3.7) driver for recent kernels.
cd iomemory-vsl4/

less README

make dpkg
make dkms

# I did this one, not sure if the above make dpkg is enough...
dpkg-buildpackage -rfakeroot -b


# Modules are built and placed in parent dir.
cd ..

# Install modules
dpkg -i iomemory-vsl4-6.5.11-7-pve_4.3.7.1205-1.0_amd64.deb
dpkg -i iomemory-vsl4-config-6.5.11-7-pve_4.3.7.1205-1.0_amd64.deb
dpkg -i iomemory-vsl4-initrd-6.5.11-7-pve_4.3.7.1205-1.0_amd64.deb
dpkg -i ./fio-util_4.3.7.1205-1.0_amd64.deb

# Reboot to load module

# Create local zfs pool (GUI won't let you) and name it the same as your other pools so you can migrate to it via GUI
zpool create localSSD /dev/fioa

# Add 3rd host to the pool in /etc/pve/storage.cfg
zfspool: localSSD
pool localSSD
content images,rootdir
mountpoint /localSSD
nodes prox,prox2,prox3

# Restart pve to push changes
systemctl restart pvedaemon

# Enable power override
root@prox3:/etc/pve# cat /etc/modprobe.d/iomemory-vsl4.conf
# Enable pci-e power override
options iomemory-vsl4 external_power_override=1621D0390:75

# Reboot to reload driver and enable power-override
 
  • Like
Reactions: Marsh
You must log in or register to reply here.
Top Bottom