Proxmox VE v5.x + Mellanox ConnectX-3
- Thread starter awedio
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
I also posted this question on the Proxmox forums.
The Mellanox site only has drivers for Debian 8.3.
I tried using the 8.3 drivers but I'm getting some errors.
# ./install --distro debian8.3
Log: /tmp/ofed.build.log
Logs dir: /tmp/mlnx-en.31026.logs
Below is the list of mlnx-en packages that you have chosen
(some may have been added by the installer due to package dependencies):
ofed-scripts
mlnx-en-utils
mlnx-en-dkms
mstflint
This program will install the mlnx-en package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with mlnx-en, do not reinstall them.
Do you want to continue?[y/N]:y
Checking SW Requirements...
One or more required packages for installing mlnx-en are missing.
/lib/modules/4.10.1-2-pve/build/scripts is required for the Installation.
Attempting to install the following missing packages:
m4 autoconf swig automake dkms autotools-dev python-libxml2 graphviz debhelper chrpath dpatch quilt libltdl-dev linux-headers-4.10.1-2-pve
Failed command: apt-get install -y m4 autoconf swig automake dkms autotools-dev python-libxml2 graphviz debhelper chrpath dpatch quilt libltdl-dev linux-headers-4.10.1-2-pve
The Mellanox site only has drivers for Debian 8.3.
I tried using the 8.3 drivers but I'm getting some errors.
# ./install --distro debian8.3
Log: /tmp/ofed.build.log
Logs dir: /tmp/mlnx-en.31026.logs
Below is the list of mlnx-en packages that you have chosen
(some may have been added by the installer due to package dependencies):
ofed-scripts
mlnx-en-utils
mlnx-en-dkms
mstflint
This program will install the mlnx-en package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with mlnx-en, do not reinstall them.
Do you want to continue?[y/N]:y
Checking SW Requirements...
One or more required packages for installing mlnx-en are missing.
/lib/modules/4.10.1-2-pve/build/scripts is required for the Installation.
Attempting to install the following missing packages:
m4 autoconf swig automake dkms autotools-dev python-libxml2 graphviz debhelper chrpath dpatch quilt libltdl-dev linux-headers-4.10.1-2-pve
Failed command: apt-get install -y m4 autoconf swig automake dkms autotools-dev python-libxml2 graphviz debhelper chrpath dpatch quilt libltdl-dev linux-headers-4.10.1-2-pve
should be apt-get install pve-headers-4.10.1-2-pve
eventually the ofed installer will still struggle due to weird dependencies.
i gave up on ofed with proxmox due to them and live happy with in-tree drivers on cx2 and cx3
eventually the ofed installer will still struggle due to weird dependencies.
i gave up on ofed with proxmox due to them and live happy with in-tree drivers on cx2 and cx3
the drivers/kernel modules/tools that ship with proxmox/it's kernel, these should be ,ok' and there is no need to build/re-compile with each kernel upgrade. they can be installed and upgraded with apt.
how do you plan to use your connectx-3 cards, ib/rdma or ethernet?
what type of cards?
how do you plan to use your connectx-3 cards, ib/rdma or ethernet?
what type of cards?
you should make sure the ports are set to ethernet mode and not infiniband.
guess you need some packages, mlx4_core mlx4_en at least.
you can also have a look here or just search for instructions for debian or ubuntu, should be 99% the same with proxmox.
HowTo Change Port Type in Mellanox ConnectX-3 A... | Mellanox Interconnect Community
guess you need some packages, mlx4_core mlx4_en at least.
you can also have a look here or just search for instructions for debian or ubuntu, should be 99% the same with proxmox.
HowTo Change Port Type in Mellanox ConnectX-3 A... | Mellanox Interconnect Community
@_alex You were absolutely correct. Both ports were set to "IB".
Changed both ports to "Eth" & life is good!
Thanks!!
By installing MFT (if you can't install MLNX_OFED), you can now change the port settings.
Ophir from Mellanox has now updated the thread.
See: HowTo Change Port Type in Mellanox ConnectX-3 A... | Mellanox Interconnect Community
Ophir from Mellanox has now updated the thread.
See: HowTo Change Port Type in Mellanox ConnectX-3 A... | Mellanox Interconnect Community
Oh, nice ...
a bit 'old fashioned' but works
- load modules in /etc/modprobe
- configure ports/other settings in /etc/rc.local
a bit 'old fashioned' but works
- load modules in /etc/modprobe
- configure ports/other settings in /etc/rc.local