Search results

  1. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Here's one important thing to notice: If the "fast start-up" (fast boot) is enabled in windows, the mlx vpi device will get the error 43, requiring reboot or disable/enable toggling. That is, with a patched kernel driver or a working linux ofed (if such a thing even exists anymore).
  2. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    The microsoft driver should work, so it points in the direction of linux ofed :( I did get that error 10, when I had winof driver on the VPI and microsoft one on the ethernet device.
  3. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Yes, after uninstalling them do the hw scan from the menu. The thing is, if you have driver version mismatch between the vpi & ethernet device, it will not work. So uninstall + remove driver and begin with the ethernet adapter.
  4. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Yes, the "other" is in System Devices
  5. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    @mimino I think I know what's happening with your error code 10. There's two devices in windows, vpi and ethernet. If the VPI device is loaded with microsoft provided driver and the ethernet has winof driver, it'll give you the error code 10. Uninstall the devices and check the delete driver...
  6. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Just patch the stock kernel driver :)
  7. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    This is very weird, as if Mellanox (nvidia) is trying to sabotage their older products. I downloaded the 4.9-3.1.5.0 LTS. By looking at the source code, it seems incompatible with the Windows driver. :oops:
  8. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Well, you've got the LTS version of OFED. The latest is 5.3-1, where yours is the LTS version 4.9-3 ? Yes, it should work with 5.4 also! I just realised, they've dropped support for connectx-3 on 5.1. Kind of weird :oops:
  9. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Here's the kernel driver patch attached.
  10. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    It's either patch the stock kernel driver or install the latest & greatest ofed. :( That vhcr cmd, it failed executing, was MLX4_CMD_ALLOC_RES. So, before doing anything, I'd try that param.
  11. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    I see a problem there, the windows driver doesn't play nice with that linux driver version. You could try adding the log_num_mgm_entry_size=-1 parameter to mlx4_core. I'd expect it to fail on port query, though.
  12. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Can you do modinfo mlx4_core, just to verify the driver version you're on.
  13. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Are you using some non-default parameters for the mlx4_core module? Here's mine: msi_x=1 enable_4k_uar=1 enable_qos=1 log_num_mac=7 log_num_mgm_entry_size=-1 log_mtts_per_seg=4
  14. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    does mlx4_core spit anything out in the kernel output, when the vm tries to bring the device online?
  15. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    I did it last summer so, no. It was just a quick way to get it to work. It'd require extensive testing & cleaning up. For me, it's been working fine. <interface type='hostdev' managed='yes'> <source> <address type='pci' domain='0x0000' bus='0x83' slot='0x00' function='0x1'/>...
  16. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Did you assign mac-address to the vf?
  17. O

    Mellanox Connectx-3 passthru

    Are you trying to pass the main pci-device or a vf? You should disable the host/vmware driver (nmlx4-core&net) anyways, it has no role in passthrough, if it's not a vf. It's probably preventing it.
  18. O

    Mellanox Connectx-3 passthru

    https://forums.servethehome.com/index.php?threads/help-with-connectx-3-sr-iov-with-linux-host-and-windows-guest-via-kvm.28956
  19. O

    Very slow ZFS RaidZ2 Performance on TrueNAS 12

    The cause is right in front of you. The actual problem is the OpenZFS code quality. It uses the Solaris Porting Layer, which is basically a not-so-great wrapper around linux kernel threading api. - It doesn't know NUMA at all - It doesn't understand cache locality nor hierarchy - It does...
  20. O

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    Here's a dirty patch for the old (5.5.15) mainstream kernel driver. It might apply successfully to later kernels as well. From d4fbe94b4adb16a5cc094a50215dc39496c9fc66 Mon Sep 17 00:00:00 2001 From: Mikko Hissa <okrasit@gmail.com> Date: Tue, 28 Jul 2020 05:35:13 +0000 Subject: [PATCH] MLX4...