Search results

  1. D

    FCoE

    it might be, but you will need to use some PLX riser to bifurcate the only slot (it will cost like cx3-/4 + fc card)
  2. D

    FCoE

    yes, i agree, since my question was about the pci-e slot, and cx-3 can boot via iscsi even (but im not sure the EMC AX4-5F supports it, i also could find the 4gb FCoE only)
  3. D

    FCoE

    To run NVMe over Fabrics (with a TCP backend), you generally just need a TCP/IP-capable NIC and an NVMe Target Server. As far as I know, the EMC AX4-5F might not support this feature, or FCoE. Its iSCSI support is a good sign, though I couldn't find clear confirmation of that online. Using your...
  4. D

    FCoE

    hello, is nvme-of-tcp not an option?
  5. D

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    hello, 1) regarding eth - you are not obliged to bind/rebind driver just for vlan/link state update (vlan and link state attributes can be changed online) - you are not obliged to set mac address on VF and use random if it is acceptable for your software - you can assign required mac for VF via...
  6. D

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    hello, i've taken a look to code and found two issues regarding to override functionality:: 1) at line 176 - it tries to resolve /sys/bus/pci/devices/0000:11:22:33.4/driver, but there is not particular attribute `driver` for mlx4_core driver (the attribute appears after driver bound) 2) at lines...
  7. D

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    i just wanted to say: 1) vfio-pci (ids, new_id) and driver bind/unbind are not idempotent 2) use driver_override instead of vfio-pci ids/new_id and driver_probe instead of bind/unbind option `driver_override` is recommended to use since 3.16 for pci devices, and in 6.6 this mechanism is finally...
  8. D

    Help with ConnectX 3 SR-IOV with Linux host and windows guest via kvm

    hello, @klui @naptastic on modern kernels (if im not mistaken - 6+) this way still works, but there is a little bit different (and recommended) approach; - do not directly bind VFs to driver, otherwise use `driver_override` to use particular driver (mlx4_core, vfio-pci, etc) echo mlx5_core >...
  9. D

    ConnectX-3 and SR-IOV on modern kernels?

    hello, 1) the module parameters triplet is important: you can use the third one only when both ports are in ETH mode 2) i guess the 2/4k MTU limitation is about of CX-3 VF using. on RDMA it has practically ~no impact, you can try to use CONNECTED mode if MTU matters (i.e. for non-RDMA...
  10. D

    ConnectX-3 and SR-IOV on modern kernels?

    hello, here is my conf, absolutely in-tree driver, no patches, the kernel is `6.19.12`, but the ~same with `7.0.12`-`7.1.5` dual port CX-3 card with None: in my case inbox opensm doesnt work with IB VF, i just installed nvidia's opensm and just drop single `opensm` binnary from `nvidia's...
  11. D

    RDMA and CX4LX Troubleshooting

    hello, it looks like something wrong with infiniband libs/deps
  12. D

    How to Connect a ConnectX Adapter Using an NVMe PCIe 5.0 x4 via 3 MCIO Connector

    hello, you need something like this: https://aliexpress.com/item/1005009798069379.html and https://aliexpress.com/item/1005007212456065.html
  13. D

    RDMA and CX4LX Troubleshooting

    just plug cx4lx it looks like you have some kind of library/dependencies problem, did you try to boot your favorite (live) distro and check the cards?
  14. D

    RDMA and CX4LX Troubleshooting

    are there any errors in dmesg (afrter reboot)? dmesg | grep 'mlx5\|DMAR' if you have virtualization enabled, like this intel_iommu=on iommu=pt pci=realloc intremap=no_x2apic_optout please check you also have "Enable 4G encoding' in BIOS
  15. D

    RDMA and CX4LX Troubleshooting

    ok, you have both base guid&mac defined so the problem might be about loaded modules, like # lsmod | grep 'mlx5_ib\|uverb' mlx5_ib 487424 0 ib_uverbs 200704 2 mlx4_ib,mlx5_ib ib_core 516096 10...
  16. D

    RDMA and CX4LX Troubleshooting

    hello, what does [mst]flint show? you also can restore guids by macs, i had the similar problem when was reflashing cx3 pro en to vpi fw
  17. D

    Infiniband or iSCSI

    almost all of my cards (about 20) are from ali, and 4-5 from ebay, connectx-3/4/5/6 the same about transceivers and fibers, as well as switches
  18. D

    Infiniband or iSCSI

    hello, yes that is please take into account: 1) connectx4-lx is more energy efficient i.e. - from offical docs Specifications - from user experience Connectx-4 Lx 2x 25GbE with ASPM support + idle power consumption measurements 2) connecting via chipset can utilize just 32 Gbps (theoretically)...
  19. D

    Infiniband or iSCSI

    hello, 1) if you buy connecx4 (not lx version) it can be reflashed to vpi mode (ethernet+infiniband) 2) on any connectx (3/4/5 pro/lx/ex etc) you can use RDMA over ethernet (RoCE v1/v2) on iSCSI (iser) 3) as i remember correctly connectx does not support aspm