1. G

    Mellanox 10G Dual-Port card does not do any routing with IPoIB?

    Hey folks, I have 3 servers equipped with the Connect-X2 MT26418, all running Proxmox (debian11). The cards are all 2 Port ones, and I got 2 cables running between them like this: +---------------------------+ | serv1 | +---------------------------+ | Port 2 |...
  2. bryan_v

    Juniper LX4 optics & Mellanox CX3/SX6036 compatibility

    Quick question: Has anyone had any success with 40G LX4 transceivers in Mellanox CX3 cards and SX6036 switches (specifically Juniper QSFP+ 40G LX4 LC transceivers)? I'm about to run optics throughout the house, and MTP/MPO cables are so much more expensive than LC/LC cables for multiple runs...
  3. C

    How to use the mlx5_core driver with Mellanox ConnectX-4 Lx in Debian?

    I just got these ConnectX-4 Lx in the mail, but not getting very far. I've been treating this like the ixgbe driver: I see Kernel driver in use: mlx5_core and am trying to create virtual functions in /etc/modprobe.d/mlx5_core.conf with some options like options mlx5_core num_vfs=12...
  4. J

    Mellanox ConnectX-3 Showing wrong bus type

    I just installed a Mellanox ConnectX-3 two port in my Windows 2016 server. Haven't installed the fiber cable or SFP+ transceivers yet, however the Bus Type is showing as x8 when I know it's x16 slot any idea what's up or is that just because it's the max the card supports so the "card" is x8...
  5. bryan_v

    Is RoCE v2 worth it?

    Hey everyone, I'm trying decide between the Mellanox MCX354A-FCBT and MCX354A-FCCT, the second which supports RoCE v2 and commands a higher price. Has anyone used RoCE v2 and noticed an appreciable improvement either in Quality of Life or performance in a homelab setting? Cheers, Bryan
  6. bryan_v

    Accoustics of MSX6036 for homelab

    Hi everyone, I'm contemplating getting a Mellanox SX6036 to act as a 40G core switch paired with CX3 QSFP cards for my VM hosts and storage nodes, however I'm concerned with the acoustics of the switch. The data room in my house draws in air from the hallway next to my office, so I've replaced...
  7. E

    Mellanox ConnectX-4, how to apply settings permanently?

    I have many Mellanox ConnectX-4 25G cards on bare-metal Windows machines, which need FEC settings applied via 'mlxlink' utility in order to link to a particular switch. Trouble is, these settings do not get saved, and are lost after a reboot. How can I make sure that they are permanently applied?
  8. M

    Looking for HP z840 workstation users with Infiniband network cards

    Hi all, I have a couple z840 workstations that I'd like to connect with Mellanox ConnectX-3 VPI 40Gb network cards. These are PCIe gen3 cards that the mobo on the z840 supports (I have tested the PCI slots with two graphics cards and the machines boot fine). When I install the network card...
  9. M

    ConnectX-3 VPI (MCX354A-FCBT or 649281-B21) - Support new OS

    Who already uses this nic. Are the drivers mlnx_ofed, mlnx_en, mlx4 and winof compatible for ConnectX-3 VPI with the new os? I think, many will find it useful to know this before buying. Win Serv 19, 22 Debian 10.11 / 11.1 Proxmox VE 7.1 Other OS and Kernel And does it work in them - sr-iov...
  10. L

    Mellanox (NVIDIA) NIC reliability & support contracts

    I generally buy Intel NICs but am looking to buy some ConnectX-5 NICs (new, not EBay) for the first time for use in a new TrueNAS box. While the Intel NICs have "limited lifetime" warranties and unlimited access to firmware/software updates, it appears the ConnectX cards only have a 1 year...
  11. J

    Mellanox SN2410 SSD

    I have a 2 SN2410 of which the SSD's are fried. It's a known issue with these disks, Mellanox stopped shipping these switches with these disks: InnoDisk 3ME 32G (MEM000089) • Failures have been observed (lower than 0.8%), with correlation to excessive SSD disk write operations • Symptoms: SSD...
  12. Z

    [Solved] Mellanox SX6012 u-boot password removal without bash access?

    Does anyone know how to remove the u-boot password on SX6012 without the access to bash? I'm aware of the methods posted by @necr, but I only have access to MLNX-OS 3.5.1006 which doesn't allow to reset the bootmgr pwd from the shell. This is the EMC switch and I converted with the manufacturing...
  13. L

    Mellanox ConnectX-4s Not Working

    So I've spent many hours on this completely unnecessary home network upgrade from 40 gigabit to 100 gigabit and it has been a very frustrating journey. I recently acquired a Celestica DX010 100 gigabit switch loaded with SONiC, two Mellanox ConnectX-4s (Exact model for both appears to be...
  14. S

    Which switch for connecting ConnectX2 and ConnectX3 cards ?

    Hello, My lab is a HyperV cluster composed of 4 x HP DL170e servers and 1 x Dell RX720XD with Windows Server 2019. HP Servers > HyperV Hosts Dell Server > Storage Server (ISCSI and SMB). I purchased 1 ConnectX3 40GB dual port card for the Dell server (part number MCX354A-QCBT). HP servers...
  15. 1

    Cost effective 40Gbe and 10Gbe - Brocade ICX6610 to FreeBSD, VMWare ESXi, and Windows

    I have been scrubbing the forum looking for good and cost effective choices to connect a ICX6610 to FreeBSD (TrueNAS) at 40gbe. Is the best cost effective approach to use a Mellanox MCX354A-QCBT (cross-flashed to MCX354A-FCBT)? Or is this solution too old now. I'm having a hard time finding a...
  16. N

    Getting a 100gbe link between Celstica DX010 and Mellanox ConnectX-4?

    Howdy! I've got a Celestica SeaStone DX010 running SONiC, which is working well with a couple QSFP+ 40gbe AOCs and ConnectX-3 adapters. I've picked up a few ConnectX-4 adapters (HP 840QSFP28, CX455A) and three different brands of DACs - but I can't get a link to come up at 100gbe. If I...
  17. A

    Mellanox Onyx update

    I was able to get a good deal on SN2700 switch, but it has a pretty old MLNX-OS release on it. Does anyone have access to a recent release and willing to share?
  18. Sniper_X

    Stumped by 10Gb speed limit on 40/56gb IB cards...

    I'm using a pair of dual port Connectx-3 cards (Part# MCX354A-FCBT on FW 2.42.5000 ) and I seem to be stuck at 10Gb. My ultimate goal is to get to pure Infiniband connectivity between the servers and the storage. I have Dell PowerConnect 8132(N4032) switches AND a pair of Mellanox IS5022...
  19. gb00s

    EU (CLOSED) Giving away broken EMC SX6012 (ES)

    Giving away the possible 'broken' EMC SX6012 from this thread: . Take note it's an 'Engineering Sample'. The Delta fans are still working. The screws for the case are gone. I will ship the broken 2nd...
  20. S

    Can't find updated firmware for Mellanox SFP+ ConnectX3 MCX341A-XCE Card

    My Freenas server is not detecting my new (from ebay) SFP+ card. It is the Mellanox ConnectX3 MCX341A-XCE, but I haven't been able to find an updated firmware for it anywhere. If I try the other firmware's for other variations of the card, I get this error -E- PSID mismatch. The PSID on flash...