1. K

    Mellanox MSN2700-CS2R_SS1 Installation

    Hello, Can anyone help get me the mellanox onyx installer for this switch? It seems like it is booting to ONIE and has no NOS. Does anyone know about this model? I cannot seem to find it in the documentation—its mfg date is from 2020.
  2. U

    Mellanox ConnectX-3 Windows 11

    None of Nvidia's WinOF drivers support Windows 11. I tried to install a Win10 driver and it gave errors, like 1719 "Windows Installer service could not be started." Upon trying to close out the failed installation, I can't even close it, it gives 2732 "Directory Manager not initialized"...
  3. U

    Mellanox ConnectX-3 Wake on LAN

    Hi, I'm using ConnectX-3's to implement 40Gbit networking throughout my home, and one of the important nodes is the HTPC. As you might imagine I think that it would be swell given the lack of HDMI-CEC on Nvidia to be able to boot up the PC without getting off the couch. The first step for that...
  4. C

    NVIDIA PNY T1000 8GB GPU, 40G Mellanox ConnectX-3, 2x Dell Precision 3650 Workstations (GPU compatible + In Warranty!)

    Dell Precision 3650 Tower Workstation $1200 quantity: 2 8 core Xeon W-1370 / 24GB DDR4 / 256GB NVME with Windows 10 + 12TB HDD + **Dell warranty** valid through **3/09/2025** (see bottom for more specs) Buy both $2350 NVIDIA PNY T1000 8GB $375 quantity: 1 Plays Elden Ring on Ultra at 1080...
  5. U

    Mellanox ConnectX3 MCX353A-QCAT? -FCBS?

    What are the difference between these and the -FCBT? I have some 354A-FCBT and -QCBT dual port NICs and I like them because they work and I'm familiar with their management now. I have need for the even smaller profile single slot QSFP+ cards like the above, which I see on EBay, and I'd like to...
  6. VMman

    Recommended 10Gb NIC for Proxmox + Ceph Lab

    Hi All, I'm currently in the planning phase of setting up a lab using 2 Proxmox hypervisors connected to 3 Ceph storage hosts. I wanted to get everyone's opinion on what is considered the best NIC value or otherwise to connect these servers together using a pair of ICX 6650's that I have. I...
  7. G

    Mellanox 10G Dual-Port card does not do any routing with IPoIB?

    Hey folks, I have 3 servers equipped with the Connect-X2 MT26418, all running Proxmox (debian11). The cards are all 2 Port ones, and I got 2 cables running between them like this: +---------------------------+ | serv1 | +---------------------------+ | Port 2 |...
  8. bryan_v

    Juniper LX4 optics & Mellanox CX3/SX6036 compatibility

    Quick question: Has anyone had any success with 40G LX4 transceivers in Mellanox CX3 cards and SX6036 switches (specifically Juniper QSFP+ 40G LX4 LC transceivers)? I'm about to run optics throughout the house, and MTP/MPO cables are so much more expensive than LC/LC cables for multiple runs...
  9. C

    How to use the mlx5_core driver with Mellanox ConnectX-4 Lx in Debian?

    I just got these ConnectX-4 Lx in the mail, but not getting very far. I've been treating this like the ixgbe driver: I see Kernel driver in use: mlx5_core and am trying to create virtual functions in /etc/modprobe.d/mlx5_core.conf with some options like options mlx5_core num_vfs=12...
  10. J

    Mellanox ConnectX-3 Showing wrong bus type

    I just installed a Mellanox ConnectX-3 two port in my Windows 2016 server. Haven't installed the fiber cable or SFP+ transceivers yet, however the Bus Type is showing as x8 when I know it's x16 slot any idea what's up or is that just because it's the max the card supports so the "card" is x8...
  11. bryan_v

    Is RoCE v2 worth it?

    Hey everyone, I'm trying decide between the Mellanox MCX354A-FCBT and MCX354A-FCCT, the second which supports RoCE v2 and commands a higher price. Has anyone used RoCE v2 and noticed an appreciable improvement either in Quality of Life or performance in a homelab setting? Cheers, Bryan
  12. bryan_v

    Accoustics of MSX6036 for homelab

    Hi everyone, I'm contemplating getting a Mellanox SX6036 to act as a 40G core switch paired with CX3 QSFP cards for my VM hosts and storage nodes, however I'm concerned with the acoustics of the switch. The data room in my house draws in air from the hallway next to my office, so I've replaced...
  13. E

    Mellanox ConnectX-4, how to apply settings permanently?

    I have many Mellanox ConnectX-4 25G cards on bare-metal Windows machines, which need FEC settings applied via 'mlxlink' utility in order to link to a particular switch. Trouble is, these settings do not get saved, and are lost after a reboot. How can I make sure that they are permanently applied?
  14. M

    Looking for HP z840 workstation users with Infiniband network cards

    Hi all, I have a couple z840 workstations that I'd like to connect with Mellanox ConnectX-3 VPI 40Gb network cards. These are PCIe gen3 cards that the mobo on the z840 supports (I have tested the PCI slots with two graphics cards and the machines boot fine). When I install the network card...
  15. M

    ConnectX-3 VPI (MCX354A-FCBT or 649281-B21) - Support new OS

    Who already uses this nic. Are the drivers mlnx_ofed, mlnx_en, mlx4 and winof compatible for ConnectX-3 VPI with the new os? I think, many will find it useful to know this before buying. Win Serv 19, 22 Debian 10.11 / 11.1 Proxmox VE 7.1 Other OS and Kernel And does it work in them - sr-iov...
  16. L

    Mellanox (NVIDIA) NIC reliability & support contracts

    I generally buy Intel NICs but am looking to buy some ConnectX-5 NICs (new, not EBay) for the first time for use in a new TrueNAS box. While the Intel NICs have "limited lifetime" warranties and unlimited access to firmware/software updates, it appears the ConnectX cards only have a 1 year...
  17. J

    Mellanox SN2410 SSD

    I have a 2 SN2410 of which the SSD's are fried. It's a known issue with these disks, Mellanox stopped shipping these switches with these disks: InnoDisk 3ME 32G (MEM000089) • Failures have been observed (lower than 0.8%), with correlation to excessive SSD disk write operations • Symptoms: SSD...
  18. Z

    [Solved] Mellanox SX6012 u-boot password removal without bash access?

    Does anyone know how to remove the u-boot password on SX6012 without the access to bash? I'm aware of the methods posted by @necr, but I only have access to MLNX-OS 3.5.1006 which doesn't allow to reset the bootmgr pwd from the shell. This is the EMC switch and I converted with the manufacturing...
  19. L

    Mellanox ConnectX-4s Not Working

    So I've spent many hours on this completely unnecessary home network upgrade from 40 gigabit to 100 gigabit and it has been a very frustrating journey. I recently acquired a Celestica DX010 100 gigabit switch loaded with SONiC, two Mellanox ConnectX-4s (Exact model for both appears to be...
  20. S

    Which switch for connecting ConnectX2 and ConnectX3 cards ?

    Hello, My lab is a HyperV cluster composed of 4 x HP DL170e servers and 1 x Dell RX720XD with Windows Server 2019. HP Servers > HyperV Hosts Dell Server > Storage Server (ISCSI and SMB). I purchased 1 ConnectX3 40GB dual port card for the Dell server (part number MCX354A-QCBT). HP servers...