1. A

    About the non-standard OSFP cage in mellanox Connectx-7 400G Network Adaptor

    I recently got some connectx-7 from local distributer that used to sell unlisted lab equipment from some of the big companyies. As the price was fair enough, i just immediately purchased it without further testing and evalutation. Unfortunately, when I tried to connect two of the NICs , I...
  2. A

    [FS] [US-MN] Multiple 100GBE network cards, PCIe Bifurcation cards, DDR4 ram

    Getting rid of some miscellaneous equipment, let me know if you have any questions. 1. Mellanox ConnectX-6 CX653105A single port 100GBE QSFP28 PCIe 3.0 x16 - $425 shipped 2. Two Mellanox ConnectX-4 CX455A single port 100GBE QSFP28 PCIe 3.0 x16 cards with two CWDM4 LC transceivers - $325...
  3. J

    What would cause a command to be added/removed from a Mellanox switch config?

    I have an MSB7800 running 3.10.2102 code and it seems like the following config line at the end is being added and removed randomly. Does anyone know why this would happen? I have confirmation from the team no one is doing it manually and I am trying to get the configs into git for archive...
  4. B

    Mellanox IB SB7800 Drive Failure

    Good morning, We've had an SSD on our sb7800 infiniband switch fail, have installed a new drive with a cloned image from one of our other identical switches. So far everything looks ok in the management interface, aside the Host ID. Though we do get 1003, 1006 errors when trying to modify...
  5. M

    EXPIRED Mellanox MCP1600-E002 QSFP28 IB EDR (100Gb/s) 2M DAC used - $12 OBO Looks like it was discounted to $12 few days ago. Which is an attractive price for EDR DAC. From quick research MCP1600-Exxx should support 56G Ethernet and FDR 56Gbit/s IB both. Ordered 4 for myself.
  6. K

    Mellanox MSN2700-CS2R_SS1 Installation

    Hello, Can anyone help get me the mellanox onyx installer for this switch? It seems like it is booting to ONIE and has no NOS. Does anyone know about this model? I cannot seem to find it in the documentation—its mfg date is from 2020.
  7. U

    Mellanox ConnectX-3 Windows 11

    None of Nvidia's WinOF drivers support Windows 11. I tried to install a Win10 driver and it gave errors, like 1719 "Windows Installer service could not be started." Upon trying to close out the failed installation, I can't even close it, it gives 2732 "Directory Manager not initialized"...
  8. U

    Mellanox ConnectX-3 Wake on LAN

    Hi, I'm using ConnectX-3's to implement 40Gbit networking throughout my home, and one of the important nodes is the HTPC. As you might imagine I think that it would be swell given the lack of HDMI-CEC on Nvidia to be able to boot up the PC without getting off the couch. The first step for that...
  9. C

    NVIDIA PNY T1000 8GB GPU, 40G Mellanox ConnectX-3, 2x Dell Precision 3650 Workstations (GPU compatible + In Warranty!)

    Dell Precision 3650 Tower Workstation $1200 quantity: 2 8 core Xeon W-1370 / 24GB DDR4 / 256GB NVME with Windows 10 + 12TB HDD + **Dell warranty** valid through **3/09/2025** (see bottom for more specs) Buy both $2350 NVIDIA PNY T1000 8GB $375 quantity: 1 Plays Elden Ring on Ultra at 1080...
  10. U

    Mellanox ConnectX3 MCX353A-QCAT? -FCBS?

    What are the difference between these and the -FCBT? I have some 354A-FCBT and -QCBT dual port NICs and I like them because they work and I'm familiar with their management now. I have need for the even smaller profile single slot QSFP+ cards like the above, which I see on EBay, and I'd like to...
  11. VMman

    Recommended 10Gb NIC for Proxmox + Ceph Lab

    Hi All, I'm currently in the planning phase of setting up a lab using 2 Proxmox hypervisors connected to 3 Ceph storage hosts. I wanted to get everyone's opinion on what is considered the best NIC value or otherwise to connect these servers together using a pair of ICX 6650's that I have. I...
  12. G

    Mellanox 10G Dual-Port card does not do any routing with IPoIB?

    Hey folks, I have 3 servers equipped with the Connect-X2 MT26418, all running Proxmox (debian11). The cards are all 2 Port ones, and I got 2 cables running between them like this: +---------------------------+ | serv1 | +---------------------------+ | Port 2 |...
  13. bryan_v

    Juniper LX4 optics & Mellanox CX3/SX6036 compatibility

    Quick question: Has anyone had any success with 40G LX4 transceivers in Mellanox CX3 cards and SX6036 switches (specifically Juniper QSFP+ 40G LX4 LC transceivers)? I'm about to run optics throughout the house, and MTP/MPO cables are so much more expensive than LC/LC cables for multiple runs...
  14. C

    How to use the mlx5_core driver with Mellanox ConnectX-4 Lx in Debian?

    I just got these ConnectX-4 Lx in the mail, but not getting very far. I've been treating this like the ixgbe driver: I see Kernel driver in use: mlx5_core and am trying to create virtual functions in /etc/modprobe.d/mlx5_core.conf with some options like options mlx5_core num_vfs=12...
  15. J

    Mellanox ConnectX-3 Showing wrong bus type

    I just installed a Mellanox ConnectX-3 two port in my Windows 2016 server. Haven't installed the fiber cable or SFP+ transceivers yet, however the Bus Type is showing as x8 when I know it's x16 slot any idea what's up or is that just because it's the max the card supports so the "card" is x8...
  16. bryan_v

    Is RoCE v2 worth it?

    Hey everyone, I'm trying decide between the Mellanox MCX354A-FCBT and MCX354A-FCCT, the second which supports RoCE v2 and commands a higher price. Has anyone used RoCE v2 and noticed an appreciable improvement either in Quality of Life or performance in a homelab setting? Cheers, Bryan
  17. bryan_v

    Accoustics of MSX6036 for homelab

    Hi everyone, I'm contemplating getting a Mellanox SX6036 to act as a 40G core switch paired with CX3 QSFP cards for my VM hosts and storage nodes, however I'm concerned with the acoustics of the switch. The data room in my house draws in air from the hallway next to my office, so I've replaced...
  18. E

    Mellanox ConnectX-4, how to apply settings permanently?

    I have many Mellanox ConnectX-4 25G cards on bare-metal Windows machines, which need FEC settings applied via 'mlxlink' utility in order to link to a particular switch. Trouble is, these settings do not get saved, and are lost after a reboot. How can I make sure that they are permanently applied?
  19. M

    Looking for HP z840 workstation users with Infiniband network cards

    Hi all, I have a couple z840 workstations that I'd like to connect with Mellanox ConnectX-3 VPI 40Gb network cards. These are PCIe gen3 cards that the mobo on the z840 supports (I have tested the PCI slots with two graphics cards and the machines boot fine). When I install the network card...
  20. M

    ConnectX-3 VPI (MCX354A-FCBT or 649281-B21) - Support new OS

    Who already uses this nic. Are the drivers mlnx_ofed, mlnx_en, mlx4 and winof compatible for ConnectX-3 VPI with the new os? I think, many will find it useful to know this before buying. Win Serv 19, 22 Debian 10.11 / 11.1 Proxmox VE 7.1 Other OS and Kernel And does it work in them - sr-iov...