infiniband

I have a little issue while testing InfiniBand connections. While system (Bios-wise) is set to performance and the CPU governance is set to performance, I still receive the message/alert/warning of CPU frequency mismatches. Both CPU's are E5-2673v3's. The cards are old 10EUR eBay findings of SUN...

Dear Friends I want to hook up around 10 individual powerful PCs and create a cluster out of them. My main use for this cluster is running CFD models which can run through MPI over Ethernet of Infiniband networks. As far as I know, Infiniband is superior for this purpose, but I dont have any...

I'm setting up a vSAN network with 3 Nodes. I have set up a vSAN cluster with 10G Ethernet (base-T) 2 years ago and wanted to delve into infiniband for a new test environment. I've done some reading up on this but wanted to get some tips from people who have successfully implemented a similar...

Hi, I bought the Switch on eBay. I factory resettet it, installed the latest FW (3.9.1) and configured everything regarding the documentation. But it does not enable the ports. I connected Melanom Connectix-3 Adapters on Windows and vSphere, but both do not show a link. The cables are fibre and...

I have noticed a few 12$ ib cards from qlogic and cx2 rebaged as sun. is there any reason on the ib side to get a cx3 pro vs 3 reg or 2? I was not seeing any documentation related to ib performance only the v1 vs v2 roce. For now the only concern i have is ib related performance on these...

I am planning on building my own 4 port 40 Gb Infiniband, 2 Port 10 Gb SFP+, 1 Port 10 GbE Switch after watching an article on how to do so on the net, brilliantly cheap way of getting really high speed transfers between my soon to be Storage server and my two clients. I brought 8 MHQH19B-XTR's...

Hey folks, *I could be way off even my own topic here too as my personal experience with this model and even the fact it's not even delivered yet is a troubling issue to my factual experience on the matter, nice disclaimer eh.. (he says) The Voltaire 4036 switch even for its age is a very...

I have been running Mellanox QDR Infiniband primarily due to the magnitude of difference in latency using RDMA and its use via SRIOV in containers and VMs. Unfortunately Infiniband in the industry is almost entirely proprietary and PCIE fabrics have been a promising future for years now without...

Hi Folks Need help getting SR-IOV (inifiniband) to work with Proxmox-ve. I’ve installed MLX_OFED_LINUX ver 4.3-1.0.1.0 installed on both the host (proxmox ve 5.4-3 and the guest (ubuntu 18.04). I’ve enabled SR-IOV in the firmware of the card, SR-IOV on MLX_OFED Driver and go the point of...

According to german it news site Golem and their source Intel bids 6billion $ for Mellanox. Sources: Intel Among Bidders for Chipmaker Mellanox - CTech Rechenzentren: Intel bietet wohl 6 Milliarden Dollar für Mellanox - Golem.de (German) I'm wondering what will happen if Mellanox accepts the...

So, through much blood, sweat, and tears, I have my NAS and ESXi recognizing their new-ish ConnectX-3 cards in IB mode. I can send pings across the links and everyone looks happy. However, after adding my NFS storage to ESXi, I am now having another problem. Access to the NFS share from ESXi is...

Hello, We bought a used IS5030 switch from ebay. But only the top 18 ports can work and the rate is only 20Gb/s per port, even though we used QFP cables. It looks like a license problem. I found several numbers on the pull out tab. But it always shows Malform license key error when I tried to...

I picked up a Mellanox IS5600 and do not have any personal use for it. Total Newb when talking about commercial-ish switches. This unit taken out of a working environment. Retailed for $500,000 retail LOL. Seriously. Half a milli! Listed for $3,000 USD plus shipping. Paypal or local cash...

Hi there all, I've just finished setting up a nice AIO proxmox server and between it and my workstation (direct connection), it saturates 10Gb or gives a faster then local SSD speed depending on work load (NFS maxes out at 4GB/s but thats a different thread)......and now I want more. Well not...

Hello all, I was thinking, for my living room where I have quite a few servers running (silent !), to do a custom 10gbe IPoIB setup. In my lab room I could use the Voltaire 4036 (non-E version unfortunately) which I still need to find a way to silence. It's way too noisy even for the server...

Hello! I have the opportunity to purchase some used dual-port Mellanox MCX354A-QCBT ConnectX-3 cards with VPI network adapter. (10 GbE Ethernet and 40 Gbps InfiniBand.) These would be perfect for my cluster after correct configuration.. They were originally pulled from EMC systems and...

Hi, I have a weird problem. I purchased 4x used identical Sun QDR X4242A Infiniband cards (Mellanox MHQH29B rebrands) for my 4 node supermicro 6027TR-HTR (X9DRT-HF) server. 3/4 cause boot to hang at post code 91, which is when the PCI stuff is loaded. The fourth one seems to let the node boot...

Good Afternoon everyone. You will have to excuse my ignorance when it comes to infiniband/ SFP/ SFP+ and QSFP. I have purchased two Mellanox MHQH29C-XSR ConnectX-2 VPI 10/40 Gbit Low Profile Dual-Port Adapter Cards for use in my home network (purely for learning purposes and for transferring...

Unbelievable deal! These are low profile dual port cards capable of 56Gbps IB and 40Gbps Ethernet. I messaged the seller and bought 32 cards for $25 each plus shipping. He said he has 1300 in stock so there's plenty for everyone :) They can be flashed from their HP firmware to stock Mellanox...

Build’s Name: RCluster (30 nodes) Operating System/ Storage Platform: Proxmox VE 5.1 CPU: AMD Ryzen 7401P Motherboard: Supermicro H11SSL-C Chassis: RMC3116-670-HSE with SAS3 expander Drives: (16) Seagate ST6000NM0034 SAS3 6TB RAM: DDR4 2666mhz ECC Reg 64GB x4 and 64GB x8 Network Cards...