infiniband

  1. M

    Best 300TB storage for $100K challenge

    Hi, I work for a research hospital. I would appreciate any comments or advice for file system and hardware design for a high-end storage solution. We have a few microscopes (running windows 10), each can produce 3-4GB/s of image data that needs to written to the shared storage and to be...
  2. S

    Understanding Infiniband IPOIB vs ETH mode

    I looking for a better understanding on the difference between IPOIB and ETH mode on ConnectX-3 adapters (40/40 IB/Eth) Couldn't find answers to my questions online; if someone could point me towards the right direction, it would be much appreciated. 1. My understanding is the IPOIB runs on top...
  3. L

    Infiniband Network for small GPU-Cluster

    Hi everyone, I am currently in the process of planning an Infiniband Network for our new GPU cluster. We want to be able to run simulation / AI workloads across the big GPU nodes and are hence looking to connect everything with a 100 Gb Infiniband-Network. At first, I was thinking of the...
  4. M

    Looking for HP z840 workstation users with Infiniband network cards

    Hi all, I have a couple z840 workstations that I'd like to connect with Mellanox ConnectX-3 VPI 40Gb network cards. These are PCIe gen3 cards that the mobo on the z840 supports (I have tested the PCI slots with two graphics cards and the machines boot fine). When I install the network card...
  5. Sniper_X

    Stumped by 10Gb speed limit on 40/56gb IB cards...

    I'm using a pair of dual port Connectx-3 cards (Part# MCX354A-FCBT on FW 2.42.5000 ) and I seem to be stuck at 10Gb. My ultimate goal is to get to pure Infiniband connectivity between the servers and the storage. I have Dell PowerConnect 8132(N4032) switches AND a pair of Mellanox IS5022...
  6. AveryFreeman

    Anyone using IB over ConnectX-3 w/ ESXi 7.0U1?

    Hey, I've never used an IB adapter before, but I was thinking of doing some direct-attached IB adapters for my 3 ESXi hosts now that I learned my 3 Intel 82598EB adapters I dug out of storage are now no longer supported. I do have VCSA if that matters. I have 82599 10GbE for basic networking...
  7. lihp

    Mellanox IS5022 InfiniBand Switch 8 ports managed - QDR

    No, not my offer. Since I am on EDR at home by now, I don't need it... Mellanox IS5022 InfiniBand Switch 8 ports managed | eBay
  8. gb00s

    Mellanox/EMC SX6012 - Revival Attempt

    New project. I received an SX6012 for shipping costs only. Tried to get it running and to get a console today. First of all, the switch starts. Both fans in the middle are running 100%. Both fans in front of the power supplies are running on minimum rpm. Mgmt port shows straight green and...
  9. V

    EU WTB: Mellanox SX6036 FDR Infiniband Switch

    Hello again! Probably should have consolidated the FS and WTB ads in one post but for clarity I figured I'd post twice. If you're here from my FS post, I am looking for the above equipment. Does anyone have one for sale? Please let me know. Happy new year!
  10. V

    EU FS: Mellanox Infiniscale IV IS5030 18-port DDR switch

    Hello boys and girls, I have here for sale the above item. I want to move up to an FDR switch as part of upgrading my home lab, so if anyone wants to get started, play around and learn about Infiniband networking with a cheap switch (DAC cables go for pretty cheap on eBay as well), this might...
  11. gb00s

    EU WTB: Mellanox 6012/6015/6036 (temporary done)

    It's Christmas ... Looking for cheap un-/managed 6012/6015's here in EU only. If you can include cables you were working with, SUPER !!! Ok, if you pay the shipping 'Overseas' are welcome as well ;) If you have a 6036 for cheap, I will consider your offer as well.
  12. gb00s

    IB - CPU Frequency Mismatches - Sun Falcon (Mellanox MHRH2A-XSR)

    I have a little issue while testing InfiniBand connections. While system (Bios-wise) is set to performance and the CPU governance is set to performance, I still receive the message/alert/warning of CPU frequency mismatches. Both CPU's are E5-2673v3's. The cards are old 10EUR eBay findings of SUN...
  13. A

    Question about best hardware components for cluster computing

    Dear Friends I want to hook up around 10 individual powerful PCs and create a cluster out of them. My main use for this cluster is running CFD models which can run through MPI over Ethernet of Infiniband networks. As far as I know, Infiniband is superior for this purpose, but I dont have any...
  14. XeonSam

    Advice on vSAN networking : Infiniband

    I'm setting up a vSAN network with 3 Nodes. I have set up a vSAN cluster with 10G Ethernet (base-T) 2 years ago and wanted to delve into infiniband for a new test environment. I've done some reading up on this but wanted to get some tips from people who have successfully implemented a similar...
  15. S

    Help! Voltaire 4036 does not enable Ports

    Hi, I bought the Switch on eBay. I factory resettet it, installed the latest FW (3.9.1) and configured everything regarding the documentation. But it does not enable the ports. I connected Melanom Connectix-3 Adapters on Windows and vSphere, but both do not show a link. The cables are fibre and...
  16. K

    Infiniband adapter question. mellinox vs other brands and cx2vs 3

    I have noticed a few 12$ ib cards from qlogic and cx2 rebaged as sun. is there any reason on the ib side to get a cx3 pro vs 3 reg or 2? I was not seeing any documentation related to ib performance only the v1 vs v2 roce. For now the only concern i have is ib related performance on these...
  17. F

    You are going to think I am insane but...

    I am planning on building my own 4 port 40 Gb Infiniband, 2 Port 10 Gb SFP+, 1 Port 10 GbE Switch after watching an article on how to do so on the net, brilliantly cheap way of getting really high speed transfers between my soon to be Storage server and my two clients. I brought 8 MHQH19B-XTR's...
  18. D

    DIY Voltaire 4036+e web2cmd interface.

    Hey folks, *I could be way off even my own topic here too as my personal experience with this model and even the fact it's not even delivered yet is a troubling issue to my factual experience on the matter, nice disclaimer eh.. (he says) The Voltaire 4036 switch even for its age is a very...
  19. E

    Open Hardware for PCIE fabric

    I have been running Mellanox QDR Infiniband primarily due to the magnitude of difference in latency using RDMA and its use via SRIOV in containers and VMs. Unfortunately Infiniband in the industry is almost entirely proprietary and PCIE fabrics have been a promising future for years now without...
  20. G

    Help getting Connectx-3 working in SR-IOV under proxmox-ve.

    Hi Folks Need help getting SR-IOV (inifiniband) to work with Proxmox-ve. I’ve installed MLX_OFED_LINUX ver 4.3-1.0.1.0 installed on both the host (proxmox ve 5.4-3 and the guest (ubuntu 18.04). I’ve enabled SR-IOV in the firmware of the card, SR-IOV on MLX_OFED Driver and go the point of...