rdma

Dear Forum Members, I hope this message finds you well. I’m reaching out to seek your advice on a challenge I’m facing regarding the reception of a substantial volume of packets while working within constraints on completion queue size. In my current project, I’m encountering difficulties...

I’m currently working on receiving UDP data packets over Ethernet using RoCEv2, but I’ve encountered a challenge in my code. I’m trying to continuously receive data packets emitted by an FPGA, utilizing RoCEv2. I attempted to modify the code for multicast given in following document ,to unicast...

Hello Community, I am reaching out to seek recommendations on the best resources to learn RoCE. Whether it's online courses, tutorials, books, or any other learning materials, I would greatly appreciate your insights and suggestions. If you have personal experiences with particular resources...

For my setup, I already have (4) CX3 non-pro cards. I needed to get a single CX4 card in order to support the mlx5 driver that is now bundled with macOS and my macs can be on the network now via thunderbolt. I am interested in setting up a simple point to point network at home where the...

Edit: Everything below is... well, it's not wrong, except RDMA is actually not working for ANY hosts. Everything is going through IPoIB and it's awful. I thought it was working for other hosts but it is not. Original post: This host ("shark") currently has four RDMA-aware functions, and none...

I'm currently using FreeNAS for iSCSI ZFS block storage for my ESXi server. The FreeNAS box is getting old so I'm looking for a replacement. I'm not currently using any of the other FreeNAS features besides ZFS and iSCSI. I was initially planning to upgrade the FreeNAS box to TrueNAS Core or...

Hey, I've never used an IB adapter before, but I was thinking of doing some direct-attached IB adapters for my 3 ESXi hosts now that I learned my 3 Intel 82598EB adapters I dug out of storage are now no longer supported. I do have VCSA if that matters. I have 82599 10GbE for basic networking...

Hello, I am seeking advice or consensus if we should try to engineer our Nutanix cluster with RDMA. It stands now the nodes are fairly beefy: Each node is 2x 6226R and 1.5TB ram each,12x3.84TB SAS12 SSDs (no NVME drives). The load is a mixed server load, various biz apps, SQL databases, some...

Hi. I want to try RoCE2 in my homelab to get better performance on my glusterfs vm storage. Are there any DCB capable switches under $500 used? 10gb ports is fine. (Not sure if DCB is required for RoCE2, actually. but hey, thats why i need smth to experiment on)

Hi all, n00by here... but this place has a wonderful amount of knowledge. I asked a similar question in the Mellanox forums here: https://community.mellanox.com/s/question/0D51T00007CGDzUSAX/which-cables-for-a-pair-of-mcx353afcbt I have two MCX353A-FCBT cards on the way and I want to connect...

We have a series of 13 Dell R610 hosts with ConnectX-2 EN cards running ESXi 6.7. We have been trying to get RDMA (more importantly the iSER adapter) to enable and show up, however we have been unsuccessful. RDMA is supported by the FW, as we had gotten it to show up before with different...

Hi Folks, Anyone come across or try out a killer use case for RDMA / RoCE? I know there are custom builds of some hadoop based apps but what have you found to be a great use of this tech? I have the box, now need the project. :) James

I want to share following testing with you 4 PVE Nodes cluster with 3 Ceph Bluestore Node, total of 36 OSD. OSD: st6000nm0034 block.db & block.wal device: Samsung sm961 512GB NIC: Mellanox Connectx3 VPI dual port 40 Gbps Switch: Mellanox sx6036T Network: IPoIB separated public network &...

Using Windows Server 2016DC, I'm building a 3-node Hyper-converged, Storage Spaces Direct failover cluster. It seems to work good and passes validation using an ordinary D-Link 10GB switch except the fact that I can't see any RDMA traffic over the switch. I confirmed that I can see RDMA activity...

Hello, I got two MCX314 cards, and I am trying to make SMB direct in windows 10 working. I enabled DCB and SMB Direct in windows features, updated firmware to 2.42 and installed WinOF 5.35 In Mellanox guide for Windows 2012, It is specified that for WinOF 4.9 or higher, RoCE version has to be...

Hi, kind of an odd question - I know SMB3 is being touted as a VM datastore-worthy transfer protocol on Windows Server platforms with RDMA / SR-IOV capabilities, etc. but is the same thing available for Linux? Do I understand it in that 'Samba' for Linux is a userland-only protocol and...

Hi Everyone, hope everything is going well on your side of the fence. I am new here and in need of help getting RDMA to work on my little storage/SAN project on Windows Server 2016. I bought 2 of these Mellanox MHQH29C-XTR cards on Ebay I have updated the firmware to 2.9.8350 My client is...