Infiniband Multicast

[jwc]

Hi,

I'm running:

• ESXi, 6.5.0, 4887370
• OFED 1.8.2.5
• ConnectX-2 MHQH29C with 2.9.1000
• IS5035 switch with subnet manager

I followed the instructions here: 10Gb SFP+ single port = cheaper than dirt

I am able to get IPoIB working and all of the hosts can ping each other.

However, multicast is not working. How can I get multicast to work? It is preventing vSAN from working.

 

[Rand__]

1. Newer versions of vSan don't need multicast any more so that should not be the issue (even if the test is still there)
2. Is that the only test that fails?
3. Have you set up the vsan enabled vmk's?`

 

[jwc]

1. I'm installing vcenter and esxi 6.5 w/ update 1 now
2. The alarms are: "Active multicast connectivity check" & "Multicast assessment based on other checks"
3. Yes, all vmk nics have vSAN checked

The symptom is the storage is partitioned.

 

[Rand__]

yes it might be U1 (vsan 6.6 iirc) that does away with multicast

vmkping is working fine?

 

[jwc]

Yes, all hosts respond to ping and vmkping

 

[Rand__]

So how is the partition? 1:2?
Whats your setup 2 node with witness, 3 nodes+?

 

[jwc]

yes it might be U1 (vsan 6.6 iirc) that does away with multicast

vmkping is working fine?

Hi Rand__, you are correct, the solution was to upgrade to ESXi 6.5 update 1 which contains vSAN 6.6. vSAN 6.6 removes the need for multicast. Thank you.