IB Open SM setup questions (where not how)

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
Hi,
since it seems to be difficult to obtain a cheap 5xxx/6xxx IB switch with subnet manager I was thinking about running opensm (dont need 36 ports and huge power consumption, so not really considering a 4036).

Now there would be several theoretical options given my environment (3 ESX hosts):
1. No IB Switch at all
A: Triangle setup - Run on each ESX (I know there is a vib which is known to work on 5.5 (connect X2 level iirc), but no feedback on newer ESX releases or compatibility with newer hardware. Unfortunately it is not fit for vsan either according to the author.
B: Run OpenSM on a dedicated box (eg some kind of low power mini pc with two PCIe slots ), i.e. Build your own switch"
-What is the cpu utilization for the open SM process on various CPUs? I.e. whats my CPU requirement
-Do I need a same gen Card to tap into the IB network as the other cards? Or can I use a ConnectX2 card to run openSM for a FDR
C: Can I tap into the network with using
C1: SRV-IO ? IE passing a virtual nic to a VM?
C2: Passing an actual nic to a VM?

2. Using an unmanaged IB Switch
Options B & C from above (B with a single pci-e slot)
 

Rand__

Well-Known Member
Mar 6, 2014
6,634
1,767
113
Stuck for now.
ESX6.5 native does not know max_vfs option and no Mellanox OEFD driver for 6.5 available yet (though mft& nmst tools are there) :/

OpenSM does not seem to run on ESX 6.5 either, not sure whether its due to its age or missing Mellanox OEFD driver though