Hey,
I've never used an IB adapter before, but I was thinking of doing some direct-attached IB adapters for my 3 ESXi hosts now that I learned my 3 Intel 82598EB adapters I dug out of storage are now no longer supported. I do have VCSA if that matters.
I have 82599 10GbE for basic networking, but I wanted to use IB for a HA storage backend sync + HA heartbeat on something like drbd, ctl HA + hast, Starwind vSAN, something I can rig up on OmniOS, etc. (haven't decided which yet, if you have any war stories feel free to share).
I'm really cheap, but I'd like to get an adapter that has some longevity, given that VMware likes to depreciate hardware fairly expediently. I noticed there's lots of ConnectX-2 adapters still supported by ESXi, but I doubt for long (RIP my 82598s, *sniff*). So I was thinking of going with the MCX354A-FCCT Dual QDR+ 40/56, which looks like the best combination of supported + inexpensive I've found so far.
My questions:
Any guidance much appreciated. Thanks!
I've never used an IB adapter before, but I was thinking of doing some direct-attached IB adapters for my 3 ESXi hosts now that I learned my 3 Intel 82598EB adapters I dug out of storage are now no longer supported. I do have VCSA if that matters.
I have 82599 10GbE for basic networking, but I wanted to use IB for a HA storage backend sync + HA heartbeat on something like drbd, ctl HA + hast, Starwind vSAN, something I can rig up on OmniOS, etc. (haven't decided which yet, if you have any war stories feel free to share).
I'm really cheap, but I'd like to get an adapter that has some longevity, given that VMware likes to depreciate hardware fairly expediently. I noticed there's lots of ConnectX-2 adapters still supported by ESXi, but I doubt for long (RIP my 82598s, *sniff*). So I was thinking of going with the MCX354A-FCCT Dual QDR+ 40/56, which looks like the best combination of supported + inexpensive I've found so far.
My questions:
- Am I making a good choice for a cheapskate for adapter? Any other recommendations? What about this re-flashing I've been reading about, how extensive is that, and how hard is it?
- What sort of protocol should I be using w/ ESXi (IPoIB, RDMA, etc.)?
- I'm totally new to IB and have no idea what I'd be doing other than I know one of the machines needs a subnet manager - are multiple subnet managers possible? I'm trying to set up an HA environment, after all...
- Any other guidance like what cables I should be using, etc. All the machines are in the same room but one is about 15ft from the others
- I assume dual port cards are necessary for a DAC setup, for the "hub" machine (set up as a gateway + SN manager), at the very least - I was thinking 1x dual w/ 1x going to each of the two other machines. Would dual port cards be helpful for either of the other two machines?
Any guidance much appreciated. Thanks!