I have read things about using virtual networking via RDMA using these. It seems a little complicated, and probably wouldn't have been an ideal use case when they were new, but considering how cheap they are on eBay now it almost seems wasteful to not at least consider them for that application.For Omni-Path? Maybe.
Do you want Omni-Path? If you have to ask, then most definitely not.
Those cards only do Omni-Path. No Ethernet.
At least for hobbyists.
My only problem is, I wouldn't even know where to start with that.