I currently have an NFS server running ZoL 0.7.11. Works okay, but I'd prefer some Illumos based distro. My ESXi host is connected to the storage server using a point to point QSFP cable and 2 Mellanox Connectx-3 EN cards. Works okay, but I'd like to dabble with RDMA. The most visible products are Mellanox Connectx-4 LX cards, but Mellanox seems long ago to have given up any interesting in any Solaris based product. Chelsio makes a couple of interesting cards (t5 and t6 asics), but chelsio seems to be riding the iwarp bandwagon, which is broken down by the side of the road (even Intel seems to have ditched it.) So now, chelsio is pushing things like iSCSI offload, although I guess you might be able to do RDMA NFS from ESXi => Solarish server. But vmware has stated that from 6.7 forward, they will only support RoCE (implying no support for iWARP). If illumos provides drivers for esxi and solarish, I guess I'd be fine with that, but it's hard to tell from reading illumos HCL what they in fact support. The chelsio driver claims to support everything up to t6 asic, but that doesn't mean it supports all the fancy offload stuff. Sorry for the TL;DR 
If anyone has any concrete recommendations, I'd much appreciate them. If I can't find out anything useful, I might need to stick with Linux and go with the recent Mellanox cards.
If anyone has any concrete recommendations, I'd much appreciate them. If I can't find out anything useful, I might need to stick with Linux and go with the recent Mellanox cards.