Mellanox ConnectX-2 and ESXi 6.0 - Barely Working - Terrible Performance

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
H

humbleThC

Member
Nov 7, 2016
99
9
8
48
mpogr said:
I think we won't know until actual measurements are performed. The idea of SLOG is to make sure access to the main drives in the pool is not broken into small transactions when sync writes are requested by the client. So, potentially, an SLOG can have positive effect even when it's not faster than the main pool drives. But this is just theory, and we don't really know how it going to look like in the real world...
Click to expand...
Just posted my initial testing results.
ZFS Advice for new Setup
 
W

wiser212

New Member
Feb 7, 2017
2
1
1
52
humbleThC said:
I've been messing around for quite some time.... Trying to figure out the correct set of firmware & drivers to use on these ConnectX-2 dual port QDR cards (MT26428) on ESXi 6.0, with the best results.

Currently running 2.10.720 firmware on all adapters. (tried 2.9.1200 and 2.9.1000)
Currently testing 1.8.2.5 OFED drivers for ESXi (tried 1.8.2.4 alot, just moved on to 1.8.2.5)

ESX List of Drivers Installed
net-ib-cm 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-ib-core 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-ib-ipoib 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-ib-mad 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-ib-sa 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-ib-umad 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-memtrack 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-mlx4-core 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
net-mlx4-ib 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05
scsi-ib-srp 1.8.2.5-1OEM.600.0.0.2494585 MEL PartnerSupported 2016-11-05

I've got the links up, and stable (via 4036E switch with Subnet Mgr)

My Windows 2012 R2 Server is working perfectly (NAS)
My Windows 10 Desktop is working perfectly, and achieving amazing performance from the NAS
- (Yeah I threw a QDR IB in my desktop, for faster NAS access)

My ESXi 6.0 U2 server however, is having massive performance issues.
I originally tried a NFS mount from Windows 2012 R2 to the ESX Server.
- I get about 3MBytes/s or 24Mbit bandwidth
- vMotions, Deploying OVFs, etc all time-out

The storage is basically unusable, thus my entire ESXi lab is unusable. Any advice would be greatly appreciated.
Click to expand...
Do you have a copy of the 1.8.2.5 driver? For the life of me, I can't find it anywhere on Mellanox's website.

Thanks
 
You must log in or register to reply here.
Top Bottom