NFSv4 weirdness in ESXi 6.5 and FreeNAS

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
Folks:

Have anyone tried to mount an NFSv4 hosted on FreeNAS 11 on an ESXi 6.5U2 hypervisor instance?
I keep getting my NFS4 mount as read-only. Not sure why that's happening or if there's something that can be done about it.
 

svtkobra7

Active Member
Jan 2, 2017
362
87
28
Folks:

Have anyone tried to mount an NFSv4 hosted on FreeNAS 11 on an ESXi 6.5U2 hypervisor instance?
I keep getting my NFS4 mount as read-only. Not sure why that's happening or if there's something that can be done about it.
Yes and you are unfortunately seeing the "expected result" ... I tried to make it work a few months back but my understanding from other's experiences is that ESXi is running a more recent version (4.1) and Freenas is at v4. For some reason they are only compatible to the extent that a v4 NFS share can be mounted as ro in ESXi.

(if implemented in FreeNAS it would have been in 11.2U1, but nearly certain the issue remains)
 

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
They "supposedly" fixed it on 11.2U2 by merging an upstream patch from FreeBSD, but upgrading the release train to FreeNAS 11.2U2 has been seriously buggy for me - reboots takes almost 30 minutes (middlewared failed to start?), the new UI feels slow/unresponsive, and removing a boot environment triggered an unprompted reboot, which is surprising (did it crash?!). I am also not sure if the patch-up would work (the merge comment notes that the functionality is broken on ESXi 6.5). I also recall someone mentioning that USB 2.0 boot drives are no longer sufficient, so I might have to upgrade the chassis from an N40L to an HPMSG8, or at least buy a PCIe x1 based USB3 adapter that can allow booting.

I don't want to use that release chain if that's the case.

Man, what is up with FreeNAS?
 
Last edited:

svtkobra7

Active Member
Jan 2, 2017
362
87
28
They "supposedly" fixed it on 11.2U2 by merging an upstream patch from FreeBSD, but upgrading the release train to FreeNAS 11.2U2 has been seriously buggy for me - reboots takes almost 30 minutes (middlewared failed to start?), the new UI feels slow/unresponsive, and removing a boot environment triggered an unprompted reboot, which is surprising (did it crash?!). I am also not sure if the patch-up would work (the merge comment notes that the functionality is broken on 6.5). I also recall someone mentioning that USB 2.0 boot drives are no longer sufficient, so I might have to upgrade the chassis from an N40L to an HPMSG8.

I don't want to use that release chain if that's the case.
Ouch sorry to hear about all of your problems - and I've had some as well with 11.2, but nothing so severe. It seems like every update recently has caused some new problem, requiring a clean install.

Man, what is up with FreeNAS?
I don't know, but it certain feels like something is. It feels like the update cadence increased significantly with 11.2 so I looked the updates and their release dates to see if anything could be discerned:
  • 11.2-RELEASE - 11.2-U.2.1 = Average days between updates = 21 days.
    • 11.2 RELEASE - 11.2-U4 = If the 11.2-U3 & 11.2-U4 targets are met, average days between updates = 19.5 days
  • 11.1-RELEASE - 11.1-u7 = Average days between updates = 45.2
    • 11.1-RELEASE - 11.1-u8 = If the 11.1-U8 target is met, average days between updates - 49.5.
I don't know that there is really a story to be told here, but two opinions: (1) It seems like the current update cadence for 11.2 is quite brisk for this type of software, and (2) I'd much prefer having to deal with updates less often as with 11.1.

I suppose that 11.2 did bring a number of new feature updates so more frequent updates are to be expected?

But the last update, U2.1, was a hotfix to address 2 regressions. Considering that fact and with 11.3-BETA1 due in 16 days, I wonder if there is contention for resources and which is negatively affecting 11.2 maintenance?


[likely trying to connect dots where there is no causality though]
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Don't you jinx it for me, I want 11.3 ! (to work great obviously, but it has features I really really look forward to)

But I had issues with 11.2 too and was glad when my esx decided to recover my lost OS drive [long weird story] which included my 11.1U7 installation - have not updated since;)
 

WANg

Well-Known Member
Jun 10, 2018
1,302
967
113
46
New York, NY
Ouch sorry to hear about all of your problems - and I've had some as well with 11.2, but nothing so severe. It seems like every update recently has caused some new problem, requiring a clean install.


I don't know, but it certain feels like something is. It feels like the update cadence increased significantly with 11.2 so I looked the updates and their release dates to see if anything could be discerned:
  • 11.2-RELEASE - 11.2-U.2.1 = Average days between updates = 21 days.
    • 11.2 RELEASE - 11.2-U4 = If the 11.2-U3 & 11.2-U4 targets are met, average days between updates = 19.5 days
  • 11.1-RELEASE - 11.1-u7 = Average days between updates = 45.2
    • 11.1-RELEASE - 11.1-u8 = If the 11.1-U8 target is met, average days between updates - 49.5.
I don't know that there is really a story to be told here, but two opinions: (1) It seems like the current update cadence for 11.2 is quite brisk for this type of software, and (2) I'd much prefer having to deal with updates less often as with 11.1.

I suppose that 11.2 did bring a number of new feature updates so more frequent updates are to be expected?

But the last update, U2.1, was a hotfix to address 2 regressions. Considering that fact and with 11.3-BETA1 due in 16 days, I wonder if there is contention for resources and which is negatively affecting 11.2 maintenance?


[likely trying to connect dots where there is no causality though]
Well, I booted it back down to 11.1U7 and backed off the upgrade chain - my guess is that since 11.2 features a new kernel (one that can do kernel modesetting as the display on the boot looks different from the old text mode), new UI and new zpool features, all the newness created a crapload of regressions. The most serious one is probably the need to switch from USB2 to USB3 for the boot drive (if you don't then you're gonna see major issues with middlewared seemingly frozen).

I am seriously thinking of mirroring the drive to my SATA SSD and running it off the optical bay to avoid the USB2/3 insanity. I am not sure if I can boot off a USB3 port hung off a PCIe USB3 expansion card. Considering that the entire upgrade chain was triggered by a desire to migrate away from 40GbE iSCSI and onto an NFSv4 based datastore for ESXi (and the patch might only work in ESXi 6.7, triggering yet another upgrade on the hypervisor side), I am really loathe to do that. If it wasn't for the fact that Mellanox ConnectX3 VPI at 40GbE seem to run cooler than the 10GbE Solarflare I was using, I would not have ventured this far down the rabbit hole here.