NFSv3 or NFSv4 for home NAS

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

ullbeking

Active Member
Jul 28, 2017
506
70
28
45
London
Hey all,

I have a home NAS that is essentially a Debian machine with OpenZFS running with 4x 8 TB WD Reds in RAIDZ2 as one pool. I plan to add another similar pool. I have a lot of multimedia content, especially music, which is very important. (That is, it's not just a bunch of box set torrents.)

I want to share these pools to my family and friends seamlessly. They are not techies and use Mac OS X and Windows primarily. This may sound like overkill for a home network, but the point is to make this a pleasant experience for all to use, rather than having my family and friends worrying about mismatches permissions, stale file handles, etc. In other words, this needs to be implemented well to be a compelling experience for them.

I would like to use NFS. I understand that NFSv4 with LDAP+Kerberos is the proper way to map users between the NAS file server and other machines. I have a couple of few questions:
  • Can you recommend good resources for implementing NFSv4 with OpenLDAP and Kerberos with OpenZFS on Linux? Everything I've found so far is very light on.
  • If, instead of using OpenZFS, I were to use mdadm+LVM+file_system, then would this affect the user-visible functionality of the system as far as mounting shares and ensuring permissions are correct, is concerned?
  • I think that auth services like OpenLDAP and Kerberos should not be virtualized, because if the virt server goes offline and other services depend on it, I could be stuck in a chicken-and-egg scenario. Therefore, would it be wise to implement the LDAP+KRB server/s on bare metal? Or can they be implemented in a VM on a properly architected system?
Thanks!!

@ullbeking
 

kapone

Well-Known Member
May 23, 2015
1,095
642
113
Any particular reason you're looking for this kind of permissions control for "family and friends"? Wouldn't "readonly" for everybody except you, be enough?
 

ullbeking

Active Member
Jul 28, 2017
506
70
28
45
London
@kapone I want them to be able to write to the server, and they want that too. In fact, that's a large part of the reason for having it. Many of my friends want to be able to write large amounts of large images, videos, and audio files (music and field recordings, multitrack recordings) directly to the NAS.
 

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
What you're talking about will work, but there are some things to consider.

1) Setup info is more difficult to find vs other setups. As you're finding.

2) Windows doesn't come with an NFS client, so now you have to talk to them about setting that up.

3) Permissions are a bit, iffy, on NFS without LDAP etc..

4) There is little performance advantage for NFS these days. It's a bit lighter, but on a modern LAN it's a non-issue.


IMO, the best way to do what you want (file shares with permissions for OSX and Windows clients) is SMB/Samba. It's more compatible, and fast enough, particularly if you don't have >1Gb. Even on my 10Gb links, the difference is well within what I consider margin for error. And Samba easily maps users and permissions to Linux users. Just make a new ZFS filesystem (zfs create) for each user, Samba share it for each user, so they can't write each other's files, and you're off. And as the admin you can set up per-user backups, snapshots, quotas, etc. using the tree.

Another ease of use option would require a client application, but something like Syncthing works much like Dropbox. A directory just gets synchronized with the server automatically. Point it at the /Users/ tree and you have a reasonably decent backup solution. And it just happens, the users don't have to remember to copy things around.
 

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Presumably when you say sharing with family and friends, you mean sharing over your WAN and not just on your home LAN...?

NFS isn't the right fit for doing this, especially if you want to complicate it further with user accounts and kerberos auth; even getting windows and OSX clients working with the auth will be far from simple, and even then I don't think either will work with NFSv4. You're better off sticking to CIFS.

And then, if you are talking about publishing this outside of your home LAN, if you want to do it in a fashion that won't have you exposing a filesystem to the internet you'll need to set up some form of VPN for your users to connect to first.

Personally, I used NFSv4 with krb auth in conjunction with my samba4 AD at home mostly as a learning experiment. It worked, but it was far from seamless.

I'd second kapone's recommendation of simplyfying things immensely by publishing this media read-only so that fine-tuning of permissions aren't needed so much - take it from me, WAF doesn't belong in the same sentence as permissions (the missus is still cheesed off connection to home still requires any sort of authentication at all even in the form of a never-expiring-password looked after by a password manager). People that aren't StH users just don't want to think about stuff like that in their leisure time. If you do want users to write stuff to it, set up a separate limited area (perhaps user-specific) that they can write to and then you can copy incoming files to the "main" media shares if you want, but if you want to stay on people's christmas card lists I'd steer clear from trying to implement granular permissions as much as possible.
 

cageek

Active Member
Jun 22, 2018
94
105
33
I wouldn't do NFS either - having done it for software development. Samba shares would typically be better. Have you thought about running your own personal cloud (e.g. NextCloud). It is a little difficult to set up, but your users would probably find it more usable.
 
  • Like
Reactions: ullbeking

poutnik

Member
Apr 3, 2013
119
14
18
I would also vouch for NextCloud/OwnCloud - if it's more than LAN sharing with family (SMB/Samba for local.). Users can decide themselves what to share with whom (file only, folder - with a specific user, a group, over an email link...). And it's especially well suited for non-techies - they just have to have a client on their machine, and copy/move whatever they want to be stored on the server to that folder.

Jiri
 
  • Like
Reactions: ullbeking

ullbeking

Active Member
Jul 28, 2017
506
70
28
45
London
Presumably when you say sharing with family and friends, you mean sharing over your WAN and not just on your home LAN...?
Over the WAN I mean sharing using HTTP, perhaps FTP, perhaps WebDAV, and other protocols that are designed for uploading and downloading over the internet. This is going to be mediated in something like a VLAN or DMZ, with the actual data that others are interested in proxied in from the internet network.

NFS isn't the right fit for doing this, especially if you want to complicate it further with user accounts and kerberos auth; even getting windows and OSX clients working with the auth will be far from simple, and even then I don't think either will work with NFSv4. You're better off sticking to CIFS.
I'll take this advice on board. I was actually not going to use Kerberos5, just LDAP. And then to secure the network using IPSec.

And then, if you are talking about publishing this outside of your home LAN, if you want to do it in a fashion that won't have you exposing a filesystem to the internet you'll need to set up some form of VPN for your users to connect to first.
The kind of use cases I have in mind don't necessarily involve a VPN... at least not what "VPN" makes me think of, i.e., interactive SSH logins, etc. I'm considering IPSec as an alternative.

Personally, I used NFSv4 with krb auth in conjunction with my samba4 AD at home mostly as a learning experiment. It worked, but it was far from seamless.
I've decided that if I were to use NFS, then it would be v3, not v4. CIFS/SMB is still an option.

But if the network is protected with IPSec (for example), and if the client IP address is granted unlimited read-write access to their exclusive NFSv3 share, how is this a problem, even potentially?

I'd second kapone's recommendation of simplyfying things immensely by publishing this media read-only so that fine-tuning of permissions aren't needed so much - take it from me, WAF doesn't belong in the same sentence as permissions (the missus is still cheesed off connection to home still requires any sort of authentication at all even in the form of a never-expiring-password looked after by a password manager). People that aren't StH users just don't want to think about stuff like that in their leisure time.
Please explain to me... if there's a never-expiring-password setup, how does this cheeze people off? What goes wrong to annoy regular users?

If you do want users to write stuff to it, set up a separate limited area (perhaps user-specific) that they can write to and then you can copy incoming files to the "main" media shares if you want, but if you want to stay on people's christmas card lists I'd steer clear from trying to implement granular permissions as much as possible.
Yes, the user-specific area is starting to become part of the design... I'm working on a post about this right now actually.

Thank you so much for all you help and opinions.
 

ullbeking

Active Member
Jul 28, 2017
506
70
28
45
London
Have you thought about running your own personal cloud (e.g. NextCloud). It is a little difficult to set up, but your users would probably find it more usable.
I have considered NextCloud but personally I all of these "out of the box" solutions don't really rub me the right way. I'm sure it's a fine product, but I prefer to set these things up myself.
 

ullbeking

Active Member
Jul 28, 2017
506
70
28
45
London
I would also vouch for NextCloud/OwnCloud [...] And it's especially well suited for non-techies - they just have to have a client on their machine, and copy/move whatever they want to be stored on the server to that folder.
As per my previous reply (to @cageek) these kinds of solutions aren't really the way I like to do things.

For example, I already have the whole system set up, but I need to do some fine tuning to make auth easier. One example is that I might get rid of KRB5 altogether, another is relaxing the need for users to even have to think about auth but making it non-necessary. In other words, the internal LAN would be less secure this way, but if you're already on my LAN then I already implicitly trust you.
 

Blinky 42

Active Member
Aug 6, 2015
615
232
43
48
PA, USA
Who in this model needs to actually write to the shares?

If you are maintaining a common pool of content for family & friends than it may be easier to make it read-only across the board, and then only you or the small subset of in-home people that can write to the set that is shared with everyone has any form of write access.

What are the clients using to actually consume the media on their devices? for non-techy people, sharing a multi TB audio tree of music with tens of thousands of files without and say have at it is a bit unfriendly from a non tech use case (from first hand experience) Mac people may want it all in iTunes for example and be able to rate/remove things they don't care about make playlists etc. Figure out what the users want/need and choose an implementation that lines up with those best. For example are you better off (== happier users) serving up that music library with itunes running in a vm and then sharing that library through itunes' internal thing? Need to serve up things over DLNA? Do you need a slew of various things, and it is easier to expose your content through a boxed distro that someone setup for oyu already and you rurn tha tin a VM with a mega NFS pool exposed to it beind the scenes and it exposes it through other protocols to your end users?

NFS + Windows = sad pain filled waste of energy not worth the time or effort. SMB/CIFS is the way to go. Also works on mac's and most mobile devices too. Easy to lock down to read-only public browsable shares so you don't have to fight setting things up, and (possibly non-browsable) and authentication required shares that have varying levels of access.

Save yourself the pain and keep permissions share level vs individual file. If you are using ZFS anyway just carve out some space for ullbeking's photos that only your family has read/write to but other have read for example, then force the user and group to one value for all files and directories under that share. I would *really* think about cases where you NEED to have files owned by different users within the same directory in this type of model, and if a project/task based share exposed to the members that need access to it isn't a better model overall.

If you do end up doing file-level security and access control it can quickly degrade into a time suck resolving problems with non tech users, and if it for fun at home, why sign up for that?
 
  • Like
Reactions: ullbeking

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
I'm no fan of them myself - they rub me up the wrong way too - but stuff like owncloud are specifically designed to do the sort of thing you're wanting to do, namely presenting a bunch of files to remote users in a somewhat usable fashion.

I do do the whole "IPsec-or-OpenVPN VPN into the DMZ, where selected services are presented" thing but this is only possible because the only users are me and my partner and I rule over our computer equipment with an iron hoof. It's not an approach I'd recommend unless you're happy to do all the tech support as well.

I'll also echo BLink saying that NFS in conjunction with anything windows-ey is, in my limited experience, a world of pain. It's hammering a square peg into a round hole made out of nitroglycerin.

Over the WAN I mean sharing using HTTP, perhaps FTP, perhaps WebDAV, and other protocols that are designed for uploading and downloading over the internet. This is going to be mediated in something like a VLAN or DMZ, with the actual data that others are interested in proxied in from the internet network.
That's basically my approach; VPN gets you in to the DMZ, from which point there's a) some read-only samba shares and b) a writable "drop" share. All auth on the file shares is done through a samba4 AD domain, auth to the DMZ is done on a certificate plus password basis. Whilst you can use samba to act as the KDC for your NFSv4 auth system, it's not exactly seamless - but I think it'd be much more streamlined than any LDAP-and-NFS stuff you might cook up.

I'll take this advice on board. I was actually not going to use Kerberos5, just LDAP. And then to secure the network using IPSec.
I imagine you're already aware, but LDAP auth means creds can be sent in the clear over the network, so you need to take steps that the user's never prompted for creds until the IPsec tunnel is established. Kerberos is significantly superior to LDAP auth for this reason, but you can always insist on LDAPS I suppose. But what are you going to be using on the client side that supports NFS with LDAP auth but not SMB and Kerberos?

The kind of use cases I have in mind don't necessarily involve a VPN... at least not what "VPN" makes me think of, i.e., interactive SSH logins, etc. I'm considering IPSec as an alternative.
I'm using VPN in this context to mean "an encrypted connection from 'possibly hostile networkX' to 'my friendly network at home'. IPsec is usually my go-to for this since it's well supported, but if you want things like J Random "free" wi-fi to work you'll probably want to set up a VPN-over-HTTPS alternative as well, since many foreign networks will stomp on attempts to VPN out. SSL/OpenVPN egress is generally not blocked anywhere near as much.

I've decided that if I were to use NFS, then it would be v3, not v4. CIFS/SMB is still an option.
NFSv3 really only works on "host" authentication, i.e. "does this host have the right IP address, if so then it's allowed". It's terribly insecure to run over anything that's not a wholly trusted network. I don't even like running it at home, but then I take paranoia pills for a living.

But if the network is protected with IPSec (for example), and if the client IP address is granted unlimited read-write access to their exclusive NFSv3 share, how is this a problem, even potentially?
Not hugely if you can guarantee that the NFSv3 shares will only be accessible to machines coming in over the VPN, i.e. expose your NFS exports only to the VPN network. But this means your VPN is the sole authentication system, and the client systems are configured to manage the VPN connection and NFS mounts seamlessly - that's not especially easy TTBOMK.

Please explain to me... if there's a never-expiring-password setup, how does this cheeze people off? What goes wrong to annoy regular users?
Because most people I've met hate any sort of password input at all. If you've got people who are happy enough when things don't automagically work, count yourself lucky :)

Like I say, I don't use it myself but for the sake of KISS I'd look into implementing something like OwnCloud behind a simple VPN and see how that works out for you first, otherwise you're looking into a lot of hard work to get things working.
 
  • Like
Reactions: ullbeking

ullbeking

Active Member
Jul 28, 2017
506
70
28
45
London
Thank you @EffrafaxOfWug , @Blinky 42 , @cageek , and @poutnik . Especially for how considered and reasoned they are, I appreciate you taking the time out to write this down.

I'm going to take this on board and seriously consider changing my auth system to this CIFS/SMB over NFS. However, there's another large factor that I need to consider, and I have several other threads in progress on STH, other forums, etc.

For example, in this thread I tried to describe my overall problem and a preliminary solution: https://forums.servethehome.com/ind...s-and-syncing-with-upstream-remote-nas.24573/ but I didn't do a very good job at job at it. Briefly, I'm trying to find a way to make my NAS quieter. I don't think it will affect the overall approach of user-level auth, but it IS an issue that I need to think about from a top-down solution-focused approach. I'm going to try to simplify my problem statement and then get back to this.

Things are coming very well so far, however.

Edit: I will update this thread when I have made some progress. Thanks all, again.
 
Last edited:

EffrafaxOfWug

Radioactive Member
Feb 12, 2015
1,394
511
113
Samba has the advantage that it'll play nice with linux and windows; even if you do want to go NFS with auth (thus requiring NFSv4), you'll already have a KDC up and running in the form of a DC. Depends what your itch is really and what you have set up currently.

Incidentally, all of my linux kit has been samba4 domain integrated for years now; even though I'm no longer using windows in anger, AD it still makes centralised auth nice and easy solely on linux (since almost everything supports it one way or another).