Got a chance to set up the Plex server and experiment. Took a while to scrape all my media, but that was expected. Overall I like the server side. It set up pretty easy in Proxmox. The biggest annoyance was networking related. It opened a port in my pfSense with uPNP, then refused to use it. Not sure what that's about. I had to try a few times with a static port assignment to get it to work. It's possible it was working and it's a UI bug, so I can cut them some slack here.
My first victim was the XBox One. I discovered that it has an IR receiver, so I set up my Harmony remote to control it as well. Overall, it works pretty well. Only a few things are annoying about it as a client for Plex.
1) It's transcode happy. This could be a Plex thing, but media that my Pi machines can play fine without transcoding in Kodi get transcoded in Plex. Stupidly, subtitles are done this way. It should be overlaying them on the client. I don't mind transcoding loads on the server if there's a decent reason for it, but if a Pi can play it back, the XBone certainly should be capable.
2) No DTS. This is Microsoft's fault, according to Plex. Not that huge of a deal, except that they changed the transcoder a while back to go from DTS to AAC. WTF? I hacked the profile to disable AAC and get AC3 now, much better. Having to transcode audio is no big deal, so I could live with this, but Plex really should default to AC3. Ideally, it would bitstream the audio to my receiver, but apparently that's Microsoft's issue as well.
3) Some of the navigation is inconsistent. Page up/down don't behave the same in some screens. In the movie list, it works as you would expect. In the episode list, page down works right, page up goes up one entry...
I also tried Android. The 1 minute playback timer is a bit short for evaluating transcode performance etc, but it worked well enough I guess. I don't watch a ton of video on my phone, but it looked good over wifi and LTE. Browsing around the library was pretty snappy, overall I like it. I'm more interested in the audio side on Android, which works well other than the 1 minute timer. Would be nice if they let me have music for testing, but I don't mind overall. I told it to import iTunes playlists, but it didn't. Meh. Could be user error. I can live without them.
I'll post again when I get a Pi running with it. Perhaps it will behave a little better with the things I didn't like about the XBox One. I'm sure it will find all new ways to annoy me though.