Interesting to see this topic bumped a few years later and that I didn't reply with what I put (hacked) into place for my aging parents while they were still alive. (They have passed on but their systems outlived them.)
It first started out with something dead simple since their needs were dead simple and browsers at the time of this era still had full access to the Internet--3x Neoware xp embedded thin clients in stock form with enhanced write filter on. 2x were set up as workstations with a usb drive for portable programs and shortcuts set up on the desktop, 1x was simply a flash drive and then shared on the network for 'storage'.
The next iteration was using xp on some old P4-era pentiums I found in a utility chase. After loading them up with 2GB of ram and imaging off their stock software on their hard drives, I loaded a fresh install of xp and used the xp steadystate add-on to configure an experience similar to their Neoware thin-client 'workstations'. Because these pentiums were full computers, I was able to add gpus that accelerated pdf viewing and they had faster NICs (gigabit vs 100Mb), so they were quite a bit faster. I partitioned the hard drives so now the portable applications lived on the same physical drive, but on a different non-write protected partition that I could add and remove applications as needed. I still kept the UI pretty bare with only a few items in the start menu and desktop--just the basics they needed. After discovering my father's browser getting corrupted after a few weeks of use (he would click on anything), I used sfk set up in a batch file to always copy a 'clean' version of the browser the first time he was using it for the day--fixed that problem permanently.
On the 'server' side, I actually didn't do much--added a gigabit NIC in the single PCI slot in these devices, upgraded the usb flash drive to an external hard drive and added 2x more that were backups of the first that refreshed every 15 minutes via native xcopy in a batch file, and that's it.
While my parents have both passed, these systems and setup was their primary computers from about 2012 to 2019 and 2020 respectively for my mom and dad. They still work today even though some hardware has been replaced along the way (1 power supply, 1 cd-rom drive).