If I'm reading correctly, you have 3 main applications
- Active Directory
- File Sharing
- SQL Server
And you want to know what to buy. 1 server or 2. You also want to know if there's an open source version.
I think it will all run on one box. Another option is to split it up, especially if you want higher uptime. Ideally you would have a NAS box that can also run a directory server and two hypervisor nodes (or Windows Server 2016 Standard nodes with Hyper-V.
The first cost is going to be licensing. With 30 users you've moved out of Windows Server 2016 Essentials land and so I can see why you might want to go open source. If you NEED active directory and SQL server then you are stuck with Microsoft. If you can use MySQL or a derivative then that will save you a ton of money.
For File Sharing, how much data do you have? Maybe something like a FreeNAS mini would work?
https://www.amazon.com/FreeNAS-Mini-Network-Attached-Diskless/dp/B00EQJ1BTU/ or a QNAP
https://www.amazon.com/QNAP-Professional-Grade-Attached-Supports-TS-453A-4G-US/dp/B017YB7T6U/
For the AD the other option is Open LDAP but that's only if you don't need AD.
Finally, what about using Azure + local storage? File servers locally is easy with lots of options and is good since you avoid WAN transfers. Azure for AD and SQL may be cheaper in terms of licensing and gets you a benefit of letting Microsoft deal with much of the configuration and redundancy.