what is the suggested amount of RAM to put in one of these machines
That really depends. A bare-metal firewall, with just the basics is running on the minimum recommended specs, depending on the firewall you want to use.
But with the prices for RAM are being very low right now, I would suggest at least 8GB, better 16GB. Just grab cheaper (but still compatible) RAM, you would probably not notice it.
Running e.g. virtualization or server services, voip telephony for a huge amount of phones, managing torrents directly on the router (not on clients behind it - not recommended), managing databases ON the router etc. needs more RAM. But these use cases are frowned upon sometimes, for different reasons, mainly because of security issues coming with that.
But if you just want to replace your ISP's crashing box, my recommendation would be to just grab an ARM powered, 4+GB mini Router, like the ones from NanoPi, and you are good to go with the openWRT (they call it friendlywrt) releases from them. Lower power-footprint than a X86 box, lower price-entry - as low as 60$ (with case - only 2x1Gbit) for an
R4S (good and secure kernel) or as low as 140 $ (with case) for an
R6S (still no good kernel there, but 2x 2.35Gbit and 1x1Gbit) - and you will be fine and happy (especially with the number of connections these mini-routers can handle with ease).
Most people buying a x86 box need the "punch" for virtualization or other demanding services that ARM based routers can struggle with. Reading your "use-case", that is not the case on your side. I would guess your router crashes because of the number of connections BT sometimes relates with.
If you are wondering about wifi support on any routers, stop wondering and grab a dedicated wifi AP and put it in your LAN with a switch or on the 3rd port of the R6S (something like
this) instead of using an USB solution with experimental driver support and not the most recent wifi standards and speeds. Just as a recommendation.
But all of that depends, if your router needs a modem to connect with the ISP.