so anyone have feedback if my crazy dream of using this as a 10G switch is good crazy or bad crazy?
(I don't have any 10G networking equipment in my house right now, except for cat 6 and 6a cables).
You could make it functional in that role fairly easily. It would work.
Performance would be limited by the fact that it would require all packets to be processed by the CPU. In a "real" switch the layer-2 functions happen in a matter of uSeconds and the first bits leave the switch almost immediately after the CRC is checked, so a packet comes in and leaves very quickly and very little inter-packet delay is introduced.
With this simulating a switch the whole packet needs to be received before it is processed by the CPU, the CPU needs to decode it and then send it back out on the other "ports". This creates single packet latency which is AT LEAST the time it takes to read a whole packet and likely adds 10's of ms additional delay. Note that this is not really as bad as it sounds because the NICs in this box do some offloading and some parallel processing gets done - e.g., the NIC is reading and preparing the next packet while the CPU is processing the last one - but the inter-packet delays introduced will dramatically limit throughput.
You will ultimately be limited in total throughput by the CPUs capacity.
In a home environment with very little of your traffic actually requiring 10Gbe performance and almost certainly never more than 2 hosts talking to each other at that rate you may be OK. It will never be really satisfying for benchmarks but it will work decently for "real" home/lab network workloads. It could really work rather well if you also use this device as your VM host for most services or stack disks onto it as a NAS because that will make most of your traffic most of the time actually terminating to this device rather than using it as a "switch".
You'd also learn a LOT doing this. About how switches work, about how to simulate them on a server, the upsides and downsides of doing this, about high speed networking performance, etc. Depending on how much you experiement you could lean a lot about different options for how high speed packet processing is done and the toolsets to implement it (e.g., VPP vs DPDK vs BPF, and combinations of these). Some of those lessons might be time consuming and perhaps even painful...
If it were me and I needed a low-cost entry into 10Gbe networking I'd just buy one of these:
MikroTik