So I think I will end up using the E3C246D2I my planned Silverstone CS01-HS build. This is similar to Patrick's converged server build from last year. For such a build IMHO 10 Gbps networking is a bit overkill, especially since one would need to probably place a SLOG and L2ARC in front of the Seagate 15mm 2.5" drives. Considering that the 15mm drives are SMR, the IOPS of the RAIDZ2 vDev probably won't be anything to write home about.
I had run CrystalDiskMark on the 5TB drives I've got, and while decent, the performance definitely drops off during sustained transfers. I probably won't need a L2ARC since I'll be maxing out the system with 32GB RAM. Perhaps placing an Optane 905P for SLOG would help with sync writes so it at least can support better utilization of the NICs... Therein lies a problem; the M.2 slot is only a 2242.
I've seen a couple M.2 slot extensions floating around (
an example). Does anyone have experience with these? Apparently one cuts the side that mounts on your motherboard to fit your particular needs. The description from all the sellers seem quite poor, with no mention of supported maximum length of the M.2 module. It does look like it's only 22x80mm though
Another option is to use a carrier card that uses the split 8/8 lanes from the PCIe x16 slot, however I'm not sure if ASRock has enabled bifurcation on this series, and documentation is lacking. IIRC, bifurcation was enabled on their C232/C236 series boards. Perhaps I'll reach out to ASRock again about this question, but I've leave that for another day.