OK...I've finally isolated the issue. The slowdown only occurs if the client in Site 1 is connected to the LAN via 10Gb. Both my Unraiid01 server and my everyday PC in Site 1 have 10Gb (server is fiber, PC is Ethernet) connectivity and when I do transfers over their 10Gb connections, I only get roughly 1/3 line speed (200Mb). If I connect my PC to the network via 1Gb, I get the full VPN speed (600+Mb).
Now the obvious culprit would be jumbo frames but I have checked, double checked and tripled checked every NIC on both my PC, Unraid server, 10Gb switch, and pfSense and all are using the standard 1500MTU.
If I manually set the link speed of my PC to LAN connection from auto (10Gb) to 1Gb, I get full speed across the VPN.
Anyone have any other ideas as to why clients connected via 10Gb to the LAN would be seeing this kind of behavior?