Zombie thread resurrection.. as I don't particularly want to start a new one.
I upgraded this past weekend from 150/150 to gig (FIOS). Router throughput was absolutely abysmal. 300 to 400Mbit between my gateway and my AWS server which VZ peers with directly, so random latency congestion through multiple networks is not the issue. Over the VPN link, it is even worse, around 80 to 100Mbit.
Current setup is a Supermicro C2558 with 16GB using the built in nics. I know I should be seeing much better rates.. watching the cpu, it does peg when running iperf over the VPN link, and that is with cryptodev.
I have an unused older dual L5640 with 32GB (DL180G6) which even with it's age should be overkill, which I might try.
As far as the VPN, at least while at work today (I cannot change the settings for my pfsense gateway remotely because.. work.. stupid filters (I could have done ssh forwarding, but I figured I would just do command line testing for now). Spun up a new fbsd 12 AWS instance (1 CPU, 1GB, 40GB disk), ran openvpn from the command line, and then did the same on my PFSEnse gateway from the console. playing with MTUs I managed to get it up to 200Mbit across teh VPN with basic settings, cryptodev and aes268cbc cipher.. but that is still pretty damn slow....
I currently am playing with it on my VM server (16 gig 8 cores assigned to the VM.. the machine is a DL380P G8 with 128GB and 2 8 core E5.. I know, small by most peoples count.. at least here). I noticed an improvement on the non VPN speed test using speedtest-cli, getting around 750mbit down 500ish up.... PFsense, Sophos, generic linux and freebsd, but still not seeing full or even reasonably close to gig speeds.
Looking for any insights anyone may have.. I really think the C2558 should be more than enough for just