Single connection bandwidth is slow

rhi_np6 · Sep 15, 2016, 4:20 PM

Hello,

I have been trying to get good performance between our site and google cloud vpn. With iperf single threaded benchmarks I am seeing 15Mbit/second transfer speed from our site to GCE on a 800Mbit uplink. CPU use on the pfSense is very low.

If I benchmark using multiple threads (32) transfer speed is up to 400Mbit/second and the pfSense CPU is naturally quiet busy but the speed up from 1 thread to 32 is almost linear. pfSense is definitely capable of handling the ipsec encryption load!!

What I cannot understand is why we cannot get better speed on the single threaded benchmark. There is no traffic shaping in place that would limit the speed of individual tcp connections.

When routing normal, non ipsec, traffic to public servers (e.g. google drive) transfer on a single connection speed is ~150Mbit/second. Again, low cpu use. I am not looking to improve this speed, it is just for comparison.

Would anyone have any ideas on why we are experiencing such slow single connection transfers.

Thanks

Rob

rhi_np6 · Sep 20, 2016, 9:59 AM

Thanks for looking! We have found that this is not an issue with IPSec, but a general issue with our connection to google compute platform.

iperf benchmarking and wireshark analysis hints to a TCP window scaling problem. Not related to pfSense, not sure what it is related to honestly!!!