Poor throughput to remote site
I'm having a problem getting decent performance to one of our remote sites. Initially, I thought it was an OpenVPN issue, but I'm having the problem with SSH, too. I've attached a (very) simplified diagram of our network - main office in on the left, remote office on the right.
When I ssh directly computer C to computer A, the transfer is slow. However, transferring from A to B is fast, and transferring from B to C is fast. The real head-scratcher, though, is that if I do:
ssh B 'ssh A "cat file"' > file
from computer C, the transfer is fast. We have no traffic shaping set up.
I've been trying to solve this problem for over a year, and I'm at a complete loss as to how to explain this behavior. Originally, I thought it might be an OpenVPN issue, since that's what we're using to connect the two sites, but I guess that's not the case, since I'm routing connections to A and B around the VPN when I do this test. Other things we've tried are replacing OpenVPN with IPsec (no difference), whitelisting the remote site in Snort (no difference) and, before I discovered the problem with SSH, a whole pile of changes to OpenVPN (MTU size, mssfix, etc). As this point, I'm not even sure it's a pfSense problem, but since it's one of the boxes the traffic is passing through, I feel I should at least check with you guys.
Anyone have any ideas?
Ok, so I assume A to B is local traffic, not via VPN?
And B to C is also not via the VPN?
What speeds to you see from C to A compared with B to C? Is it the same A to C or C to B.
I would try testing directly between the pfSense firewalls using iperf3 on each both inside and outside the VPN to see if you can pin down the throttle point.
pkg install iperf3 rehash