Slow traffic in one direction

[swimboy]

I have 4 locations with T1 connections to the internet, running pfSense 2.0.1 on Alix boxes. All four locations have IPsec VPNs between them. Everything has been running smoothly for over a year, until last week, when I started having problems between sites A and B. Whenever there would be any significant traffic from B to A, the bandwidth from A to B would plummet, and I'd start seeing connection errors on client workstations. I installed iperf on servers in each location to measure the extent of the problem. Between all pairs other than A and B, I could run bi-directional simultaneous tests with iperf and get 1.25 - 1.50 Mb/sec throughput every time. Between A and B, if I ran the tests sequentially, I could get similar numbers. However, if I ran the tests sequentially, I'd get the same 1.25 - 1.50 Mb/sec from B to A, but only 10 - 100 kb/sec from A to B. Restarting IPsec on the router at B seems to clear up the problem for a short period of time, but the problem always comes back. It might also be coincidence, because I think the problem has come and gone without restarting IPsec as well.

I forwarded port 5001 at each end, and tested iperf over a plain internet connection without going through the VPN, and the problem never manifested itself in that configuration. At first, I thought that indicated that it was a problem with the firewall itself; but now I'm starting to wonder if there's something wrong with the internet connection between the two locations that is causing the VPN to misbehave, but I have no idea what to look for. The two sites have different ISPs, so I need to have a good idea of what's wrong before I go down that path.

I also investigated whether I have my VPN misconfigured. I've deleted and reconfigured the IPsec configurations at both ends several times; compared the settings to the other locations that aren't having problems, and even restored the configuration from a backup that predated the start of this issue, to no avail.

Does anybody have an idea of where I should look next?

[SeventhSon]

MTU problems maybe?

You can try the "MSS clamping on VPN traffic" option on site A.

Otherwise, do packet capture on the WAN while testing the port forwarding way and a pcap on the IPSec interface when testing iperf through the tunnel. You should see what's happening