I have been dealing with a similar problem.. and ip fast forwarding did not help much.
Essentially, I have BGP setup, 6 pfSense boxes connected in a full mesh with some backend MPLS as the primary connectivity, but OpenVPN tunnels from everything to everything as a backup. 6 OpenVPN tunnels all TCP (BGP doesn't seem to want to play nice with UDP tunnels) and one of them, but only one, exhibits this problem. It is in fact a general problem with that pfSense box, as all OpenVPN tunnels out/in are slow, even though it is very new hardware. I have pored over it extensively, and I can't see anything that would alter its network behavior.
I am on 2.0.1-release, and I am really not sure where to go from where I am now…
I have done everything except look at frame sizes and packet traces. If anyone can give me a pointer in the right direction it would be sincerely appreciated!
EDIT- This has included taking all encryption off the tunnel, so it is definitely not to do with encryption load. I am getting 140ms transit times and low bandwidth (4 +/- .5 MBps) when I am getting 94ms WAN to WAN