Invisible traffic shaping on VPN tunnels?
I've encountered a problem that I first brought up in the OpenVPN section, but I am repeating it here since I have discovered the problem is not specific to OpenVPN. I am experiencing the same behavior on both OpenVPN and IPSEC site-to-site VPNs.
One pfsense 2.0.1 router at main office with WAN,OPT1, and LAN interfaces. OPT1 is default gateway and connected to Internet with 10.5Mbps (7xT1) up/down connection.
Remote office with pfsense 2.0.2 RC router with same setup, except Internet connection is a 10Mbps up/down metro-Ethernet.
IPSEC VPN between the offices on the OPT1 interface of both routers.
Router hardware is AMD Athlon 64 with 2MB Ram on both routers with Intel dual gigabit Ethernet card.
Local net in main office is 192.168.0.0/24
Local net in remote office is 192.168.101.0/24
The Traffic Shaper is NOT enabled on either router
My problem is that traffic that traverses the IPSEC (also tested with OpenVPN) is being shaped on a per session basis. From an individual computer, bandwidth through the tunnel is being limited by pfSense to approximately 1/5 to 1/4 of the bandwidth available. The speed of my VPN between offices is a LOT more important than speed for any other Internet access. How can I remove or modify this invisible traffic shaper in pfSense? I have already tried setting the fastforward tunable to 1, which breaks the IPSEC tunnel, and had no effect on the OpenVPN tunnel.
I tested a large file-copy from a windows workstation on one network to a workstation on the remote network, and vice-versa. In both cases, the file copy speed was about 260 to 300 KBps on a 10.5 Mbps connection with no other traffic on the network (doing this at night). Further testing shows:
1. If I open multiple file copy processes on one workstation, they are all throttled such that the sum of the bandwidth of all the simultaneous copies is about 250-300 KBps.
2. If I open a copy process on separate workstations, each one gets a bandwidth of approximately 250-300 KBps, so the total in the tunnel increases from 250KBps to 500KBps to 750KBps and so on up to the 10.5Mbps capacity.
3. If I transfer a large file to/from a single machine from outside the networks (that is, the traffic does not traverse the VPN) then the file transfers at the full 10Mbps bandwidth.
So, there is definitely some form of traffic shaping per session is occurring when traversing the tunnel.
How can I change this? I really, really need my site-to-site VPN to have access to most of my bandwidth for a single machine. This appears to me to be a built-in limitation by design in pfSense. Is that true?
So called network "experts" keep harping on me to go buy Cisco ASA 5505 appliances to replace my pfSense routers. Will that fix this problem? Will I then have control over the bandwidth per session in my VPN?
Through further testing, I discovered that this issue only occurred when doing SMB file copies from a Win7 machine to a Samba server. The issue was caused by the settings of SO_SNDBUF and SO_RCVBUF in Samba. The recommended settings of 8192 cause a significant performance hit when transferring files over a VPN. Changing the settings to 65536 cured the problem completely.