Site to site OpenVPN slow performance (2.7.2 CE)
-
Recently I have become aware of an issue with slow performance between our two sites over VPN. Both locations have 1Gbps fiber connections and speed tests are in the expected range (800Mbps-950Mbps) however when I run iperf tests between a client in the remote site and our file server the results are never over 50Mbit/s. We are using OpenVPN for a site to site connection and both firewalls are sufficiently powerful. I'm scratching my head as to what the issue could be. Any thoughts?
-
My first thought is MTU. Incorrect MTU value would have impact on transfers or on application responsiveness.
So make sure you have the proper MTU set for OVPN.https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#mss-clamping
MSS clamping is a pretty good way of getting around what most likely would be fragmentation along the link. Most applications set the DF bit which prevents the packet from being broken up when its to large hence slow transfers.
Additional questions i have is are you running iperf on servers at each site or on the firewalls themselves?
-
@michmoor would MTU really have that drastic impact on bandwidth? I think everything is at default values for MTU. I know there may be values for that on our modems and we want that to match what is set on firewall? We are using OpenVPN site to site tunnel so I don't think MSS clamping applies? Correct me if I'm wrong but that seems to be for IPsec tunnels. These iperf tests were run from a client in the brance office to our main corporate server. I just ran a test directly from one firewall to the other and it's better but still not great.
-
@snewby here is more verbose output from the firewall.
-
@snewby said in Site to site OpenVPN slow performance (2.7.2 CE):
We are using OpenVPN site to site tunnel so I don't think MSS clamping applies? Correct me if I'm wrong but that seems to be for IPsec tunnels
It applies to OVPN as well.
Do not run iperf on the firewalls.
You need to start digging into what the best MTU would be for a given situation.
ping x.x.x.x -f -l 1472 if you start getting messages that DF is set, reduce the mtu value.
ping x.x.x.x -f -l 1440 and so on...Once you figure out the MTU you can determine the optimal MSS value. Take the result of the above test which lets say is 1400 Bytes. Subtract the TCP and IP header value. 1400 - (IP header 20 bytes +tcp header 20 bytes) = 1360 bytes which is the optimal MSS.
-
@michmoor I just ran the test and it does look like there is an issue. Default MTU on pfense is 1500. It seems okay when I lower to 1472. Do I change that value on firewall WAN interface, WAN or both? Also if I change it does that normally cause the network connection to drop? And to be clear for MSS value I would just do 1472 - 40 = 1,432?
-
@snewby Turn on MSS Clamping and try a value of 1400 to start and restest again.
How to enable MSS clamping i provided the link above.
-
@michmoor which interface do I enable that on? I'm guessing WAN connection? Also I guess I need to lower MTU value there to 1472. Lastly do you know if changing that setting would cause a network drop?
-
@snewby
not there.https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#mss-clamping
System > Advanced > Firewall & NAT > VPN Packet Processing > Maximum MSS
A value of 1400 should be good enough.
-
@michmoor okay so I don't touch MTU settings on interfaces at all and just change this on both firewalls? In the future I may be implementing a wireguard VPN which this setting does not seem to apply to.
-
@michmoor I changed Maximum MSS on both firewalls to 1400 as you suggested. I then restarted the VPN tunnel. Now when I run pings the speed has improved (about doubled) but it's still nowhere close to the line speed of almost 1Gbps. Yes, I know VPN adds some overhead, but it shouldn't be this much right? I would think I'd at least get 500 Mbits/sec. As I comparison I ran a ping to a server on the LAN. The internet speed at both sites is 1Gbps fiber. I included a speedtest from the remote site where you can see it is in the 900+ Mbps. Where else might there be a bottleneck? Do I need to change MTU value on the actual interface? I don't think it's a hardware bottleneck as both routers have powerful processors.
-
@snewby it’s possibly you are at a hardware bottleneck. Can you give a bit more details about your hardware?
-
-
@snewby review the following from documentation
Short of changing MSS, Options to scale ovpn are quite limitedhttps://docs.netgate.com/pfsense/en/latest/vpn/performance.html#scaling-openvpn
