Limiting Traffic Branch to Branch OpenVPN
Hi everyone, I have been using pFsense at this site for years, but the Main Router at head office was an old Dlink DFL-860e. Because it was slow, we had no problems. Recently I replaced it with an Intel-i7 with pFsense to match all of the other gear.
Now I have a problem, the new router "Main Firewall" is quick enough to saturate the ISP's connection (200mbps Telstra Fibre Optic). All branches are connected via OpenVPN, and this is not an issue with the other branches, as they are slower 100mbps links, but moving data to the CoLo' through OpenVPN now causes data loss as the Telstra 200mbps link drops data if we exceed the purchased speed.
I have tried applying limiters to the DMZ interface on the main router, I have also tried applying limiters to the LAN and/or OpenVPN interface on the 2nd router that does the branch to branch tunnels. But I am still getting data loss. I do not know how to measure the loss, but the MS Exchange servers are at the CoLo and the staff at the head office get connection failures on their Outlook clients. If I swap back in the old slow router (replacing the main firewall) then everybody is happy ..... slow but Outlook works again.
I have created two limiters for inbound and outbound traffic, and set the limit to 190mbps, but also tried dropping them down to 150mbps.
Any advice on how best to limit the WAN traffic on the 200mbps link would be greatly appreciated, also if anyone can advise how best to test/measure rather than just seeing if Outlook works would also be very helpful.
I guess the primary key to what I am asking is: should the firewall rule for the limiter be on the LAN interface, or the OpenVPN interface?
As a general rule of thumb, you should rate limit to just shy of your line rate so you can control bloat and loss.
I assume your limiter should be on whichever interface is physically connected and in the same broadcast domain as your WAN link.
I read somewhere that the Limiter does not work well on the WAN port if it is using NAT (I may be wrong on that one) so other forum entries were suggesting to use the LAN side with the limiter. This is where the waters get muddy for me. As I assume the outbound packets first pass through the LAN port then the OpenVPN port ?? So I am unsure which one to apply the Limiter rule to.
I'm not familiar with limiters, I'm waiting for 2.4.4 to drop with fq_Codel in the UI before I start. Is it possible the issue with NAT+Limiters is that most people using limiters want to bucket by LAN device and you can't do that? If that's the case, then who cares. If it doesn't actually work with NAT, it's going to be mostly useless for anyone other than transparent firewalling.
What I have found out so far (by teseting every combination)
- Limiters dont seem to work on the WAN port
- A limiter on the LAN port can throttle inbound to LAN traffic aka Internet Upload speed.
- A limiter on the OpenVPN port can throttle inbound to the OpenVPN port aka outside traffic coming in to the LAN
But it does not work consistently for me....
I am still struggling to throttle outbound traffic especially UDP OpenVPN
I cannot help past theory as I have no experience. Hopefully someone else jumps in. Have a great weekend :-)
Thank you everyone, I now think the LImiter function is not working because the default gateway is the primary router and there are static routes to the second for inter branch traffic.
Some how, that process allows the traffic to bypass the Limiter rule on the LAN port on the 2nd router. I will change the default gateway to the 2nd router and see if the Limiter kicks in correctly. Very Strange behavior!!