Performance mystery with PIA on pfsense
-
Ok then I'll do some more specific reading about multi WAN and how to set things up. I'd be very interested to learn about the shortcomings of such things and how to get around them! Then I'll also be setting up a path around the VPN and straight out for Netflix so that that still works.
Appreciate any thoughts and assistance. I use pfSense because its the best, it does what I need, but also because it lets me try things and learn!
I'll also try and check the graphs but I'm fairly certain that everything I was doing was all going out the one pipe.
Thank you again all!
-
I'm curious as to the drawbacks of gateway grouped VPN clients? I haven't noticed anything.
-
OK so I've got two VPN links and both come up. Each of these is associated with an interface… They are bolted into a gateway group both as Tier 1. I have 3 manual NAT outbound rules one for each VPN interface and one for the actual WAN. I then have my pass rule in my WIFI_TRUSTED interface set to use that gateway group.
Right after things come up it was working for a bit... I ran a couple of speed tests but didn't see any proxy warnings. Then things stopped working (I had no internet access). One of the VPN links went down. But it seems like it should have continued working with the other one. However no.
So I then added the actual WAN as Tier 2 into the gateway group. Still nothing would get through.... I then disabled both of the VPN connections and things started working through the WAN. I then deleted the WAN out of the gateway group (and expected things to stop working). However, it continued. I'd expect that it wouldn't be able to get out if the rule that I'm heading out through is locked to go out through a gateway that doesn't include the WAN...
Strange.
-
Well I now believe that I have things much closer…
I now have 3 gateway groups. 1 with both as Tier 1. Then 2 more with one as tier 1 and the other as tier 2 and vice versa.
I then created 3 firewall rules one for each of these gateways.
I also added a separate DNS server for each of the VPN links and explicitly set them to use the gateway for each VPN. This is in addition to the 8.8.8.8 and 8.8.4.4 servers that are allowed on any gateway. I'm also using 8.8.8.8 and 8.8.4.4 for the "monitor" IPs for the two VPN links.
Things come up and work quite well. If I load a what is my IP page from a few different tabs I'll see each of the IPs that I'm linking to randomly...
However after a short bit one of the links goes down and then just continually retries and never then comes back up...
I get errors in the log like these during the trouble:
write UDPv4: Permission denied (code=13)And even at one point got an error that read:
write UDPv4: No buffer space available (code=55)This box has 4GB of RAM and right now only shows 13% usage. If I go in and change the IP address of the sever that is no longer connecting it works again... But only for a short while.
~Brett
-
What packages are you running?
-
Here's the packages listing from the dashboard of what I have running and version numbers.
 -
Try disabling pfBNG & Snort (also clear out the snort2c table) and see if it works for you.
It's possible that one of those is misconfigured, or configured in such a way that it conflicts with your VPN.
-
My guess is that a rule in snort is flagging your VPN traffic. So when you change the servers IP address, it works for a short period before it triggers your snort rule and is blocked again.
Again, make sure you clear your snort2c table after disabling snort. If you can identify snort as the problem then you can reenable it as an IDS only until you identify the rule(s) generating false positives on your VPN and remove them.
-
Ah yes great idea! I didn't think to look there being that a single VPN link never triggered it and I could go for days without issue but Snort can be a fickle one from time to time… I've got too much work going on tonight and the next day to play with it but I'll certainly have time over the rainy weekend to give this a go. Thanks for the ideas!
~Brett
-
Aaaand its working!!!
Got around the Snort issue by adding my VPN server IPs to the Snort Whitelist. I have found that using IPs makes life easier than using PIA's server URL's and even PIA's tech support recommends doing so.
I've got all my interfaces updated to use the 3 gateway groups that I've created (its my understanding that I need to have 3 gateway groups in order for failover to work properly). Anyway once that was all done and working I tested Netflix and Amazon Prime and they were failing because I was through a VPN. I then created some aliases for these two networks in pfBlocker and then created some forward around rules and that is now all working great. Then created the rules to allow my local networks to still see one another once I deleted the rules that allowed routing before hand. After that a bit of clean up to allow remote VPN connections into the house to go back out through the VPN except for Netflix and Amazon AWS. A bit of testing and all seems to do what I'd expect.
Now to wait a few days and see if it all remains but now that its been working for over an hour I'm thinking all will remain.
THANK YOU ALL SO MUCH!
-
I'm glad you got it all working, your configuration is very similar to mine!
-
Just thought I'd chime in and say I resolved a similar issue by disabling
1:2200073 SURICATA IPv4 invalid checksum
It was blocking PIA.