VPN up Gateway up - No Internet
-
I have been running my pfSense router for over two years with my WAN as the primary and selective routing of specific devices (via IP address) out the VPN connection that I have established. I only do this for a couple devices and use it maybe once a month. I don't know when it started happening, that it doesn't work, but between November and this week it no longer works.
Any devices that are added so that they route out the VPN will no longer get internet access. The logs don't show any rejects for packets. The gateway is up, the outbound NAT appears to still be there an without issue. Any ideas what may be happening?
The only change that I believe I have made in this time is to add a traffic shaper to help with the issues I was having with my WiFi Calling.
-
@vmac good evening!
Are you tagging the VPN traffic and using a floating rule to kill access to WAN?
I ask because I do just that and I accidentally tagged traffic that I shouldn't have and prevented them from accessing several of my gateways with my floating rule.
If this doesn't apply, I'm sorry I couldn't have been more help; I'm rather new to all of this. Check all of your Firewall rules and make sure all of the appropriate Outband NAT is enabled.
v/r,
JG -
@apsis-im
Yes VPN traffic is being tagged NO_WAN_EGRESS and there is a floating rule that stops it from going out the WAN, but that is the way it has always been setup. It should exist the VPN Gateway instead of the WAN gateway. So there are no changes there.Here is the Outbound NAT, again this hasn't changed.
-
@vmac right on... I thought it was worth a mention... I literally did it to myself an hour before I came to read the boards. I deleted one of the LAN to TUN firewall rules that I actually needed and mistakenly built it from one of the existing rules that had the tagging and ended up blocking unintended traffic.
NAT's there the LAN>WAN / LAN>VPN rules are as they should be; it "should" work.
Let's try to establish a known good. When these devices you're adding now aren't added to your LAN>VPN Rule are those devices able to get out? I'm assuming they are.
If this is the case I'd say it's worth taking a look at your LAN>VPN Rule (or all that's applicable o those IPs) They may have an unintended setting OR those IPs match another rule that applies before the pass rule you're expecting the IPs to use.
v/r,
JG
Edit:
You did mention the known good; sorry. just stepping through it in my head... again, I'm not that savvy. -
Sticky Connections: Do you have that on? It was suggested in a guide for multi-wan/VPN setups. It didn't function as I expected; it prevents devices from using the new available states as they became available (until the states expire they're stuck using the old/down/blocked stuff). I would have to reset states or reboot the firewall to get the traffic to flow after I made changes.
it's:
system > advanced > misc > load balancing
v/r,
JG
-
@apsis-im said in VPN up Gateway up - No Internet:
@vmac right on... I thought it was worth a mention... I literally did it to myself an hour before I came to read the boards. I deleted one of the LAN to TUN firewall rules that I actually needed and mistakenly built it from one of the existing rules that had the tagging and ended up blocking unintended traffic.
NAT's there the LAN>WAN / LAN>VPN rules are as they should be; it "should" work.
Let's try to establish a known good. When these devices you're adding now aren't added to your LAN>VPN Rule are those devices able to get out? I'm assuming they are.
If this is the case I'd say it's worth taking a look at your LAN>VPN Rule (or all that's applicable o those IPs) They may have an unintended setting OR those IPs match another rule that applies before the pass rule you're expecting the IPs to use.
v/r,
JG
Edit:
You did mention the known good; sorry. just stepping through it in my head... again, I'm not that savvy.So they way I have this setup is that I created an Alias "VPN_Devices". When I want to add a device to go out of the VPN I add it's IP address to the Alias. When I do that it will reload the filter. After that is done I can then verify it is going out the VPN by doing a quick bounce to what is my ip. The devices work perfectly fine prior to me adding them to this alias. Once I add them to the alias the Internet immediately drops. Don't know if that answered your question though. It looks like this:
Also to you other post, no I don't have Load Balancing checked.
-
@vmac my configuration is similar (naming aside), just with a couple more tunnels and aliases to bypass/utilize the different tunnels ...
There are fewer rules than I imagined. So, with only one rule to contend with and no sticky connections we can move on...
Does your DNS lookup/Resolver/Forwarder continue to function for the VPN traffic when those devices are added to your alias? That would hamstring the test scenario you described.
does whatsmyIP simply not load? I'm going to try and recreate what you're seeing...
edit:
I'm at a loss, lol! Are we certain that the vpn_devices firewall rule is set to pass and pointed at a gateway that exists/is up? I deleted a gateway to rename it earlier today and the LAN>VPN rules that used it were set to "default"; that hosed me for a while.
-
It should. I have the DNS resolver working on all interfaces.
yes it doesn't load when I try it. Nor can I ping any IP (ex. 8.8.8.8)
I don't know what firewall rule other than the above would be necessary. The gateway that should pass it is showing up and green. Nevermind the 3rd gateway which shows down but works fine smh...
-
@vmac said in VPN up Gateway up - No Internet:
I don't know what firewall rule other than the above would be necessary. The gateway that should pass it is showing up and green. Nevermind the 3rd gateway which shows down but works fine smh...
None should be... I'm just biased by my configuration and I wanted to make it clear that that's why I had even expected more.
This is weird... your rule has states and you're not getting rejects in the logs.
I was unsuccessful in breaking my configuration in a way applicable to your situation. Someone's having a similar problem on another new post. They're not using NordVPN (wireguard), but they suspect the VPN isn't up as all the statuses indicate. They tore their configuration down to the barebones to test and still can't pass traffic.
this discussion is here if you want to follow:
https://forum.netgate.com/topic/160378/wg-not-routing-or-sending-traffic
What does the traffic shaper do? Would it play a role?
edit:
Wait; there are no states... Hmmm... gonna google stuff, lol
and his issue is not similar in anyway... wireguard isn't an openvpn service...
-
@vmac said in VPN up Gateway up - No Internet:
This does show the gateway up...
Have we verified that this is the gateway VPN _Devices rule is set to?
edit:
Duh you have... sorry.
-
Do you have alternate servers you can try? maybe there's an issue with the one your client is connected to?
-
@Apsis-IM
Yes tried multiple. -
@vmac is your traffic shaper configured? Perhaps considerations for your tunnel need appropriate configurations.
v/r
JG -
@apsis-im
I removed the shaper on both the WAN and NORDVPN and it still is blocking.
I'm at a loss too, was hoping someone had some insight on what i might need to try. -
@vmac seems we both took L's last night... I failed miserably at my little project for 10 hours straight. There's a lot of action on these boards... Someone may come through and grace this thread with some knowledge.
-
@vmac may try seeing how this plays out?
https://forum.netgate.com/topic/160257/lan-connection-drops-when-openvpn-client-connected/4
There are several apparently informed individuals helping this person out. The problem seems similar to yours.
-
@johnpoz @viragomann
Can either of you help here? -
@vmac
I don't know, how you've configured the DNS on the affected machines, but since your LAN rules don't allow DNS requests to internal servers, they can only access external ones, but must be configured to do so or you do some forwarding to an external server.
So check if you can resolve hostnames on a computer which belongs to the VPN_devices alias. -
@viragomann I'm confused what you are stating here.
Here is a copy of my current LAN rules. Doesn't the last rule allow access from any device on my LAN vLAN to any device?
Are you stating that I need to add another rule to allow DNS traffic?
-
@vmac
Yes, it does. But this rule will never be applied, since that one above matches to any traffic and directs it to the vpn gateway.Rules are processed from the top downwards. If one matches it is applied and other rules are ignored.
Still don’t know, how you do DNS resolution. But assumed it“s done by pfSense, add a pass rule to the top of the rule set for TCP/UDP, dest. „This Firewall“, port DNS. So this ruhe only is applied for DNS access to pfSense, all other traffic is still directed to the vpn.
-
@viragomann
I'm confused as to what you mean about the order. since VPN devices would only match specific devices. However, I made the change and still can't get any Internet when VPN connected:
To answer your other question, I have my pfSense resolving all DNS queries:
-
I'm starting to wonder if it has something to do with the Automatic Outbound NAT not working.
When I check the OpenVPN logs I see this noted:
When I go to Outbound NAT I don't see this interface in the "automatic" generated outbound.
Could this be the issue?
-
@vmac said in VPN up Gateway up - No Internet:
I'm confused as to what you mean about the order. since VPN devices would only match specific devices. However, I made the change and still can't get any Internet when VPN connected:
To answer your other question, I have my pfSense resolving all DNS queries:
The question is if your VPN devices are configured to resolve host names. If they are set to use pfSense you need this rule, otherwise DNS requests are directed to the VPN provider, while the destination address is pfSense and resolution fails.
Since the rule shows some matches I assume the devices are set to use pfSense for DNS resolution.@vmac said in VPN up Gateway up - No Internet:
I'm starting to wonder if it has something to do with the Automatic Outbound NAT not working.
When I check the OpenVPN logs I see this noted:When I go to Outbound NAT I don't see this interface in the "automatic" generated outbound.
I cannot find what's really wrong there. The virtual interface port is ovpnc3. This one you should have assigned to NordVPN in interfaces > assignments.
The only weird thing in the outbound NAT is in the automatically generated rules: 192.168.3.1/26 and 192.168.3.0/26. No idea where the .1 is from.
Your VPN devices may be in 192.168.1.0/24 and 192.168.3.0/24? However, the latter may not really be defined on your system, since it isn't shown in automatic rules.
-
@viragomann
Well I'm at a loss.Yes the devices are set to use pfSense for DNS.
That is correct ovpnc3 is assigned to NORDVPN.
I'm assuming this is where the 192.168.3.1 comes from Interfaces->IoT:
Yes I have some devices that I want to use the VPN that are in my IoT vLAN, and I have ones that are regular LAN vLAN that I want to use the VPN. The device that I'm testing right now is on the LAN vLAN and still can't hit the VPN. I just tried the IoT vLAN and same issue.
The really weird thing is nothing has been changed except me adding a traffic shaper. Before my config has worked for literally 2-3 years.
-
I just updated to 2.5.0 RC and have the same problem.
NordVPN used to work perfect, now I don't get internet through it. The interface comes up with an IP but when I look at the traffic graph I only see outgoing traffic (no incoming).
It was a straight upgrade, I made no changes to the pfSense config.
-
@sensecanuck could that be the problem? I've been using 2.5 beta since June/July due to an issue with miniupnp and my ps4s. Maybe something that was changed in November/December changed something?
-
@vmac could be.
This is the 3rd time I've tried upgrading to 2.5.0 and have always had this VPN issue. I just assumed it was on the to-be-fixed list but apparently it's something else since we're now into RCs.All three of my 2.5.0 attempts have been December or later.
-
@vmac said in VPN up Gateway up - No Internet:
I'm assuming this is where the 192.168.3.1 comes from Interfaces->IoT:
What I wanted to point out is that the outbound NAT shows these entries among other: 192.168.3.1/26, 192.168.3.0/26. But normally it only shows network address, 192.168.3.1/26 is none, but it's part of 192.168.3.0/26.
Post your routing table to get closer.Do a packet capture on the NortVPN interface, while you try to access something from a concerned device, to see if the packets are natted well.
-
My setup is almost identical to @vMAC (select VPN based on a IP alias group). The only difference I have is I don't use the DNS resolver (to avoid DNS leaks) - I forward all VPN alias group DNS traffic to the VPN's DNS server.
But same problem - was working perfect until the move to 2.5.0.
Noticed though my gateway shows as offline. I switched the monitor to 1.1.1.1 but no difference.
-
@sensecanuck I thought it typical to block and forward all dns traffic to the pfsense dns resolver to avoid dns leaks...
-
I didn't want to use the VPN DNS for all LAN clients, just the few selected to use VPN. So I manually forward all port 53 traffic on those clients to the VPN provider's DNS server.
-
@sensecanuck gotcha!
-
@sensecanuck I'm also feeling your pain.
Whether I upgrade or rebuild from scratch its the same result. Tried on 2 boxes (one a vlan setup and the other multiple nics).
Also made several attempts at this. I can get the gateway up but I can't get traffic to flow through it. It seemed to me like the firewall wouldn't direct traffic into it - though I haven't looked into it very deeply.
Watching this thread closely to see if anyone can shed light. -
@dilligaf You seem to be a step above me. My gateway won't even show up. I've changed every rule I can find, changed the monitor address to 1.1.1.1, no luck. I'm so fed up with this I'm about to try out an OPNsense install.
-
@sensecanuck I've have a very similar setup by the sounds and like you use cloudflare dns. And like you changed everything I could think of.
All sorts of different issues: I've had it running but only at about 120mb/s (I'm expecting 400+). I thought at that point it was just hardware acceleration. Backed up that config but when I restored it no traffic.
And had the gateway up but no traffic like you. I was mucking about with it late last night and can't repeat anything.
I'm looking to overhaul my network as a suitable and more powerful NUC type box has come into my possession. There's an issue there with 2.5 I don't understand and want to use PFSense on it as everything generally just works, but don't really want to put the effort in on 2.4.5.
And like you I'm testing OpnSense. Not getting the same VPN performance though. Generally 15-20% less. Beware of the NordVPN guide - do not follow the guide for dns prefetch suppork and prefetch dns key. It doesn't work!
-
@viragomann
Here is the routes:
I did a packet capture, but how can I tell if it is being properly routed? If you want me to post or DM let me know, I can do that. It's trivial to get a new IP address generated from my ISP, so I'm not to worried about that.
-
@vmac said in VPN up Gateway up - No Internet:
I did a packet capture, but how can I tell if it is being properly routed?
Go to packet capture, select the NordVPN interface and set the protocol filter to ICMP and enter 8.8.8.8 at host and hit start. Than go to a devices out of the VPN group and do a ping to 8.8.8.8. Check if the ping is working. Then stop the capture and check the result.
If the policy routing and NAT are working well you should see ICMP requests from your virtual VPN IP to 8.8.8.8 and replies coming back.You can find your virtual IP in Status > OpenVPN:
-
@viragomann I don't see the ICMP replies in my capture (only sends).
- My config hasn't been touched since upgrading to 2.5.0.
- Under Status -> Gateways my VPN shows offline even with 1.1.1.1 as the monitor
- Under Status -> OpenVPN the status shows up and I get an IP
- I have some selective DNS (I don't use the DNS resolver for my VPN interface) but I'm assuming that would be irrelevant to the ICMP test?
NAT -> Outbound - If I disable the highlighted rule I get internet (bypasses my VPN). Which I find interesting because it means my NO_WAN_EGRESS tagging doesn't work.
Rules -> LAN - I always see 0/0 B on this one.
-
@sensecanuck said in VPN up Gateway up - No Internet:
I don't see the ICMP replies in my capture (only sends).
And what is the source address? I explained above how to check. Since you don't provide the infos, I can't verify.
Pinging an IP doesn't need DNS. You must see response packets. If not either the source IP isn't correct (from outbound NAT) or it is something wrong at the VPN provider.
There are other threads relating to outbound NAT on 2.5, but I did not go in.
Maybe it helps to switch the outbound NAT into another mode and back again or drop the rule and add it again. -
Recreating the NAT rule didn't make a difference.
The pings are coming from the virtual address but I don't get returns
11:34:33.915913 IP 10.8.3.5 > 8.8.8.8: ICMP echo request, id 28931, seq 601, length 40
I can also see a number of non-ICMP request being sent out (again, no returns).My gateway always shows down, whether I monitor 1.1.1.1 or 8.8.8.8 (don't know if that makes a difference).