OpenVPN clients no longer accessible from LAN after upgrade to pfSense 2.7
-
My settings are as follow:
Openvpn clients 172.16.12.64/27
LAN 192.168.1.0/24Before 2.7, devices on LAN can access devices from Openvpn. After 2.7, it was broken.
Openvpn clients can be ping from pfSense but not from any devices on LAN.traceroute from LAN device shows nothing:
traceroute to 172.16.12.67 (172.16.12.67), 30 hops max, 46 byte packets 1 * * * 2 *^Cwhile gateway is accessible from LAN device:
--- 172.16.12.65 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.278/0.278/0.278/0.000 msWhat can be done to resolve the problem?
-
Most likely there is a configuration problem that has always been wrong but some change on the backend changed and now your previously "working" settings which happened to be incorrect in some way stopped working.
A few common things we have seen are:
- SSL/TLS setups where people had filled in a tunnel network on the client when they should not
- SSL/TLS setups with a /24 tunnel network where the Client-Specific Overrides were not setup correctly breaking LAN-to-LAN routing
- Static Key configurations using the wrong subnet size for the tunnel network (e.g. /24 when it should have been /30)
- Not explicitly setting the same topology on both sides
- Some other routing conflict preventing the correct entries from being in the tables
- A configuration that worked by chance before that was never correct (e.g. routes in System > Routing instead of in OpenVPN natively)
- Policy routing rules overriding the VPN and sending the client traffic in some unexpected path
- Missing or incorrectly configured default gateway (e.g. set to auto when it should be set to a WAN or WAN failover group)
Compare your setup against the reference here: https://docs.netgate.com/pfsense/en/latest/recipes/openvpn-s2s-tls.html
There are a lot of troubleshooting suggestions for that sort of stuff at https://docs.netgate.com/pfsense/en/latest/troubleshooting/connectivity.html
But to boil that down a bit, you should check:
- Look at the OS routing table on both sides, make sure there are entries for the default route and opposite side LAN(s) and that those routes are pointing to the correct OpenVPN interface(s).
- When you ping from the firewall make sure to ping from both the OpenVPN interface itself (default source) and again using the LAN interface as a source. That tests routing between the LANs in both directions, not just to/from the OpenVPN interface directly, which is a much different test.
- When pinging from a client on the LAN, look at its states under Diagnostics > States on both firewalls, there should be two entries on each, one as it enters the firewall and one as it exits the firewall. If something like outbound NAT is catching it, the NAT would show in these states. If the traffic is taking the wrong path, that would also show (e.g. it should go in LAN, out VPN, in VPN, out LAN).
- If the packets are exiting a WAN unexpectedly it may be from those clients hitting a policy routing firewall rule, so you might need to add a rule above whatever rule it's hitting to pass VPN traffic without a gateway set.
That should give you a better idea of what's going on and what needs fixed.
-
Hi Jmp,
I am starting to interact with this some problem on my machines. I am tantalizingly close. So I have 6 client openvpn pfSense servers connecting to my main pfSense server that has the port open to the Internet and the openvpn server. All 6 can talk to each other through the main server, the pfSense unit that has the openvpn server. However, any client computer on the pfSense box that has the openvpn server cannot talk to any of the other servers.
We do have two wans and rules for bandwidth sharing, and rollover, and a rule to send traffic to the gateway group. The queres might not be reaching the correct wan, which is NOT the default wan in any case, but another wan/modem. Ok, now I struggled with something similar on a mutliwan CARP setup. You say to install a rule to capture VPN traffic and send it to the default wan before it is captured by the rule that sends it to the gateway group, or in my case send it to the the designated openvpn modem.
In trying to write such a rule I am going to display my ignorance. I don't know how to capture the openvpn traffic. Should I use the IPs for the other sites themselves? Is there a choice that I am missing in writing this rule?
e.g. Server XYZ has 10.xx.xy.1 address
Should I trap for this address 10.xx.xy.1 and send it to the wan that is the designated openvpn WAN?
Thanks for your patience. !Duh! maybe, but I still want to ask.
Roy
-
@reberhar "Leave as 'default' to use the system routing table", say the comment in the rule wizard. So perhaps by the "policy routing". my openvpn designated wan, as I was suggesting above; was mistaken. I am assuming that I want to use the system routing table for this.
-
@reberhar Worked ... Thanks for your posting. It pushed me in the right direction.
-
@jimp You are quite right that my problem was caused by "wrong" settings in 2.6. It seems to me that routing to openvpn clients were done "implicitly" without firewall rules/gateways restriction in 2.6.
In 2.7, things are strict to firewall rules. Things work after fixing the firewall rules. -
@bchan Ditto ... There turned out to be serveral other things that needed fixing because of this change from 2.6. to 2.7. E.g. we have more than one lan on our servers, and to allow access from the main private lan to the other lans, I also needed to put rules in. Actually, working with the tunnels, there were any number of things that fell into this category.
For the tunnels, because there were so many broken ips, I put aliases in the firewalls of my servers, and leveraging this feature, I only had to put one rule in the firewall of each server to make sure all those broken addresses worked again.
Of course this was only necessary on the servers that use policy routing. The others fall through without problems.
-
@bchan If possible, please be more specific in terms of what needed to be done. This would greatly help the others in the same boat!
-
@michaelschefczyk So you have to go into firewall rules / lan and you need to insert a rule at least below the lockout rules. For me it was below my pfBlocker rules and above any policy rules.
Let's make up an address. Say you have a server on your network that has a number 10.7.0.1, and that you have a multiwan system with a rule that grabs your traffic and sends it out to the multiwans, so that you can do traffic balancing and rollover, which is a really cool feature of pfSense. At the bottom of the rule way way down there, you have something like...
Gateway "HereIsMyGatewayGroup"
Leave as 'default' to use the system routing table. Or choose a gateway to utilize policy based routing.
Gateway selection is not valid for "IPV4+IPV6" address family.But when you put the 10.7.0.1 address into your browser that address also gets sent out to your gateway group or the gateway you have chosen in this rule.
So your gateway group or other gateway is a policy based routing.
Ok again, really cool... but also read what it say... "Leave as 'default' to use the system routing table."
When you do a policy routing like this whatever matches the rule uses that policy routing ... and it NEVER gets to the system routing table which contains the information for your tunnels. So your request goes to the gateway group or other gateway or gateways your have chosen, and they say, "HUH?" and probably sends it out to the Internet where it is lost.
You WANT your tunnel requests to get to the system routing table, because that is the place the mapping happens for your VPNs.
Soooo, you have to put a rule above your policy routing server rules that matches your VPN server address 10.7.0.1 and NOT have a policy routing at the bottom. Doing that will bypass your policy wan routing rules and send your 10.7.0.1 request to the system routing tables.
And Shazam, it works!
If you have a lot of rules, you can create an alias in the firewalll menu and use that alias in place of 10.7.0.1. You might call it "MyVPNAddresses" so that you don't have to make separate rules for each address, which can really bloat your LAN table.
Remember that your policy routing, your gateway groups, whatever you choose in the Gateway line, will EAT or receive everything that comes its way, whether you want it to or not.
If you have more than one lan and try to access across lans, the same thing will happen.
-
This post is deleted! -
@michaelschefczyk The cause was exactly what @reberhar has just explained. To prevent LAN devices from crossing to other subnets, I had put WAN as gateway to the allow rules. In 2.6 access from LAN to vpn clients managed to get thru this rule. But in 2.7, I need a rule with default gateway to let the LAN to VPN traffic to get thru.
-
@reberhar see my case:
My Pfsense have two VLAN more one tunnel site-to-site OpenVPN to comunnication only with VLAN20.
This configuration is:
LAN firewall: 10.1.1.1/24
VLAN10: 192.168.15.0/24
VLAN20: 172.24.0.0/24LAN client firewall: 10.0.0.0/24
IPv4 Tunnel Network: 192.168.210.0/30
Communication LAN - VLAN10 is blocked.
Communication LAN - VLAN20 is blocked.
Communication VLAN10 - VLAN20 is blocked.But, LAN and VLAN10 is possible ping all LAN client: 10.0.0.0/24
I can't identify why the firewall is routing the other side of the tunnel to an IP range that is not included in the OpenVPN settings.
-
@patrick-pesegodinski Hello Patrick,
It sound like you are having a frustrating time.
With the information you have given it is difficult to understand exactly what your problem is. I can make a few suggestions and ask some questions.
Please be more specific in your description of the problem, including when it started and also the version of pfSense you are using. What are the firewall rules involved in this problem? What is the mask that you have for each of these address?
Is this access available in a similar manner other than ICMP - ping, on the other protocols, TLS, UDP etc? How have you configured your clients and your servers? How and why did your block access across these lans?
From one of my servers, a piece of client configuration.
IPV4 Remote Servers 10.3.0.0/21,10.0.0.0/23,10.1.0.0/22,10.2.0.0/21
This is an important line and might hold the key to what your problem is. You should only include the servers in this line that you want to talk to each other.
I work in Spanish and English and I know that when I write in Spanish that I have to err on the side of providing more detail for my friends and coworkers to understand.
-
I'm having a very similar problem after upgrading from Pfsense 23.05 to 23.09
I have a policy based route that directs traffic from specific source LAN IPs over an OpenVPN client connection. The VPN host is a generic privacy service (Privado), and its IP is dynamic.
The rule is triggering - I can see the byte counts going up, but pings through it from hosts have no connectivity.
If I ssh into PFsense, I can ping from the interface (ping -s vpnclientip 8.8.8.8) and I can see replies over that interface just fine, so it's not an issue with the client settings.I rolled back to 23.05 and all is well again, but i would like to understand how to fix it.
-
@djrobx I use the community addition, but I know that the route to Open VPN is through the system routing table, not with a policy rule. The policy rule will prevent your quieres from getting to open VPN.
From my post above
So your gateway group or other gateway is a policy based routing.
Ok again, really cool... but also read what it say... "Leave as 'default' to use the system routing table."
When you do a policy routing like this whatever matches the rule uses that policy routing ... and it NEVER gets to the system routing table which contains the information for your tunnels. So your request goes to the gateway group or other gateway or gateways your have chosen, and they say, "HUH?" and probably sends it out ...
I hope this helps
-
I found the cause of my issue but I don't totally understand it.
I have two OpenVPN clients - one to the privacy service (PrivadoVPN) and another "site to site" client connection to another friend's PFSense router that I open and create rules for only when necessary. That second client is disabled at the moment, but its outbound NAT rule still existed.
If I move the Privado NAT rule ahead of the private client NAT rule, my policy-based route starts working. So somehow the inactive client is interfering with the policy route function in the newer PFSense where it didn't before.
-
@djrobx The new release of pfsense does strictly enforce the firewall rules where the earlier version did not. Your action makes perfect sense.
-
@reberhar The policy based rule was preventing your Private NAT rule from reaching the system table.
-
@reberhar
To all, I have this same situation and I am using the System tables, however the Routes set up within the system tables are using the same IP address as the default gateway for each Point-2-Point VPN connection: 10.0.4.2Common Name: Remote sites IP DG in the <Diagnostics-IPV4 Routes>
P2P_Maine 10.10.60.0/24 10.0.4.2
P2P_NH 10.10.70.0/24 10.0.4.2
P2P_DBS 192.168.11.0/24 10.0.4.2
In the server configuration: VPN/OpenVPN/Server
IPV4 Tunnel Network: 10.0.4.0/24
IPV4 Local Networks(s) 10.10.30.0/24, 192.168.11.0/24, 10.10.60.0/24, 10.10.70.0/24
IPv4 Remote network(s) 192.168.11.0/24, 10.10.60.0/24, 10.10.70.0/24
Gateway creation: IPv4 onlyunder Status/OpenVPN
Common Name Virtual Address
P2P_Maine 10.0.4.3
P2P_NH 10.0.4.2
P2P_DBS 10.0.4.4The hose: 10.10.30.16 has a virtual address on the VPN interface of 10.0.4.1
The problem I have is the same as discussed, I cannot connect to any device on the client sites.
I have read the TLS configuration and many documents regarding the settings and cannot find any inconsistencies.
This worked pre-2.7.1Looking for any assistance to identify if this is the same issue as above or completely different.
Thanks in advance.
-
@itinfo Hi itinfo,
So I am trying to wrap my mind around all that you have said here. First do you have any policy routings at all? You were running a pre 2.7.1 version of pfSense and upgraded to 2.7.1 right? Were you running 2.7.0? Do you have Suricata or Snort or pfBlocker? Are you getting any error messages or do your attempts just time out? What are the logs telling you? Have you tried a trace route to see where the system thinks it is going? What does ping do? What about your output wan that is sending out and receiving the queres? Has that changed? I recently changed my Internet facing wan and had some configuration to cover that. It does not seem like that is your problem however. Do you have a DMZ? How many lans do you have?
2.7.0 began to strickly enforcing firewall rules, That was why the work around is needed for anyone who was using policy routing rules., but if you were already running 2.7.0 without your OpenVPN problems, then the problem is elsewhere.
One of the problems I had in upgrading to 2.7.1 was that Suricata began to block some of my IPs. There have been several Suricata updates since then.
How far are you getting? Are you able to look at the pfSense gui on the other side?
