Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71
-
Hey All,
I posted a question originally in the OpenVPN forum list about my problem, but it looks like the problem seems to be with the backup/restore of my configuration. Here is the post in
openvpn.The problem is that since the upgrade, routing to my VPN Gateway do no work anymore. I use selective routing based on an Alias, but even if I route everything to my VPN Gateway it still does not work.
I checked all OpenVPN Setting and looked at the logs, and everything there appears to be OK. The connection to the VPN Server (Commercial VPN Provider) is established.
Routing appears to be fine, but for some reason I think that the rules in my VPN point to the incorrect Gateway, even though it appears to be pointing to the correct one:
Here is a pic of my rule and the pop over when I hover:
And this is the link under the the
VPN_SYD_VPNV4in the rule:
But when I click on the gateway link, it takes me to a different gateway definition with id=3, not the VPN Gateway, which should be
id=2:
I have tried to clean up my gateways and OpenVPN configurations to get back to just having a WAN, LAN and IOT VLAN, but when I removed the
VPN_SYD_VPNV4gateway and interface using the UI, all internet traffic stops working.. i.e. it is as if it has deletedPPPoEinterface and gateway rather than theVPN_SYD_VPNV4one?!?I tried restoring my config a few times, but the problem persists. I can get my internet back by restoring, but routing through OpenVPN just fails.
Should I just start with a fresh install rather than trying to fix it?
-
J jimp moved this topic from Problems Installing or Upgrading pfSense Software on
-
Check the actual generated ruleset in /tmp/rules.debug.
Is the correct gateway actually set on the route-to rules that should apply to?
Steve
-
@stephenw10 said in Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71:
/tmp/rules.debug
This is now getting back to my OpenVPN problem and the clients not getting routed to the VON Gateway..
I've had a look and found this in rules.debug :
# Gateways GWWAN_PPPOE = " route-to ( pppoe0 <ISP.IP.ADDRESS> ) " GWVPN_CLIENT = " route-to ( vmx1 192.168.1.201 ) " GWVPN_SYD_VPNV4 = " "Where the
GWVPN_SYD_VPNV4does not have a route.. could that be because the VPN route is dynamic?And the Rule definitely routes to that empty definition:
pass in quick on $LAN $GWVPN_SYD_VPNV4 inet proto { tcp udp } from $THRU_VPN to any tag "NO_WAN_EGRESS" ridentifier 1618222021 keep state label "USER_RULE: THRU_SYD Addresses" label "id:1618222021" label "gw:VPN_SYD_VPNV4"I expect that this rules.debug file gets generated and applied before the VPN is active and therefore does not contain the route to the VPN?
If I look at the active routing table I see this:
[2.7.1-RELEASE][root@pfSense.home]/etc: netstat -rWn Routing tables Internet: Destination Gateway Flags Nhop# Mtu Netif Expire default <ISP.IP.ADDRESS> UGS 9 1492 pppoe0 10.8.0.0/24 192.168.1.201 UGS 1 1500 vmx1 10.9.0.0/24 192.168.1.201 UGS 1 1500 vmx1 <VPN.REMOTE.IP> link#10 UH 10 1500 ovpnc2 <VPN.LOCAL.IP> link#5 UHS 11 16384 lo0 <ISP.IP.ADDRESS> link#9 UH 7 1492 pppoe0 127.0.0.1 link#5 UH 2 16384 lo0 192.168.0.0/23 link#2 U 4 1500 vmx1 192.168.0.1 link#5 UHS 5 16384 lo0 192.168.100.0/24 link#8 U 6 1500 vmx2.100 192.168.100.1 link#5 UHS 3 16384 lo0 <EXT.IP.ADDRESS>. link#5 UHS 8 16384 lo0So all the routes are definitely there when the VPN is up..
Not sure where to go next.. I wonder if my install of pfSense is borked somehow and I should just try reinstalling it reupload my config file?
-
A little more info.. I had a look at the active rules and see these rules taht correspond to the rule above:
pass in quick on vmx1 inet proto tcp from <THRU_VPN> to any flags S/SA keep state label "USER_RULE: THRU_SYD Addresses" label "id:1618222021" label "gw:VPN_SYD_VPNV4" ridentifier 1618222021 tag NO_WAN_EGRESS pass in quick on vmx1 inet proto udp from <THRU_VPN> to any keep state label "USER_RULE: THRU_SYD Addresses" label "id:1618222021" label "gw:VPN_SYD_VPNV4" ridentifier 1618222021 tag NO_WAN_EGRESS\and it sees like the
$GWVPN_SYD_VPNV4had not been resolved to point to a the vpn gateway!I checked simmilar rules that use
$GWVPN_CLIENTor$GWWAN_PPPOEas defined above like:pass out quick on { pppoe0 } $GWWAN_PPPOE inet from <MY-INTERNET-IP> to any ridentifier 1665212341 keep state dnqueue( 2,1) label "USER_RULE: CoDel Limiters" label "id:1665212341" label "gw:WAN_PPPOE"and the actual that is active shows:
pass out quick on pppoe0 route-to (pppoe0 <MY-ISP-INTERNAL-IP>) inet from <MY-INTERNET-IP> to any flags S/SA keep state label "USER_RULE: CoDel Limiters" label "id:1665212341" label "gw:WAN_PPPOE" ridentifier 1665212341 dnqueue(2, 1)So it does get resolved in that rule, but not in my VPN routing rule!
So something in some definition of Gateways in my configuration (or stored in some config DB) seems to be borked.
Strangeness continues!
-
@grillp I have exactly the same issue. Since 2.7.1 the routing over an OpenVPN client does not work.
I have exactly the same rules as you. Using PIA as VPN provider.Does your THRU_NL network is on another subnet as your main LAN? Or is it in a seperate VLAN?
Looks like I -can- route and ping from my LAN to the VPN gateway, but not from the seperate VLAN subnet to the VPN gateway.
I do however can ping the VPN gateway from the VLAN subnet, but not 8.8.8.8. -
A VPN gateway like that is dynamic so you would expect to see it empty like that before the VPN connects. But as soon as it connects it should regenerate the rules to include that.
That only happens for an assigned interface.
Did the interface or gateway get renamed?Try running Status > Filter Reload and then rechecking.
-
I did some other tests. Looks like routing to VPN gateway from the default LAN is working.
Routing to the VPN gateway from another subnet or VLAN is not.All firewall rules are still the same as they were upgrading from 2.6.0 to 2.7.0 and started after upgrading to 2.7.1
Status > Filter Reload did not help, interface and gateway still have the same name. Nothing has changed since 2.6.0
-
Ok check the rulset file in /tmp/rules.debug.
Do you have the VPN gateway populated?
Is that also added to the policy routing rules on LAN? On other internal interfaces?
Trying to replicate this here now.
-
@stephenw10 Yes, it is populated.
GWPRIVATEVPN_VPNV4 = " route-to ( ovpnc2 10.32.x.x ) " nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535 # VLANVPN to PRIVATEVPN pass in quick on $VLANVPN $GWPRIVATEVPN_VPNV4 inet proto { tcp udp } from $OPT12__NETWORK to any ridentifier 1496691583 keep state label "USER_RULE: VLANVPN to PRIVATEVPN outgoing ports IP4 TCP/UDP" label "id:1496691583" label "gw:PRIVATEVPN_VPNV4" pass in quick on $VLANVPN $GWPRIVATEVPN_VPNV4 inet proto icmp from $OPT12__NETWORK to any icmp-type echoreq ridentifier 1502282393 keep state label "USER_RULE: VLANVPN to PRIVATEVPN outgoing ports ICMP" label "id:1502282393" label "gw:PRIVATEVPN_VPNV4" -
@stephenw10 And for the LAN
nat on $PRIVATEVPN inet from 192.168.1.0/24 to any -> 10.32.x.x/32 port 1024:65535 # LAN to PRIVATEVPN pass in quick on $LAN $GWPRIVATEVPN_VPNV4 inet from $VPNClients to any ridentifier 1422085516 keep state label "USER_RULE: LAN to PRIVATEVPN outgoing ports IP4" label "id:1422085516" label "gw:PRIVATEVPN_VPNV4" -
@stephenw10 Did see the difference between the LAN and VLAN rule. Then changed TCP/UDP in the VPN VLAN rule to * and it looks like it's working now, also from a subnet.
Why? -
How are you testing?
-
@digdug3 said in Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71:
@stephenw10 Did see the difference between the LAN and VLAN rule. Then changed TCP/UDP in the VPN VLAN rule to * and it looks like it's working now, also from a subnet.
Why?No looks like it does not work again. Probably due to a reload.
Saw another post about the NAT rules. I did have a disabled OPT1, the nat rule was directly after it:scrub on $PRIVATEVPN2 inet6 all fragment reassemble # Missing interface 'opt1' for rule 'LAN to OPT1'nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535 # VLANVPN to PRIVATEVPN nat on $PRIVATEVPN inet from 192.168.1.0/24 to any -> 10.32.x.x/32 port 1024:65535 # LAN to PRIVATEVPNTotally removed the interface, also from hybrid NAT generated rules and the VPN works again...
Now the debug looks like:scrub on $PRIVATEVPN2 inet6 all fragment reassemble nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535 # VLANVPN to PRIVATEVPN nat on $PRIVATEVPN inet from 192.168.1.0/24 to any -> 10.32.x.x/32 port 1024:65535 # LAN to PRIVATEVPNAnd it works again.
-
@digdug3 said in Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71:
Missing interface 'opt1' for rule 'LAN to OPT1'nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535 # VLANVPN to PRIVATEVPN
Aha, looks like a missing
/nsomewhere. Hmmm -
Did you have outbound NAT in manual mode? In hybrid mode the auto rules should still have translated that.
-
Added a bug to track: https://redmine.pfsense.org/issues/15024
-
@stephenw10 I've always had them in "Hybrid Outbound NAT" mode.
-
Hmm, then I would have expected the auto rules to apply that translation even if the manual rule you added was not being applied.
Do you see an equivalent rule in the listed out OBN rules?
-
@stephenw10 Just checked the OBN rules again and the VPN nat rule was added manually (years ago):
No other rules were commented out. -
If it's in hybrid mode though you should also have auto rules added for the VLANVPN subnet on the PrivateVPN interface. They should be shown below the manual rules.
