Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71

digdug3 · Nov 22, 2023, 3:09 PM

@stephenw10 Yes, it is populated.

GWPRIVATEVPN_VPNV4 = " route-to ( ovpnc2 10.32.x.x ) "
nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535  # VLANVPN to PRIVATEVPN
pass  in  quick  on $VLANVPN  $GWPRIVATEVPN_VPNV4 inet proto { tcp udp }  from $OPT12__NETWORK to any ridentifier 1496691583 keep state label "USER_RULE: VLANVPN to PRIVATEVPN outgoing ports IP4 TCP/UDP" label "id:1496691583" label "gw:PRIVATEVPN_VPNV4"
pass  in  quick  on $VLANVPN  $GWPRIVATEVPN_VPNV4 inet proto icmp  from $OPT12__NETWORK to any icmp-type echoreq ridentifier 1502282393 keep state label "USER_RULE: VLANVPN to PRIVATEVPN outgoing ports ICMP" label "id:1502282393" label "gw:PRIVATEVPN_VPNV4"

digdug3 · Nov 22, 2023, 3:13 PM

@stephenw10 And for the LAN

nat on $PRIVATEVPN inet from 192.168.1.0/24 to any -> 10.32.x.x/32 port 1024:65535  # LAN to PRIVATEVPN
pass  in  quick  on $LAN  $GWPRIVATEVPN_VPNV4 inet from $VPNClients to any ridentifier 1422085516 keep state label "USER_RULE: LAN to PRIVATEVPN outgoing ports IP4" label "id:1422085516" label "gw:PRIVATEVPN_VPNV4"

digdug3 · Nov 22, 2023, 3:20 PM

@stephenw10 Did see the difference between the LAN and VLAN rule. Then changed TCP/UDP in the VPN VLAN rule to * and it looks like it's working now, also from a subnet.
Why?

stephenw10 · Nov 22, 2023, 4:05 PM

How are you testing?

digdug3 · Nov 22, 2023, 4:08 PM

@digdug3 said in Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71:

@stephenw10 Did see the difference between the LAN and VLAN rule. Then changed TCP/UDP in the VPN VLAN rule to * and it looks like it's working now, also from a subnet.
Why?

No looks like it does not work again. Probably due to a reload.
Saw another post about the NAT rules. I did have a disabled OPT1, the nat rule was directly after it:

scrub on $PRIVATEVPN2 inet6 all    fragment reassemble
# Missing interface 'opt1' for rule 'LAN to OPT1'nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535  # VLANVPN to PRIVATEVPN
nat on $PRIVATEVPN inet from 192.168.1.0/24 to any -> 10.32.x.x/32 port 1024:65535  # LAN to PRIVATEVPN

Totally removed the interface, also from hybrid NAT generated rules and the VPN works again...
Now the debug looks like:

scrub on $PRIVATEVPN2 inet6 all    fragment reassemble
nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535  # VLANVPN to PRIVATEVPN
nat on $PRIVATEVPN inet from 192.168.1.0/24 to any -> 10.32.x.x/32 port 1024:65535  # LAN to PRIVATEVPN

And it works again.

stephenw10 · Nov 22, 2023, 4:11 PM

@digdug3 said in Routing/INterface/Gateway issues after updating from CE 2.7 -> 2.71:

Missing interface 'opt1' for rule 'LAN to OPT1'nat on $PRIVATEVPN inet from 192.168.14.0/24 to any -> 10.32.x.x/32 port 1024:65535 # VLANVPN to PRIVATEVPN

Aha, looks like a missing /n somewhere. Hmmm

stephenw10 · Nov 22, 2023, 4:14 PM

Did you have outbound NAT in manual mode? In hybrid mode the auto rules should still have translated that.

stephenw10 · Nov 22, 2023, 4:26 PM

Added a bug to track: https://redmine.pfsense.org/issues/15024

digdug3 · Nov 22, 2023, 5:38 PM

@stephenw10 I've always had them in "Hybrid Outbound NAT" mode.

stephenw10 · Nov 22, 2023, 5:47 PM

Hmm, then I would have expected the auto rules to apply that translation even if the manual rule you added was not being applied.

Do you see an equivalent rule in the listed out OBN rules?

digdug3 · Nov 22, 2023, 6:55 PM

@stephenw10 Just checked the OBN rules again and the VPN nat rule was added manually (years ago):

No other rules were commented out.

stephenw10 · Nov 22, 2023, 7:16 PM

If it's in hybrid mode though you should also have auto rules added for the VLANVPN subnet on the PrivateVPN interface. They should be shown below the manual rules.

digdug3 · Nov 23, 2023, 7:34 AM

@stephenw10 No, they aren't, probably because the VPN client has "Don't pull routes" checked. I also only want these rules for two of my subnets, not all of them.

Next "issue" I found was when deleting an interface the manual created OBN rule wasn't removed (just like the firewall rules). It should be easy to replicate.

And last, the comment in the debug said "missing interface", shouldn't it be "disabled interface" when the interface is disabled?
"missing" is more correct when the interface is deleted but the manual OBN rule is still there.

stephenw10 · Nov 23, 2023, 1:07 PM

The pf process doesn't see any difference between disabled or entirely removed interfaces. It just sees referenced to an interface that isn't defined.

It is interesting that it's not removed like a firewall rule would be though.

digdug3 · Nov 23, 2023, 1:18 PM

@stephenw10 At least it's fixed now for me and looks like the missing /n after the "missing interface" comment was the culprit.