Policy based routing and VPN again
-
I have been reading so many articles and so many posts here about how to set up VPN and direct a set of LAN clients to use only that single VPN connection.
I've tried the tag method, I've tried the 2 rule method; one where you enforce the gateway, and the one after it to reject in case the gateway goes down.
I've tried articles from PIA, These Forums, this other one that was very education: https://www.infotechwerx.com/blog/Prevent-Any-Traffic-VPN-Hosts-Egressing-WAN
I'm at my wits end, it has been 2 weeks, and about 12 hours of trying to figure this out. Port Forwarding on the VPN link to the 2 LAN clients. Currently outbound works on the 2 clients, they go out the correct VPN interface, their IP resolves, I am able to do speed tests. But the Port forwarding is all sorts of wacked. I originally configured my interfaces, but toyed with the OpenVPN rule set, not realizing that I had named my interface OPENVPN. After this realization I removed my rules from the incorrect instance, renamed the interfaces, and Applied it to AirVPN.
When I have an allow default on the OpenVPN interface, I see traffic hit my 2 clients, but the clients response goes out the WAN (igb2), no bueno. If I remove the any to any rule on OpenVPN I get the AirVPN IP and the Redirected to IP on the AirVPN interface (ovpnc3)
09:02:17.224896 IP 198.199.98.246.38628 > 10.4.10.131.58910: Flags [s], seq 3287419867, win 14600, options [mss 1288,sackOK,TS val 1799929115 ecr 0,nop,wscale 8], length 0 09:02:17.224939 IP 198.199.98.246.38628 > 192.168.0.47.58910: Flags [s], seq 3287419867, win 14600, options [mss 1288,sackOK,TS val 1799929115 ecr 0,nop,wscale 8], length 0 09:02:18.220811 IP 198.199.98.246.38628 > 10.4.10.131.58910: Flags [s], seq 3287419867, win 14600, options [mss 1288,sackOK,TS val 1799929365 ecr 0,nop,wscale 8], length 0 09:02:18.220835 IP 198.199.98.246.38628 > 192.168.0.47.58910: Flags [s], seq 3287419867, win 14600, options [mss 1288,sackOK,TS val 1799929365 ecr 0,nop,wscale 8], length 0 To me seems mind boggling if it was doing the translation and dropping the translation in AirVPN interfaces. I've toyed with floating rules, but since removed them, because I think it's dirty to apply rules specific to an interface on something so generic. If I change my default route to AirVPN through the System -> Routing, Selecting AirVPN and assign it as Default, magically everything is functional, but I don't want all my clients going over the VPN. I've posted my rules.debug for posterity, and edited masked my PUBLIC_IP [code] set optimization normal set limit states 1634000 set limit src-nodes 1634000 #System aliases loopback = "{ lo0 }" SPECTRUM = "{ igb2 }" LAN = "{ ix0 }" WORKVPN = "{ ovpnc2 }" AIRVPN = "{ ovpnc3 }" OpenVPN = "{ openvpn }" #SSH Lockout Table table <sshlockout> persist table <webconfiguratorlockout> persist #Snort tables table <snort2c> table <virusprot> table <bogons> persist file "/etc/bogons" table <negate_networks> # User Aliases revo_cameras = "{ 8200 8016 10019 }" table <transmission> { 192.168.0.44 192.168.0.47 } transmission = "<transmission>" Webconsole = "{ 22 1337 }" # Gateways GWSPECTRUM_DHCP = " route-to ( igb2 70.125.128.1 ) " GWAirVPN = " route-to ( ovpnc3 10.4.10.131 ) " GWOPENVPN_DHCP = " route-to ( ovpnc2 10.0.130.1 ) " set loginterface ix0 set skip on pfsync0 scrub on $SPECTRUM all fragment reassemble scrub on $LAN all fragment reassemble scrub on $WORKVPN all fragment reassemble scrub on $AIRVPN all fragment reassemble no nat proto carp no rdr proto carp nat-anchor "natearly/*" nat-anchor "natrules/*" # Outbound NAT rules (manual) nat on $AIRVPN from 192.168.0.0/23 to any -> 10.4.10.131/32 port 1024:65535 nat on $WORKVPN from 192.168.0.0/23 to any -> 10.0.130.11/32 port 1024:65535 nat on $SPECTRUM from 192.168.0.0/23 to any -> PUBLIC_IP/32 port 1024:65535 # Load balancing anchor rdr-anchor "relayd/*" # TFTP proxy rdr-anchor "tftp-proxy/*" # NAT Inbound Redirects rdr on ovpnc3 proto { tcp udp } from any to 10.4.10.131 port 57695 -> 192.168.0.44 # Reflection redirects rdr on { ix0 openvpn } proto { tcp udp } from any to 10.4.10.131 port 57695 tag PFREFLECT -> 127.0.0.1 port 19000 rdr on ovpnc3 proto { tcp udp } from any to 10.4.10.131 port 58910 -> 192.168.0.47 # Reflection redirects rdr on { ix0 openvpn } proto { tcp udp } from any to 10.4.10.131 port 58910 tag PFREFLECT -> 127.0.0.1 port 19001 rdr on ix0 proto tcp from any to (self) port 80 -> 192.168.0.1 port 8080 # Reflection redirects rdr on openvpn proto tcp from any to (self) port 80 tag PFREFLECT -> 127.0.0.1 port 19002 rdr on igb2 proto { tcp udp } from any to 70.125.154.54 port 34197 -> 192.168.1.183 # Reflection redirects rdr on { ix0 openvpn } proto { tcp udp } from any to PUBLIC_IP port 34197 tag PFREFLECT -> 127.0.0.1 port 19003 rdr on igb2 proto tcp from any to PUBLIC_IP port $revo_cameras -> 192.168.0.98 # Reflection redirects rdr on { ix0 openvpn } proto tcp from any to PUBLIC_IP port 8200 tag PFREFLECT -> 127.0.0.1 port 19004 rdr on { ix0 openvpn } proto tcp from any to PUBLIC_IP port 8016 tag PFREFLECT -> 127.0.0.1 port 19005 rdr on { ix0 openvpn } proto tcp from any to PUBLIC_IP port 10019 tag PFREFLECT -> 127.0.0.1 port 19006 rdr on igb2 proto { tcp udp } from any to PUBLIC_IP port 25565 -> 192.168.1.156 # Reflection redirects rdr on { ix0 openvpn } proto { tcp udp } from any to PUBLIC_IP port 25565 tag PFREFLECT -> 127.0.0.1 port 19007 # UPnPd rdr anchor rdr-anchor "miniupnpd" anchor "relayd/*" anchor "openvpn/*" anchor "ipsec/*" # Allow IPv6 on loopback pass in quick on $loopback inet6 all tracker 1000000001 label "pass IPv6 loopback" pass out quick on $loopback inet6 all tracker 1000000002 label "pass IPv6 loopback" # Block all IPv6 block in log quick inet6 all tracker 1000000003 label "Block all IPv6" block out log quick inet6 all tracker 1000000004 label "Block all IPv6" # block IPv4 link-local. Per RFC 3927, link local "MUST NOT" be forwarded by a routing device, # and clients "MUST NOT" send such packets to a router. FreeBSD won't route 169.254./16, but # route-to can override that, causing problems such as in redmine #2073 block in log quick from 169.254.0.0/16 to any tracker 1000000101 label "Block IPv4 link-local" block in log quick from any to 169.254.0.0/16 tracker 1000000102 label "Block IPv4 link-local" #--------------------------------------------------------------------------- # default deny rules #--------------------------------------------------------------------------- block in log inet all tracker 1000000103 label "Default deny rule IPv4" block out log inet all tracker 1000000104 label "Default deny rule IPv4" block in log inet6 all tracker 1000000105 label "Default deny rule IPv6" block out log inet6 all tracker 1000000106 label "Default deny rule IPv6" # IPv6 ICMP is not auxilary, it is required for operation # See man icmp6(4) # 1 unreach Destination unreachable # 2 toobig Packet too big # 128 echoreq Echo service request # 129 echorep Echo service reply # 133 routersol Router solicitation # 134 routeradv Router advertisement # 135 neighbrsol Neighbor solicitation # 136 neighbradv Neighbor advertisement pass quick inet6 proto ipv6-icmp from any to any icmp6-type {1,2,135,136} tracker 1000000107 keep state # Allow only bare essential icmpv6 packets (NS, NA, and RA, echoreq, echorep) pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {129,133,134,135,136} tracker 1000000108 keep state pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {129,133,134,135,136} tracker 1000000109 keep state pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000110 keep state pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000111 keep state pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {128,133,134,135,136} tracker 1000000112 keep state # We use the mighty pf, we cannot be fooled. block log quick inet proto { tcp, udp } from any port = 0 to any tracker 1000000113 label "Block traffic from port 0" block log quick inet proto { tcp, udp } from any to any port = 0 tracker 1000000114 label "Block traffic to port 0" block log quick inet6 proto { tcp, udp } from any port = 0 to any tracker 1000000115 label "Block traffic from port 0" block log quick inet6 proto { tcp, udp } from any to any port = 0 tracker 1000000116 label "Block traffic to port 0" # Snort package block log quick from <snort2c> to any tracker 1000000117 label "Block snort2c hosts" block log quick from any to <snort2c> tracker 1000000118 label "Block snort2c hosts" # SSH lockout block in log quick proto tcp from <sshlockout> to (self) port 22 tracker 1000000301 label "sshlockout" # webConfigurator lockout block in log quick proto tcp from <webconfiguratorlockout> to (self) port 1337 tracker 1000000351 label "webConfiguratorlockout" block in log quick from <virusprot> to any tracker 1000000400 label "virusprot overload table" antispoof log for $SPECTRUM tracker 1000001570 # allow our DHCP client out to the SPECTRUM pass in on $SPECTRUM proto udp from any port = 67 to any port = 68 tracker 1000001591 label "allow dhcp client out SPECTRUM" pass out on $SPECTRUM proto udp from any port = 68 to any port = 67 tracker 1000001592 label "allow dhcp client out SPECTRUM" # Not installing DHCP server firewall rules for SPECTRUM which is configured for DHCP. antispoof log for $LAN tracker 1000002620 # allow access to DHCP server on LAN pass in quick on $LAN proto udp from any port = 68 to 255.255.255.255 port = 67 tracker 1000002641 label "allow access to DHCP server" pass in quick on $LAN proto udp from any port = 68 to 192.168.0.1 port = 67 tracker 1000002642 label "allow access to DHCP server" pass out quick on $LAN proto udp from 192.168.0.1 port = 67 to any port = 68 tracker 1000002643 label "allow access to DHCP server" antispoof log for $WORKVPN tracker 1000003670 antispoof log for $AIRVPN tracker 1000004720 # loopback pass in on $loopback inet all tracker 1000005811 label "pass IPv4 loopback" pass out on $loopback inet all tracker 1000005812 label "pass IPv4 loopback" pass in on $loopback inet6 all tracker 1000005813 label "pass IPv6 loopback" pass out on $loopback inet6 all tracker 1000005814 label "pass IPv6 loopback" # let out anything from the firewall host itself and decrypted IPsec traffic pass out inet all keep state allow-opts tracker 1000005815 label "let out anything IPv4 from firewall host itself" pass out inet6 all keep state allow-opts tracker 1000005816 label "let out anything IPv6 from firewall host itself" pass out route-to ( igb2 70.125.128.1 ) from PUBLIC_IP to !70.125.128.0/19 tracker 1000005911 keep state allow-opts label "let out anything from firewall host itself" pass out route-to ( ovpnc2 10.0.130.1 ) from 10.0.130.11 to !10.0.130.11/32 tracker 1000005912 keep state allow-opts label "let out anything from firewall host itself" pass out route-to ( ovpnc3 10.4.10.131 ) from 10.4.10.131 to !10.4.0.0/16 tracker 1000005913 keep state allow-opts label "let out anything from firewall host itself" # make sure the user cannot lock himself out of the webConfigurator or SSH pass in quick on ix0 proto tcp from any to (ix0) port { 1337 22 } tracker 10000 keep state label "anti-lockout rule" # NAT Reflection rules pass in inet tagged PFREFLECT tracker 1000006231 keep state label "NAT REFLECT: Allow traffic to localhost" # User-defined rules follow anchor "userrules/*" # array key "enc0" does not exist for "" in array: {SPECTRUM LAN WORKVPN AIRVPN OpenVPN } label "USER_RULE" pass in quick on $SPECTRUM reply-to ( igb2 70.125.128.1 ) inet proto icmp from any to any tracker 1485194335 keep state label "USER_RULE: Allow ping" pass in quick on $SPECTRUM reply-to ( igb2 70.125.128.1 ) inet proto tcp from 216.69.255.55 to PUBLIC_IP port $Webconsole tracker 1435184149 flags S/SA keep state label "USER_RULE: Remote Admin" pass in quick on $SPECTRUM reply-to ( igb2 70.125.128.1 ) inet proto tcp from any to 192.168.0.98 port $revo_cameras tracker 1460580583 flags S/SA keep state label "USER_RULE: NAT Revo Cameras" pass in quick on $SPECTRUM reply-to ( igb2 70.125.128.1 ) inet proto { tcp udp } from any to 192.168.1.156 port 25565 tracker 1460580584 keep state label "USER_RULE: NAT Minecraft" pass in quick on $SPECTRUM reply-to ( igb2 70.125.128.1 ) inet proto { tcp udp } from any to 192.168.1.183 port 34197 tracker 1466545887 keep state label "USER_RULE: NAT Factorio headless" pass in quick on $LAN inet from $transmission to <negate_networks> tracker 10000001 keep state label "NEGATE_ROUTE: Negate policy routing for destination" pass in quick on $LAN $GWAirVPN inet from $transmission to any tracker 1493862877 keep state label "USER_RULE" block in log quick on $LAN inet from $transmission to any tracker 1493864751 label "USER_RULE" pass in quick on $LAN inet proto tcp from any to 192.168.0.1 port 8080 tracker 1478203523 flags S/SA keep state label "USER_RULE: NAT " pass in quick on $LAN inet from any to any tracker 0100000101 keep state label "USER_RULE: Default allow LAN to any rule" pass in quick on $AIRVPN $GWAirVPN inet proto { tcp udp } from any to 192.168.0.47 port 58910 tracker 1493878657 keep state label "USER_RULE: NAT Transmission The Mixing Bowl" pass in quick on $AIRVPN $GWAirVPN inet proto { tcp udp } from any to 192.168.0.44 port 57695 tracker 1493878050 keep state label "USER_RULE: NAT Transmission Radarr/Sonarr" pass in quick on $AIRVPN reply-to ( ovpnc3 10.4.10.131 ) inet from any to any tracker 1493886673 keep state label "USER_RULE" # VPN Rules anchor "tftp-proxy/*" anchor "miniupnpd" [/code] Any possible insight on why/how I've messed this up? I'm on 2.3.4[/s][/s][/s][/s]</negate_networks></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></transmission></transmission></negate_networks></bogons></virusprot></snort2c></webconfiguratorlockout></sshlockout>
-
All you do is set the gateway in advanced settings on your firewall rules.
-
That's what I thought, but what about port forwarding, everything coming in through the vpn is trying to go back out the WAN, but the device can ping out through the VPN without a problem, all traffic originating from the device is going through the VPN tunnel as expected, but if the traffic is initiated from VPN, it tries to go back out the WAN.
There are no entries in the negate_networks.
In the rules.debug gateways are set, and assigned to my two rules in in the $LAN, and the 2 rules: Everything for the 2 devices is configured for going out the vpn. Ideas?
pass in quick on $LAN $GWAirVPN inet from $transmission to any tracker 1493862877 keep state label "USER_RULE" block in log quick on $LAN inet from $transmission to any tracker 1493864751 label "USER_RULE" pass in quick on $LAN inet proto tcp from any to 192.168.0.1 port 8080 tracker 1478203523 flags S/SA keep state label "USER_RULE: NAT " pass in quick on $LAN inet from any to any tracker 0100000101 keep state label "USER_RULE: Default allow LAN to any rule" pass in quick on $AIRVPN $GWAirVPN inet proto { tcp udp } from any to 192.168.0.47 port 58910 tracker 1493878657 keep state label "USER_RULE: NAT Transmission The Mixing Bowl" pass in quick on $AIRVPN $GWAirVPN inet proto { tcp udp } from any to 192.168.0.44 port 57695 tracker 1493878050 keep state label "USER_RULE: NAT Transmission Radarr/Sonarr" pass in quick on $AIRVPN reply-to ( ovpnc3 10.4.10.131 ) inet from any to any tracker 1493886673 keep state label "USER_RULE"
-
tl;dr.
To me, best option is to create an alias with machine's ips that should use VPN. Then create the rules in lan interface and set gateway on that rules. Take care about the order. Don't forget that as soon as a rule matches, it will be applied.
Connections initiated from outside, will not pass to the network as long as you don't nat them. Connections initiated from inside will go thru that gateway as long as it's initiated from one ip in that alias.
-
"everything coming in through the vpn is trying to go back out the WAN"
Well then sounds like your issue is on the remote side..
-
tl;dr.
To me, best option is to create an alias with machine's ips that should use VPN. Then create the rules in lan interface and set gateway on that rules. Take care about the order. Don't forget that as soon as a rule matches, it will be applied.
Connections initiated from outside, will not pass to the network as long as you don't nat them. Connections initiated from inside will go thru that gateway as long as it's initiated from one ip in that alias.
Word for word, my config says that's what I'm doing
pass in quick on $LAN $GWAirVPN inet from $transmission to any tracker 1493862877 keep state label "USER_RULE"
"$transmission" is myAlias table list of hosts that are supposed to use the "$GWAirVPN" gateway on "$LAN" as the FIRST rule.
When someone says
To me, best option is to create an alias with machine's ips that should use VPN. Then create the rules in lan interface and set gateway on that rules. Take care about the order. Don't forget that as soon as a rule matches, it will be applied.
Am I not doing that? I'm confused, as I thought that's what I was doing. I must be missing something.
"everything coming in through the vpn is trying to go back out the WAN"
Well then sounds like your issue is on the remote side..
How could this be on the Remote side? Just like the other topic with a similar issue, this is a VPN service. If I set my default gateway to that of the VPN service, all port forwarding responses and initiated traffic goes through the VPN as one would assume should work.
-
tl;dr.
To me, best option is to create an alias with machine's ips that should use VPN. Then create the rules in lan interface and set gateway on that rules. Take care about the order. Don't forget that as soon as a rule matches, it will be applied.
Connections initiated from outside, will not pass to the network as long as you don't nat them. Connections initiated from inside will go thru that gateway as long as it's initiated from one ip in that alias.
Word for word, my config says that's what I'm doing
pass in quick on $LAN $GWAirVPN inet from $transmission to any tracker 1493862877 keep state label "USER_RULE"
"$transmission" is myAlias table list of hosts that are supposed to use the "$GWAirVPN" gateway on "$LAN" as the FIRST rule.
When someone says
To me, best option is to create an alias with machine's ips that should use VPN. Then create the rules in lan interface and set gateway on that rules. Take care about the order. Don't forget that as soon as a rule matches, it will be applied.
Am I not doing that? I'm confused, as I thought that's what I was doing. I must be missing something.
Well… to be fair, I did put tl;dr there didn't I? ;)
Can you post the GUI rule?
-
that Tl;dr was too far up, I typically skip to the bottom to see if it was there. bleh.
![Screenshot from 2017-05-26 09-07-54.png](/public/imported_attachments/1/Screenshot from 2017-05-26 09-07-54.png)
![Screenshot from 2017-05-26 09-07-54.png_thumb](/public/imported_attachments/1/Screenshot from 2017-05-26 09-07-54.png_thumb) -
Seems ok to me. Do you have NAT rules (port forward to inside ip) also?
-
Added an allow rule to see if that would help, not like the results make sense anyways.
![Screenshot from 2017-05-26 09-48-46.png](/public/imported_attachments/1/Screenshot from 2017-05-26 09-48-46.png)
![Screenshot from 2017-05-26 09-48-46.png_thumb](/public/imported_attachments/1/Screenshot from 2017-05-26 09-48-46.png_thumb) -
Post a pic of your NAT rules.
-
Simple stuff
![Screenshot from 2017-05-26 13-11-18.png](/public/imported_attachments/1/Screenshot from 2017-05-26 13-11-18.png)
![Screenshot from 2017-05-26 13-11-18.png_thumb](/public/imported_attachments/1/Screenshot from 2017-05-26 13-11-18.png_thumb) -
I'm no pro, and maybe it's different for your config. But based on my working config, I think the "simple stuff" is where you messed up.
What about localhost?
My outbound NAT looks more like this.
VPN 127.0.0.0/8 * * 500 VPN Address * static=yes
VPN 127.0.0.0/8 * * * VPN Address * static=no
VPN (subnet a,b,c, etc) * * 500 VPN Address * static=yes
VPN (subnet a,b,c, etc) * * * VPN Address * static=no
That's with Hybrid outbound.
-
I checked this, added in the nat rule.
Still no work. Nat rules are working as far as I can tell.
![Screenshot from 2017-05-26 14-54-48.png](/public/imported_attachments/1/Screenshot from 2017-05-26 14-54-48.png)
![Screenshot from 2017-05-26 14-54-48.png_thumb](/public/imported_attachments/1/Screenshot from 2017-05-26 14-54-48.png_thumb) -
For giggles, I added in a am AIRVPN_TAG parameter on the port forwarded rules in the AIRVPN ruleset, and a floating rule of "any any any with tagged AIRVPN_TAG" to set the gateway to GW AIRVPN, and still, traffic is trying to go out the WAN.