Static Routing between Lan Vlan OpenVPN

JonathanLee · Jan 14, 2024, 8:49 AM

@darwood what about floating rule as it’s essentially on the same interface??
Better yet just creat a NAT rule for that traffic with destinations out that other interface…

darwood · Jan 14, 2024, 4:07 PM

@JonathanLee

I have tried adding a NAT rule under outbound. Tried with VLAN30 and LAN as the NAT Address.

But this did not work. Ping to other machines on LAN from VLAN30 works, but Ping to 10.20.1.102 fails from VLAN30. Also ping to 10.0.0.13 also fails from VLAN30, but from LAN works.

I should be able to ping 10.0.0.13 from Vlan30 as I can ping other clients on LAN, but this is not the case.

What would allow a ping to clients on LAN from VLAN30, but would block a ping from VLAN30 to that client, but if the ping comes from LAN it works.

firewall/NAT/Portforward
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination 10.20.1.0/24, Destination Port *, Nat Address VLAN30 address, Nat Port *
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination Lan subnets, Destination Port *, Nat Address VLAN30 address, Nat Port *

(not at the same time)

firewall/NAT/Portforward
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination 10.20.1.0/24, Destination Port *, Nat Address LAN address, Nat Port *
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination Lan subnets, Destination Port *, Nat Address LAN address, Nat Port *

firewall/Rules/Floating
Interface VLAN30, Protocol IPv4* Source VLAN30 subnets, Port *, Destination 10.20.1.0/24, Port 0, Gateway LINK_SER, Queue none

And tested using pfsense ping from Lan/VLAN30. Same results ping successful from LAN, fails from VLAN30(it fails from anything other than LAN). Ping from Vlan30 to other LAN machines are successful.

viragomann · Jan 14, 2024, 5:25 PM

@darwood said in Static Routing between Lan Vlan OpenVPN:

I have identical rules in vlan30, basically allow all. Vlan30 clients can ping lan clients. But Vlan30 clients can not ping 10.20.1.100, and traffic does not reach the LINK_SER gateway.

Sure? Have you sniff the traffic to prove this?

I have tried adding a NAT rule under outbound. Tried with VLAN30 and LAN as the NAT Address.

Can you post a screenshot?

firewall/NAT/Portforward
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination 10.20.1.0/24, Destination Port *, Nat Address VLAN30 address, Nat Port *
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination Lan subnets, Destination Port *, Nat Address VLAN30 address, Nat Port *

(not at the same time)

firewall/NAT/Portforward
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination 10.20.1.0/24, Destination Port *, Nat Address LAN address, Nat Port *
Interface VLAN30, Source VLAN30 subnets, Source Port *, Destination Lan subnets, Destination Port *, Nat Address LAN address, Nat Port *

This rules make no sense at all.

darwood · Jan 14, 2024, 6:35 PM

@viragomann

I am remote to this pfsense router atm. But I can ssh in to the pfsense.

From Vlan30 to 10.0.0.13
ping -S 10.0.30.254 10.0.0.13
PING 10.0.0.13 (10.0.0.13) from 10.0.30.254: 56 data bytes
^C
--- 10.0.0.13 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

From Vlan30 to 10.0.0.15 - same vlan to a different LAN IP
ping -S 10.0.30.254 10.0.0.15
PING 10.0.0.15 (10.0.0.15) from 10.0.30.254: 56 data bytes
64 bytes from 10.0.0.15: icmp_seq=0 ttl=64 time=0.531 ms
64 bytes from 10.0.0.15: icmp_seq=1 ttl=64 time=0.911 ms
64 bytes from 10.0.0.15: icmp_seq=2 ttl=64 time=0.361 ms

From LAN to 10.0.0.13
ping -S 10.0.0.254 10.0.0.13
PING 10.0.0.13 (10.0.0.13) from 10.0.0.254: 56 data bytes
64 bytes from 10.0.0.13: icmp_seq=0 ttl=255 time=0.749 ms
64 bytes from 10.0.0.13: icmp_seq=1 ttl=255 time=0.461 ms
64 bytes from 10.0.0.13: icmp_seq=2 ttl=255 time=0.510 ms
^C
--- 10.0.0.13 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.461/0.573/0.749/0.126 ms

Rules currently loaded.
pfctl -s rules
scrub from any to <vpn_networks> fragment no reassemble
scrub from <vpn_networks> to any fragment no reassemble
scrub on mvneta0.4090 inet all fragment reassemble
scrub on mvneta0.4090 inet6 all fragment reassemble
scrub on mvneta0.4091 inet all fragment reassemble
scrub on mvneta0.4091 inet6 all fragment reassemble
scrub on mvneta0.20 inet all fragment reassemble
scrub on mvneta0.20 inet6 all fragment reassemble
scrub on mvneta0.30 inet all fragment reassemble
scrub on mvneta0.30 inet6 all fragment reassemble
anchor "openvpn/" all
anchor "ipsec/" all
pass in quick on lo0 inet6 all flags S/SA keep state label "pass IPv6 loopback" ridentifier 1000000001
pass out quick on lo0 inet6 all flags S/SA keep state label "pass IPv6 loopback" ridentifier 1000000002
block drop in log quick inet6 all label "Block all IPv6" ridentifier 1000000003
block drop out log quick inet6 all label "Block all IPv6" ridentifier 1000000004
block drop in log quick inet from 169.254.0.0/16 to any label "Block IPv4 link-local" ridentifier 1000000101
block drop in log quick inet from any to 169.254.0.0/16 label "Block IPv4 link-local" ridentifier 1000000102
block drop in log inet all label "Default deny rule IPv4" ridentifier 1000000103
block drop out log inet all label "Default deny rule IPv4" ridentifier 1000000104
block drop in log inet6 all label "Default deny rule IPv6" ridentifier 1000000105
block drop out log inet6 all label "Default deny rule IPv6" ridentifier 1000000106
block drop log quick inet proto tcp from any port = 0 to any label "Block traffic from port 0" ridentifier 1000000107
block drop log quick inet proto udp from any port = 0 to any label "Block traffic from port 0" ridentifier 1000000107
block drop log quick inet proto tcp from any to any port = 0 label "Block traffic to port 0" ridentifier 1000000108
block drop log quick inet proto udp from any to any port = 0 label "Block traffic to port 0" ridentifier 1000000108
block drop log quick from <snort2c> to any label "Block snort2c hosts" ridentifier 1000000109
block drop log quick from any to <snort2c> label "Block snort2c hosts" ridentifier 1000000110
block drop in log quick proto tcp from <sshguard> to (self) port = 2424 label "sshguard" ridentifier 1000000301
block drop in log quick proto tcp from <sshguard> to (self) port = http label "GUI Lockout" ridentifier 1000000351
block drop in log quick from <virusprot> to any label "virusprot overload table" ridentifier 1000000400
block drop in log quick on mvneta0.4090 from <bogons> to any label "block bogon IPv4 networks from WAN" ridentifier 11001
block drop in log on ! mvneta0.4090 inet from <REDACTEDWANIP>/29 to any ridentifier 1000001470
block drop in log inet from <REDACTEDWANIP> to any ridentifier 1000001470
block drop in log on mvneta0.4090 inet6 from <REDACTEDWANIPIPV6> to any ridentifier 1000001470
block drop in log quick on mvneta0.4090 inet from 10.0.0.0/8 to any label "Block private networks from WAN block 10/8" ridentifier 12001
block drop in log quick on mvneta0.4090 inet from 127.0.0.0/8 to any label "Block private networks from WAN block 127/8" ridentifier 12002
block drop in log quick on mvneta0.4090 inet from 172.16.0.0/12 to any label "Block private networks from WAN block 172.16/12" ridentifier 12003
block drop in log quick on mvneta0.4090 inet from 192.168.0.0/16 to any label "Block private networks from WAN block 192.168/16" ridentifier 12004
block drop in log quick on mvneta0.4090 inet6 from fc00::/7 to any label "Block ULA networks from WAN block fc00::/7" ridentifier 12005
block drop in log on ! mvneta0.4091 inet from 10.0.0.0/24 to any ridentifier 1000002520
block drop in log inet from 10.0.0.254 to any ridentifier 1000002520
block drop in log on mvneta0.4091 inet6 from <REDACTEDWANIPIPV6> to any ridentifier 1000002520
pass in quick on mvneta0.4091 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server" ridentifier 1000002541
pass in quick on mvneta0.4091 inet proto udp from any port = bootpc to 10.0.0.254 port = bootps keep state label "allow access to DHCP server" ridentifier 1000002542
pass out quick on mvneta0.4091 inet proto udp from 10.0.0.254 port = bootps to any port = bootpc keep state label "allow access to DHCP server" ridentifier 1000002543
block drop in log on ! mvneta0.20 inet from 10.0.20.0/24 to any ridentifier 1000003570
block drop in log inet from 10.0.20.254 to any ridentifier 1000003570
block drop in log on mvneta0.20 inet6 from <REDACTEDWANIPIPV6> to any ridentifier 1000003570
pass in quick on mvneta0.20 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server" ridentifier 1000003591
pass in quick on mvneta0.20 inet proto udp from any port = bootpc to 10.0.20.254 port = bootps keep state label "allow access to DHCP server" ridentifier 1000003592
pass out quick on mvneta0.20 inet proto udp from 10.0.20.254 port = bootps to any port = bootpc keep state label "allow access to DHCP server" ridentifier 1000003593
block drop in log on ! mvneta0.30 inet from 10.0.30.0/24 to any ridentifier 1000004620
block drop in log inet from 10.0.30.254 to any ridentifier 1000004620
block drop in log on mvneta0.30 inet6 from <REDACTEDWANIPIPV6> to any ridentifier 1000004620
pass in quick on mvneta0.30 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server" ridentifier 1000004641
pass in quick on mvneta0.30 inet proto udp from any port = bootpc to 10.0.30.254 port = bootps keep state label "allow access to DHCP server" ridentifier 1000004642
pass out quick on mvneta0.30 inet proto udp from 10.0.30.254 port = bootps to any port = bootpc keep state label "allow access to DHCP server" ridentifier 1000004643
pass in on lo0 inet all flags S/SA keep state label "pass IPv4 loopback" ridentifier 1000005711
pass out on lo0 inet all flags S/SA keep state label "pass IPv4 loopback" ridentifier 1000005712
pass out inet all flags S/SA keep state allow-opts label "let out anything IPv4 from firewall host itself" ridentifier 1000005713
pass out route-to (mvneta0.4090 <REDACTEDWANIP>) inet from <REDACTEDWANIP> to ! <REDACTEDWANIP>/29 flags S/SA keep state allow-opts label "let out anything from firewall host itself" ridentifier 1000005811
pass in quick on mvneta0.4091 proto tcp from any to (mvneta0.4091) port = http flags S/SA keep state label "anti-lockout rule" ridentifier 10001
pass in quick on mvneta0.4091 proto tcp from any to (mvneta0.4091) port = 2424 flags S/SA keep state label "anti-lockout rule" ridentifier 10001
anchor "userrules/" all
pass out on mvneta0.30 route-to (mvneta0.4091 10.0.0.13) inet from <OPT3__NETWORK> to 10.20.1.0/24 flags S/SA keep state label "USER_RULE" label "id:1705199260" label "gw:LINK_SER" ridentifier 1705199260
pass in quick on openvpn inet all flags S/SA keep state label "USER_RULE: OpenVPN OpenVPN Corp Network wizard" label "id:1656005485" ridentifier 1656005485
pass in quick on openvpn inet all flags S/SA keep state label "USER_RULE: OpenVPN OpenVPN Secuirty wizard" label "id:1656006325" ridentifier 1656006325
pass in quick on openvpn inet all flags S/SA keep state label "USER_RULE: OpenVPN OpenVPN Secuirty wizard" label "id:1656006394" ridentifier 1656006394
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from any to (self) port 21 >< 2425 flags S/SA keep state label "USER_RULE" label "id:1705251456" ridentifier 1705251456
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto udp from any to <REDACTEDWANIP> port = openvpn keep state label "USER_RULE: OpenVPN OpenVPN Corp Network wizard" label "id:1656005484" ridentifier 1656005484
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto udp from any to <REDACTEDWANIP> port = rsf-1 keep state label "USER_RULE: OpenVPN OpenVPN Secuirty wizard" label "id:1656006324" ridentifier 1656006324
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto udp from any to <REDACTEDWANIP> port = 1196 keep state label "USER_RULE: OpenVPN OpenVPN Secuirty wizard" label "id:1656006393" ridentifier 1656006393
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from any to 10.0.0.15 port = submission flags S/SA keep state label "USER_RULE: NAT SMTP to KACKT STARTTLS" label "id:1689123396" ridentifier 1689123396
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from any to 10.0.0.15 port = smtp flags S/SA keep state label "USER_RULE: NAT " label "id:1689127029" ridentifier 1689127029
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from any to 10.0.0.15 port = imaps flags S/SA keep state label "USER_RULE: NAT " label "id:1689127190" ridentifier 1689127190
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from any to 10.0.0.15 port = imap flags S/SA keep state label "USER_RULE: NAT " label "id:1689129076" ridentifier 1689129076
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from any to 10.0.0.15 port = https flags S/SA keep state label "USER_RULE: NAT 443 to KACKT for MailPlus app" label "id:1700154071" ridentifier 1700154071
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from REDACTEDEXTERNALIP to <REDACTEDWANIP> port = 6690 flags S/SA keep state label "USER_RULE: Passed via EasyRule" label "id:1700862156" ridentifier 1700862156
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from REDACTEDEXTERNALIP to <REDACTEDWANIP> port = microsoft-ds flags S/SA keep state label "USER_RULE: Passed via EasyRule" label "id:1700865372" ridentifier 1700865372
pass in quick on mvneta0.4090 reply-to (mvneta0.4090 <REDACTEDWANIP>) inet proto tcp from REDACTEDEXTERNALIP to 10.0.0.10 port = http-alt flags S/SA keep state label "USER_RULE: NAT " label "id:1704146078" ridentifier 1704146078
pass in quick on mvneta0.4091 inet from 10.20.1.0/24 to 10.0.0.13 flags S/SA keep state label "USER_RULE: Allow from LINK_SER" label "id:1656006751" ridentifier 1656006751
pass in quick on mvneta0.4091 inet from 10.0.30.0/24 to 10.0.0.13 flags S/SA keep state label "USER_RULE: Allow from LINK_SER" label "id:1705216069" ridentifier 1705216069
pass in quick on mvneta0.4091 inet from 10.20.1.0/24 to any flags S/SA keep state label "USER_RULE: Allow from LINK_SER" label "id:1656006686" ridentifier 1656006686
pass in quick on mvneta0.4091 inet from <LAN__NETWORK> to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule" label "id:0100000101" ridentifier 100000101
pass in quick on mvneta0.20 inet proto tcp from any to 10.0.0.15 port = smtp flags S/SA keep state label "USER_RULE" label "id:1697941554" ridentifier 1697941554
pass in quick on mvneta0.20 inet proto tcp from any to 10.0.0.10 port = http-alt flags S/SA keep state label "USER_RULE" label "id:1704221032" ridentifier 1704221032
pass in quick on mvneta0.20 inet proto udp from any to 10.0.0.15 port = submission keep state label "USER_RULE" label "id:1697941439" ridentifier 1697941439
pass in quick on mvneta0.20 inet proto udp from any to 10.0.0.254 port = ntp keep state label "USER_RULE" label "id:1689820479" ridentifier 1689820479
block drop in quick on mvneta0.20 inet from any to <OPT3__NETWORK> label "USER_RULE: Block Lan" label "id:1656006984" ridentifier 1656006984
block drop in quick on mvneta0.20 inet from any to <LAN__NETWORK> label "USER_RULE: Block Lan" label "id:1656006920" ridentifier 1656006920
block drop in quick on mvneta0.20 inet proto tcp from any to (self) port = 10433 flags S/SA label "USER_RULE: Block Web Interface" label "id:1656006863" ridentifier 1656006863
pass in quick on mvneta0.20 inet from <OPT2__NETWORK> to any flags S/SA keep state label "USER_RULE" label "id:1656003821" ridentifier 1656003821
pass in quick on mvneta0.30 inet proto tcp from any to 10.20.1.0/24 flags S/SA keep state (sloppy) label "USER_RULE" label "id:1705250684" ridentifier 1705250684
pass in quick on mvneta0.30 inet all flags S/SA keep state label "USER_RULE: Allow Lan" label "id:1656007148" ridentifier 1656007148
block drop in quick on mvneta0.30 inet from any to <OPT2__NETWORK> label "USER_RULE: Block Lan" label "id:1656007200" ridentifier 1656007200
block drop in quick on mvneta0.30 inet proto tcp from any to (self) port = 10433 flags S/SA label "USER_RULE: Block Web Interface" label "id:1656007110" ridentifier 1656007110
pass quick on mvneta0.4091 inet proto tcp from 10.0.0.0/24 to 10.20.1.0/24 flags any keep state (sloppy) label "pass traffic between statically routed subnets" ridentifier 1000006241
pass quick on mvneta0.4091 inet from 10.0.0.0/24 to 10.20.1.0/24 flags S/SA keep state (sloppy) label "pass traffic between statically routed subnets" ridentifier 1000006242
pass quick on mvneta0.4091 inet proto tcp from 10.20.1.0/24 to 10.0.0.0/24 flags any keep state (sloppy) label "pass traffic between statically routed subnets" ridentifier 1000006243
pass quick on mvneta0.4091 inet from 10.20.1.0/24 to 10.0.0.0/24 flags S/SA keep state (sloppy) label "pass traffic between statically routed subnets" ridentifier 1000006244
anchor "tftp-proxy/" all

viragomann · Jan 14, 2024, 7:17 PM

@darwood
This is not helpful. We already know, what is working and what doesn't.

Run

pfctl -sn

to check out the NAT rules.

darwood · Jan 14, 2024, 7:43 PM

@viragomann said in Static Routing between Lan Vlan OpenVPN:

pfctl -sn

My bad I had as well but failed to attach it.

root: pfctl -sn
no nat proto carp all
nat-anchor "natearly/" all
nat-anchor "natrules/" all
nat on mvneta0.30 inet from any to 10.20.1.0/24 -> 10.0.30.254 port 1024:65535
nat on mvneta0.30 inet from any to <LAN__NETWORK> -> 10.0.30.254 port 1024:65535
nat on mvneta0.4091 inet from 10.0.0.0/24 to 10.20.1.0/24 -> 10.0.0.254 port 1024:65535
nat on openvpn inet from 10.0.70.0/24 to 10.20.1.0/24 -> (openvpn) port 1024:65535 round-robin
nat on openvpn inet from 10.0.80.0/24 to 10.20.1.0/24 -> (openvpn) port 1024:65535 round-robin
nat on openvpn inet from 10.0.90.0/24 to 10.20.1.0/24 -> (openvpn) port 1024:65535 round-robin
nat on mvneta0.4090 inet from <tonatsubnets> to any port = isakmp -> <REDACTEDWANIP> static-port
nat on mvneta0.4090 inet6 from <tonatsubnets> to any port = isakmp -> (mvneta0.4090) round-robin static-port
nat on mvneta0.4090 inet from <tonatsubnets> to any -> <REDACTEDWANIP> port 1024:65535
nat on mvneta0.4090 inet6 from <tonatsubnets> to any -> (mvneta0.4090) port 1024:65535 round-robin
no rdr proto carp all
rdr-anchor "tftp-proxy/*" all
rdr on mvneta0.4090 inet proto tcp from any to <REDACTEDWANIP> port = imap -> 10.0.0.15
rdr on mvneta0.4090 inet proto tcp from <REDACTEDEXTERNALIP> to <REDACTEDWANIP> port = http-alt -> 10.0.0.10
rdr on mvneta0.4090 inet proto tcp from any to <REDACTEDWANIP> port = imaps -> 10.0.0.15
rdr on mvneta0.4090 inet proto tcp from any to <REDACTEDWANIP> port = smtp -> 10.0.0.15
rdr on mvneta0.4090 inet proto tcp from any to <REDACTEDWANIP> port = submission -> 10.0.0.15
rdr on mvneta0.4090 inet proto tcp from any to <REDACTEDWANIP> port = https -> 10.0.0.15

viragomann · Jan 14, 2024, 7:53 PM

@darwood
So I cannot find a proper outbound NAT rule.

My assumption is that the gateway only allows access from your LAN subnet. If that's the case you need a rule for masquerading the traffic from VLAN 30 with the LAN address.
It should look like this:
interface: LAN
source: VLAN 30 subnet
destination:10.20.1.0/24
translation: interface address (=LAN address)

JonathanLee · Jan 14, 2024, 7:55 PM

@viragomann would it need to be floating? It’s on the same interface or no??

viragomann · Jan 14, 2024, 8:00 PM

@JonathanLee
As I understood, the gateway is connected to LAN. LAN subnet can access the remote devices, but VLAN 30 cannot.

So if he masquerade VLAN 30 traffic destined to the remote network with the LAN address it should work.
This is done with an outbound NAT rule.

darwood · Jan 14, 2024, 9:18 PM

@viragomann

You are correct, the solution was to create a outbound nat. I had tried this before but I made a mistake for the interface I selected VLAN30, but this was not the correct interface. The interface should have been LAN. Your solution pointed out that mistake.

Thank you.