Outbound NAT not being applied even though rules are there
-
Yes, it's one of the CARP ip which is being used by multiple local nets. I've attached two screenshots of my configuration.
WAN CARP status on Master device:
Outbound NAT rules:
This is neither the full CARP list nor the full Outbound NAT list, since it's too long to get into one screenshot.
Edit:
Here's also what pfctl shows:pfctl -sn ... nat on lagg0.209 inet from 192.168.32.0/22 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.32.0/22 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.38.0/24 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.38.0/24 to any -> 93.111.250.180 static-port ... -
What I today tried is to switch over to secondary firewall. We suffer the same situation there.
Then I rebooted the first firewall and switched back to it. Sadly, no improvement afterall.
Here's the full output of pfctl:[2.4.4-RELEASE][root@fw2-rx.domain1.net]/root: pfctl -sn no nat proto carp all nat-anchor "natearly/*" all nat-anchor "natrules/*" all nat on enc0 inet from 10.254.252.0/24 to 172.24.0.0/16 -> 192.168.222.1 nat on enc0 inet from 192.168.20.0/24 to 172.24.0.0/16 -> 192.168.222.1 nat on lagg0.209 inet from 192.168.13.0/24 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.13.0/24 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 127.0.0.0/8 to any port = isakmp -> 93.111.250.178 static-port nat on lagg0.209 inet from 127.0.0.0/8 to any -> 93.111.250.178 port 1024:65535 nat on lagg0.209 inet6 from ::1 to any port = isakmp -> 2a00:aaaa:0:16::2 static-port nat on lagg0.209 inet6 from ::1 to any -> 2a00:aaaa:0:16::2 port 1024:65535 nat on lagg0.209 inet from 192.168.25.0/24 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.25.0/24 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.89.0/24 to any port = isakmp -> 93.111.250.181 static-port nat on lagg0.209 inet from 192.168.89.0/24 to any -> 93.111.250.181 static-port nat on lagg0.209 inet from 172.16.0.0/22 to any port = isakmp -> 93.111.250.178 static-port nat on lagg0.209 inet from 172.16.0.0/22 to any -> 93.111.250.178 static-port nat on lagg0.209 inet from 192.168.16.0/22 to any port = isakmp -> 93.111.250.178 static-port nat on lagg0.209 inet from 192.168.16.0/22 to any -> 93.111.250.178 static-port nat on lagg0.209 inet from 192.168.12.0/24 to any port = isakmp -> 93.111.250.22 static-port nat on lagg0.209 inet from 192.168.12.0/24 to any -> 93.111.250.22 static-port nat on lagg0.209 inet from 192.168.20.0/24 to any port = isakmp -> 93.111.250.22 static-port nat on lagg0.209 inet from 192.168.20.0/24 to any -> 93.111.250.22 static-port nat on lagg0.209 inet from 192.168.102.0/24 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.102.0/24 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.38.0/24 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.38.0/24 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.26.0/24 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.26.0/24 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.98.0/23 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.98.0/23 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 10.254.252.0/24 to any port = isakmp -> 93.111.250.22 static-port nat on lagg0.209 inet from 10.254.252.0/24 to any -> 93.111.250.22 static-port nat on lagg0.209 inet from 192.168.52.0/23 to any port = isakmp -> 93.111.250.182 static-port nat on lagg0.209 inet from 192.168.52.0/23 to any -> 93.111.250.182 static-port nat on lagg0.209 inet from 192.168.40.0/23 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.40.0/23 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.32.0/22 to any port = isakmp -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.32.0/22 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.10.0/24 to any -> 93.111.250.180 static-port nat on lagg0.209 inet from 192.168.250.0/24 to any -> 93.111.250.180 static-port no rdr proto carp all rdr-anchor "relayd/*" all rdr-anchor "tftp-proxy/*" all rdr on lagg0.209 inet proto tcp from <H_RX_asterisk> to 93.111.250.183 port = https -> <H_vws> round-robin rdr on lagg0.209 inet proto tcp from <H_RX_ip_domain1_net> to 93.111.250.183 port = https -> <H_vws> round-robin rdr on lagg0.209 inet proto tcp from <H_RX_www_domain1_net> to 93.111.250.183 port = https -> <H_vws> round-robin rdr on lagg0.209 inet proto tcp from <H_RX_www_domain2_de> to 93.111.250.183 port = https -> <H_vws> round-robin rdr-anchor "miniupnpd" allI can't find any errors. Might this be a bug?
-
Does somebody have an idea? Otherwise I would open a bug report because there doesn't seem to be configuration errors.
-
So you trying to do static port? Why?? You understand that can't work without issues as soon as you have more than one client behind the nat right... As some point you will have a conflict..
You have how many clients sharing that same public IP? And you want the ports to be static - how does that work, when there is a conflict?
-
@johnpoz
Thank you. I'm aware of this page:
https://docs.netgate.com/pfsense/en/latest/nat/static-port.htmlFor testing I disabled static port for a local subnet that suffers from the problem but I can still see local source IPs leaving WAN.
What's weird is that most of the time it's only NTP that is suffering this problem and most of the times it's only two hosts that are not being NATed.14:14:54.088287 IP 192.168.38.151.2051 > 93.111.248.15.123: NTPv4, Client, length 48 14:14:58.725877 IP 192.168.38.152.2051 > 93.111.248.15.123: NTPv4, Client, length 48 14:15:04.088357 IP 192.168.38.151.2051 > 93.111.248.15.123: NTPv4, Client, length 48 14:15:08.725876 IP 192.168.38.152.2051 > 93.111.248.15.123: NTPv4, Client, length 48 14:15:14.088417 IP 192.168.38.151.2051 > 93.111.248.15.123: NTPv4, Client, length 48 14:15:18.725892 IP 192.168.38.152.2051 > 93.111.248.15.123: NTPv4, Client, length 48 14:15:24.088503 IP 192.168.38.151.2051 > 93.111.248.15.123: NTPv4, Client, length 48 -
And does that source port have any other nats assigned to it, because you were using static? Why did you only turn if off on some of your source networks? That doesn't solve the problem of a conflict when you have 1 IP, with only X number of ports to use, and multiple clients behind the nat picking source ports... At some point they are going to do the same source port... Then what happens? Maybe it goes out un natted..
What you linked too doesn't go into the problem with static ports and multiple clients..
What happens when client X wants to use source port Y, and then client Z also wants to use source port Y for its connection.. How does that work?? You can not maintain that nat..
-
I disabled "static port" for every Manual Outbound NAT except for those ISAKMP entries but still see local IPs on WAN.
-
did you clear the old states?
-
That seems to have worked, thank you so much.
I thought that the combination of Source IP, Destination IP, Source Port and Destination Port were to determine one specific connection but when the answers to my outbound traffic arrive at the firewall there are no internal IP in the headers (of course). -
I have never found the time to do full testing of what "could" or what actually should happen when you force static mappings like that and you run into a conflict..
While your correct a state would have the dest IP, so you could get by with the returning traffic source being different..
But with enough clients you could run into same dest and same source port.. Now your hosed!!
Using static ports is fine when you have a specific 1 application in mind sort of thing... But when you try to do that with a large population of clients for "all" the ports - yeah your at some point going to run into issue..
Glad you got it sorted..
-
We use pfSense for another installation with Sony Playstation clients. Those devices are really stubborn when it comes to port randomization, they just wont work with it. That's the reason why we made it the default which doesn't seem to be a good standard.
Cisco seems to be doing a mix of both with iOS. They use static ports as long as there's no conflict and only if they detect one, they gamble a different source port.
