pfr_update_stats: assertion failed
-
Hi, my system logs are getting flooded with pfr_update_stats: assertion failed errors.
Feb 17 23:29:53 kernel pfr_update_stats: assertion failed. Feb 17 23:29:53 kernel pfr_update_stats: assertion failed. Feb 17 23:29:53 kernel pfr_update_stats: assertion failed. Feb 17 23:29:53 kernel pfr_update_stats: assertion failed. Feb 17 23:29:58 kernel pfr_update_stats: assertion failed. Feb 17 23:29:58 kernel pfr_update_stats: assertion failed. Feb 17 23:29:58 kernel pfr_update_stats: assertion failed. Feb 17 23:29:58 kernel pfr_update_stats: assertion failed.I remember playing with pfBlockerPG a week or two ago, but didn't go much further, so I uninstalled the package. A few days ago I was checking out console and logs and was surprised to see this flooding my logs. Searching through the forums, it does lead to a lot of posts regarding this error.. but most still have pfBlocker running.
I installed pfBlocker again and tried the following...
I tried following this thread and enabling suppression, then force update/reload... no dice. Still floods my logs.
I checked to see if there are any rules for loopback address using the commands below.. nothing showed up.grep "^127\." /var/db/pfblockerng/deny/* grep "^127\." /var/db/aliastables/*Then I removed pfBlocker, also making sure to check the box that deletes all settings. Logs still filled with the same error.
From past few weeks, the only thing I really setup has been another VLAN for a test environment and sending out remote logs to ElasticSearch (That's when I was playing around with pfBlocker). Both I've tried deleting/stopping and errors still persists.
So far the error just show up and don't actually affect any part of my network... but flooding my logs with that error makes it hard to parse any other information.
Appreciate any information on this... searching only turns up pfBlocker, but I've seemingly fully removed it, so I don't see how it can be the issue?
-
@nd-t said in pfr_update_stats: assertion failed:
pfr_update_stats: assertion failed
It does seem to be a bad forwarding rule some where. Can we see what port forwards you have?
Steve
-
Hi,
I haven't changed any of the ports I've forwarded for months so I don't think that would be the issue?Reverse Proxy
Wireguard
Plex/Torrent
PS5 RemotePlay
-
Must be expanding in some odd way perhaps. Can we see the rdr section of the actual ruleset in /tmp/rules.debug?
-
I managed to narrowed it down quite a bit last night and I believe it definitely has to do something with NAT reflection.
I started turning off the forwarded ports one by one, when I got to the 443 rule for 10.0.11.10, the errors stopped right away! I have Pure NAT enabled to hairpin some services to use internally. I tried turning off NAT reflection and the errors stopped. This also caused my services to stop working internally.. I narrowed it down further and it's my Nextcloud instance.
Doing a bunch of further digging, I read that NAT reflection is a bit... 'messy' and isn't the most elegant method as it adds unnecessary connections to the firewall. I turned off NAT reflection and added the FQDN's as local DNS entries pointing to my reverse proxy and everything worked internally again. Would that be the 'proper' way of doing things?
That being said, I'm still mystified by why NAT reflection would cause the flood of errors, so in hopes of helping finding a possible bug (or my own misconfiguration), I'll return to my previous configuration...
- Turn on NAT Reflection, Pure NAT with Enable NAT Reflection for 1:1 NAT and Enable automatic outbound NAT for Reflection checked.
- Enable Pure NAT for the 443 HTTPS rule
- Deleted local DNS entry for my Nextcloud
pfr_update_stats error popped up right away.
RDR section in /tmp/rules.debug prior to NAT Reflection (sanitized my own IP)
# TFTP proxy rdr-anchor "tftp-proxy/*" # NAT Inbound Redirects rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 443 -> 10.0.11.10 rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 80 -> 10.0.11.10 rdr on vmx0 inet proto udp from any to [MY_IP] port 51820 -> 192.168.1.103 rdr on vmx0 inet proto udp from any to [MY_IP] port 123 -> 192.168.1.103 port 51820 rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 32400 -> 10.0.100.12 rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 56881 -> 10.0.100.11 rdr on vmx0 inet proto tcp from any to [MY_IP] port 9295 -> 10.0.110.82 rdr on vmx0 inet proto udp from any to [MY_IP] port 987 -> 10.0.110.82 rdr on vmx0 inet proto udp from any to [MY_IP] port 9295:9297 -> 10.0.110.82 rdr on vmx0 inet proto udp from any to [MY_IP] port 9303:9304 -> 10.0.110.82 # UPnPd rdr anchor rdr-anchor "miniupnpd"RDR section after enabling NAT Reflection
# TFTP proxy rdr-anchor "tftp-proxy/*" # NAT Inbound Redirects rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 443 -> 10.0.11.10 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto { tcp udp } from any to [MY_IP] port 443 -> 10.0.11.10 no nat on vmx1.1001 proto { tcp udp } from (vmx1.1001) to 10.0.11.10 port 443 nat on vmx1.1001 proto { tcp udp } from 10.0.11.0/24 to 10.0.11.10 port 443 -> 10.0.11.1 port 1024:65535 rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 80 -> 10.0.11.10 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto { tcp udp } from any to [MY_IP] port 80 -> 10.0.11.10 no nat on vmx1.1001 proto { tcp udp } from (vmx1.1001) to 10.0.11.10 port 80 nat on vmx1.1001 proto { tcp udp } from 10.0.11.0/24 to 10.0.11.10 port 80 -> 10.0.11.1 port 1024:65535 rdr on vmx0 inet proto udp from any to [MY_IP] port 51820 -> 192.168.1.103 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto udp from any to [MY_IP] port 51820 -> 192.168.1.103 no nat on vmx1 proto udp from (vmx1) to 192.168.1.103 port 51820 nat on vmx1 proto udp from 192.168.1.0/24 to 192.168.1.103 port 51820 -> 192.168.1.1 port 1024:65535 rdr on vmx0 inet proto udp from any to [MY_IP] port 123 -> 192.168.1.103 port 51820 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto udp from any to [MY_IP] port 123 -> 192.168.1.103 port 51820 no nat on vmx1 proto udp from (vmx1) to 192.168.1.103 port 51820 nat on vmx1 proto udp from 192.168.1.0/24 to 192.168.1.103 port 51820 -> 192.168.1.1 port 1024:65535 rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 32400 -> 10.0.100.12 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto { tcp udp } from any to [MY_IP] port 32400 -> 10.0.100.12 no nat on vmx1.100 proto { tcp udp } from (vmx1.100) to 10.0.100.12 port 32400 nat on vmx1.100 proto { tcp udp } from 10.0.100.0/24 to 10.0.100.12 port 32400 -> 10.0.100.1 port 1024:65535 rdr on vmx0 inet proto { tcp udp } from any to [MY_IP] port 56881 -> 10.0.100.11 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto { tcp udp } from any to [MY_IP] port 56881 -> 10.0.100.11 no nat on vmx1.100 proto { tcp udp } from (vmx1.100) to 10.0.100.11 port 56881 nat on vmx1.100 proto { tcp udp } from 10.0.100.0/24 to 10.0.100.11 port 56881 -> 10.0.100.1 port 1024:65535 rdr on vmx0 inet proto tcp from any to [MY_IP] port 9295 -> 10.0.110.82 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto tcp from any to [MY_IP] port 9295 -> 10.0.110.82 no nat on vmx1.110 proto tcp from (vmx1.110) to 10.0.110.82 port 9295 nat on vmx1.110 proto tcp from 10.0.110.0/24 to 10.0.110.82 port 9295 -> 10.0.110.1 port 1024:65535 rdr on vmx0 inet proto udp from any to [MY_IP] port 987 -> 10.0.110.82 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto udp from any to [MY_IP] port 987 -> 10.0.110.82 no nat on vmx1.110 proto udp from (vmx1.110) to 10.0.110.82 port 987 nat on vmx1.110 proto udp from 10.0.110.0/24 to 10.0.110.82 port 987 -> 10.0.110.1 port 1024:65535 rdr on vmx0 inet proto udp from any to [MY_IP] port 9295:9297 -> 10.0.110.82 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto udp from any to [MY_IP] port 9295:9297 -> 10.0.110.82 no nat on vmx1.110 proto udp from (vmx1.110) to 10.0.110.82 nat on vmx1.110 proto udp from 10.0.110.0/24 to 10.0.110.82 -> 10.0.110.1 port 1024:65535 rdr on vmx0 inet proto udp from any to [MY_IP] port 9303:9304 -> 10.0.110.82 # Reflection redirect rdr on { vmx1 vmx1.3101 vmx1.100 vmx1.1000 vmx1.1001 vmx1.101 vmx1.3000 vmx1.110 vmx1.4000 } inet proto udp from any to [MY_IP] port 9303:9304 -> 10.0.110.82 no nat on vmx1.110 proto udp from (vmx1.110) to 10.0.110.82 nat on vmx1.110 proto udp from 10.0.110.0/24 to 10.0.110.82 -> 10.0.110.1 port 1024:65535 # UPnPd rdr anchor rdr-anchor "miniupnpd"Wow... NAT reflection really does add a whole bunch of routes... I hope this helps, let me know if there's anything else needed!
-
Yeah, split DNS is definitely preferred over NAT reflection if you can use it.
When we've seen this error before it was caused by something incorrectly expanded into the ruleset, like interpreting a port number as an IP address or an invalid IPv6 address. But I'm not seeing that here. It was definitely only the first forward there that triggered it?
Steve
-
I'm fairly certain it's only that first forward there.
I left all other rules enabled and turned off that one, the error stops. I turn on that rule and errors start right away.
If I turn off NAT Reflection on that rule, errors stop.
Also note that if I turn off my NextCloud VM, the errors stops... so it seems something funky is going on with Nextcloud, my reverse proxy, and when NAT reflection is on.
-
Hmm, Nextcloud adding forwards with UPnP?
