Duplicated firewall rules
-
Hi,
I have the following problem with my pfsense 2.2.6:
2.2.6-RELEASE (amd64)
built on Mon Dec 21 14:50:08 CST 2015
FreeBSD 10.1-RELEASE-p25- dual WAN (load balancing)
- multiple vlan interfaces on the lan side
firewall rules from both wan interfaces are multiplied by at least 1000 times (4-5 rules multiplied 1000 times)
how can delete the duplicated rules from both interfaces without reinstall? the pfsense is in production and I cannot afford a downtime for the moment.
any useful help needed.
Liviu
-
Select all the ones you want to delete and click the X at the bottom of the screen.
Any idea how you ended up with duplicated rules?
-
@cmb:
Any idea how you ended up with duplicated rules?
Indeed. That's what would worry me more than the task of removing the duplicates.
-
Wonder if someone held down F5 and re-posted a creation event.
-
@cmb:
Select all the ones you want to delete and click the X at the bottom of the screen.
Any idea how you ended up with duplicated rules?
hi, same problem duplicated multiple rules 512 times, config file xml has 110.000 lines , when accesing webgui php-fpm goes 100% on cpu and timeout…is there a way to delete rules from ssh ....or restore good config without webgui ?
thanks
-
+1
My System
pfSense CE v2.2.3
FreeBSD gateway.mlan.local 10.3-RELEASE-p5 FreeBSD 10.3-RELEASE-p5 #0 7307492(RELENG_2_3_2): Tue Jul 19 13:29:35 CDT 2016 root@ce23-amd64-builder:/builder/pfsense-232/tmp/obj/builder/pfsense-232/tmp/FreeBSD-src/sys/pfSense amd64Packages Installed
- Avahi
- Squid
- Lightsquid
- pfBlockerNG
Observations
- The number of rules listed against the WAN interface grew to 1000+, causing php-fpm to peg at 100% CPU usage any time a config change was made in the web-gui
- A free install of pfSense and a re-load of the old config caused the issue to re-occur (config had 1000+ rules)
- After removing all of the duplicates, re-installing pfSense and then loading the 'cleaned' config the issue re-occurs
- 9am: pfSense re-installed (35 rules - counted using 'grep -c tracker /conf/config.xml')
- 6pm: Now 155 rules (all dupes in the WAN section - counted using 'grep -c tracker /conf/config.xml')
- 9am following day: Now 283 rules (all dupes in the WAN section - counted using 'grep -c tracker /conf/config.xml')
- The duplicates all have the same <tracker>values as their originals
- The multiplier of duplicates I get is always a multiple of 2 (32, 64, 256, 512, 1024, etc)
grep tracker /conf/config.xml | sort | uniq -c | sort -n | tail -10 1 <tracker>1455574900</tracker> 1 <tracker>1470906216</tracker> 32 <tracker>1422442622</tracker> 32 <tracker>1422442623</tracker> 32 <tracker>1447961159</tracker> 32 <tracker>1447961390</tracker> 32 <tracker>1447961411</tracker> 32 <tracker>1448888848</tracker> 32 <tracker>1448888874</tracker> 32 <tracker>1448888893</tracker>Theories Tested:
- Thought it might be a browser issue or plugin. Tried with Firefox, Chrome and event 'Lynx' - same issue
- Thought it might be pfBlockerNG. Disabled it. Issue still occurs
Sample Rule:
<rule><id><tracker>1448888893</tracker> <type>pass</type> <interface>wan</interface> <ipprotocol>inet</ipprotocol> <tag><tagged><max><max-src-nodes><max-src-conn><max-src-states><statetimeout><statetype>keep state</statetype> <os><protocol>tcp</protocol> <source> <any><destination><address>192.168.1.40</address> <port>1234</port></destination> <updated><time>1448888893</time> <username>admin@192.168.1.254</username></updated> <created><time>1448888893</time> <username>admin@192.168.1.254</username></created></any></os></statetimeout></max-src-states></max-src-conn></max-src-nodes></max></tagged></tag></id></rule> ```</tracker> -
For anyone having the same issue you can temporarily fix the issue (and cleanup the rules) using the following steps if you access to a Linux box
You can install the necessary utils on Ubuntu with "sudo apt-get install xgrep libxml2-utils"
1. Build a list of unique rules in your config
xmllint --c14n --noblanks config-<date>.xml \ | xgrep -x '/pfsense/filter/rule' -t \ | sort \ | uniq \ | xmllint --c14n --noblanks -</date>2. Replace all the text between the tags in your config file with the output from the command above and save it for future use (eg. config-cooked.xml)
3. Using the Diagnostics -> Backup & Restore menu. Select your 'cooked' config and load only the Firewall Rules section from the config
-
You can also determine how many instances of the duplicate rules you have with the command below;
grep tracker /conf/config.xml | sort | uniq -c | sort -n | tail -15No Problems Here!
[root@gateway ~]# grep tracker /conf/config.xml | sort | uniq -c | sort -n | tail -15 1 <tracker>1470906216</tracker> 1 <tracker>1770008480</tracker> 1 <tracker>1770008555</tracker> 1 <tracker>1770008584</tracker> 1 <tracker>1770008659</tracker> 1 <tracker>1770008755</tracker> 1 <tracker>1770008797</tracker> 1 <tracker>1770008859</tracker> 1 <tracker>1770008861</tracker> 1 <tracker>1770008901</tracker> 1 <tracker>1770008941</tracker> 1 <tracker>1770008946</tracker> 1 <tracker>1770008965</tracker> 1 <tracker>1770009045</tracker> 1 <tracker>1770009050</tracker>Time to Fix the Rules!
[root@gateway ~]# grep tracker /conf/config.xml | sort | uniq -c | sort -n | tail -15 1 <tracker>1770008901</tracker> 1 <tracker>1770008941</tracker> 1 <tracker>1770008946</tracker> 1 <tracker>1770008965</tracker> 1 <tracker>1770009045</tracker> 1 <tracker>1770009050</tracker> 512 <tracker>1422442622</tracker> 512 <tracker>1422442623</tracker> 512 <tracker>1422442625</tracker> 512 <tracker>1447961159</tracker> 512 <tracker>1447961390</tracker> 512 <tracker>1447961411</tracker> 512 <tracker>1448888848</tracker> 512 <tracker>1448888874</tracker> 512 <tracker>1448888893</tracker> -
Possibly Fixed with the complete removal of pfblockerNG
- I've upgraded to v2.3.2_1 [https://blog.pfsense.org/?p=2122]
- I then un-installed pfBlockerNG
- Restored my Firewall Rules from 'good' config
- Removed the pfBlockerNG rules (no aliases any more) from Firewall
- Reboot.
No re-occurrences in 9hrs and counting (always had at least 1 occurrence after a reboot)
-
Hi, I have the exact same issue. I can't delete the rules form the UI as CPU is 100% and impossible to access the Firewall Rules configuration page.
First How can I delete the duplicates and end how can I prevent that from re-occuring?Thanks
-
you can't
2.3.2 eol since October 31, 2018.
you need to install 2.4.5-p1 if your system is 64bit capable otherwise it's time to upgrade you hardware
New vulnerabilities are discovered continually, so the longer an unsupported release is in use, the more likely it is to be affected
