Delete state, Reject & Block rules work perfectly fine
-
2.2.2 AMD 64 (Latest release version)
Set up a pinger to ping something continuously, in my case I was using the wan IP address assigned by the ISP to my bridged modem.
Initially the default block rule is in effect and will block the pinger.
#1
Add rule:
Action: pass
TCP/IP Version IPv4
Protocol Any (havent tried anything more restrictive}
Source Any
Destination WAN net
Log: TickedSave rule & click Apply Changes button.
Pinger starts getting TTL's back and logs show ping is passing through in the fw logs
Now repeat #1 but change the rule from pass to block, save and click Apply Changes Button.
Pinger still getting TTL's back and still shows ping passing through in the fw logs.#2
Open up Diagnostics:Show States webpage, locate ICMP state with corresponding IP address of device running the pinger and delete the state. Press F5 to reload webpage and the state reappears and the pinger still shows TTL's without any delays or other impedence expected from a block rule.Repeat #1 as above but change the rule from Pass to Reject, pinger still returns TTL's without any problems.
Repeat #2 as above, locate & delete the state and still pinger returns TTL's.Only by clicking on the Reset Button in Diagnostics:Show States, Reset States tab will the reject or block rule take effect.
But then try this for #1.
Using the a Reject or Block rule working properly ie its blocking the pinger, change it to Pass and the Pinger will start working after you click Aply Changes, returning a TTL and the fw log will show the ping passing through as well.
At the moment it seems only by resetting all states (Reset Button in Diagnostics:Show States, Reset States tab) or rebooting pfsense after any reject or block rule can you be a little surer the rules are working properly, unless you repeat a myriad of tests (pen tests) to see which rules are working and which are not, or check the fw logs as carefully as possible considering the restrictions the fw logs have when lots of data is passing around.
[Edit]
So why does the Delete State not work when testing a Reject or Block rule?
SO why does the Delete State not reload the loaded Reject or Block rule?
[/Edit]Why does the Apply Changes work for Pass rules but not Reject or Block rules?
-
Confirmed.
-
Additional test
s.Create a Pass Rule that is initially disabled.
Pinger is blocked by default deny rule.
Enable the Pass rule, Apply Changes and pinger works returning TTL's.
Disable the pass rule, Apply Changes and pinger still returns TTL's.Resetting State table stops pinger returning TTL's.
So the Enable & Disable rule doesnt work as expected either.
I have not tested earlier versions of pfsense but I think I have found out why I kept being hacked, which those who just set rules and dont test or monitor the logs will not know about, thus putting their customers/users at risk.
-
Confirmed in 2.1.5 as well.
-
Wonder if this could be spread to other basic functions of pfsense??
Handling packets, blocking and response?
-
I'm about to test a long free film from youtube and let it run all the way this time as the first test I did using youtube stopped the rules from reloading as it didnt stop the youtube, however the youtube file does have a buffer of a few minutes worths I need to leave this running for a longer time to be sure as I could still see data coming in from the wan, but the fw log only shows the rule match once for a session.
A simple webpage is no enough data and so a socket session (windows speak if its different in Linux) is what I suspect is stopping the rules from working properly. A film will keep a long socket session open, an online newspaper wont be enough data to keep the socket open for long and javascript can open new sockets as can Ajax, plus the browser is essentially stateless anyway unlike Pinger which will run a session until its stopped, so webpages are harder to test, but so far I have seen https from youtube delay the loading of rules for a few minutes.
-
Well changing the rules to block, reject, allow and alot of other settings like restricting source and destination with an accompanying click of the Apply Changes button did nothing for the 2hrs 20mins the youtube film ran for.
With regard to states, deleting the outward bound state did nothing either for all the various rule changes I had tried.
The only time deleting a state stopped the film from playing was when I deleted the inward bound state and then the rules work as expected, but lets face it who goes through their state table and deletes the relevant state?
So thats the low down so far.
Its been confirmed in 2.1, I wonder if this problem goes back further than 2.1 and I also wonder if its something exclusive to pfsense or something in FreeBSD which will affect other firewalls also using FreeBSD as its OS, and/or is it related to a package being used by pfsense which ports to other Linux OS's.
Potentially there could be an awful lot of firewalls from various suppliers & not just pfsense affected by this for this simple reason:
Killing the outward bound state doesnt let the rules work as expected. With this in mind, any malware on an internal system could initiate a state which is kept open, perhaps even the Keep Alive in http(s) might do the trick ( I havent tested yet)?
One obvious way is by monitoring some API's in the OS, could be enough for any malware/botnet/whatever-you-want-to-call-it on a system behind the firewall to initiate a state kept open before downloading new instructions/payload in which to evade further capture from AV & AntiMalware software and/or carry out any other nefarious operation, but do bear in mind it can take weeks/months/years before some unknown communicating software is reverse engineered and considered good or bad, just look at how long it took AV companies to find Stuxnet and Flame for some examples.
Anyway workaround is to reset the entire state table after applying any rules changes just to increase your piece of mind.
Off to test some other firewalls and versions now to see how big this bug is.
-
Confirmed issue in OPNSense latest version as well.
-
2.2.2 AMD 64 (Latest release version)
Set up a pinger to ping something continuously, in my case I was using the wan IP address assigned by the ISP to my bridged modem.
Initially the default block rule is in effect and will block the pinger.
#1
Add rule:
Action: pass
TCP/IP Version IPv4
Protocol Any (havent tried anything more restrictive}
Source Any
Destination WAN net
Log: TickedSave rule & click Apply Changes button.
Pinger starts getting TTL's back and logs show ping is passing through in the fw logs
Now repeat #1 but change the rule from pass to block, save and click Apply Changes Button.
Pinger still getting TTL's back and still shows ping passing through in the fw logs.What was running a "pinger"? Which interface was it connecting through? Which interface is the rule added to?
I assume the rules on your LAN interface and pinging from a network device and attempting to ping your WAN IP.
So the issue seems to be adding a block rule for ICMP then manually clearing the individual states doesn't seem to stop the ping from still getting through?
I wonder if it has to do with how ICMP "states" are being handled. ICMP is stateless protocol with no ports. The network stack may be misrepresenting what a state really is for ICMP just to keep the format the same.
Try setting your ping to only ping once every 10 seconds, giving you enough time to delete all of the related states. It could be that as long as ANY ICMP state for the target rule exists, the firewall can't tell the difference. The default ping rate of once per seconds may not be enough time to let you delete all relevant states.
Summary is my guess is this is something specific to ICMP and as long as any related ICMP state still exists, it'll continue to let "new States" to be created.
-
Even if you delete the rule, it keeps responding to ICMP.
-
Is ICMP handled in the networking stack before handing packets off to PF or anything that has hooks in?
-
Its a network layer protocol so IMHO it should be.
-
Everyone knows deleting/changing rules doesn't kill existing states. Not a bug.
-
@mer:
Is ICMP handled in the networking stack before handing packets off to PF or anything that has hooks in?
ICMP can be blocked, so it has to go through PF.
Even if you delete the rule, it keeps responding to ICMP.
This is a guess, but this is what I was getting after.
With TCP or UDP, it's not an issue because all states are a combination of IP and port and destroying a state actually destroys the flow. ICMP may internally map multiple flows through the same state and make it look like there are multiple states when there is really only one. You might be destroying pseudo-states, but not actually destroying the underlying state, which means new flows can be created. Resetting the state table forces all states to be destroyed, so it's not an issue.
My guess is if you do a "ping -t 8.8.8.8", then create a rule to block pings to 8.8.0.0/16, then attempt to ping 8.8.4.4, you won't be able to ping 8.8.4.4, but will be able to continue pinging 8.8.8.8 because a state already exists. But making "new" pings to 8.8.8.8 will work because of the magic happening that makes ICMP look like it actually has ports.
PF represents ICMP states in a strange way that looks like they have ports. My guess is it has something to do with how ICMP does not have a real layer4 and ICMP states are misrepresented to make you think you're destroying the underlying state.
LAN icmp 8.8.8.8:11309 <- 192.168.1.2:11309 0:0
LAN icmp 8.8.8.8:1 <- 192.168.1.2:1 0:0Since ping defaults to every one second, if you destroy one of those states, but not both, with in one second, a new ping will go out and create a new pseudo state. Try setting ping to every 10 seconds, giving you enough time to destroy all of the ICMP states before another ping goes out.
I'd test it out myself, but I once made changes to the firewall that I forgot to apply, and a long while later created some other rules, applied them and broke the Internet for a few moments. Wife was furious because she was in the middle of something. Well, she's been in the middle of something non-stop, so I can't risk testing myself.
-
But wouldnt it be great if the script actually took all states with either DST or SRC and killed every last one of them when you pressed apply when you change the rule?
I think thats what we actually want to happen when doing so.
Everyone knows deleting/changing rules doesn't kill existing states. Not a bug.
-
It doesn't do that. That would be a new feature request, not a bug.
-
i don't think i would want it to kill related states when creating a new rule …. this "feature' has saved me a couple of times when i did something stupid. (not locking myself out when i made a typo)
anyhow, i prefer to clear the states myself
-
Summary is my guess is this is something specific to ICMP and as long as any related ICMP state still exists, it'll continue to let "new States" to be created.
Its not just ICMP, HTTPS which I have tested with the long youtube film that ran for over 2hrs 20mins despite various rules was also affected. As its WAN states that need to be deleted the recent change to unbound/ DNS resolver ie the change to the DNS service also means lots of WAN states kept open which means lots of firewall rules wont be updated. I've got over 2000 wan states open to various name servers since I started this thread all of a sudden and my CPU load has gone up to just under 50% load which will further prevent any rule changes from working properly unless I reset all states.
Everyone knows deleting/changing rules doesn't kill existing states. Not a bug.
I didnt and I dont see any mention of deleting states in numerous online blogs detailing how to do XYZ, nor is it in the pfsense book and I dont see it mentioned often in the forums either. So whether "everyone" knows is a matter of opinion.
With ISA server we used to just restart the service which had the same effect of deleting the states, but if anything the Apply Changes button in the pfsense gui is a little misleading imo.
I will say that in the Reset States webpage in the gui, there is some text which says it may be neccessary to reset the states so if anything that text could also be reiterated on the rules webpage for safety sake although not my preferred option which is below.
i don't think i would want it to kill related states when creating a new rule …. this "feature' has saved me a couple of times when i did something stupid. (not locking myself out when i made a typo)
anyhow, i prefer to clear the states myself
A new rule perhaps, but if you change an existing rule or even simply disable/enable an existing rule what then? Jump through some more hoops just to get the change to work, not everyone gets to charge by the hour.
And what if you have hundreds or thousands of states to delete one by one, it could be a never ending process because as you delete one state another could pop up from another device when a network and various devices have all been compromised with malware, ergo you never get to delete ALL the states so the new rules can never take effect.
So this leads you to one course of action, resetting all states in one go, which then renders the argument for a new rule perhaps mute point.
I've done similar but with multiple windows programs that check for the existence of each other, if one is not running, one of the running apps fires it up, if its been deleted it restores it, so not only can you not stop the various apps from running you cant even delete them, a similar technique is used with malware & viruses which you may or may not have had the pleasure of trying to fix for customers.
Besides there is also the point of inconsistent behaviour, why does deleting outward bound states not work, but deleting inbound states does work?
Inconsistent behaviour and misleading wording is the undoing of any good system.
But wouldnt it be great if the script actually took all states with either DST or SRC and killed every last one of them when you pressed apply when you change the rule?
I think thats what we actually want to happen when doing so.
It doesn't do that. That would be a new feature request, not a bug.
How about this for a feature request. Two or three buttons that pop up when a rule change has occurred which can be:
1. Apply Changes - No State changes
2. Apply Changes - reset only affected states
3. Apply Changes - reset all states.I would suggest it would be easiest and quickest to do buttons 1 & 3 for now because the code already exists in the existing Apply Changes button and the same for Reset Button in Diagnostics:Show States, Reset States tab so I doubt it would be too time consuming or difficult to merge the code together.
At least this way the gui becomes a reminder for breaking old habits and improves the experience for novice users who want to use pfsense in anger. I know CLI's have their fans, but Apple havent got to where they are to day by ignoring the gui experience.
I still havent got to the bottom of where this problem stems from though, as its been confirmed in OPNsense we still dont know if this is restricted to freebsd or a package used by freebsd which means other firewalls in other OS which use the package have this "backdoor" weakness, but might explain why the NSA were recommending opensource firewalls a while back if they have had their grubby mits over some code in advance.
Edit.
One other thought, is if the rules are only reset when the inward bound wan state is deleted, what if you are using an OPTx for an additional internet connection?
Do deleting the states not work for those interfaces as this appears to be the case currently when deleting outward bound states on OPTx interfaces. This could potentially be a bigger headache still unless all states are reset after every rule change. -
Summary is my guess is this is something specific to ICMP and as long as any related ICMP state still exists, it'll continue to let "new States" to be created.
Its not just ICMP, HTTPS which I have tested with the long youtube film that ran for over 2hrs 20mins despite various rules was also affected. As its WAN states that need to be deleted the recent change to unbound/ DNS resolver ie the change to the DNS service also means lots of WAN states kept open which means lots of firewall rules wont be updated. I've got over 2000 wan states open to various name servers since I started this thread all of a sudden and my CPU load has gone up to just under 50% load which will further prevent any rule changes from working properly unless I reset all states.
The original discussion was about how the block rule was not taking effect and allowing new ICMP states. Now that the discussion is changing into "how to kill existing states for a block rule", the topic has changed.
I think they already have a script for this and use it for scheduled rules. Maybe we just need a "kill all states" button under the edit window screen, allowing you to kill existing states for the current rule.
-
Its simple. When changing one specific rule, all states for that rule should be killed. On all interfaces.
Only way the integrity of the firewall is kept.
