Firewall rules hitcount for pfSense 2.1.5 and 2.2.4



  • @jimp:

    Why use a hash there? In 2.2.x all rules have a unique tracker ID already, no need for an extra hash.

    Hi Jimp, Thank's for the feedback. I'm testing with tracker right now.

    It looks like rules created by nat does not have this tracker(testing with 2.2.4).

    Using tracker field, it reduced to just one new file and two patches.

     <rule><source>
                                    <any><interface>wan</interface>
                            <protocol>tcp</protocol>
                            <destination><address>127.0.0.1</address>
    
                                    <port>80</port></destination> 
    
                            <associated-rule-id>nat_55d21c4e8742b7.64890792</associated-rule-id>
                            <created><time>1439833166</time>
                                    <username>NAT Port Forward</username></created></any></rule> 
    
    


  • I've updated the code for 2.2.4 to use tracker and included counter to all rules on gui. See the topic above for instructions.



  • Thanks, works great 2.2.4.
    One question though, shouldn't we rather use 644 permissions? At least for /etc/inc files ?


  • Banned

    @athurdent:

    One question though, shouldn't we rather use 644 permissions? At least for /etc/inc files ?

    Yeah, you should. Also, you can diff the new file against /dev/null, put it into System Patches and forget about Filer altogether.



  • @athurdent:

    One question though, shouldn't we rather use 644 permissions? At least for /etc/inc files ?

    Updated to 0644, thanks for the feedback.



  • Something seems odd with pfSense handling auto-added Port Forward rules. They get a name like "USER_RULE: NAT …" and there's no tracker ID. As I use Multi-WAN and some Port Forwards were just copied from WAN1 to WAN2, pfctl -vvsr shows two rules with the same name. That breaks the new counters...
    Edit: Put in unique descriptions, still no luck.



  • On to the next problem, Port Aliases. While pf let's us write single-line rules with something like

    port {  25  465  587 }
    

    it automatically creates a separate rule for every single port. pfctl will show three rules for the example above whilst our ruleset only has a single rule for this.



  • Updated the code for 2.2.4 to include kill states option for specific rule.




  • @athurdent:

    On to the next problem, Port Aliases. While pf let's us write single-line rules with something like

    port {  25  465  587 }
    

    it automatically creates a separate rule for every single port. pfctl will show three rules for the example above whilst our ruleset only has a single rule for this.

    I'll try to simulate it.



  • @marcelloc:

    Updated the code for 2.2.4 to include kill states option for specific rule.

    Wasn't able to try it out yet. But if that means, the byte counter does not show anymore when hovering there with the mouse, that would be too bad. Maybe the Hits field could show
    Packets(Bytes)/States
    or something similar?



  • @athurdent:

    Wasn't able to try it out yet. But if that means, the byte counter does not show anymore when hovering there with the mouse, that would be too bad. Maybe the Hits field could show
    Packets(Bytes)/States
    or something similar?

    I'ts still working inside the td with mouse over, I've just included the option to kill when there are active states and only over the numbers.



  • @athurdent:

    Something seems odd with pfSense handling auto-added Port Forward rules. They get a name like "USER_RULE: NAT …" and there's no tracker ID.

    Edit and save rule created by nat to force the trackerid



  • @athurdent:

    On to the next problem, Port Aliases. While pf let's us write single-line rules with something like

    port {  25  465  587 }
    

    it automatically creates a separate rule for every single port. pfctl will show three rules for the example above whilst our ruleset only has a single rule for this.

    check if new code fixes it. (v0.31)

    @78(1439883226) pass in quick on em0 reply-to (em0 172.17.12.1) inet proto tcp from any to (self:3) port = http flags S/SA keep state label "USER_RULE: 1439883226"
      [ Evaluations: 17        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: pid 23535 State Creations: 3319624344]
    @79(1439883226) pass in quick on em0 reply-to (em0 172.17.12.1) inet proto tcp from any to (self:3) port = https flags S/SA keep state label "USER_RULE: 1439883226"
      [ Evaluations: 17        Packets: 763       Bytes: 442121      States: 6     ]
      [ Inserted: pid 23535 State Creations: 3319624408]
    @80(1439883226) pass in quick on em0 reply-to (em0 172.17.12.1) inet proto tcp from any to (self:3) port = ssh flags S/SA keep state label "USER_RULE: 1439883226"
      [ Evaluations: 1         Packets: 555       Bytes: 81812       States: 1     ]
      [ Inserted: pid 23535 State Creations: 3321614424]
    @81(1439883226) pass in quick on em0 reply-to (em0 172.17.12.1) inet proto tcp from any to (self:3) port = 8443 flags S/SA keep state label "USER_RULE: 1439883226"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: pid 23535 State Creations: 3321614400]
    @82(1439883226) pass in quick on em0 reply-to (em0 172.17.12.1) inet proto tcp from any to (self:3) port = domain flags S/SA keep state label "USER_RULE: 1439883226"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: pid 23535 State Creations: 3321614376]
    
    




  • Thanks, works great.
    But one odd thing I noticed is, that rules with multiple ports like those you fixed seem to get their counter cleared on every filter reload. At the same time, my other "normal" rules do not suffer from this.
    Are you seeing this too, or am I doing something wrong?



  • More features on version 0.4 for pfsense 2.2, now we can view hit count, list and kill States(with ajax to keep it light and fast).






  • Very nice!

    Good job as always, marcelloc!

    []`s
    Jack



  • marcelloc, the lastest update works great, thanks again.

    Still, the new hitcount feature reveals that some rules with multiple ports like "{  25  465  587 }" or multiple protocols like "{ tcp udp }" get their packet/byte counter reset on reload. Evaluations / State Creations seem not affected. Nobody else seeing this?



  • Awesome feature!

    Can't wait to see this merged upstream!  ;D



  • Good job.
    Test-driving it on a 2.2.4. Works as advertised.

    Git->Clone->Patch->Push-request this  :)



  • +1 thanks Marcello, really useful feature.
    Love to see that merged also  ;D



  • Pull request sent do 2.3-DEVELOPMENT

    https://github.com/pfsense/pfsense/pull/1892



  • Very, very nice.

    (Also: One more column of info feels like a step toward widescreen display!)



  • I suggest all who cant test to do it on 2.2 and/or 2.3-devel and comment on pull request. Maybe with more people testing, it get merged faster…



  • @marcelloc:

    I suggest all who cant test to do it on 2.2 and/or 2.3-devel and comment on pull request. Maybe with more people testing, it get merged faster…

    Done.

    The more I play with this the more I like it.
    On mouseover, I get the data I was trying (unsuccessfully) to glean out of pfTop.



  • @LinuxTracker:

    The more I play with this the more I like it.

    me too.  :)

    @LinuxTracker:

    On mouseover, I get the data I was trying (unsuccessfully) to glean out of pfTop.

    Great!



  • Send a new pull request with almost all code working on 2.3 with bootstrap.

    I'll need some help to adjust the code to popup traffic on rule click.
    The code is there but is not visible after ajax return.

    https://github.com/pfsense/pfsense/pull/1901



  • I noticed that 2.3 is nearing Beta stage. Has your great improvement been merged with 2.3(and approved)?



  • @brandur:

    I noticed that 2.3 is nearing Beta stage. Has your great improvement been merged with 2.3(and approved)?

    unfortunately no. :(



  • @marcelloc:

    @brandur:

    I noticed that 2.3 is nearing Beta stage. Has your great improvement been merged with 2.3(and approved)?

    unfortunately no. :(

    That is just very sad news  :'(
    Was there any particular reason it didn't "make it"?

    (Then the question becomes. Are you going to try to get it committed for a higher version?)



  • Hmm, not nice because this was a super no brainer feature that was very very helpful  >:(
    I see the pull request has a "CLA label". I have no idea for what CLA stands in this case  :-[ Could someone shed a light here?



  • I see the pull request has a "CLA label". I have no idea for what CLA stands in this case  :-[ Could someone shed a light here?[/quote]
    That means that the contributor has correctly completed the relevant licensing agreement. So that is a good thing.

    From the comments on https://github.com/pfsense/pfsense/pull/1901 it seems that there is some thought to add some support in binaries to make it more efficent to do. But for some reason progress in those comments stops in late Sep 2015.



  • Cool feature  ;)

    However on 2.2.6 x32, with pfBlockerNG, it does break pftop/Label
    Before it was : USERRULES: pfB_PR, after patching it shows :USERRULES: 1770001532

    and the Status: System logs: Firewall Rule column
    Instead of displaying pfB_PRI3 auto rule (1770001532) it shows 1770001532 (1770001532)



  • The trackerid is used to count rules match.
    If it get merged one day we will need to change the way pfblocker get his rules.



  • @bennyc:

    Hmm, not nice because this was a super no brainer feature that was very very helpful  >:(

    I feel for marcelloc and everyone who finds this useful, as I know this is not the first time he's tried to get this merged.

    I feel too for the core developers, as they face a difficult balance between trying to cram in extra features and trying to get 2.3 released as soon as possible. There seems now to be a real determination to get 2.3 released so that there is no need to revisit 2.2.x any further.

    Based on a discussion I started in the 2.3 forum, I believe any new features or major changes have now missed the cut for 2.3. The RFC 4638 support I contributed was close to missing the cut, and only made it because it was a complete implementation, had no conflicts with the master branch and caused no regressions.

    Hopefully, once 2.3 has released, there will be opportunity for this to be revisited by the necessary people.

    Meanwhile, if marcelloc fixed the conflict(s) with master, those who find this useful could install this via System Patches (add .diff to the end of the pull request URL in a web browser, then create a patch using the URL that is shown in your web browser with a base directory of / and a path strip count of 2). This won't work until the conflict(s) are fixed.


  • Moderator

    @marcelloc:

    The trackerid is used to count rules match.
    If it get merged one day we will need to change the way pfblocker get his rules.

    pfBlockerNG is already using tracker IDs…

    The issue with the current "Rule Count" code, is that its modifying the Description field in certain coditions which removes the human-readable text.... So I don't think the code in the pfBNG package needs to be changed. Let me know if you see it differently and I will consider making changes to the pfBNG code.

    Thanks



  • Great. But first the code merge :)



  • @marcelloc:

    The trackerid is used to count rules match.
    If it get merged one day we will need to change the way pfblocker get his rules.

    But the patches changes things in "Status: System logs: Firewall" and "pftop/Label", not the pfBlockerNG tabs.


  • Netgate

    I'm working to merge this request, but in a way it doesn't break any of existing code.

    We'll have this for 2.3.

    Thanks for this great contribution.



  • @loos:

    I'm working to merge this request, but in a way it doesn't break any of existing code.

    We'll have this for 2.3.

    Thanks for this great contribution.

    Sounds great, thanks !  :) :)




Log in to reply