PfBlockerNG


  • Moderator

    Arbor Networks Feeds discontinued:
    https://forum.pfsense.org/index.php?topic=122237.0



  • I know that this thread is a little older now.
    I utilized the php script that Dok presented.
    Should I go through and delete the failed downloads or what?
    [ pfB_MAIL - ToastedSpam ] Download FAIL [ 03/19/17 18:50:22 ] 
    [ pfB_MAIL - VirBL ] Download FAIL [ 03/19/17 18:45:01 ]
    [ pfB_SEC1 - MalwareGroup ] Download FAIL [ 03/19/17 18:35:56 ]
    [ pfB_PRI3 - Geopsy ] Download FAIL [ 03/19/17 18:35:33 ]
    [ pfB_PRI3 - VMX ] Download FAIL [ 03/19/17 18:35:31 ]
    [ pfB_PRI3 - DangerRulez ] Download FAIL [ 03/19/17 18:35:03 ]
    [ pfB_PRI2 - HoneyPot ] Download FAIL [ 03/19/17 18:34:41 ]
    [ pfB_PRI1 - dShield_Block ] Download FAIL [ 03/19/17 18:33:09 ]
    [ pfB_PRI1 - Abuse_Spyeye ] Download FAIL [ 03/19/17 18:32:51 ]
    [ pfB_MAIL - URIBL ] Download FAIL [ 03/19/17 18:29:37 ]
    [ pfB_MAIL - ToastedSpam ] Download FAIL [ 03/19/17 18:29:35 ]
    [ pfB_MAIL - VirBL ] Download FAIL [ 03/19/17 18:25:05 ]
    [ pfB_SEC1 - MalwareGroup ] Download FAIL [ 03/19/17 18:15:43 ]
    [ pfB_PRI3 - Geopsy ] Download FAIL [ 03/19/17 18:15:03 ]
    [ pfB_PRI3 - VMX ] Download FAIL [ 03/19/17 18:14:55 ]
    [ pfB_PRI3 - DangerRulez ] Download FAIL [ 03/19/17 18:14:17 ]
    [ pfB_PRI2 - HoneyPot ] Download FAIL [ 03/19/17 18:13:51 ]
    [ pfB_PRI1 - dShield_Block ] Download FAIL [ 03/19/17 18:11:35 ]



  • I went through each one that failed, retried them, and disabled the ones that failed again,
    Then I attempted to go to each site, and if the site was no longer in existence, then I deleted.
    Answered my own question without getting yelled at!


  • Moderator

    @mtarbox:

    I went through each one that failed, retried them, and disabled the ones that failed again,
    Then I attempted to go to each site, and if the site was no longer in existence, then I deleted.
    Answered my own question without getting yelled at!

    The script is a little dated. Some feeds have changed…. some down... so always best to open the site in a browser and check it out....

    The next version of the package will have an automated Feeds Management Tab, which should make this process easier to manage.



  • The next version of the package will have an automated Feeds Management Tab, which should make this process easier to manage.

    +1 from me for that! This might be a cool rich Option for us all, to get taps named like Malware, Virus, ect.
    and insert there the links or URLs to several lists and perhaps a little "+" button to create his own new tap!
    Perhaps I mean ;)



  • Dropbox is block.  NET::ERR_CERT_AUTHORITY_INVALID

    grep result.

    grep "dropbox.com" /var/unbound/pfb_dnsbl.conf
    local-data: "dl.dropbox.com 60 IN A 10.10.10.1"
    local-data: "dropbox.com.azamgold.com 60 IN A 10.10.10.1"
    local-data: "dropbox.com.paki-library.com 60 IN A 10.10.10.1"
    local-data: "dropbox.com.sexvoorlichting.com 60 IN A 10.10.10.1"
    local-data: "www.dropbox.com 60 IN A 10.10.10.1"

    It seems Iblock is causing the issue. DNSBL alerts says the domain is already in the whitelist.

    And now google is doing the same thing.


  • Moderator

    @SLIMaxPower:

    Dropbox is block.  NET::ERR_CERT_AUTHORITY_INVALID

    grep result.

    grep "dropbox.com" /var/unbound/pfb_dnsbl.conf
    local-data: "dl.dropbox.com 60 IN A 10.10.10.1"
    local-data: "dropbox.com.azamgold.com 60 IN A 10.10.10.1"
    local-data: "dropbox.com.paki-library.com 60 IN A 10.10.10.1"
    local-data: "dropbox.com.sexvoorlichting.com 60 IN A 10.10.10.1"
    local-data: "www.dropbox.com 60 IN A 10.10.10.1"

    It seems Iblock is causing the issue. DNSBL alerts says the domain is already in the whitelist.

    And now google is doing the same thing.

    IBlock feeds are IP based… What your showing here is from DNSBL...

    grep "dropbox" /var/db/pfblockerng/dnsbl/*
    

    If you have "dropbox.com" in the whitelist, that won't remove "www.dropbox.com"
    Re-edit the DNSBL Whitelist and remove the dropbox entry, then use the Alerts Tab "+" whitelist icon to whitelist it again…. You can also whitelist all sub-domans if you wish... Follow the whitelist prompts.



  • The firewall is now integrated into the home network.

    iBlock lists have been disabled as this was causing too many https blocks.

    Firewall IPv6 stuff:

    Jun 3 16:59:23 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:42220   [2600:1406:32::43]:53 UDP
    Jun 3 16:59:23 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:34926   [2600:1401:2::43]:53 UDP
    Jun 3 16:59:23 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:65407   [2600:1408:1c::43]:53 UDP
    Jun 3 16:59:23 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:53784   [2600:1480:1::43]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:33499   [2a01:111:2032:1::5]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:58694   [2620:0:37::53]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:52594   [2620:0:32::53]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:29910   [2600:1480:1::c0]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:64372   [2600:1406:32::c2]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:4792   [2600:1480:e800::c0]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:56634   [2620:0:32::53]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:62068   [2620:0:34::53]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:33253   [2600:1401:1::41]:53 UDP
    Jun 3 16:59:22 ► WAN Block all IPv6 (1000000004)   [fd2a:bc2b:39d8:0:215:17ff:fe93:cc34]:6709   [2600:1401:2::42]:53 UDP



  • I just installed pfBlockerNG and tried to configure pfBlockerNG, IPV4, and DNSBL using google searches.  Needless to say, I followed some contradicting and outdated "tutorials".  So I decided that I will start at the beginning of this thread and read the entire thread before I try to configure anything.

    To revert my pfSense back to prior installing and configuring pfBlockerNG and all the other components (i.e. IPV4, DNBL), I uninstalled pfBlockerNG and deleted all the auto generated rules and aliases.  Is there anything else I need to undo?


  • Moderator

    @mifronte:

    To revert my pfSense back to prior installing and configuring pfBlockerNG and all the other components (i.e. IPV4, DNBL), I uninstalled pfBlockerNG and deleted all the auto generated rules and aliases.  Is there anything else I need to undo?

    There are notes in the General tab on how to do that… Uncheck "Enable" and "Keep" then save... This will clear out all the downloaded files... Then re-enable both followed by a Force Update command....

    If you want to start from complete scratch. Then Uncheck "Keep",  save, uninstall the pkg. Then re-install it ...

    Hope that helps!



  • Having a problem with GSN Gaming Casino on Facebook with pfblockerNG enabled on my server. Tried whitelisting everything I thought could be the cause but still not getting through. Has this been reported by anyone else and/or resolved? I did a fairly extensive search but didn't turn up anything useful (to me) about how to allow the traffic.



  • Issue with pfBlockerNG + Unbound + DNSBL when RAM Disk is enabled.

    Unbound fails to start after system reboot with RAM Disk enabled.
    User must force Update + Reload CRON for Unbound to successfully startup.

    The file '/var/unbound/pfb_dnsbl.conf' is removed upon system reboot with RAM Disk enabled thus forced CRON to regenerate file is the only way to get Unbound started again.

    We need the option to move this file inside pfBlocker's settings OR an option to preserve this file (and other settings/caches within pfBlocker) across system reboots when RAM Disk is enabled.

    Please as this currently requires manual user input every time the machine is rebooted to restore services!!


  • Banned

    Hmmm, I'm using pfBNG, DNSBL, Unbound - Resolving, and RAM disks and I don't have to do any of that, not for a normal reboot, a hard shutdown, or a power outage.

    I do have servicewatchdog monitoring Unbound, but that wouldn't Force Update & Reload.

    I am using 2.4.0 BETA, possibly something has been changed there that allows this to work?


  • Moderator

    @kklouzal:

    Issue with pfBlockerNG + Unbound + DNSBL when RAM Disk is enabled.

    Unbound fails to start after system reboot with RAM Disk enabled.
    User must force Update + Reload CRON for Unbound to successfully startup.

    The file '/var/unbound/pfb_dnsbl.conf' is removed upon system reboot with RAM Disk enabled thus forced CRON to regenerate file is the only way to get Unbound started again.

    We need the option to move this file inside pfBlocker's settings OR an option to preserve this file (and other settings/caches within pfBlocker) across system reboots when RAM Disk is enabled.

    Please as this currently requires manual user input every time the machine is rebooted to restore services!!

    See the following open Redline:  https://redmine.pfsense.org/issues/6603
    This can only be fixed in the pfSense Unbound code.



  • @BlueKobold:

    The next version of the package will have an automated Feeds Management Tab, which should make this process easier to manage.

    Any update on progress on that? Just curious, it sounds to be a very nice addition…


  • Moderator

    @occamsrazor:

    @BlueKobold:

    The next version of the package will have an automated Feeds Management Tab, which should make this process easier to manage.

    Any update on progress on that? Just curious, it sounds to be a very nice addition…

    I took some holiday time but hopefully in the next few weeks if all goes as plan.



  • I'm not sure if this is necessarily the best place to ask/report this, but in the process of attempting to reduce the maximum log file size below the current minimum of 5000 lines, I believe that I have discovered that this limit is only enforced every time the pfBlockerNG cron task executes.  This determination was made experimentally as well as by taking a look at where pfb_log_mgmt is called in https://github.com/pfsense/FreeBSD-ports/blob/75dcc67f66100f7ef6bbf201c6562bdcb02c3177/net/pfSense-pkg-pfBlockerNG/files/usr/local/pkg/pfblockerng/pfblockerng.inc  I don't claim to know the best approach to improving this, but if the cron job frequency is set to a relatively long interval and the network size is relatively large, it appears that the associated log files could easily grow well beyond their configured limits.  I realize that it would be impractical to provide a hard guarantee that the log files will never exceed their configured sizes even for a brief time period, but I wonder whether it would be feasible to trim them every minute, for example, as opposed to it being tied to the main cron job interval.  Or I could be totally off base here and missing or failing to understand something obvious, so if that's the case, apologies in advance :)  Thanks to all the pfBlockerNG devs; it's a great package and very much appreciated.



  • @BBcan177:

    @Mr.:

    Some weird stuff  ???

    Hey Mr. J.

    You are mixing some things up here  :)

    The pfBlockerNGSuppress alias does not need to be referenced to any Firewall Rules.

    Suppression -

    Suppression process occurs when Lists are downloaded from the Threat Sources.

    When a List is downloaded, if the list contains 1.2.3.4/32 and the Suppress Alias has 1.2.3.4/32, then this IP is suppressed from the Blocklist.

    If a list has 1.2.3.4/32 and the Suppress Alias has 1.2.3.0/24, then this IP is suppressed from the Blocklist.

    If a list has 1.2.3.4/24 and the Suppress Alias has 1.2.3.4/32, then the Single 1.2.3.4/32 is suppressed, and all of the other IPs in this Range are added to the Blocklist.

    When you click on the "+" icon in the Alerts tab, it will add the IP to the Suppress Alias, and also removes the IP from the Aliastable. However, the Suppressed IP is still in the Blocklist, and will be removed from the List at the Next Cron Update for the particular List. This will prevent these Suppressed IPs from being blocked.

    Whitelisting -

    When you whitelist, you are creating a new pfBNG alias and typically set it for "Permit Outbound". You can enter the Whitelisted IPs in the custom Box in the alias.

    The best method is to suppress the IP above. But if you have a Block occuring from a CIDR under a /24, you can't suppress that (ie /20 etc…) To overcome that, you need to allow the IP "Permit Outbound" which will create a state in the pfSense State table that allows the return of that IP without being Blocked by the pfBNG Block/Reject rules. In the Alerts Tab, you can see the List that Blocked the IP, if no IP is shown below the List, then the Block occurred by a /32 Blocklist entry. If its blocked by a CIDR, it will show the IP and CIDR below the List. You then can decide if its a /24 to use Suppression, or use the Whitelist for other CIDR ranges.

    Other questions -

    The Permit Rules need to be above the Block/Reject rules. Ensure that in the Alias, you set "Logging" or enable Global logging in the General Tab which will enable Logging for all Aliases globally.

    When you add a manual Rule, it can't have "pfB_" in the description, these will be removed by the Cron task each hour. To create "Alias" type rules, you need to enter the Description starting with "pfb_" (Lowercase)… This is explained in detail in the Alias "List Action" Section.

    You cannot Use Domain names with pfBlockerNG currently. You will need to convert the domain into an IP and add that to a Custom list. In v2.0 I will also have Domain Name Blocking (DNSBL).

    You can use a service like Hurricane Electric to collect IPs for Domain names that are changing more frequently and collect the list with the "html" format.

    http://bgp.he.net/search?search[search]=twitter&commit=Search
        http://bgp.he.net/search?search[search]=facebook&commit=Search
        http://bgp.he.net/search?search[search]=spotify&commit=Search
        http://bgp.he.net/search?search[search]=dropbox&commit=Search

    Please advise  https://forum.pfsense.org/index.php?topic=137017.0

    Thank you.



  • Hello friends, i love this package but i have one question or one suggestion to add

    I see in the queues of my mail server, mails in queue because can not connect to ip x.x.x.x , i not view this ip in country list ipv4 denegated, but maybe is in my personal ipv4 lists (dshield, etc..)  and the question is:

    How can i check if one ip is denegated in pfblockerng ???  there's no search ip engine ???



  • pfBlockerNG has an issue in the new 2.4.0 and 2.4.1 version of Pfsense, It has updated to BSD 11.1 from 11.0 and now if you have pfBlockerNG installed it will "Eventually" Lock up the whole kernal with a 502 Bad Gateway error.  Here is a connected thread on this issue.

    https://forum.pfsense.org/index.php?topic=137103.0


  • Banned

    @BreeOge:

    It has updated to BSD 11.1 from 11.0 and now if you have pfBlockerNG installed it will "Eventually" Lock up the whole kernal with a 502 Bad Gateway error.  Here is a connected thread on this issue.

    Humpf… 502 Bad Gateway is not a kernel message. It comes from nginx. Has nothing to do with kernel really.



  • Well then I guess the blue fairies that control Nginx have an issue with the newest version of BSD 11.1.  Because it locks up the Kernel, PHP, and all when it crashes. Nginx is just a symptom.  and the only common thing between all of them is PfblockerNG.  502 is just a symptom of the lockup not the cause or even the correct reason for the lockup.  If you get a fever from a cold, you could think its the flu, or anything but you dont know because its only a symptom, not the cause.

    Also when it freezes if you log in local, you can not run any commands, nothing.  It will execute the command but never return anything.. Find command, even the directory change doesn't even show the directory anymore.  Seems to me more than a Nginx issue..


  • Moderator

    @BreeOge:

    Well then I guess the blue fairies that control Nginx have an issue with the newest version of BSD 11.1.  Because it locks up the Kernel, PHP, and all when it crashes. Nginx is just a symptom.  and the only common thing between all of them is PfblockerNG.  502 is just a symptom of the lockup not the cause or even the correct reason for the lockup.  If you get a fever from a cold, you could think its the flu, or anything but you dont know because its only a symptom, not the cause.

    Also when it freezes if you log in local, you can not run any commands, nothing.  It will execute the command but never return anything.. Find command, even the directory change doesn't even show the directory anymore.  Seems to me more than a Nginx issue..

    Haven't tested 2.4.1 (FBSD 11.1) yet…. I had one of my testers confirm this issue but at a glance have not found whats causing it....

    I'll send you a PM to help troubleshoot this issue better...

    Thanks!



  • I've got the same problem. Anything I can do to help? Other than reboot every 3 hours of course  ::)

    Doug


  • Moderator

    Can you remove the pfBlockerNG widget from the dashboard and see if that helps. Also before a reboot or running option 11/16 from the console. Please review the /tmp/PHP_Error file for any details.



  • Okay I will do this and report back. I might add that doing "Firewall > pfBlockerNG" Disable didn't help.

    Doug



  • I updated to yesterdays build.

    The issue persists. 3+ hours after booting the GUI and serial console become unresponsive. Interesting to note that  from a network point of view everything still works. WAN and LAN connectivity is normal.

    VPN in does not work.

    I monitored "/tmp/PHP_error.txt" until it became unresponsive but there were no entries.

    Doug



  • To me it seems like some counter / process in the kernel with Pfblockerng is not releasing like it should.  I have multiple boxes, with different loads, all same config.  All of them will eventually lock up, but the time it takes is related to how much load is on the system.

    If you try to SSH in when it locks you will notice you can not restart, start or do anything with any service, it will freeze.  Also if you change directory the directory your in is never shown.  FYI if you do SSH in, you will need to CTRL-Z to drop out of the menu execution as it will lock up as well.



  • Well I got home from work and all appears to be working. GUI responsive, Serial console okay. VPN answered several times during the day. Still I think I'll follow others and reinstall and restore a recent config. Good time to switch to ZFS.

    Thanks for listening.

    Doug



  • @john_galt:

    Well I got home from work and all appears to be working. GUI responsive, Serial console okay. VPN answered several times during the day. Still I think I'll follow others and reinstall and restore a recent config. Good time to switch to ZFS.

    Thanks for listening.

    Doug

    Please keep us updated, I was wondering that myself if a fresh install to the newest release would fix issues. I know they had a major issue the other day and had to revert due to upgrade problems.



  • Jinxed myself. 502 Bad Gateway nginx

    Going to nurse it along until the weekend when I can get an 18 pack
    and re-install.

    Doug





  • BBcan177

    Just letting you know of another thread on this, trying to narrow the issue down.  Wanted to make you aware of the other thread.

    https://forum.pfsense.org/index.php?topic=137103.msg754247#new

    Thank you sir.
    BreeOge



  • I made the edits in this post:

    https://forum.pfsense.org/index.php?topic=137103.msg754338#msg754338

    Now my pfSense isn't locking up as I had reported before.

    Doug



  • I have reinstalled 2.4.0-R with ZFS and restored my previous config file. I reset pfBlockerNG.
    I will report any of the previous symptoms if the occur.

    Doug


  • Moderator

    @BreeOge:

    BBcan177

    Just letting you know of another thread on this, trying to narrow the issue down.  Wanted to make you aware of the other thread.

    https://forum.pfsense.org/index.php?topic=137103.msg754247#new

    Thank you sir.
    BreeOge

    Thanks for helping work thru the issue… I will work on converting that to use an SQLite3 database as recommended by jimp... I have just shied away in the past on using any SQL dbs...

    For those that remove that section of code in the index.php, that will stop the widget DNSBL counters from populating, however all else will still function as-is...



  • Hey, don't know what thread to post this on, but I am using a nanobsd install of pfsense and want to upgrade. I just want to know if pfblockerng is running okay with the new release before switching and if I do a backup of my config, and import, will that be ok without issues?



  • If you use the ZFS file system (If you do a full reload) It will work just fine. If you just upgrade or re-install with the UFS file system, then pfBlockerNG will lock up the system with a 502 bad gateway error.

    https://forum.pfsense.org/index.php?topic=137103.0

    In that thread there is a current work around, but like BBcan stated, if you do the work around your widget will not update properly.

    BreeOge



  • I wanted to chime in here as I just updated from a month old RC to 2.4.0-RELEASE last night and ran into this problem today.

    I haven't read through all of the many pages of the many threads that seem related to this issue (show how popular pfBNG is!), so maybe this has already been covered.

    But I've seen several people state that this doesn't happen on ZFS - I have a raidz2 ZFS install, and this happened to me, just throwing that out there.



  • @belt9:

    I wanted to chime in here as I just updated from a month old RC to 2.4.0-RELEASE last night and ran into this problem today.

    I haven't read through all of the many pages of the many threads that seem related to this issue (show how popular pfBNG is!), so maybe this has already been covered.

    But I've seen several people state that this doesn't happen on ZFS - I have a raidz2 ZFS install, and this happened to me, just throwing that out there.

    That is good to know. Thank you for the report.  BBcan177 is currently updating it to use SQLlite and this should fix any issues in the future.  In the other thread there is a temp fix posted..

    https://forum.pfsense.org/index.php?topic=137103.75

    Thank you
    BreeOge


Log in to reply