pfBlockerNG IP Reputation



  • This picture is worth many words!

    Screen Shot 2020-05-12 at 4.53.45 PM.png



  • No one else reproduced the above result? Is it that IP reputation had been removed from the package?



  • @NollipfSense said in pfBlockerNG IP Reputation:

    reputation

    Noop.

    6c14cb53-872b-4241-979a-5cc33fc8f5a1-image.png

    Works for me.

    Btw : The php file exists :

    965f291e-e41f-4b09-b7c1-21f3424b7992-image.png



  • I had this some month ago, installation was corrupt.



  • @Gertjan Wow ... thank you for sharing. What folder did you select to view the file?



  • @Bob-Dig I'll reinstall pfBlockerNG.



  • Reinstalling pfBlockerNG did not fix; so, I'll need to reinstall pfSense.



  • @NollipfSense said in pfBlockerNG IP Reputation:

    Reinstalling pfBlockerNG did not fix

    When removing pfBlocker, the usr/local/www/pfblockerng/ directory should be gone, or, at least empty.
    When you re install, that file doesn't come back ??

    Re installing pfSEnse will not generate these files.

    File system problems ?



  • @Gertjan said in pfBlockerNG IP Reputation:

    @NollipfSense said in pfBlockerNG IP Reputation:

    Reinstalling pfBlockerNG did not fix
    

    When removing pfBlocker, the usr/local/www/pfblockerng/ directory should be gone, or, at least empty.
    When you re install, that file doesn't come back ??

    Re installing pfSEnse will not generate these files.

    File system problems ?

    Okay Gertjan, I did not completely remove it ... just reinstalled it and it's the same. So, I will remove it then check to be sure the directory is gone or empty. Thank you!

    Completely removed pfBlockerNG, keep setting, and reinstall ... same issue. The directory was there but empty. I'll try removing again then delete the directory.



  • This is the only file in the directory when pfBlockerNG is removed.

    Screen Shot 2020-05-22 at 2.07.18 PM.png

    Then, when reinstalling pfBlockerNG, some how IP reputation.php is not included.

    Screen Shot 2020-05-22 at 2.19.36 PM.png



  • Hello!

    Did you run a full update of pfbng? I think that file might be dynamically generated at some point during that process.

    John



  • @NollipfSense said in pfBlockerNG IP Reputation:

    Then, when reinstalling pfBlockerNG, some how IP reputation.php is not included.

    Yep, I confirm.
    It's not included in the package file - neither referenced in the manifest.
    It's a package build issue.
    Some one should inform @BBcan177.

    Maybe the reputation functionality was removed from pfBlocker ?

    I can rename any file in that directory, re install pfBlockerNG, and the file will reapaer.
    Except the pfblockerng_reputation.php file.

    @serbus said in pfBlockerNG IP Reputation:

    I think that file might be dynamically generated at some point during that process.

    It's a static web page.
    pfBlockerNG does not generate this page.
    An exception exists : the /usr/local/www/pfblockerng/www/dnsbl_active.php page, which is copied from the dnsbl_default.php file by pfBlockerNG itself. This is the way we select the default "Blocked Webpage" - made our own one.

    18acd772-82a8-4906-84fb-ab8c41b8f7f6-image.png



  • @Gertjan said in pfBlockerNG IP Reputation:

    It's a static web page.
    pfBlockerNG does not generate this page.

    Hello!

    I am new to pfsense and still working through much of the code. Maybe you can help.

    How and when are the pfblocker reputation and country xml files generated?
    What is the purpose of the pkg.php system?
    Does pkg ever save/cache the php it dynamically generates from xml for performance or other reasons?

    Thanks!

    John



  • @NollipfSense said in pfBlockerNG IP Reputation:

    No one else reproduced the above result? Is it that IP reputation had been removed from the package?

    Hello!

    You could try running :

    php /usr/local/www/pfblockerng/pfblockerng.php gc

    To recreate the reputation and country php files.

    John



  • @serbus said in pfBlockerNG IP Reputation:

    I think that file might be dynamically generated at some point during that process.

    I have been thinking along those lines because I noticed also that Maxmind data had not loaded despite providing my key.



  • @Gertjan Do you also have the MaxMind feed loaded?



  • @serbus said in pfBlockerNG IP Reputation:

    php /usr/local/www/pfblockerng/pfblockerng.php gc

    Hey John, I am reporting that worked. However, the command result shows geolocation files not found confirmed that IP reputation list is derived from geolocation feed. Just discovered making the page had produced a crash

    Screen Shot 2020-05-24 at 10.05.09 PM.png

    Screen Shot 2020-05-24 at 10.06.55 PM.png

    Screen Shot 2020-05-24 at 10.23.49 PM.png



  • @NollipfSense :

    [2.4.5-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: php /usr/local/www/pfblockerng/pfblockerng.php gc
     Creating pfBlockerNG Continent PHP files
     IPv4 Africa                     [ 05/25/20 07:16:41 ]
     IPv6 Africa                     [ 05/25/20 07:16:43 ]
     IPv4 Antarctica
     IPv6 Antarctica
     IPv4 Asia
     IPv6 Asia                       [ 05/25/20 07:16:46 ]
     IPv4 Europe                     [ 05/25/20 07:16:47 ]
     IPv6 Europe                     [ 05/25/20 07:16:55 ]
     IPv4 North America              [ 05/25/20 07:16:58 ]
     IPv6 North America              [ 05/25/20 07:17:04 ]
     IPv4 Oceania                    [ 05/25/20 07:17:05 ]
     IPv6 Oceania                    [ 05/25/20 07:17:06 ]
     IPv4 South America
     IPv6 South America              [ 05/25/20 07:17:07 ]
     IPv4 Proxy and Satellite
     IPv6 Proxy and Satellite        [ 05/25/20 07:17:08 ]
     IPv4 Top Spammers
     IPv6 Top Spammers
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    

    and

    [2.4.5-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: php /usr/local/www/pfblockerng/pfblockerng.php ugc
    Country code update Start [ 05/25/20 07:42:27 ]
     Processing ISO IPv4 Continent/Country Data
     Processing ISO IPv6 Continent/Country Data [ 05/25/20 07:42:55 ]
     Creating pfBlockerNG Continent PHP files
     IPv4 Africa                     [ 05/25/20 07:43:05 ]
     IPv6 Africa                     [ 05/25/20 07:43:06 ]
     IPv4 Antarctica
     IPv6 Antarctica
     IPv4 Asia
     IPv6 Asia                       [ 05/25/20 07:43:09 ]
     IPv4 Europe                     [ 05/25/20 07:43:10 ]
     IPv6 Europe                     [ 05/25/20 07:43:19 ]
     IPv4 North America              [ 05/25/20 07:43:21 ]
     IPv6 North America              [ 05/25/20 07:43:27 ]
     IPv4 Oceania                    [ 05/25/20 07:43:29 ]
     IPv6 Oceania
     IPv4 South America
     IPv6 South America              [ 05/25/20 07:43:30 ]
     IPv4 Proxy and Satellite        [ 05/25/20 07:43:31 ]
     IPv6 Proxy and Satellite
     IPv4 Top Spammers
     IPv6 Top Spammers               [ 05/25/20 07:43:32 ]
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    

    @serbus said in pfBlockerNG IP Reputation:

    Did you run a full update of pfbng? I think that file might be dynamically generated at some point during that process.

    Well .... @servus is right.
    I was wrong.
    This file "reputation" IS actually regenerated out of /usr/local/www/pfblockerng/pfblockerng.php ...

    @NollipfSense : Yes, I have an activated MaxiMind account.

    2728b5ee-17bb-4618-9abc-63157d3a2ad5-image.png

    @NollipfSense : How many files - what ype of files do you have here /usr/local/share/GeoIP/cc/ ?

    I have more the 1500 files - it depends probably on which regions I've selected, I guess.
    Some of them have a time stamp like 03/09/2019 - others 07/05/2020 - and the better part was downloaded just today : 25/05/2020 - 07h17.

    Also : /usr/local/share/GeoIP/ ? This directory gets filled with files from MaxMind - if you have an account with them.

    So : files actually get downloaded , No file system full ? Run a "fsck" just to be sure.

    Btw : The Diagnostic > Command propmpt : never use that one. It could hide stuff. Keyboard command belong on a real command line. It's one click away with Putty or any other SSH client. Better get used to it ^^



  • @Gertjan said in pfBlockerNG IP Reputation:

    How many files - what ype of files do you have here /usr/local/share/GeoIP/cc/ ?

    Appeared empty!

    Screen Shot 2020-05-26 at 9.25.17 PM.png

    @Gertjan said in pfBlockerNG IP Reputation:

    Also : /usr/local/share/GeoIP/ ?

    Also, appeared empty!

    Screen Shot 2020-05-26 at 9.35.36 PM.png



  • Hello!

    php /usr/local/www/pfblockerng/pfblockerng.php dc

    should re-download the maxmind files. They should appear in the /usr/local/share/GeoIP folder.

    the "dc" command will also run the "ugc" functions after the download.

    John



  • @NollipfSense : do not use the GUI for this. Use the console/SSH.

    There is a little surprise here, see the last two lines :

    [2.4.5-RELEASE][admin@pfsense.brit-hotel-fumel.net]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc
    
    Download Process Starting [ 05/27/20 09:48:21 ]
     /usr/local/share/GeoIP/GeoLite2-Country.tar.gz         200 OK
     /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip                200 OK
    Download Process Ended [ 05/27/20 09:48:26 ]
    
    Country code update Start
     Processing ISO IPv4 Continent/Country Data
     Processing ISO IPv6 Continent/Country Data [ 05/27/20 09:48:55 ]
     Creating pfBlockerNG Continent PHP files
     IPv4 Africa                     [ 05/27/20 09:49:03 ]
     IPv6 Africa                     [ 05/27/20 09:49:04 ]
     IPv4 Antarctica
     IPv6 Antarctica
     IPv4 Asia
     IPv6 Asia                       [ 05/27/20 09:49:08 ]
     IPv4 Europe
     IPv6 Europe                     [ 05/27/20 09:49:17 ]
     IPv4 North America              [ 05/27/20 09:49:20 ]
     IPv6 North America              [ 05/27/20 09:49:25 ]
     IPv4 Oceania                    [ 05/27/20 09:49:27 ]
     IPv6 Oceania
     IPv4 South America              [ 05/27/20 09:49:28 ]
     IPv6 South America
     IPv4 Proxy and Satellite        [ 05/27/20 09:49:29 ]
     IPv6 Proxy and Satellite        [ 05/27/20 09:49:30 ]
     IPv4 Top Spammers
     IPv6 Top Spammers
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    


  • @serbus and @Gertjan I used the CLI and here is the result:

    [2.5.0-DEVELOPMENT][admin@NollipfSense.nollipfsense.lan]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc

    Download Process Starting [ 05/27/20 10:55:35 ]
    /usr/local/share/GeoIP/GeoLite2-Country.tar.gz 401 Unauthorized

    Failed to Download GeoLite2-Country.mmdb
    /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip 401 Unauthorized

    Failed to Download
    Download Process Ended [ 05/27/20 10:55:36 ]

    [2.5.0-DEVELOPMENT][admin@NollipfSense.nollipfsense.lan]/root:

    What I don't understand is I have a registered key; so, not sure what the unauthorized is all about nor what to do to resolve.

    Screen Shot 2020-05-27 at 10.58.12 AM.png



  • Goto Maxminds and check your account and Download History.



  • @RonpfS Last download was on May 5, 2020 at 14.56pm ... so, I guess I'll have to wait for June. I had to reinstall a fresh pfSense 2.5-dev so may explain why I haven't got the feed since its once per month.



  • It changes every 6 days, do you see the md5 download every day ?



  • Hello!

    Is there a limit to the number of times per month you can download the files from maxmind?

    I use the same license key in a number of different routers and routinely download "off schedule" when setting things up or troubleshooting.

    Maybe you could create a new license key at the maxmind site and try that in your router.

    John



  • @NollipfSense said in pfBlockerNG IP Reputation:

    @RonpfS Last download was on May 5, 2020 at 14.56pm ... so, I guess I'll have to wait for June. I had to reinstall a fresh pfSense 2.5-dev so may explain why I haven't got the feed since its once per month.

    Run this command from the command prompt to force Maxmind to update: php /usr/local/www/pfblockerng/pfblockerng.php dc . This should force the Maxmind.com database to update.



  • @jdeloach said in pfBlockerNG IP Reputation:

    Run this command from the command prompt to force Maxmind to update: php /usr/local/www/pfblockerng/pfblockerng.php dc . This should force the Maxmind.com database to update.

    He did ( see above ) :

    @NollipfSense said in pfBlockerNG IP Reputation:

    [2.5.0-DEVELOPMENT][admin@NollipfSense.nollipfsense.lan]/root: php /usr/local/www/pfblockerng/pfblockerng.php dc
    Download Process Starting [ 05/27/20 10:55:35 ]

    He wasn't authorized.

    /usr/local/share/GeoIP/GeoLite2-Country.tar.gz 401 Unauthorized
    Failed to Download GeoLite2-Country.mmdb
    /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip 401 Unauthorized
    Failed to Download
    Download Process Ended [ 05/27/20 10:55:36 ]



  • So, I contacted MaxMind support that confirmed that somehow when I did the force the update, it kept downloading last month's (April) database ... which is not available ... hence, the unauthorized message.

    Support suggested "If you alter your download URL and remove the 'date' parameter entirely, that will make it download the most recent database available rather than a specific database version. Alternatively, you can use the 'Get permalinks' link in your Download Files page to get a permanent download URL that you can use."

    So, m question: where would I find the download file to change or replace with "permalinks."



  • Maybe it's time to move this topic to pfblockerNG forum.



  • @RonpfS That's okay with me ... admin.



  • @NollipfSense

    Hello!

    Pfb uses the maxmind permalink url for retrieving the maxmind db. It does not look like it asks for a specific version or month.

    Here is the url from the pfb code:

    https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-Country&license_key=MAXMIND_KEY&suffix=tar.gz

    You should be able to replace MAXMIND_KEY with your key and try the url in your browser.

    John



  • @serbus Hello John, I preferred to let pfSense do the downloading instead of downloading it by way of the browser. I looked at this file: /usr/local/www/pfblockerng/pfblockerng_feeds.php ... however, no MaxMind url was in the file.

    Alternatively, I could wait until next Thursday when the new file would be available.



  • Hello!

    The maxmind urls are in usr/local/www/pfblockerng/pfblockerng.php

    Loading that link in your browser would just be a general test for your maxmind account, license key, and networks access to the download.

    John



  • @serbus Well John, early this morning I tried again and got same unauthorized ... so, I tried the browser and got invalid key; so, I just generated a new key ... all is good.


Log in to reply