pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!


  • Moderator

    pfBlockerNG users

    If you use the GeoIP functionality of pfBlockerNG or if you use the "IP Reputation" component of pfBlockerNG or if you want to continue to see the Country for IP blocked events in the Reports Tab, then you must register for a free MaxMind account and obtain a License key.

    As per MaxMind Blog
    https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

    Starting December 30, 2019, we will be requiring users of our GeoLite2 databases to register for a MaxMind account and obtain a license key in order to download GeoLite2 databases. We will continue to offer the GeoLite2 databases without charge, and with the ability to redistribute with proper attribution and in compliance with privacy regulations. In addition, we are introducing a new end-user license agreement to govern your use of the GeoLite2 databases. Previously, GeoLite2 databases were accessible for download to the public on our developer website and were licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.Starting December 30, 2019, downloads will no longer be served from our public GeoLite2 page, from geolite.maxmind.com/download/geoip/database/*, or from any other public URL. See the section below for steps on how to migrate to the new download mechanism.

    INSTRUCTIONS:

    1. Sign up for a MaxMind account at: [ Registration should be done ASAP ]
      https://www.maxmind.com/en/geolite2/signup

      Note: They seem to not allow registrations from "VPN IPs" or using single-use email addresses.

    ec365794-adac-41ee-9d2d-60f647356a0b-image.png

    1. Click "Generate new license key"

    922599e7-5f49-4471-9e19-bf004b93aa0f-image.png

    1. Enter a "License key description", Select "yes" for "GeoIP Update", and select the License key for "version 3.1.1 or newer" and confirm.

    9629bdc2-2bf0-4707-97a8-70087fa1fb34-image.png

    1. Copy/paste the new "License Key" into pfBlockerNG

      pfBlockerNG - General Tab
      pfBlockerNG-devel - IP Tab

      Note: When pasting, ensure there are no trailing spaces. Best to paste "as plain text"

    9fce4b51-749d-4399-b11f-73548c279ad1-image.png

    ===========================================

    The new code (PRs) to support these changes is currently under review by the pfSense devs and should hopefully be available soon.

    pfBlockerNG v2.1.4_19
    https://github.com/pfsense/FreeBSD-ports/pull/734

    pfBlockerNG-devel v2.2.5_28
    https://github.com/pfsense/FreeBSD-ports/pull/738

    ===========================================

    Next pfBlockerNG Cron runs to update MaxMind databases:

    MaxMind generally updates on the first Tuesday of each month.

    pfBlockerNG v2.1.4_18 and below is set to update on January 7th, 2020

    pfBlockerNG-devel v2.2.5_27 and below is set to update on January 9th, 2020

    Going forward both pfBlockerNG and pfBlockerNG-devel will update on the first Thursday of each month since there have been some short delays by MaxMind.

    ===========================================

    Any issues or feedback, please let me know.

    Continue to follow here in the pfSense forum and on Twitter [ @bbcan177 ], Reddit [ /r/pfBlockerNG ]
    and Patreon for upcoming changes to pfBlockerNG.

    Thanks!



  • Thank You!



  • This kind of rapid response and support is why I use Patreon to support this package. Nice work BBcan!



  • Ok I've been here

    https://www.maxmind.com/en/geolite2/signup

    And done this

    "Generate new license key"

    Then go to

    pfBlockerNG-devel - IP Tab

    to

    Copy/paste the new "License Key" into pfBlockerNG

    I can't find anywhere to paste this license key.

    So am I missing something here or what?

    Thanks



  • Need to wait till the new pfBlockerNG update is out.



  • Right don't remember reading that anywhere, thanks...



  • Awesome support as usual :-)
    I just hope that Netgate does not take too long to review the package ...



  • @jacotec said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    I just hope that Netgate does not take too long to review the package ...

    It's a package.
    I guess @BBcan177 pushes out an update as soon as he finished writing and testing.



  • @Gertjan He already pushed it ;-)
    Netgate just needs to approve it ...



  • Now available.



  • @NogBadTheBad im using pfBlockerNG-devel 2.2.5_27 version, but i cant see the update in the package manager.



  • @havastamas

    Screenshot 2020-01-02 at 18.18.42.png

    Maybe wait a short while.



  • I just updated, already put my license key and it's working.



  • @NogBadTheBad Maybe because im using 2.4.x development version? (2.4.5?)



  • Their cloudflare antiddos such pain. Doesn't know why but it has half hour to register. Auch



  • @havastamas

    Maybe.



  • They mention "up to 25 keys"...do you happen to know if we need a separate key for each router, if we have more than one?



  • @teamits it's more about control over your account API usage. So it up to you to decide how much control you need. 1 key per software or 1 key per hardware instance or one key for all. If you directly know where you use it fine. When time is come to rotate the keys you want not to miss where you was used them :)



  • So I went ahead and installed pfBlockerNG 2.1.4_19 on my pfsense 2.4.4-RELEASE-p3 machine and signed up for an account with MaxMind and obtained a license key. I started downloading from MaxMind and am running into an issue with extracting data from the tarred database.

     UPDATE PROCESS START [ 01/02/20 15:29:54 ]
    
    ===[  DNSBL Process  ]================================================
    
      DNSBL: Flush DNSBL_IP
    Clearing all DNSBL Feeds...  completed
    Validating database... completed
    Reloading Unbound.... completed
    DNSBL update [ 0 | PASSED  ]... completed [ 01/02/20 15:29:55 ]
    ------------------------------------------
    
    ===[  Continent Process  ]============================================
    
    MaxMind Database downloading and processing ( approx 4MB ) ... Please wait ...
    
    Download Process Starting [ 01/02/20 15:29:55 ]
     /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
    . /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
    .tar: Failed to set default locale
    tar: Failed to set default locale
    
    Download Process Ended [ 01/02/20 15:29:56 ]
    
    Country code update Start
     [ MAXMIND UPDATE FAIL, Language File Missing, using previous Country code database ]
     Creating pfBlockerNG Continent XML files
     IPv4 Africa			
    grep: /usr/local/share/GeoIP/cc/Africa_v4.txt: No such file or directory
     IPv6 Africa			
    grep: /usr/local/share/GeoIP/cc/Africa_v6.txt: No such file or directory
     IPv4 Antarctica		
    grep: /usr/local/share/GeoIP/cc/Antarctica_v4.txt: No such file or directory
     IPv6 Antarctica		
    grep: /usr/local/share/GeoIP/cc/Antarctica_v6.txt: No such file or directory
     IPv4 Asia			
    grep: /usr/local/share/GeoIP/cc/Asia_v4.txt: No such file or directory
     IPv6 Asia			
    grep: /usr/local/share/GeoIP/cc/Asia_v6.txt: No such file or directory
     IPv4 Europe			
    grep: /usr/local/share/GeoIP/cc/Europe_v4.txt: No such file or directory
     IPv6 Europe			
    grep: /usr/local/share/GeoIP/cc/Europe_v6.txt: No such file or directory
     IPv4 North America		
    grep: /usr/local/share/GeoIP/cc/North_America_v4.txt: No such file or directory
     IPv6 North America		
    grep: /usr/local/share/GeoIP/cc/North_America_v6.txt: No such file or directory
     IPv4 Oceania			
    grep: /usr/local/share/GeoIP/cc/Oceania_v4.txt: No such file or directory
     IPv6 Oceania			
    grep: /usr/local/share/GeoIP/cc/Oceania_v6.txt: No such file or directory
     IPv4 South America		
    grep: /usr/local/share/GeoIP/cc/South_America_v4.txt: No such file or directory
     IPv6 South America		
    grep: /usr/local/share/GeoIP/cc/South_America_v6.txt: No such file or directory
     IPv4 Proxy and Satellite	
    grep: /usr/local/share/GeoIP/cc/Proxy_and_Satellite_v4.txt: No such file or directory
     IPv6 Proxy and Satellite	
    grep: /usr/local/share/GeoIP/cc/Proxy_and_Satellite_v6.txt: No such file or directory
     IPv4 TOP 20			
    grep: /usr/local/share/GeoIP/cc/Top_20_v4.info: No such file or directory
     IPv6 TOP 20			
    grep: /usr/local/share/GeoIP/cc/Top_20_v6.info: No such file or directory
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    
    
    
    ===[  Aliastables / Rules  ]==========================================
    
    No changes to Firewall rules, skipping Filter Reload
    No Changes to Aliases, Skipping pfctl Update
    
     UPDATE PROCESS ENDED [ 01/02/20 15:29:56 ]
    
    [2.4.4-RELEASE][admin@host.example.org]/usr/local/share/GeoIP: ls -la
    total 24332
    drwxr-xr-x   3 root  wheel       512 Jan  2 14:49 .
    drwxr-xr-x  44 root  wheel      1024 Jan  2 14:42 ..
    -rw-r--r--   1 root  wheel  16645188 Dec 31 08:32 GeoLite2-Country-CSV.zip.orig
    -rw-r--r--   1 root  wheel   1981295 Jan  2 15:29 GeoLite2-Country-CSV.zip.raw
    -rw-r--r--   1 root  wheel   4034560 Dec 31 08:32 GeoLite2-Country.tar.gz.orig
    -rw-r--r--   1 root  wheel   2044326 Jan  2 15:29 GeoLite2-Country.tar.gz.raw
    drwxr-xr-x   2 root  wheel       512 Jan  2 14:42 cc
    [2.4.4-RELEASE][admin@host.example.org]/usr/local/share/GeoIP: cd cc
    [2.4.4-RELEASE][admin@host.example.org]/usr/local/share/GeoIP/cc: ls -la
    total 8
    drwxr-xr-x  2 root  wheel  512 Jan  2 14:42 .
    drwxr-xr-x  3 root  wheel  512 Jan  2 14:49 ..
    [2.4.4-RELEASE][admin@host.example.org]/usr/local/share/GeoIP/cc: 
    

    It appears the files have been downloaded but not untarred. The error appears to have to do with setting up the default locale for the tar command. The language on pfsense is set to English and so is MaxMind localization language. What do I need to do to fix this?

    ~Doug


  • Moderator

    @dougs
    Can you try uninstalling pfBlockerNG and trying pfBlockerNG-devel?



  • @BBcan177
    Uninstalled pfBlockerNG and installed pfBlockerNG-devel 2.2.5_28. Ran update as follows:

     UPDATE PROCESS START [ 01/02/20 16:36:16 ]
    
    ===[  DNSBL Process  ]================================================
    
    Saving DNSBL database... completed
    
    Clearing all DNSBL Feeds completed
    Reloading Unbound Resolver..... completed
    DNSBL update [ 0 | PASSED  ]... completed
    ------------------------------------------------------------------------
    
    ===[  GeoIP Process  ]============================================
    
    MaxMind Database downloading and processing ( approx 4MB ) ... Please wait ...
    
    Download Process Starting [ 01/02/20 16:36:16 ]
     /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
     /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
    Download Process Ended [ 01/02/20 16:36:17 ]
    
    Country code update Start
     Processing ISO IPv4 Continent/Country Data
     Processing ISO IPv6 Continent/Country Data [ 01/02/20 16:36:23 ]
     Creating pfBlockerNG Continent PHP files
     IPv4 Africa			 [ 01/02/20 16:36:24 ]
     IPv6 Africa			 [ 01/02/20 16:36:25 ]
     IPv4 Antarctica		
     IPv6 Antarctica		
     IPv4 Asia			
     IPv6 Asia			
     IPv4 Europe			
     IPv6 Europe			 [ 01/02/20 16:36:27 ]
     IPv4 North America		
     IPv6 North America		 [ 01/02/20 16:36:28 ]
     IPv4 Oceania			
     IPv6 Oceania			 [ 01/02/20 16:36:29 ]
     IPv4 South America		
     IPv6 South America		
     IPv4 Proxy and Satellite	
     IPv6 Proxy and Satellite	
     IPv4 Top Spammers		
     IPv6 Top Spammers		
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    
    
    
    ===[  IPv4 Process  ]=================================================
    
    [ DNSBLIP_v4 ]			 Downloading update [ 01/02/20 16:36:29 ] .
    [ DNSBLIP_v4 ] file_get_contents(/var/db/pfblockerng/DNSBLIP_v4.txt): failed to open stream: No such file or directory
    
    
     [ pfB_DNSBLIP_v4 - DNSBLIP_v4 ] Download FAIL
       Local File Failure
    
    
    The Following List has been REMOVED [ DNSBLIP_v4 ]
    
    
    ===[  Aliastables / Rules  ]==========================================
    
    No changes to Firewall rules, skipping Filter Reload
    No Changes to Aliases, Skipping pfctl Update
    
     UPDATE PROCESS ENDED
    
    

    Looks like an error with DNSBLIP?

    ~Doug


  • Moderator

    OK I found the issue with pfBlockerNG and the "tar" error.

    I posted a fix here (v. 2.1.4_20)
    https://github.com/pfsense/FreeBSD-ports/pull/742/files

    Would still recommend that everyone move to pfBlockerNG-devel. Will be ending pfBlockerNG version soon.

    The DNSBL Error might be fixed with a "Force Reload - All"



  • Let's to note that another one great DB exist ip2location company.
    Like in MaxMind for end user there are free DB to download.



  • Thank you so much!



  • @mcury said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    I just updated, already put my license key and it's working.

    How do you know it worked?

    I am on pfBlockerNG-devel 2.2.5_28, added MaxMind License Key, ran update and reload, but when look at
    ls -lah /usr/local/share/GeoIP/
    my files dated Dec 3 2019

    Did it work ?



  • @chudak Probably "working" in that it no longer throws the error. MaxMind updates their data first Tuesday of every month, but pfBNG doesn't until the following Thursday in case there's a delay. Check again on the 9th. If I'm incorrect please let me know.



  • @provels
    Will do, thx



  • @chudak said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @mcury said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    I just updated, already put my license key and it's working.

    How do you know it worked?

    I am on pfBlockerNG-devel 2.2.5_28, added MaxMind License Key, ran update and reload, but when look at
    ls -lah /usr/local/share/GeoIP/
    my files dated Dec 3 2019

    Did it work ?

    You can run the following from the command prompt to see if it updates:
    "php /usr/local/www/pfblockerng/pfblockerng.php dc", copy and paste without the quotes.



  • @chudak I'm using 2.4.4p3 with pfBlocker-NG-devel 2.2.5_28

    Check if the files downloaded from maxmind are located at:

    ls -lah /usr/local/share/GeoIP/

    Check the logs during the maxmind download, as you can see, it downloaded a 4MB database file.

    **Saving configuration [ 01/05/20 12:32:23 ]**
    
    MaxMind Database downloading and processing ( approx 4MB ) ... Please wait ...
    
    Download Process Starting [ 01/05/20 12:32:24 ]
     /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
     /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
    Download Process Ended [ 01/05/20 12:32:28 ]
    
    Country code update Start
     Processing ISO IPv4 Continent/Country Data [ 01/05/20 12:32:31 ]
     Processing ISO IPv6 Continent/Country Data [ 01/05/20 12:33:08 ]
     Creating pfBlockerNG Continent PHP files
     IPv4 Africa			 [ 01/05/20 12:33:18 ]
     IPv6 Africa			 [ 01/05/20 12:33:19 ]
     IPv4 Antarctica		
     IPv6 Antarctica		
     IPv4 Asia			
     IPv6 Asia			 [ 01/05/20 12:33:22 ]
     IPv4 Europe			 [ 01/05/20 12:33:23 ]
     IPv6 Europe			 [ 01/05/20 12:33:33 ]
     IPv4 North America		 [ 01/05/20 12:33:35 ]
     IPv6 North America		 [ 01/05/20 12:33:42 ]
     IPv4 Oceania			 [ 01/05/20 12:33:44 ]
     IPv6 Oceania			
     IPv4 South America		
     IPv6 South America		 [ 01/05/20 12:33:45 ]
     IPv4 Proxy and Satellite	 [ 01/05/20 12:33:46 ]
     IPv6 Proxy and Satellite	
     IPv4 Top Spammers		
     IPv6 Top Spammers		
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    


  • @jdeloach said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @chudak said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @mcury said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    I just updated, already put my license key and it's working.

    How do you know it worked?

    I am on pfBlockerNG-devel 2.2.5_28, added MaxMind License Key, ran update and reload, but when look at
    ls -lah /usr/local/share/GeoIP/
    my files dated Dec 3 2019

    Did it work ?

    You can run the following from the command prompt to see if it updates:
    "php /usr/local/www/pfblockerng/pfblockerng.php dc", copy and paste without the quotes.

    yup that worked, thx !



  • @BBcan177 , on pfBlockerNG, v2.1.4_20, it looks like if I open the plugin page (Firewall/pfBlockerNG) the MaxMind License Key field is blank. maxmind_key is populated in the saved config file, which I saved after entering it...I'm thinking maybe it's not populating correctly on the page?

    (saw your comment about upgrading/moving to -devel in all cases, just haven't gotten to that yet)



  • @jdeloach said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    You can run the following from the command prompt to see if it updates:
    "php /usr/local/www/pfblockerng/pfblockerng.php dc", copy and paste without the quotes.

    confirmed: running 2.4.4-RELEASE-p3 -> pfBlockerNG 2.1.4_20 (updated, paste the MaxMind "License Key") worked!

    Thank you very much!


  • Netgate Administrator

    I'll just add that I updated the package, filled in the key and ran an update. All went smoothly. 👍

    Steve



  • @dougs said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @BBcan177
    Uninstalled pfBlockerNG and installed pfBlockerNG-devel 2.2.5_28. Ran update as follows:

     UPDATE PROCESS START [ 01/02/20 16:36:16 ]
    
    ===[  DNSBL Process  ]================================================
    
    Saving DNSBL database... completed
    
    Clearing all DNSBL Feeds completed
    Reloading Unbound Resolver..... completed
    DNSBL update [ 0 | PASSED  ]... completed
    ------------------------------------------------------------------------
    
    ===[  GeoIP Process  ]============================================
    
    MaxMind Database downloading and processing ( approx 4MB ) ... Please wait ...
    
    Download Process Starting [ 01/02/20 16:36:16 ]
     /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
     /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
    Download Process Ended [ 01/02/20 16:36:17 ]
    
    Country code update Start
     Processing ISO IPv4 Continent/Country Data
     Processing ISO IPv6 Continent/Country Data [ 01/02/20 16:36:23 ]
     Creating pfBlockerNG Continent PHP files
     IPv4 Africa			 [ 01/02/20 16:36:24 ]
     IPv6 Africa			 [ 01/02/20 16:36:25 ]
     IPv4 Antarctica		
     IPv6 Antarctica		
     IPv4 Asia			
     IPv6 Asia			
     IPv4 Europe			
     IPv6 Europe			 [ 01/02/20 16:36:27 ]
     IPv4 North America		
     IPv6 North America		 [ 01/02/20 16:36:28 ]
     IPv4 Oceania			
     IPv6 Oceania			 [ 01/02/20 16:36:29 ]
     IPv4 South America		
     IPv6 South America		
     IPv4 Proxy and Satellite	
     IPv6 Proxy and Satellite	
     IPv4 Top Spammers		
     IPv6 Top Spammers		
     pfBlockerNG Reputation Tab
    Country Code Update Ended
    
    
    
    ===[  IPv4 Process  ]=================================================
    
    [ DNSBLIP_v4 ]			 Downloading update [ 01/02/20 16:36:29 ] .
    [ DNSBLIP_v4 ] file_get_contents(/var/db/pfblockerng/DNSBLIP_v4.txt): failed to open stream: No such file or directory
    
    
     [ pfB_DNSBLIP_v4 - DNSBLIP_v4 ] Download FAIL
       Local File Failure
    
    
    The Following List has been REMOVED [ DNSBLIP_v4 ]
    
    
    ===[  Aliastables / Rules  ]==========================================
    
    No changes to Firewall rules, skipping Filter Reload
    No Changes to Aliases, Skipping pfctl Update
    
     UPDATE PROCESS ENDED
    
    

    Looks like an error with DNSBLIP?

    ~Doug

    Having the same issue



  • @bjurkovski said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @dougs said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @BBcan177
    Uninstalled pfBlockerNG and installed pfBlockerNG-devel 2.2.5_28. Ran update as follows:

    Looks like an error with DNSBLIP?

    ~Doug

    Having the same issue

    Maybe an Force Update can help?
    Something else - I switched from pfblockerng to pfblockerng-devel an there is a increase of RAM usage from about 13% - is it also in your case?

    Thanks,
    fireodo



  • @teamits said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    @BBcan177 , on pfBlockerNG, v2.1.4_20, it looks like if I open the plugin page (Firewall/pfBlockerNG) the MaxMind License Key field is blank. maxmind_key is populated in the saved config file, which I saved after entering it

    I checked several times yesterday and the field had the key in it each time I logged in. On a hunch waited until overnight again, and this morning the field is blank in the GUI. Seems so unlikely but is there some way it could disappear overnight? If I export the config file this morning it's missing, compared to my last export 3 days ago:

    <enable_log/>
    <skipfeed>0</skipfeed>
    <log_maxlines>20000</log_maxlines>
    <maxmind_key>****</maxmind_key>
    <maxmind_locale>en</maxmind_locale>
    <database_cc/>
    <inbound_interface>wan</inbound_interface>

    vs

    <enable_log/>
    <maxmind_locale>en</maxmind_locale>
    <database_cc/>
    <skipfeed>0</skipfeed>
    <log_maxlines>20000</log_maxlines>
    <inbound_interface>wan</inbound_interface>

    FWIW this is on an SG-4860 and it hasn't restarted in months.



  • Hi,
    just a quiston: If Idon't want to use GeoIP and set "Disable MaxMind CSV database Updates" checkbox I'm already get an notification every hour that "MaxMind need a license". Is there a way to block this spam without disabling the notification or uninistalling pfBlockerNG?
    Cheers
    Marco


  • Moderator

    @MarcO42
    You need to disable the "Action" settings in the GeoIP tab and any IPv4/6 aliases that use the "GeoIP" state for the notifications to stop.



  • Instead of sign up Maxmind, can we use IP2Location database as replacement?



  • @christopa1999 said in pfBlockerNG MaxMind Registration required to continue to use the GeoIP functionality!:

    Instead of sign up Maxmind, can we use IP2Location database as replacement?

    Why avoid registration?
    Just using MaxMind at this moment and after IP2locations integration ready, - just choose. ;)


Log in to reply