Update Frequency of MaxMind's GeoIP database

mpfrench

I'm running pfBlockerNG 3.2.0_3 on a Netgate SG-1100

There seems to be a disagreement on the update frequency of the GeoIP database between what is written in pfBlockerNG and MaxMind's web site. Namely,

at the tab Firewall/pfBlockerNG/IP/GeoIP

"The GeoIP database is automatically updated the first Tuesday of each month. (To avoid any MaxMind update delays, update is now scheduled for the first Thursday of each month.)"

However, MaxMind's web site says the database is updated twice weekly.

Frequency of update conflicts with https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en
"The GeoLite2 Country, City, and ASN databases are updated twice weekly, every Tuesday and Friday."

It seems as though the update schedule in pfBlockerNG needs to be changed to match MaxMind's schedule.

Gertjan

@mpfrench said in Update Frequency of MaxMind's GeoIP database:

However, MaxMind's web site says the database is updated twice weekly.

If the GeoLite2 free license permits such an access, twice a week instead of ones per month, the cron job can be changed.

If you don't have it, install the pfSense cron package.

You'll find :

That's the GeoLite2 update cron task.
Why 'dcc' ? The file /usr/local/www/pfblockerng/pfblockerng.php tells me this.

Let's decode "0 2 * * 5" : https://crontab.guru/#0_2___5
Note that the "2" is a random hour, between 0 and 23. So Your can have any number (hour) between 0 and 23.

That is in conflict with what the GUI said : it is updated in reality weekly, on day 5, or Friday, at a random (for me : 2 AM) - that's not once a month but once a week.

Go look/edit here : /usr/local/pkg/pfblockerng/pfblockerng.inc line 10580

You could edit line 10588, the :

		$pfb_gwday	= '5';

for

		$pfb_gwday	= '2,4';

and that means "Tuesday and Thursday".

Instead of :

		$pfb_ghour	= rand(0,23);

change that for

		$pfb_ghour	= '23';

so you'll be sure not to update to early ;)

There is an easy way to fact-check all this.
Go to the place with the answers : look at the /var/log/pfblockerng/extras.log log file.
You can see it with the GUI.

and it shows me at the end :

.....
Download Process Starting [ 03/3/23 08:00:00 ]
 /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
 /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
......

Download Process Starting [ 03/10/23 02:00:00 ]
 /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
 /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
.....

Download Process Starting [ 03/17/23 02:00:00 ]
 /usr/local/share/GeoIP/GeoLite2-Country.tar.gz		200 OK
 /usr/local/share/GeoIP/GeoLite2-Country-CSV.zip		200 OK
.....

That is ones a week, every Friday, at 2 AM ;)

Edit : if your happy with ones a week, at Friday, there is nothing to do.

bigsy

@mpfrench pfBlockerNG-devel Maxmind update frequency was changed to weekly a few months ago. See these release notes for v3.1.0_7 / v3.1.0_14, "MaxMind GeoIP (when enabled with key) is downloaded each Friday".

Gertjan

@bigsy

Hummm. Smart move, looking at the doc. Should have thought about that one. Way faster to deduct the same ting.
Anyway, the code (the real documentation) says the same thing.

mpfrench

@mpfrench said in Update Frequency of MaxMind's GeoIP database:

at the tab Firewall/pfBlockerNG/IP/GeoIP
"The GeoIP database is automatically updated the first Tuesday of each month. (To avoid any MaxMind update delays, update is now scheduled for the first Thursday of each month.)"

It appears that the only thing that needs to be changed is the verbiage in the GUI to match what the code is doing. Weekly updates at 2am Friday is fine with me.

mpfrench

On second thought, since MaxMind is updating on Fridays, it would make better sense to change pfB's schedule to update on Saturdays to make sure that we don't miss getting the latest data. Alternatively, if the CRON job remains updating on Fridays, change the time from 2am to 11pm(23:00).

ipinfo_devrel

I am the DevRel of IPinfo. We launched an open access (CC BY-SA 4.0) full accuracy database for country and ASN information, that has daily updates.

I talked a bit with PFblockerNG's core developer. The issue is that even though the database is free and provides daily updates, they are not a drop-in replacement. If anyone wants to crack at integrating our databases in pfB, let me know. We provide the database in an MMDB format, however the data schema is different.

Docs: https://ipinfo.io/developers/ip-to-country-asn-database