Update lists fail

Qinn · May 7, 2018, 10:58 AM

I am getting more fails on the lists download in pfBLockerNG 2.1.2_3, anyone experiencing the same?

[DNSBL_BBcan177 - MS_2 ] Download FAIL [ 05/07/18 00:03:56 ]
[ DNSBL_Ads - Disconnect ] Download FAIL [ 05/07/18 00:03:17 ]
[ DNSBL_Ads - WinTelemetry ] Download FAIL [ 05/07/18 00:03:00 ]
[ DNSBL_Ads - Quidsup ] Download FAIL [ 05/07/18 00:02:42 ]
[ DNSBL_Ads - Disconnect_Ad_Filter ] Download FAIL [ 05/07/18 00:02:25 ]
[ DNSBL_Ads - Disconnect_Trackers ] Download FAIL [ 05/07/18 00:02:08 ]
[ DNSBL_Ads - PiHole_StevenBlack ] Download FAIL [ 05/07/18 00:01:43 ]

When I look in the log I see, they are mostly related to SSL certificate problem

[ WinTelemetry ] Downloading update [ 05/07/18 00:02:45 ] . cURL Error: 60
SSL certificate problem: unable to get local issuer certificate Retry in 5 seconds…
. cURL Error: 60
SSL certificate problem: unable to get local issuer certificate Retry in 5 seconds...
. cURL Error: 60
SSL certificate problem: unable to get local issuer certificate Retry in 5 seconds...
.. unknown http status code

And that Maxmind is old...

MaxMind GeoLite2 Date/Time Stamp
Last-Modified: Mon, 06 Nov 2017 19:15:47 GMT
Duplicate Represented IP4 Networks: 31851
Duplicate Represented IP6 Networks: 3052

Kai_null · May 10, 2018, 5:52 AM

Is the date time correct on the pfsense box?

Qinn · May 10, 2018, 9:24 AM

@Kai_null:

Is the date time correct on the pfsense box?

Yup, the dashboard pfS reads "Current date/time Thu May 10 11:23:16 CEST 2018"

btw what is your MaxMind from, here it reads in the dashboard "MaxMind: Last-Modified: Mon, 06 Nov 2017 19:15:47 GMT"

Kai_null · May 10, 2018, 3:17 PM

MaxMind: Last-Modified: Tue, 03 Apr 2018 22:41:00 GMT

I am out of good ideas to help you :(.

I did see this which I thought was interesting:
https://serverfault.com/questions/681835/freebsd-pfsense-root-ca-fails

Suggests to me your problem is not with pfblockerng but rather with openssl. Perhaps it is unable to update its cert store?

other ideas which may or may not help you:

https://www.google.com/search?q=pfblockerng+%22unable+to+get+local+issuer+certificate%22&oq=pfblockerng+%22unable+to+get+local+issuer+certificate%22

https://www.google.com/search?q=pfsense+%22unable+to+get+local+issuer+certificate%22&oq=pfsense+%22unable+to+get+local+issuer+certificate%22

https://www.google.com/search?q=freebsd+%22unable+to+get+local+issuer+certificate%22&oq=pfsense+%22unable+to+get+local+issuer+certificate%22

https://www.google.com/search?q=openssl+%22unable+to+get+local+issuer+certificate%22&oq=pfsense+%22unable+to+get+local+issuer+certificate%22

wish I had answers for you.

Curious to know what you eventually find.

ps: I have found that a 2 am reinstall in some cases is a quicker bug fix then diving down the rabbit hole. I guess it depends on how stock your config is.

BBcan177 · May 11, 2018, 2:46 AM

@Qinn

All of those feeds are hosted on Github and/or Amazon, so I would assume that a DNSBL Feed or an IP Blocklist is blocking access on download… Check the pfBlockerNG Alerts Tab...

Might need to whitelist:

raw.githubusercontent.com
s3.amazonaws.com

or wildcard whitelist the whole domain

.githubusercontent.com
.amazonaws.com

For the MaxMind issue, from the pfSense box, check to see if you can access the MaxMind site:

host -t A geolite.maxmind.com
geolite.maxmind.com has address 104.16.37.47
geolite.maxmind.com has address 104.16.38.47

Then try to ping the resulting IPs and get a reply.

The MaxMind download errors are reported to the error.log file…

Once you have fixed connectivity, you can manually download the MaxMind database with this command:

php -f /usr/local/www/pfblockerng/pfblockerng.php dc

Qinn · May 12, 2018, 7:34 AM

@Kai_null:

MaxMind: Last-Modified: Tue, 03 Apr 2018 22:41:00 GMT

I am out of good ideas to help you :(.

I did see this which I thought was interesting:
https://serverfault.com/questions/681835/freebsd-pfsense-root-ca-fails

Suggests to me your problem is not with pfblockerng but rather with openssl. Perhaps it is unable to update its cert store?

other ideas which may or may not help you:

https://www.google.com/search?q=pfblockerng+%22unable+to+get+local+issuer+certificate%22&oq=pfblockerng+%22unable+to+get+local+issuer+certificate%22

https://www.google.com/search?q=pfsense+%22unable+to+get+local+issuer+certificate%22&oq=pfsense+%22unable+to+get+local+issuer+certificate%22

https://www.google.com/search?q=freebsd+%22unable+to+get+local+issuer+certificate%22&oq=pfsense+%22unable+to+get+local+issuer+certificate%22

https://www.google.com/search?q=openssl+%22unable+to+get+local+issuer+certificate%22&oq=pfsense+%22unable+to+get+local+issuer+certificate%22

wish I had answers for you.

Curious to know what you eventually find.

ps: I have found that a 2 am reinstall in some cases is a quicker bug fix then diving down the rabbit hole. I guess it depends on how stock your config is.

Thanks for your time and thoughts.

cheers Qinn

Qinn · May 12, 2018, 7:34 AM

@BBcan177:

@Qinn

All of those feeds are hosted on Github and/or Amazon, so I would assume that a DNSBL Feed or an IP Blocklist is blocking access on download… Check the pfBlockerNG Alerts Tab...

Might need to whitelist:
raw.githubusercontent.com
s3.amazonaws.com
or wildcard whitelist the whole domain
.githubusercontent.com
.amazonaws.com
For the MaxMind issue, from the pfSense box, check to see if you can access the MaxMind site:
host -t A geolite.maxmind.com
geolite.maxmind.com has address 104.16.37.47
geolite.maxmind.com has address 104.16.38.47
Then try to ping the resulting IPs and get a reply.

The MaxMind download errors are reported to the error.log file…

Once you have fixed connectivity, you can manually download the MaxMind database with this command:
php -f /usr/local/www/pfblockerng/pfblockerng.php dc

Thanks, I will try and report back.

Cheers Qinn

Qinn · May 12, 2018, 8:02 AM

a```
host -t A geolite.maxmind.com

returns```
geolite.maxmind.com has address 10.10.10.1

If I disable DNSBL I get

host -t A geolite.maxmind.com
geolite.maxmind.com has address 104.16.38.47
geolite.maxmind.com has address 104.16.37.47

I added .geolite.maxmind.com to the Custom Domain Whitelist in DNSBL, but nothing changes, still I get the ip of the VIP, instead op 104.16.etc.

BBcan177 · May 12, 2018, 2:13 PM

@Qinn:

a```
host -t A geolite.maxmind.com
returns```
geolite.maxmind.com has address 10.10.10.1
If I disable DNSBL I get
host -t A geolite.maxmind.com
geolite.maxmind.com has address 104.16.38.47
geolite.maxmind.com has address 104.16.37.47
I added .geolite.maxmind.com to the Custom Domain Whitelist in DNSBL, but nothing changes, still I get the ip of the VIP, instead op 104.16.etc.

When you manually add a domain or an IP to a whitelist, you need to run a Reload to get it to apply the change or wait until the next cron run to execute… Alternatively, when you whitelist from the Alerts Tab, the whitelist/suppression will take effect immediately.

Qinn · May 12, 2018, 2:32 PM

Thanks yeah I was pulling my hair ;) After I added it to the whitelist, I did a "Select 'Reload' option" All and hit run got a "exists" in return. Then I realized it and a reload solved it.

Now

host -t A geolite.maxmind.com
geolite.maxmind.com has address 104.16.37.47
geolite.maxmind.com has address 104.16.38.47

Now I works, btw I can do a manual update of MaxMind, but when will pfblockerNG do it automatically?

Thanks BBCan177 I hope everything it going well with the next major release, thumbs up!!

BBcan177 · May 12, 2018, 2:39 PM

@Qinn:

Thanks yeah I was pulling my hair ;) After I added it to the whitelist, I did a "Select 'Reload' option" All and hit run got a "exists" in return. Then I realized it and a reload solved it.

Now
host -t A geolite.maxmind.com
geolite.maxmind.com has address 104.16.37.47
geolite.maxmind.com has address 104.16.38.47
Now I works, btw I can do a manual update of MaxMind, but when will pfblockerNG do it automatically?

Thanks BBCan177 I hope everything it going well with the next major release, thumbs up!!

MaxMind is updated once per month. You will see the cron task in pfSense for that…

Thanks! Its been submitted and awaiting review by the devs...

Qinn · May 12, 2018, 2:55 PM

Thanks, did a manual update and worked like a charm.

 php -f /usr/local/www/pfblockerng/pfblockerng.php dc
Country code update Start
 Converting MaxMind Country databases for pfBlockerNG.
 Processing ISO IPv4 Continent/Country Data
 Processing ISO IPv6 Continent/Country Data [ 05/12/18 16:45:27 ]
 Creating pfBlockerNG Continent XML files
 IPv4 Africa                     [ 05/12/18 16:45:48 ]
 IPv6 Africa                     [ 05/12/18 16:45:49 ]
 IPv4 Antarctica                 [ 05/12/18 16:45:50 ]
 IPv6 Antarctica
 IPv4 Asia
 IPv6 Asia                       [ 05/12/18 16:45:59 ]
 IPv4 Europe                     [ 05/12/18 16:46:01 ]
 IPv6 Europe                     [ 05/12/18 16:46:30 ]
 IPv4 North America              [ 05/12/18 16:46:40 ]
 IPv6 North America              [ 05/12/18 16:47:01 ]
 IPv4 Oceania                    [ 05/12/18 16:47:04 ]
 IPv6 Oceania                    [ 05/12/18 16:47:06 ]
 IPv4 South America
 IPv6 South America              [ 05/12/18 16:47:08 ]
 IPv4 Proxy and Satellite        [ 05/12/18 16:47:10 ]
 IPv6 Proxy and Satellite
 IPv4 TOP 20
 IPv6 TOP 20
 pfBlockerNG Reputation Tab
Country Code Update Ended [ 05/12/18 16:47:11 ]

[2.4.3-RELEASE][root@pfSense.localdomain]/root:

Just to be sure I added the MaxMind IP's to my IPV4 whitelist also.

Qinn · May 20, 2018, 4:11 PM

@BBcan177:

MaxMind is updated once per month. You will see the cron task in pfSense for that…

I did a```
crontab -l


Cheers Qinn

RonpfS · May 20, 2018, 4:26 PM

Did you leave MaxMind Updates unticked ?

There is a Cron package you can install ;)

The job run from 4th to 10th of the month

/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php dcc >> /var/log/pfblockerng/extras.log 2>&1

Qinn · May 21, 2018, 1:31 PM

@RonpfS:

Did you leave MaxMind Updates unticked ?

There is a Cron package you can install ;)

The job run from 4th to 10th of the month
/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php dcc >> /var/log/pfblockerng/extras.log 2>&1

Thanks for you reply RonpfS. Nope I haven't disabled it, I just thought a```
crontab -l

RonpfS · May 21, 2018, 5:14 PM

On my system I get :

crontab -l

crontab: no crontab for root
```  :o

Qinn · May 22, 2018, 3:13 PM

So what does the MaxMind updates trigger?

Qinn · Jun 5, 2018, 4:09 PM

@qinn Is there a way to check when the next update of MaxMind will run?

RonpfS · Jun 5, 2018, 4:09 PM

@qinn You can check /var/log/pfblockerng/extras.log to see when the last update was done.

Qinn · Jun 5, 2018, 4:31 PM

Thanks, nice overview btw of the last 2 years, seems I have to wait for 11 juni (when updating comes once a month)