PfblockerNG does not respect update frequency?
-
I have been using pfblockerNG for quite a while now but recently noticed some of my feeds are not being updated (see errors on pfsense's homepage).
[ DNSBL_alexa - phishtank ] Download FAIL [ 03/25/18 14:00:05 ]
For example Phishtank no longer updates.
http://data.phishtank.com/data/online-valid.php_serialized.bz2
When I manually try to access the link in the feed with my browser, I see:
You have exceeded the request rate limit for this method. Please see the response headers for usage stats. For more information about rate limiting on Phishtank, please see our developer site: http://www.phishtank.com/developer_info.php
I believe this is because somehow my pfsense box abuses the Phishtank server and I get banned. The update frequency is set to Daily in the feed… (Once a day).
Why is this happening?
-
Did you register to get a application key on https://www.phishtank.com/register.php
May pfblockerNG see that the last download failed for more than one day ago and it retries at every cron update.
Disable the url for a few days to see if you get out of their blocklist at some point.
-
Been using Phishtank for months and months AFAIK never needed a API key.. Is this new?
In the meantime I changed CRON's setting to execute every 6 hrs instead of hours (can it cause other issues?) and disabled the feed entry until I see if they unblock me…
-
@lpallard:
Been using Phishtank for months and months AFAIK never needed a API key.. Is this new?
@https://www.phishtank.com/developer_info.php:
If you do intend to fetch these files automatically, please register for an application key and see below for instructions on how to use it to request files. Without this key, you will be limited to a few downloads per day.
-
I binned off phishtank a while back as it had a really high false-positive count. I got fed up with it blocking genuine content…
