Updating access lists more often than 1 hour

jsone

currently making text files available on an external webserver, these files are updated every 60 to 300 seconds, pfsense via pfblockerng only updates once an hour.

at a quicker frequency, i would like pfblockerng to be able to read these files and add the records to the tables that store those aliases without reloading the entire firewall or purging the existing data from the table, as we are dealing with a bajillion records at times, (50k to 200k)

i would like to keep the firewalls function separate from these external systems otherwise i would consider the option of an rbl on the external servers.

how best can we sync the files from a weburl at a greater frequency without a major disruption to the statefull inspection process?

Thanks!
-Jim

ps. i wrote this when i had a fever.

BBcan177

Currently its set to once an hour… May change in future, but you can add a custom script to download your feed at what ever interval is appropriate and update the Aliastable... Then once per hour, pfSense will re-download the feed and update it....

So in a shell script, after downloading the feed to a folder in pfSense, you can use this command to update the Aliastable:

pfctl -t pfB_YOUR_ALIASTABLE -T replace /var/db/pfblockerng/YOUR_FEED_NAME.txt

Hope that helps!

gessel

With the Cron package, it is pretty easy to schedule curl to pull a target block list (for example, one generated locally with fail2ban) and put it in, say /var/db/pfblockerng/original/mybl.orig, say at */1 * * * * *

This gets updated as one would expect.

However, using, say, native aliases (in order to ensure one's anti-lockout rules remain sorted first past local updates), I'm not sure how to execute whatever code parses the .orig file into the clean, IPs only .txt file in /native/

Executing pfctl on the .orig file throws an error as there are still unparsed header lines. Is there a scriptable command, something like /pfblockerng.php [dcc, cron] that would handle parsing should we do our own? (in this case, not too hard, I suppose with grep.)

then reload the table with pfctl?

gessel

Oh, also, I think the command is

pfctl -t pfB_YOUR_ALIASTABLE -T replace -f /var/db/pfblockerng/YOUR_FEED_NAME.txt

the "-f" necessary to specify the target file.