PfBlockerNG
-
I what to see the complete list of IP's on a alias. So i'm looking at Diagnostics > Tables and selecting the alias table. When I do this I get a blank page and in pfsense the alert:
Loading really large files into the browser can cause issues… Try to use the pfBlocker Log tab (Which also might have issues with these large files) or view the files from the shell. Files are located in the following folders:
/var/db/pfblockerng/deny or /permit or /match or /native
You can view these files with your choice of file editor like vi or ee…
-
Sometimes after cron has run and i read the log file it has a very small number at the end which does not correspond to the total addition of the numbers on the block files.
This is fixed with a force reload and the number matches the number above in the log file. (its usually about 1/8th the number)
Another thing I've noticed is the number on the widget shown after deny does not correspond to either of these numbers and I'm not using de-duplication so why is that? And is there anything i can do to investigate?
Thanks
Rob
UPDATE: BBcan177 confirmed this issue is just a timing issue and how to check it.
-
Hello eveybody,
So im trying to block TOR, I wrote this script to collect TOR exit nodes, and write it to a txt in a web server. So pfBlocker goes and downloads this file and adds the IPs, as shown below;
CRON PROCESS START [ 10/22/15 16:55:48 ]
[ TOR01 ]
Remote timestamp: Thu, 22 Oct 2015 22:53:58 GMT
Local timestamp: Wed, 21 Oct 2015 21:03:50 GMT
Updates Found
UPDATE PROCESS START[ TOR01 ] Downloading New File –--------------------------------------------------------
Original Masterfile Outfile [ Post Duplication count ]
–--------------------------------------------------------
7375 6416 6416 [ Passed ]
–--------------------------------------------------------===[ Aliastables / Rules ]================================
No Changes to Firewall Rules, Skipping Filter Reload
Updating: pfB_TOR
1131 addresses added.783 addresses deleted.===[ FINAL Processing ]=====================================
[ Original count ] [ 7375 ]
[ Processed Count ] [ 6416 ]
===[ Deny List IP Counts ]===========================
6416 /var/db/pfblockerng/deny/TOR01.txt
====================[ Last Updated List Summary ]==============
Oct 22 16:53 TOR01
Sanity Check (Not Including IPv6) ** These two Counts should Match! **
–----------
Masterfile Count [ 6416 ]
Deny folder Count [ 6416 ]Duplication Sanity Check (Pass=No IPs reported)
–----------------------
Masterfile/Deny Folder Uniq check
Deny Folder/Masterfile Uniq checkSync Check (Pass=No IPs reported)
IPv4 Alias Table IP Total
6416
IPv6 Alias Table IP Total
0
Alias Table IP Counts
6416 /var/db/aliastables/pfB_TOR.txt
pfSense Table Stats
table-entries hard limit 2000000
Table Usage Count 9765UPDATE PROCESS ENDED [ 10/22/15 16:55:50 ]
Then it adds the rules to the firewall, but is not blocking any of the IPs. Can you guys help me … what Im doing wrong??
Thanks
-
Can you show the rules that are added?
-
Can you show the rules that are added?
Thank you for your reply, here you go
- The first screenshot is WAN, to block from the outside and just added outbound too.
- Next is LAN with it its default rules plus the auto block.
- The last one is for my office branch, in which the rule for 10.4.2.3 is to get acces to the internet via squid/dansguardian at port 3128 on that pfsense.
-
Where are you seeing these, can you post the log showing the TOR traffic entering your network?
Also should the WAN only be blocking TCP TOR traffic? (because it is, not * like the others)
Rob
-
Is the Reputation Settings supposed to give the option to choose all 3 :
Individual List Reputation, Collective List Reputation (pMAX), Collective List Reputation (dMAX)Reading the descriptions it seems as though the dMAX makes the most sense but I can't tell what the other 2 would mean– so then I don't know if I quite understand what's going on here.
Seems like that whole feature needs some fresh eyes to write the descriptions because it doesn't answer the what's and why's.
Also, on the same page, is the entire Emerging Threats IQRISK IPv4 Reputation section a subscriber feature or just portions of it? For clarity, if the section could be hidden unless a checkbox is checked or something it would help in knowing if it's relevant to those who don't pay for it. Just a thought.
Thanks!
-
[ [color=red]XML RPC Sync ] Improvements to http/https and Username. The 'General Tab settings' can also be excluded from the sync to allow for Site specific customizations.
Is the XML RPC Sync meant to support sync to an IPv6 address? When I try it with just an IP v6 address the sync times out because it is trying to use the address in a non-literal format. When I enter the address in the literal form with [] around it, I get the following error (which is the same when trying to use a host name).
php-fpm[26593]: /pkg_edit.php: [pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address or Port settings.I am already synchronising some of the pfSense config over v6 successfully.
-
php-fpm[26593]: /pkg_edit.php: [pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address or Port settings.Did you enter the port field? It will log that error message if the address is not valid (IPv4/6) or if the port is not valid…
Here is a snippet of that validation code:
// Validate Replication Target IP Address and Port Settings if (!is_ipaddr($sync_to_ip) || !is_port($port)) { log_error("[pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address or Port settings."); $success = FALSE; return $success; } -
Is the Reputation Settings supposed to give the option to choose all 3 :
For Reputation, 'Max', 'dMax' and 'pMax' are all individual settings.
'Max' is for each individual List, while the other two are done after downloading of the lists has completed. 'dmax' and 'pMax' Reputation processes analyze all of the lists as a collective.
-
Did you enter the port field? It will log that error message if the address is not valid (IPv4/6) or if the port is not valid…
Yep, and if I change the IPv6 address for an IPv4 one and change nothing else, it works without complaining. The logs show it is using the port - it just does not like a v6 address, with [] or without.
-
May I suggest using a hostname? :P
-
May I suggest using a hostname? :P
I tried it (when I was just going to use the dynamic v4 for the sync) but it gives the same error as the literal IPv6 address.
-
Maybe you'd rather post a screenshot of what you actually configured.
-
Maybe you'd rather post a screenshot of what you actually configured.
Some background on the setup with screenshots coming up :)
So, the setup is two pfSense boxes running the latest release connected to two separate cable connections. Both have a dynamic v4 WAN address and both have v6 through a tunnelbroker. THere is a v4 and v6 IPSEC VPN between the boxes. This is the XML RPC Sync settings I have for pfBlocker and this is what I have for syncing just the aliases with the pfSense XML RPC Sync.
As per this post in the docs, comunications between the pfSense boxes over IPSEC won't work without additional routes. With the routes in place, both the pfBlocker and system XML RPC sync work with the LAN IPv4 address (10.x.x.x) where the IPv6 address is in both screenshots BUT the routes break routing for some other machines that need to use the VPN so having them in full-time is not an option so my plan was to just shove the sync traffic over the WAN, either v4 or v6.
With the system sync settings, I can stick in an IPv6 address or a hostname and it works. With the pfBlocker sync, if I put in a v6 address as 2001:xxx:xxx:xxx::xxx (without []) I get this in the log:
php-fpm[63574]: /pkg_edit.php: [pfBlockerNG] XMLRPC syncing to https://2001:xxx:xxx:xxx::xxx:443.To me, this looks like pfBlocker wants the literal address in the config as the system's sync gives this entry in the log and works:
php-fpm[46766]: /rc.filter_synchronize: Filter sync successfully completed with https://[2001:xxx:xxx:xxx::xxx]:443.So I put in a literal address as [2001:xxx:xxx:xxx::xxx] in the pfBlocker sync settings and I get this entry in the logs:
php-fpm[24610]: /pkg_edit.php: [pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address or Port settings.If I try a host name instead of the IPv6 literal address, I get the same error message. If I put in the public dynamic IPv4 address, it works. For clarity, I am NOT changing any sync settings apart from the destination address each time.
So, what works for the system sync is IPv4 address, v6 address and hostname. What works for me with the pfBlocker sync is IPv4 address only.
-
Try this with System Patches:
--- a/usr/local/pkg/pfblockerng/pfblockerng.inc 2015-10-24 23:39:52.249959365 +0200 +++ b/usr/local/pkg/pfblockerng/pfblockerng.inc 2015-10-24 23:39:52.274963703 +0200 @@ -2897,8 +2897,12 @@ } // Validate Replication Target IP Address and Port Settings - if (!is_ipaddr($sync_to_ip) || !is_port($port)) { - log_error("[pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address or Port settings."); + if (!is_ipaddr($sync_to_ip) && !is_hostname($sync_to_ip) && !is_domain($sync_to_ip)) { + log_error("[pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address."); + $success = FALSE; + return $success; + } elseif (!is_port($port)) { + log_error("[pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target Port settings."); $success = FALSE; return $success; } @@ -2907,6 +2911,10 @@ if (empty($synctimeout)) { $synctimeout = 150; } + + if (is_ipaddrv6($sync_to_ip)) { + $sync_to_ip = "[{$sync_to_ip}]"; + } $url = "{$protocol}://{$sync_to_ip}"; if ($port == "") { $port = $config['system']['webgui']['port']; }; -
With a literal v6 address, I now get:
php-fpm[17959]: /pkg_edit.php: [pfBlockerNG] XMLRPC sync terminated due to mis-configured Replication Target IP Address.With a non-literal address, it now works - thank you very much :)
-
Yeah, don't put literals there. (Also, if you could test with some hostname, would be appreciated.)
-
Yep, a host name now works as well according to the logs.
-
One minor thing. The header field was a little strange. Im used to a header meaning something to ignore as actual input. So my immediate intuition was to input something like a # because my lists did not load right and the force update said:
** TERMINATED - Header contains Blank/International/Special or SpacesBut I found in the forum that its a header like a description. So maybe a little description for header would be helpfull.
I was going to name it "filename" but I think it can also be misleading (users might think to enter the URL filename there)
There is a description of what a Header should be just above the URL settings in each alias. Suggestions are always welcome! :)
Header sounds like it's something too programmy. Why not just call it a label?
btw, I've been trying to add this ad blocking list because they have tons of formats yet get a download failed so I dunno
http://pgl.yoyo.org/adservers/iplist.php?ipformat=plain&showintro=0&mimetype=plaintexthttp://pgl.yoyo.org/adservers/ – main page which offers various list formats and flags to reformat in different ways (csv, etc)
