pfBlockerNG-devel v3.1.0_9 / v3.1.0_15

juliokele

v3.1.0_9 fix issue saving URLs in the IPv4/v6/DNSBL Tab doesn't work.

error.log:
PFB_FILTER - 2 | pfb_download_failure [ 12/21/22 16:51:46 ] Invalid URL (not allowed) [ https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts ]
PFB_FILTER - 2 | Category_edit [ 12/21/22 16:55:00 ] Invalid URL (not allowed) [ https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts ]

Edit:
solved with v3.2.0_2

dennypage

@juliokele said in pfBlockerNG-devel v3.1.0_9 / v3.1.0_15:

v3.1.0_9 fix issue saving URLs in the IPv4/v6/DNSBL Tab doesn't work.

error.log:
PFB_FILTER - 2 | pfb_download_failure [ 12/21/22 16:51:46 ] Invalid URL (not allowed) [ https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts ]

The URL is valid, and contains no escaped characters. Is it possible that you had a DNS resolution failure?

juliokele

@dennypage DNS resolution is ok, curl working just fine.

BBcan177

@juliokele said in pfBlockerNG-devel v3.1.0_9 / v3.1.0_15:

DNS resolution is ok, curl working just fine.

What does this reply with?

ping raw.githubusercontent.com

Then in pfSense Diag / Execute PHP Commands:

print_r(is_ipaddr_configured('IP FROM PING COMMAND ABOVE HERE'));

Post the Response

planedrop

FYI, still digging on this a bit, but seems I can't save alert settings to stop auto refreshing with this version. I uncheck the box, click Save Alert, but it continues to auto refresh and the box isn't unchecking.

TheXman

@planedrop
I had noticed the same thing. I'm not sure if it is the way that @BBcan177 will ultimately fix it, but I was able to resolve the issue by changing line 42 of /usr/local/www/pfblockerng/pfblockerng_alerts.php to the following:

$alertrefresh	= $pfb['aglobal']['alertrefresh']	!= ''	? $pfb['aglobal']['alertrefresh']	: 'off';  //Modified by TheXman

If $pfb['aglobal']['alertrefresh'] had not been previously set, it was defaulting $alertrefresh to "on". I changed it to default to "off". This also corrected an issue where every time the page refreshed, it did a DNS query of the pfSense host name. After making the change and refreshing the page, it should recognize and save the value of the checkbox.

alexander.deca

We have upgraded to this release and we have suddenly dropped rules as IPv4's have been deleted in the GeoIPv4 lists -> after the cron job runs to update the GeoIPv4 every 2 hours, when manually doing an update force reload it works again as it adds many address again to the different lists (see below) and it breaks again when the cron job runs again. So temporarily disabled the cron job and it is stable.

(re-installed the package but this did not help) Any advice or what we can do to troubleshoot?

Updating: pfB_PRI1_v4
1941 addresses added.9 addresses deleted.
Updating: pfB_PRI2_v4
no changes.
Updating: pfB_PRI3_v4
2647 addresses added.11 addresses deleted.
Updating: pfB_PRI4_v4
1811 addresses added.64 addresses deleted.
Updating: pfB_PRI5_v4
no changes.
Updating: pfB_GeoIP_Unifi_v4
2058 addresses added.
Updating: pfB_GeoIP_Belgium_v4
898 addresses added.
Updating: pfB_GeoIP_IPN_clients_v4
5617 addresses added.39 addresses deleted.
Updating: pfB_GeoIP_3CX_clients_v4
9496 addresses added.48 addresses deleted.
Updating: pfB_GeoIP_SBC_dyn_clients_v4
15411 addresses added.293 addresses deleted.
Updating: pfB_IPN_Client_3CX_pub_EDL_v4
no changes.
Updating: pfB_GeoIP_EU_v4
9 addresses added.88 addresses deleted.

best regards

crsesilva

Hello,

Is anyone else having this problem?

After upgrading to pfBlockerNG-devel v3.1.0_9, using UT1 blacklist.

The error only with this category the others are Ok.

Thanks.

Gertjan

@crsesilva

The issue : the "header" is "ut1_audio-video", this is checked, like our own feeds.
One of the criteria for the 'headers' is is : letters and underscores only. ( see preg_match("/\W/", $input) on line 447, pfblockerng.inc )
The thing is : "ut1_audio-video" contains a dash : '-' => the \W check fails.

Take a look at all the files here : /var/db/pfblockerng/ut1/
All the file names are correct, only '/var/db/pfblockerng/ut1/ut1_audio-video' has a dash, which should be a underscore, I guess.

So, rename the file, change the - for _, the file name becomes : ut1_audio_video
Edit /usr/local/pkg/pfblockerng/ut1_global_usage and locate (line 112 ?) "NAME: audio-video"
Change it for "NAME: audio_video".
Save.

Back in pfBlockerng :

Disable :

and a full reload.

Enable again.
What I saw : our "Audio video" is now unchecked, it was checked before -> normal, we changed the internal name.
Check it.
( I polluted the config now ?)
and a full reload.

Champagne !

( this is as it is : it worked for me [ because I found the issue during debugging, and I think I found it] - my solution is only a work around )

crsesilva

@gertjan

Thanks for responding so quickly.

I applied the mentioned changes and it worked great.

I sincerely appreciate all your efforts.

Thanks so much for your time and support.

juliokele

@bbcan177 the response is always empty:

I've figured it out, v3.1.0_9 don't work with my HA/HAProxy setup.
More precisely with Host Overrides for HAProxy on LAN-Carp-VIP Address (192.168.1.254).
v3.1.0_7 working fine.

Draco

@bbcan177 Just upgraded to 3.1.0_9 (from _7) on 22.05-RELEASE (amd64) after disabling pfBlocker before install (enable after, and ran Update). Everything looks great so far!

planedrop

@thexman I'll give this a shot and see how it goes, thank you!

JeGr

@bbcan177 Hi,

after upgrading to _9 installations that utilized the IOC lists from ThreatFox / abuse.ch
(https://threatfox.abuse.ch/export/)
won't download the list anymore. Be it the JSON file from the last 48 hours or the full data dump (zipped with "zip" not gzip), it always ends in a MIME Type Error:

[ Abuse_ThreatFox_v4 ]		 Downloading update .. 200 OK.
 PFB_FILTER - 18 | pfb_download Failed or invalid Mime Type Compressed: [application/x-decompression-error-gzip-Unknown-compression-format|0]

or

[ Abuse_ThreatFox_48h_v4 ]	 Downloading update .. 200 OK
[PFB_FILTER - 17] Failed or invalid Mime Type: [application/json|0]

 [ pfB_PRI1_v4 - Abuse_ThreatFox_48h_v4 ] Download FAIL
  DNSBL, Firewall, and IDS (Legacy mode only) are not blocking download.

That only happened recently after upgrading, before it was running fine with _6 or _7 I believe the systems were on. The old list from before the update was/is still being used so it worked before.

List URLs are working via shell/curl or in browser so no problem on that front. It's only when trying to download it with pfB that those Mime Type errors pop up.

Edit: Edit: Curl in shell sees normal content types: content-type: application/json or content-type: application/zip so no clue where that failed or invalid types come from.

Cheers
\jens

fireodo

@jegr

Hi and a good New Year.

Read this: Mime Types

I guess it adress your problem.

Cheers,
fireodo

BBcan177

@jegr
The latest code now validates the contents of all Compressed files before extraction to ensure that the file-mime type is allowed.

There is a currently an incompatibility with ZIP files and the 'file' mime-type magic database that validates the Compressed file type contents as "application/x-decompression-error-gzip-Unknown-compression-format".

I tried to see if the file maintainer could add functionality to fix this, but unfortunately I had no luck with that. So for now, I have no way to validate the ZIP file contents before extraction, so in the next version it will first Extract ZIP compressed files, and then perform the file-mime type validation on the extracted file. I will continue to see if I can find a way to validate before extraction.

The second part is that the next version will add "application/json" as a valid mime-type.

Thanks for the report!

Draco

@bbcan177 said in pfBlockerNG-devel v3.1.0_9 / v3.1.0_15:

The second part is that the next version will add "application/json" as a valid mime-type.

Does that mean a URL like MSFT Azure IP Blocks will be a valid download source? At present, I have to download this on my PC then upload it to /var/db/pfblockerng/deny on my pfSense box.

Note the MSFT Azure IP Blocks link sends you to a page where the file download starts; that page contains a link to the most current file (at present, https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public_20230102.json).

[edit: added more info on links]

sensei-two

Hi
I was reading about the DoH/DoT/DoQ Blocking feature in DNSBL SafeSearch of my
pfBlockerNG-devel v3.1.0_9.
As far as I have understood it, it blocks clients on my LAN to use DoH/DoT, so
I was wondering if this feature can also affect DoT queries from Unbound itself since I enabled DoT in its setting.

Thanks

xpxp2002

This post is deleted!

xpxp2002

@sensei-two I believe it does. I use unbound to funnel all public DNS queries to Cloudflare over DoT. But I took a different approach to blocking all other DoT/DoH.

I have pfB-NG creating a deny alias using these blocklists:
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall_ipv4
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall_ipv6

Then I created rules on the LAN-side interfaces that destination blocks the aliases that these blocklists create.

Unbound should be using your WAN interface to reach your DoT provider and it won't have an ingress interface (i.e. there's no LAN-to-WAN or WAN-to-LAN flow). The only way I think you could control the firewall's own egress-to-WAN would be using a floating rule.