pfBlockerNG/pfBlockerNG-devel v3.2.0_2
-
I have observed the situation that under pfSense 23.01 with the current version of pfBlockerNG 3.2.0_1 both in the regular version as well as the DEVEL variant the update and reload of pfBlockerNG does not work properly.
The feeds are downloaded and run through individually, but when processing at the "TLD finalize" level, it hangs indefinitely until the pfSense is restarted. I have observed this phenomenon with both pfBlockerNG variants and also with classic DNSBL unbound mode as well as Python mode.
UPDATE PROCESS START [ v3.2.0_1 ] [ 02/20/23 10:52:50 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL SafeSearch... enabled Loading DNSBL Whitelist... completed DNSBL - SafeSearch changes found - Rebuilding! [ EasyList ] Reload . completed .. Whitelist: adsafeprotected.com|amazon-adsystem.com|mail-ads.google.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 20441 20439 0 3 0 20436 ---------------------------------------------------------------------- IPv4 count=24 [ EasyPrivacy ] Reload [ 02/20/23 10:52:56 ] . completed .. Whitelist: adfox.yandex.ru|adsdk.yandex.ru|an.yandex.ru|awaps.yandex.ru|bat.bing.com|bs.yandex.ru|collector.github.com|commerce.bing.com|copilot-telemetry.githubusercontent.com|fcmatch.google.com|fcmatch.youtube.com|informer.yandex.ru|mc.yandex.com|mc.yandex.ru| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 16418 15271 16 14 0 15241 ---------------------------------------------------------------------- IPv4 count=6 [ Adaway ] Reload [ 02/20/23 10:53:02 ] . completed .. Whitelist: ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 7355 7355 382 58 0 6915 ---------------------------------------------------------------------- [ D_Me_ADs ] Reload [ 02/20/23 10:53:06 ] . completed .. Whitelist: ads.bing.com|ads.youtube.com|adserver.bing.com|amazon-adsystem.com|bs.yandex.ru|pagead.l.google.com|partnerad.l.google.com|pixel.adsafeprotected.com|video-stats.video.google.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 2701 2701 348 9 0 2344 ---------------------------------------------------------------------- [ D_Me_Tracking ] Reload [ 02/20/23 10:53:10 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 34 34 14 0 0 20 ---------------------------------------------------------------------- [ Yoyo ] Reload [ 02/20/23 10:53:14 ] . completed .. Whitelist: ads.bing.com|ads.youtube.com|adsafeprotected.com|adsdk.yandex.ru|adserver.bing.com|adservice.google.com|adservice.google.com.mt|amazon-adsystem.com|analytics.google.com|bat.bing.com|bs.yandex.ru|mail-ads.google.com|pagead.l.google.com|partnerad.l.google.com|video-stats.video.google.com|www-google-analytics.l.google.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 3680 3680 2507 16 0 1157 ---------------------------------------------------------------------- [ Abuse_ThreatFox ] Reload [ 02/20/23 10:53:17 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 26782 26642 2 0 0 26640 ---------------------------------------------------------------------- [ D_Me_Malv ] Reload [ 02/20/23 10:53:24 ] . completed .. Whitelist: ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 2735 2735 2725 9 0 1 ---------------------------------------------------------------------- [ D_Me_Malw ] Reload [ 02/20/23 10:53:27 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 1 1 1 0 0 0 ---------------------------------------------------------------------- [ Krisk_C19 ] Reload [ 02/20/23 10:53:31 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 2000 2000 0 0 0 2000 ---------------------------------------------------------------------- [ Maltrail_BD ] Reload [ 02/20/23 10:53:35 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 272221 272221 5191 0 0 267030 ---------------------------------------------------------------------- [ MVPS ] Reload [ 02/20/23 10:54:11 ] . completed .. Whitelist: ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 8730 8730 1126 43 0 7561 ---------------------------------------------------------------------- [ SFS_Toxic_BD ] Reload [ 02/20/23 10:54:16 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 43273 43272 8 0 0 43264 ---------------------------------------------------------------------- IPv4 count=2 [ Spam404 ] Reload [ 02/20/23 10:54:26 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 7066 7064 61 0 0 7003 ---------------------------------------------------------------------- [ SWC ] Reload [ 02/20/23 10:54:30 ] . completed .. Whitelist: ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 11397 11397 2666 42 0 8689 ---------------------------------------------------------------------- [ Abuse_urlhaus ] Reload [ 02/20/23 10:54:38 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 553 553 84 0 0 469 ---------------------------------------------------------------------- [ PhishTank ] Reload [ 02/20/23 10:54:42 ] . completed .. Whitelist: 543.sites.google.com|accounts.google.com|docs.google.com|drive.google.com|forms.yandex.com|play.google.com|s3.amazonaws.com|script.google.com|sites.google.com|storage.cloud.google.com|www.bing.com|www.google.com|www.sites.google.com| ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 26468 21536 66 13 0 21457 ---------------------------------------------------------------------- IPv4 count=97 [ OISD ] Reload [ 02/20/23 10:54:50 ] . completed .. Whitelist: ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 894756 894756 98405 436 0 795915 ---------------------------------------------------------------------- [ AZORult_BD ] Reload [ 02/20/23 10:56:37 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 982 982 749 0 0 233 ---------------------------------------------------------------------- [ Ponmocup ] Reload [ 02/20/23 10:56:43 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 50 50 0 0 0 50 ---------------------------------------------------------------------- [ H3X_1w ] Reload [ 02/20/23 10:56:49 ] . completed . No Domains Found! Ensure only domain based Feeds are used for DNSBL! [ Rescure_DNSBL ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 500 500 148 0 0 352 ---------------------------------------------------------------------- ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 02/20/23 10:57:07 ] TLD: TLD analysis............. completed [ 02/20/23 10:58:16 ] TLD finalize.I have tried it on existing pfSense environments but also on new installations. In both cases exactly the same.
Thanks and greetings
-
@roger-s
This is a known issue. Read the first post from @BBcan177
Full Details hereThere is still a Regression with TLD Wildcard Feature in pfSense+ (possibly in pfSense 2.6 also) due to some recent changes to FreeBSD Grep command. This is being reviewed and will provide more updates as soon as we have a solution.
The only solution for now is to disable Wildcard Blocking (TLD) in pfBlockerNG -
B Beerman referenced this topic on
-
B Beerman referenced this topic on
-
B Beerman referenced this topic on
-
So I tried to update to 3.2.0_2 but the update is failing.
>>> Upgrading pfSense-pkg-pfBlockerNG... Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. The following 1 package(s) will be affected (of 0 checked): Installed packages to be UPGRADED: pfSense-pkg-pfBlockerNG: 3.2.0_1 -> 3.2.0_2 [pfSense] Number of packages to be upgraded: 1 2 MiB to be downloaded. pkg-static: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_amd64-pfSense_plus_v23_01/All/pfSense-pkg-pfBlockerNG-3.2.0_2.pkg: Not Found FailedEDIT: Update works now please disregard.
-
Installed this version on my SG-3100, using DNSBL python mode, with geoIP.
It's working perfectly fine for me, thanks a lot for all your efforts in fixing the problems, BBcan177 and Netgate team.
-
@jrey BBcan177 posted a fix for the download problem: https://www.reddit.com/r/pfBlockerNG/comments/116fuie/temp_workaround_to_get_latest_v320_2_files/
-
Thanks, but I wasn't looking for the 3.2.0_2 package.
The issue for me had nothing to do with the lack of that package. I spent some time working through the code and providing feedback to @BBcan177 over the weekend. A local code change worked out to resolve the issue. So for my case (which was a real problem documented in this thread by the way) I'm not sure that changes made to test the change would be in the 3.2.0_2 package at this point. So I'm unlikely to even try until all my current testing is complete.
PS @BBcan177 I just waiting for a 12:00 time cycle to pass "12hours" and I'll send you the notes. Thanks JR
-
Looks like the update worked now, idk why it was not before.
Does this also fix the PHP errors and freezing of the Pfblockerng widget on the dashboard?
EDIT: Looks like this PHP error is still present, it freezes PfSense+ 23.01 so hopefully its fixed soon. I had to remove the pfblockerng widget and all is well again.
-
@raidflex
I am working on the next version. Can you test this widget patch please?curl -o /usr/local/www/widgets/widgets/pfblockerng.widget.php "https://gist.githubusercontent.com/BBcan177/6d4e9aa41cbbd802b69c0501a2c7d092/raw" -
@bbcan177 said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:
@raidflex
I am working on the next version. Can you test this widget patch please?curl -o /usr/local/www/widgets/widgets/pfblockerng.widget.php "https://gist.githubusercontent.com/BBcan177/6d4e9aa41cbbd802b69c0501a2c7d092/raw"This did fix the widget issue. I also rebooted to ensure no PHP errors are present and the freezing has also been resolved. Thank you!
-
@raidflex Thanks for testing!
-
@bbcan177 Do you have a fix to test for the URL validation errors?
-
@dennypage trying to remember which error in particular? But there was a change made to v3.2.0_2 to avoid appending any local DNS to the validation. So if the url was feed.com, the validation uses "feed.com." (added a trailing dot)
-
@bbcan177 said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:
@dennypage trying to remember which error in particular?
This one: pfBlockerNG update errors
-
@dennypage should already be fixed in v3.2.0_2
"Experience is something you don't get until just after you need it."
Website: http://pfBlockerNG.com
Twitter: @BBcan177 #pfBlockerNG
Reddit: https://www.reddit.com/r/pfBlockerNG/new/ -
@bbcan177 said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:
should already be fixed in v3.2.0_2
v3.2.0_2 isn't showing yet in 23.01, but I assume it will soon. Thanks.
-
@dennypage I updated to it on 23.01
There was error first time clicked on it that couldn't download package, but it worked 2nd time
-
@bbcan177 And of course, as soon as I post it shows up.
3.2.0_2 does indeed correct the URL issue. Thanks!
-
@BBcan177 Seems to be stuck when updating
-
I take it the TLD issue is still not fixed.
This update only addresses the GeoIP problem?
Firewall: NetGate,Palo Alto-VM,Juniper SRX
Routing: Juniper, Arista, Cisco
Switching: Juniper, Arista, Cisco
Wireless: Unifi, Aruba IAP
JNCIP,CCNP Enterprise -
B BBcan177 unpinned this topic on
-
@johnpoz
Didn't see any update after multiple refresh on "Installed Packages" so I did an force update from the console "13", then back to "Installed Packages". Now I see multiple updates ready for update, weird :)
