pfBlockerNG/pfBlockerNG-devel v3.2.0_2

tman222

Thanks @BBcan177 for the new version - I assume the issue raised here is also included in the list of fixes?

https://forum.netgate.com/topic/177632/pfblockerng-update-errors

Just curious as I ran into this today after upgrading to 23.01. Thanks in advance!

pdavis

Hi Bbcan177 - thank you for all your work on this great package!

After upgrading to pfSense 23.01, I uninstalled the devel version and moved to straight pfBlockerNG (3.2.0_1) per the Netgate upgrade documentation

I use python mode, and reenabled DCHP registration per that documentation, but got an error message that the python mode was disabled and the red x dnsbl status on the home page. When I check the DNSBL Mode, though, python is still enabled.

I assume that the error message and status are legacy, and going to be removed in future updateof pfBlockerNG? Let me know, thanks

[pfBlockerNG]: Terminating DNSBL Python mode due to DNS Resolver DHCP Registration option enabled! (include)

emikaadeo

@pdavis

jrey

@bbcan177 said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:

Fix issue with Feed URL validations failing due to local hostnames being appended to the DNS Validation query

is there a way to easily confirm if this could be the problem I am seeing?

since upgrading (22.05 -> 23.01) v3.2.0_1 of pfB ( I notice that on the dashboard wizard the updated times are not updating through the cron job. there is little in the view when the cron runs
CRON PROCESS START [ v3.2.0_1 ] [ 02/18/23 02:00:00 ]

No Updates required.
CRON PROCESS ENDED
UPDATE PROCESS ENDED

Previously it would at least list the various feeds and then say no update - somewhere else I can check?

My schedule has not changed. The feed list has not changed.
The pervious pfB was uninstalled (keep setting) before update and then reinstalled 3.2.0_1 after system update. The reload at that point worked, but hasn't done an update since. All the feeds are there and it is running and blocking based on the feeds from ~48 hours ago.

the error log has nothing special, and the pfblockerng.log just a sequence as noted above.

michmoor

@pdavis The documentation got ahead of the merge request. Thats whats causing confusion on reddit and here.
The fix is still pending.

provels

Am I the only one not seeing this being available? Pulled from Package Manager?

jrey

@provels

Lucky you, lol.

I'm really just trying to see how I can confirm if the fix that is suppose to be in it is the potential cause of the "nothing" happening with the feeds being updated that I am seeing in V3.2.0_1 With nothing in the error.log, and only cron entries that report "No Updates required" it is making difficult to see the "real" problem.

BBcan177

@jrey disable TLD Wildcard feature as there is a regression with FreeBSD Grep. Might need a reboot to clear the long running grep task. Working on a solution.

jrey

@bbcan177

Thanks but this already is and always has been disabled

Screen Shot 2023-02-18 at 3.04.30 PM.png

it is not just the DNSBL lists - custom lists where I am providing a specific url to get IP addresses to block from are it is not obtaining updates there either. (and I can confirm they have in fact been updated at source)

The system is still working on all the same lists it had before the system was updated.
(that is to say all the files in /var/db/pfblockerng/original have time stamps from before the update applied)
those would have to update first and that doesn't appear to be happening, through cron, or manual.
i was able to get one update by adding an IP to the custom list at the bottom, but the feeds on the same set above it never pulled new files.
just doesn't seem to be any error logged in any file I can find.

james2432

@tbr281 said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:

@BBcan177 I upgrated to pfsense 23.01 last night and that crashed pfblocker... im not too worried about it since i wanted a clean install of pfblocker.. just wanted to give you a heads up. hope the new build corrects the issue.

I fixed mine by accessing pfBlockerNG's settings page via the dashboard tool(wrench).

Going to "keep settings" and unchecking. Save. Uninstall pfBlockerNG, reinstall.

BBcan177

@jrey Can you pm or email (bbcan177 at gmail.com) your pfblockerng.log for review?

jrey

@bbcan177

So not sure this will help.

I went into the previously mentioned /var/db/pfblockerng/origianal directory and deleted one of the smaller files created within a larger group of feeds.

Cron didn't even know it was missing -- (maybe there is an index that needs to be adjusted, didn't look)

logged
CRON PROCESS START [ v3.2.0_1 ] [ 02/18/23 16:00:01 ]

No Updates required.
CRON PROCESS ENDED
UPDATE PROCESS ENDED
end

from here I went to the force a manual "update" the file in question didn't even appear on the list of files that "exist" but it didn't download it either... interesting.

wait a few minutes, and update again this time "reload" all file except the one I had deleted indicated "reload [date] completed"
the file I had deleted guess what...
Downloading update [ 02/18/23 16:20:15 ] .. 200 OK. completed ..

based on historical updates I would expect this particular file will update (at source) again in the next few hours, so I am going to leave everything now - and see if it updates itself next time (when the source updates).

Deleting original file
Cron - did not download the file (or recognize it as missing)
Manual update - does not download the file (didn't list it here either)
Manual reload - yeah a new updated file !!!

no changes made to the feeds or other settings at all in between tests.

jrey

@bbcan177 said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:

Can you pm or email (bbcan177 at gmail.com) your pfblockerng.log for review?

There is honestly nothing much in it since the update
every cron job that has run since the update only has entries like those already provided
CRON PROCESS START [ v3.2.0_1 ] [ 02/18/23 02:00:00 ]

No Updates required.
CRON PROCESS ENDED
UPDATE PROCESS ENDED

the reload and update I tried earlier (without the file being delete) show
either nothing downloaded just a list of the files the "exist" and then a reload compete for each of the existing files and never downloading anything.
There are not other messages of concern in these, the update and/or reload just didn't achieve the results of getting the newer files.

I finally thought just delete one of the existing files and see what happens, that is described in my other post.

What I have noticed has not been logged since the update is entries like for each file. (in my lists) but then again the logs have been pretty boring.

Remote timestamp: Thu, 16 Feb 2023 11:19:37 GMT
Local timestamp: Wed, 15 Feb 2023 11:19:43 GMT Update found

there is nothing in the error.og since the update.. prior to the update it would occasionally show (for example)
[ pfB_PRI1_v4 - Talos_BL_v4 ] Download Fail [ 02/16/23 10:00:18 ]
DNSBL, Firewall, and IDS (Legacy mode only) are not blocking download.
[ 02/16/23 10:00:18 ]
Restoring previously downloaded file contents... [ 02/16/23 10:00:18 ]

that entry is in fact is the last entry in the error log.
(I wasn't concerned about it because as several others have noted that particular feed has been up and down) a bunch and it would always get it later)
The other feeds I use have never logged any errors, prior to update or now (only the Talos one goes missing once and a while) although it also hasn't reported MIA since the update either, I don't think that has anything to do with the feed, but rather that it appears not to check at all)
My other post details the steps taken and the results.

BBcan177

@jrey next time delete from the deny folder. That's is the monitored folder. Not the original folder.

jrey

@bbcan177

Ok there was a cron within 10 minutes of your ask, so I nuked one file.

with the file now removed from the deny directory only the cron job now logged the same.
CRON PROCESS START [ v3.2.0_1 ] [ 02/18/23 18:00:01 ]

No Updates required.
CRON PROCESS ENDED
UPDATE PROCESS ENDED

and the file I removed from deny has not returned. If fact all the files in there still show the time stamp of my previous run at ~16:20 when the reload with the one file missing from original "reloaded" all the files and "downloaded" the one I had nuked.

So my next step delete the file in original (a different file from the one previously) but now the same one I just deleted from deny before cron and run a manual reload
Again all the files "reload" except the one in original that I nuked.
It downloaded
Downloading update [ 02/18/23 18:09:54 ] .. 200 OK. completed ..

and is now back in both "original" and "deny"

next test, delete another different one in deny and go directly to manual "update" not reload,
Downloading update .. 200 OK. completed ..
(no time stamp in that message however) but an update did download it and it updated the one in original as well

So at least I know at this point that I can delete the files in deny, run either a manual reload or update and they will download. kind of rules out the original question is it related to the DNS item I quoted.

Thanks
JR

johnpoz

@bbcan177 silly question, maybe I just need more coffee this morning.

I just updated to 23.01, and pfblocker seems to be working just fine and currently at 3.2.0_1, my take is that _2 just not available yet..

But my question is more to devel vs non devel version. I notice version numbers and package dependence for _1 seem to be in sync between the versions.

Is there any reason to move away from devel version and change over to just the NG version?

SteveITS

@johnpoz it was in the 23.01 release notes I believe ;) but they migrated devel to non devel so they are equivalent at the moment.

johnpoz

@steveits said in pfBlockerNG/pfBlockerNG-devel v3.2.0_2:

it was in the 23.01 release notes I believe

Well F me then ;) hehehe you are correct good sir..

https://docs.netgate.com/pfsense/en/latest/releases/23-01.html

The pfBlockerNG package has been updated to match pfBlockerNG-devel. After upgrade it is safe to uninstall pfBlockerNG-devel (keeping settings) and install pfBlockerNG instead.

How did I miss that? Doh! Thanks!

edit: successfully moved to just NG version of package vs -devel, all looking good here..

jrey

@johnpoz
@BBcan177

So I'm curious. is the cron job updating the feeds as expected?
I am on 3.2.0_1 and well as my previous posts are showing, the feeds are not updating through the cron job.
I can force them to update by deleting one or all of the txt files in the deny directory and running a manual update.
But when deleting the txt files in the deny directory and letting cron run, nothing happens - just returns "No Updates required."
I might be wasting my time, but was just going to read through the pfblockerng.php and see if I can spot anything different between the "update" and "cron" processing paths that might cause it.

looks to me list the function pfblockerng_sync_cron has a couple of points where it could not do anything and end update up exiting with only the "No required message" being logged. I might have another coffee myself and then add a couple of logging statements to at least see if there is a failure point in

Screen Shot 2023-02-19 at 10.46.38 AM.png

looks like any of those first three if statements could just cause a "silent" unlogged by-pass of the enclosed -
that would lead to what is being logged.

johnpoz

@jrey what is not updating.. Your lists? Yeah if there is nothing to update, why should they update?

Be happy to try and duplicate what you think is problem.. I ran a manual update after my change over to NG vs -devel

 [ Force Reload Task - All ]
 UPDATE PROCESS START [ v3.2.0_1 ] [ 02/19/23 09:31:58 ]

===[  DNSBL Process  ]================================================


===[  GeoIP Process  ]============================================


===[  IPv4 Process  ]=================================================

[ MA_v4 ]			 exists. [ 02/19/23 09:31:59 ]
[ MA_rep_v4 ]			 exists.
[ PlexRemoteCheck_v4 ]		 exists.
[ SCake_v4 ]			 exists.
[ UptimeRobot_v4 ]		 exists.
[ US_v4 ]			 exists.
[ SCake_v4 ]			 exists.
[ Uptime_v4 ]			 exists.
[ dohIPlist_v4 ]		 exists.
[ GreatWalldohIPlist_v4 ]	 exists.
[ BlockDOH_custom_v4 ]		 exists.
[ shodan_v4 ]			 exists.
[ stretchoid_v4 ]		 exists.
[ shadowserver_v4 ]		 exists.
[ ScanDeny_custom_v4 ]		 exists.
[ AS14061_v4 ]			 exists.
[ AS39690_v4 ]			 exists.
[ AS62567_v4 ]			 exists.
[ AS133165_v4 ]			 exists.
[ AS135340_v4 ]			 exists.
[ AS200130_v4 ]			 exists.
[ AS201229_v4 ]			 exists.
[ AS202018_v4 ]			 exists.
[ AS202109_v4 ]			 exists.
[ AS205301_v4 ]			 exists.
[ AS393406_v4 ]			 exists.
[ AS394362_v4 ]			 exists.

===[  Aliastables / Rules  ]==========================================

No changes to Firewall rules, skipping Filter Reload
No Changes to Aliases, Skipping pfctl Update

 UPDATE PROCESS ENDED [ 02/19/23 09:32:00 ]

I am by no means a pfblocker expert, but this seems normal to me..