pfBlockerNG_devel commit reverse

jrey

@FCS001FCS said in pfBlockerNG_devel commit reverse:

I just deleted the old ASN entries and then rebuilt them and did a "Reload" again and all worked

Yes,
It is important for people to realize that with the old system, each ASN was downloaded as an individual file
with the new system all the ASN data is in one file, and that one file only downloads once per day after initial load. Now when the routine updates asks for an ASN - the data is pulled from the one local master file, not the internet.
So all I am suggesting is that when you ran the first reload the data may not have been available in the master file yet. Thus the extraction part fails and it appears you get nothing.

@FCS001FCS said in pfBlockerNG_devel commit reverse:

but I do not remember if I had any special characters in the entries

On this point - the old data source didn't have any special characters, so the only way you run into the problem is if you had an ASN that under the old system would not have had any special characters but now under the new one does and ran an update (those are harder, but not impossible to find) and / or you now tried to add new one with these characters. Then those ASN's would be a problem.

The issue here is that it couldn't happen on the old data, and with the new data the underlying config save functions do not properly handle international strings and therefore those ASNs do not get saved.

It has only impacted my testing - there is no way I can use this to production (although the method provided of "Add the ASN to the custom list", yes works, it messes up my analytics over in Graylog because anything you list in custom gets attributed to custom not the ASN - so you can not longer track which specific ASN caused the event. Others who don't care about this can certainly use the custom list and get by the problem. for me it is "No Go" on production.

BBcan177

@lohphat

See this to fix ASN for IPv6

https://www.reddit.com/r/pfBlockerNG/s/Kv6252BTcK

lohphat

@BBcan177 said in pfBlockerNG_devel commit reverse:

kerNG/s/Kv6252BTcK

That did it!

I'm NOT crazy.

Today.

Popolou

Has everything now settled down so as to proceed with an update to _18 with a functional ASN lookup?

incith

Getting errors trying to remove countries from top spammers list (using _18)

Fatal error: Uncaught TypeError: array_path_enabled(): Argument #1 ($arr) must be of type array, int given, called in /etc/inc/config.lib.inc on line 1250 and defined in /etc/inc/util.inc:3662 Stack trace: #0 /etc/inc/config.lib.inc(1250): array_path_enabled(-1, 'notifications/s...', 'disable') #1 /etc/inc/notices.inc(379): config_path_enabled('notifications/s...', 'disable') #2 /etc/inc/notices.inc(662): notify_via_smtp('pfSense is rest...') #3 /etc/inc/notices.inc(151): notify_all_remote('pfSense is rest...') #4 /etc/inc/config.lib.inc(239): file_notice('config.xml', 'pfSense is rest...', 'pfSenseConfigur...', '') #5 /etc/inc/config.lib.inc(695): restore_backup('/cf/conf/backup...') #6 /usr/local/www/pfblockerng/pfblockerng_Top_Spammers.php(291): write_config('[pfBlockerNG] s...') #7 {main} thrown in /etc/inc/util.inc on line 3662 PHP ERROR: Type: 1, File: /etc/inc/util.inc, Line: 3662, Message: Uncaught TypeError: array_path_enabled(): Argument #1 ($arr) must be of type array, int given, called in /etc/inc/config.lib.inc on line 1250 and defined in /etc/inc/util.inc:3662 Stack trace: #0 /etc/inc/config.lib.inc(1250): array_path_enabled(-1, 'notifications/s...', 'disable') #1 /etc/inc/notices.inc(379): config_path_enabled('notifications/s...', 'disable') #2 /etc/inc/notices.inc(662): notify_via_smtp('pfSense is rest...') #3 /etc/inc/notices.inc(151): notify_all_remote('pfSense is rest...') #4 /etc/inc/config.lib.inc(239): file_notice('config.xml', 'pfSense is rest...', 'pfSenseConfigur...', '') #5 /etc/inc/config.lib.inc(695): restore_backup('/cf/conf/backup...') #6 /usr/local/www/pfblockerng/pfblockerng_Top_Spammers.php(291): write_config('[pfBlockerNG] s...') #7 {main} thrown Fatal error: Uncaught TypeError: array_path_enabled(): Argument #1 ($arr) must be of type array, int given, called in /etc/inc/config.lib.inc on line 1250 and defined in /etc/inc/util.inc:3662 Stack trace: #0 /etc/inc/config.lib.inc(1250): array_path_enabled(-1, 'notifications/s...', 'disable') #1 /etc/inc/notices.inc(379): config_path_enabled('notifications/s...', 'disable') #2 /etc/inc/notices.inc(662): notify_via_smtp('PHP ERROR: Type...') #3 /etc/inc/notices.inc(151): notify_all_remote('PHP ERROR: Type...') #4 /etc/inc/config.lib.inc(1154): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors') #5 [internal function]: pfSense_clear_globals() #6 {main} thrown in /etc/inc/util.inc on line 3662

jrey

@incith

and what version of pfSense are you running _18 on ?

incith

@jrey said in pfBlockerNG_devel commit reverse:

@incith

and what version of pfSense are you running _18 on ?

Sorry about that!

Version	2.7.2-RELEASE (amd64)
built on Mon Mar 4 14:53:00 EST 2024
FreeBSD 14.0-CURRENT

incith

Just noticed there's better logs --

Crash report begins.  Anonymous machine information:

amd64
14.0-CURRENT
FreeBSD 14.0-CURRENT amd64 1400094 #1 RELENG_2_7_2-n255948-8d2b56da39c: Wed Dec  6 20:45:47 UTC 2023     root@freebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/obj/amd64/StdASW5b/var/jenkins/workspace/pfSense-CE-snapshots-2_7_2-main/sources/F

Crash report details:

PHP Errors:
[07-Oct-2024 10:03:46 EST5EDT] PHP Fatal error:  Uncaught TypeError: array_path_enabled(): Argument #1 ($arr) must be of type array, int given, called in /etc/inc/config.lib.inc on line 1250 and defined in /etc/inc/util.inc:3662
Stack trace:
#0 /etc/inc/config.lib.inc(1250): array_path_enabled(-1, 'notifications/s...', 'disable')
#1 /etc/inc/notices.inc(379): config_path_enabled('notifications/s...', 'disable')
#2 /etc/inc/notices.inc(662): notify_via_smtp('pfSense is rest...')
#3 /etc/inc/notices.inc(151): notify_all_remote('pfSense is rest...')
#4 /etc/inc/config.lib.inc(239): file_notice('config.xml', 'pfSense is rest...', 'pfSenseConfigur...', '')
#5 /etc/inc/config.lib.inc(695): restore_backup('/cf/conf/backup...')
#6 /usr/local/www/pfblockerng/pfblockerng_Top_Spammers.php(291): write_config('[pfBlockerNG] s...')
#7 {main}
  thrown in /etc/inc/util.inc on line 3662
[07-Oct-2024 10:03:46 EST5EDT] PHP Fatal error:  Uncaught TypeError: array_path_enabled(): Argument #1 ($arr) must be of type array, int given, called in /etc/inc/config.lib.inc on line 1250 and defined in /etc/inc/util.inc:3662
Stack trace:
#0 /etc/inc/config.lib.inc(1250): array_path_enabled(-1, 'notifications/s...', 'disable')
#1 /etc/inc/notices.inc(379): config_path_enabled('notifications/s...', 'disable')
#2 /etc/inc/notices.inc(662): notify_via_smtp('PHP ERROR: Type...')
#3 /etc/inc/notices.inc(151): notify_all_remote('PHP ERROR: Type...')
#4 /etc/inc/config.lib.inc(1154): file_notice('phperror', 'PHP ERROR: Type...', 'PHP errors')
#5 [internal function]: pfSense_clear_globals()
#6 {main}
  thrown in /etc/inc/util.inc on line 3662



No FreeBSD crash data found.

pfSenseConfigurator
pfSense is restoring the configuration /cf/conf/backup/config-1728309802.xml @ 2024-10-07 10:03:46
PHP errors
PHP ERROR: Type: 1, File: /etc/inc/util.inc, Line: 3662, Message: Uncaught TypeError: array_path_enabled(): Argument #1 ($arr) must be of type array, int given, called in /etc/inc/config.lib.inc on line 1250 and defined in /etc/inc/util.inc:3662
Stack trace:
#0 /etc/inc/config.lib.inc(1250): array_path_enabled(-1, 'notifications/s...', 'disable')
#1 /etc/inc/notices.inc(379): config_path_enabled('notifications/s...', 'disable')
#2 /etc/inc/notices.inc(662): notify_via_smtp('pfSense is rest...')
#3 /etc/inc/notices.inc(151): notify_all_remote('pfSense is rest...')
#4 /etc/inc/config.lib.inc(239): file_notice('config.xml', 'pfSense is rest...', 'pfSenseConfigur...', '')
#5 /etc/inc/config.lib.inc(695): restore_backup('/cf/conf/backup...')
#6 /usr/local/www/pfblockerng/pfblockerng_Top_Spammers.php(291): write_config('[pfBlockerNG] s...')
#7 {main}
thrown @ 2024-10-07 10:03:47
 
S.M.A.R.T. Status  
Drive	Ident	S.M.A.R.T. Status
nvme0		PASSED
UPS Status 
Summary status:	On line

jrey

@incith

and was the item you were trying to "remove countries from top spammers list" an ASN?

can you show the screen and the item you are trying to delete ?

incith

@jrey pfblocker > IP > geoip

jrey

@incith

okay, I'm not going to be able to troubleshoot that I don't use MaxMind for GeoIP data

but there were a lot of changes under the hood with _18 related to how configuration items are handled. There are still a few issues with ASN names where for example strings contains International characters (accented) in the fields that will cause the config save to fail and auto restore said config.

So what I'm suggesting is that if anything in TopSpammers list you are trying to remove has international characters that "might" be a problem based on the issue we see on the ASN side where those strings cause similar issues.
Why I asked for the screen capture of what you are specifically trying to remove, is because I have no idea what is on that screen. I'd be looking however for international (accented) characters just to see if that "could be" the issue here.

Specifically for MaxMind in this case suggest we tag @BBcan177 for more insight.

incith

@jrey

Maybe this? Not the one I'm trying to remove (it's already unchecked that is), but maybe it's parsing through it.

jrey

@incith

that would are my guess, but sorry no way for me to troubleshoot this case specifically- it just "looks" like a similar issue to me.

if you want to see a similar error (where the config gets restored) on the ASN side
create or go to a list you already have with ASN data items (assuming you are using ASNs somewhere and have ipinfo.io up and running)

simply try to add (any ASN that contains international characters in the name) and save the list. the data won't be saved and it will "restore" the config.

the new ipinfo data ASN names contain these, the old bgpview ASN data did not.

here are a couple samples (with the new ASN data) that will cause it to fail.. (of course there are many more)

AS29571
AS273867

BBcan177

Ok there is a bug and it will be fixed in the next upcoming version.

https://github.com/pfsense/FreeBSD-ports/commit/3586bb086db9a1ff7dcddc3e2e30605ecac99796

This can be applied by the System Patches package using the commit ID and setting "Path Strip Count" to 4.

Or you can edit /usr/local/www/pfblockerng.php and change the single quotes to a double quote as per the commit above.

Once that has been patched, you can run this command to rebuild the GeoIP pages:

php -f /usr/local/www/pfblockerng/pfblockerng.php dc

incith

@BBcan177 Works great, thank you!

TommyMoo

Updated today pfblocker devel to version 3.2.0_19, all went well, no problem. Thank you very much, my impression is, that it gained some speed compared to previous version & Thanks for bugfixes

BBcan177

@TommyMoo

Thanks for reporting.

More info here:

https://www.reddit.com/r/pfBlockerNG/s/4Yrr4ffq3I

tman222

@BBcan177 said in pfBlockerNG_devel commit reverse:

@TommyMoo

Thanks for reporting.

More info here:

https://www.reddit.com/r/pfBlockerNG/s/4Yrr4ffq3I

Hi @BBcan177 - thanks for this new version; the install went smoothly. I have a quick question about the release: I see that in the release notes that setting the ASN Cache setting is no longer required. Does this mean that under IP -> ASN Configuration -> ASN Reporting, we can now choose "Disabled" and everything will continue download / work fine as before? If yes, what would be the purpose of the ASN Reporting field, i.e. when it would be useful to still enable the cache setting? Could this field be removed altogether now that the switch IPinfo has occurred? Thanks in advance.

BBcan177

@tman222

Yes you can set that to disabled and it will still download OK. That setting was more useful with BGPview, but I would still recommend to set it to atleasy 1 hour as it might give a small micro performance boost.

tman222

@BBcan177 said in pfBlockerNG_devel commit reverse:

@tman222

Yes you can set that to disabled and it will still download OK. That setting was more useful with BGPview, but I would still recommend to set it to atleasy 1 hour as it might give a small micro performance boost.

Thanks @BBcan177. Since the IPinfo ASN data is updated only once every 24 hours, do you see any additional advantage to increasing that ASN cache setting from 1 hour to e.g. 4 or 12 hours? Thanks again.