pfBlockerNG_devel commit reverse

aivxtla · Sep 24, 2024, 5:22 PM

No worries. Thanks for the fix that was fast.

tinfoilmatt · Sep 24, 2024, 5:35 PM

Simply reporting a successful (and uneventful) update from 3.2.0_8 > 3.2.0_17, running -devel package on CE 2.7.2. Update was performed via console (i.e. pkg upgrade).

lohphat · Sep 30, 2024, 12:40 PM

This post is deleted!

lohphat · Oct 3, 2024, 2:59 AM

This post is deleted!

BBcan177 · Oct 3, 2024, 3:02 AM

Try to change the ASN cache setting to one hour.

lohphat · Oct 3, 2024, 3:07 AM

@BBcan177

I figued out how to add the token to the settings but IPv6 lookups are still failing, is there a different issue with IPv6 lookups?

[ QUIC_ASN_List_custom_v6 ]	 Reload
  Collecting host IP: mask.icloud.com... completed
  Collecting host IP: mask-h2.icloud.com... completed
  Collecting ASN: AS8075... Failed to collect ASN  Collecting ASN: AS13335... Failed to collect ASN  Collecting ASN: AS15169... Failed to collect ASN  Collecting ASN: AS16509... Failed to collect ASN  Collecting ASN: AS19551... Failed to collect ASN  Collecting ASN: AS20940... Failed to collect ASN  Collecting ASN: AS15133... Failed to collect ASN... Restoring previous data
 . completed ..
[ pfB_QUIC_ASN_List_v6 QUIC_ASN_List_custom_v6 ] Custom List: No IPs found! Ensure only IP based Feeds are used! ]

BBcan177 · Oct 3, 2024, 3:13 AM

@lohphat

See the api details and try a manual lookup to see if those ASNs have any IP prefixes.

https://ipinfo.io/products/asn-api

lohphat · Oct 3, 2024, 3:21 AM

@BBcan177

Yep. Could it be the code isn't parsing the prefixes6 array vs the prefixes array?

        domain:"google.com",

prefixes6:Array[107],
Object,
netblock:"2001:4860::/32",
id:"GOOGLE-IPV6",
name:"Google LLC",
country:"US",
size:"79228162514264337593543950336",
status:"ALLOCATION",
domain:"google.com",

BBcan177 · Oct 3, 2024, 3:24 AM

@lohphat search by ASN, not domain

lohphat · Oct 3, 2024, 3:27 AM

@BBcan177

I did I just clipped the result. My config for ASN lookup is attached. Try AS15169 and it returns a bunch of IPv4 and IPv6 networks. The pfBlocker-NG IPv4 lookups work, the IPv6 lookups return no networks. The API returns a lot of IPv6 networks.

login-to-view

BBcan177 · Oct 3, 2024, 3:30 AM

@lohphat try to add those to that IPv6 alias but where you would enter URL feeds. Choose the ASN format. I will check the custom list tomorrow.

lohphat · Oct 3, 2024, 3:39 AM

@BBcan177

Nope. Adding them above as individual ASNs didn't work either:

[ AS8075_v6 ]			 Downloading update .
  Collecting ASN: AS8075... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ AS13335_v6 ]			 Downloading update .
  Collecting ASN: AS13335... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ AS15169_v6 ]			 Downloading update [ 10/2/24 23:38:10 ] .
  Collecting ASN: AS15169... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ AS16509_v6 ]			 Downloading update .
  Collecting ASN: AS16509... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ AS19551_v6 ]			 Downloading update [ 10/2/24 23:38:11 ] .
  Collecting ASN: AS19551... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ AS20940_v6 ]			 Downloading update .
  Collecting ASN: AS20940... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ AS15133_v6 ]			 Downloading update [ 10/2/24 23:38:12 ] .
  Collecting ASN: AS15133... Failed to collect ASN... Creating empty file
. completed ..
  Empty file, Adding '::127.1.7.7' to avoid download failure.

[ QUIC_ASN_List_custom_v6 ]	 Downloading update
  Collecting host IP: mask.icloud.com... completed
  Collecting host IP: mask-h2.icloud.com... completed
  Collecting ASN: AS8075... Failed to collect ASN  Collecting ASN: AS13335... Failed to collect ASN  Collecting ASN: AS15169... Failed to collect ASN  Collecting ASN: AS16509... Failed to collect ASN  Collecting ASN: AS19551... Failed to collect ASN  Collecting ASN: AS20940... Failed to collect ASN  Collecting ASN: AS15133... Failed to collect ASN... Restoring previous data
 . completed ..
[ pfB_QUIC_ASN_List_v6 QUIC_ASN_List_custom_v6 ] Custom List: No IPs found! Ensure only IP based Feeds are used! ]

FCS001FCS · Oct 3, 2024, 4:34 PM

@BBcan177

Updated today to pfBlockerNG-devel 3.2.0_18 and ASNs function is working fine again with the IPInfo.io service.

I set the ASN Cache to 1 hour and entered my IPInfp.io token in the IP setup tab. The old entries I had for the ASN lookups did not work at first, so I deleted them and then started from the beginning and re-entered the ASN numbers and then it all worked.

Many thanks to @BBcan177 for all his hard work on getting this up and running again.

leres · Oct 3, 2024, 4:17 PM

Where is the repo that has PORTREVISION 18 of pfSense-pkg-pfBlockerNG-devel? I have been using the devel branch of https://github.com/pfsense/FreeBSD-ports.git but although it contains commits from today, pfSense-pkg-pfBlockerNG-devel only shows PORTREVISION 15 (Sept 23).

jrey · Oct 3, 2024, 4:36 PM

@FCS001FCS said in pfBlockerNG_devel commit reverse:

The old entries I had for the ASN lookups did not work at first, so I deleted them and then started from the beginning and re-entered the ASN numbers and then it all worked.

you really shouldn't have to delete all the ASN numbers and reenter them -- although I could see a possibility where the new ASN data has not been downloaded,(it is one file now, not individual files) and a cron job starts to update before that download is available and therefore they fail - that case should be in the pfblockerng.log I would think.
through all the testing over the past several weeks, I've not had to reenter any ASN.

you might also want to check this as well in case anything there that may impact you

https://forum.netgate.com/topic/190361/pfblockerng-devel-3-2-0_18?_=1727956822873

FCS001FCS · Oct 3, 2024, 8:04 PM

@jrey said in pfBlockerNG_devel commit reverse:

you really shouldn't have to delete all the ASN numbers and reenter them

Yes, I assumed so also but when I updated and forced a "Reload" after the update the ASNs did not populate the related .txt file. So, I just deleted the old ASN entries and then rebuilt them and did a "Reload" again and all worked. I did not want to spend too much time trying to determine the exact issue, so that was quickest for me.

I saw the thread you referenced earlier today, but I do not remember if I had any special characters in the entries, but it could have been the issue. The new entries do not have any special characters.

jrey · Oct 3, 2024, 9:05 PM

@FCS001FCS said in pfBlockerNG_devel commit reverse:

I just deleted the old ASN entries and then rebuilt them and did a "Reload" again and all worked

Yes,
It is important for people to realize that with the old system, each ASN was downloaded as an individual file
with the new system all the ASN data is in one file, and that one file only downloads once per day after initial load. Now when the routine updates asks for an ASN - the data is pulled from the one local master file, not the internet.
So all I am suggesting is that when you ran the first reload the data may not have been available in the master file yet. Thus the extraction part fails and it appears you get nothing.

@FCS001FCS said in pfBlockerNG_devel commit reverse:

but I do not remember if I had any special characters in the entries

On this point - the old data source didn't have any special characters, so the only way you run into the problem is if you had an ASN that under the old system would not have had any special characters but now under the new one does and ran an update (those are harder, but not impossible to find) and / or you now tried to add new one with these characters. Then those ASN's would be a problem.

The issue here is that it couldn't happen on the old data, and with the new data the underlying config save functions do not properly handle international strings and therefore those ASNs do not get saved.

It has only impacted my testing - there is no way I can use this to production (although the method provided of "Add the ASN to the custom list", yes works, it messes up my analytics over in Graylog because anything you list in custom gets attributed to custom not the ASN - so you can not longer track which specific ASN caused the event. Others who don't care about this can certainly use the custom list and get by the problem. for me it is "No Go" on production.

BBcan177 · Oct 4, 2024, 2:24 AM

@lohphat

See this to fix ASN for IPv6

https://www.reddit.com/r/pfBlockerNG/s/Kv6252BTcK

lohphat · Oct 4, 2024, 2:47 AM

@BBcan177 said in pfBlockerNG_devel commit reverse:

kerNG/s/Kv6252BTcK

That did it!

I'm NOT crazy.

Today.

Popolou · Oct 4, 2024, 10:22 AM

Has everything now settled down so as to proceed with an update to _18 with a functional ASN lookup?