[Solved] pfBlockerNG-devel Not Downloading ASN Information

tman222

Does the process need to be bootstrapped somehow? I don't see references to ASN downloads or attempted downloads in the pfBlockerNG extras.log file. This is the first time I have had the ASN reporting capability enabled (i.e. I never attempted to use it with the old data source in place) - not sure if that would cause any problems.

FCS001FCS

@tman222

You need to add an entry of which ASNs you want the IP list for.

I have attached a screenshot entry for Amazon that I have set up that returns approx. 4500 IP addresses.

Note that in the source box you enter the number only and it will bring up a list to select from.

Then save the page and force an IP RELOAD.

Screenshot 2024-10-05 183811.png

jrey

@tman222

Same answer as on the thread you referenced pointing to this thread ?

tman222

@jrey said in pfBlockerNG-devel Not Downloading ASN Information:

@tman222

Same answer as on the thread you referenced to here ?

@jrey - thanks for the response. I have ASN Reporting set to "Enabled - ASN entries cached for 1 hour". I confirmed that my IPInfo token works as well. In my extras.log file all I see is references to GeoLite2 downloads, no references at all to ASN downloads. I'm quite perplexed, for some reason the ASN download isn't even attempting to start despite being enabled under IP settings.

tman222

@FCS001FCS said in pfBlockerNG-devel Not Downloading ASN Information:

@tman222

You need to add an entry of which ASNs you want the IP list for.

I have attached a screenshot entry for Amazon that I have set up that returns approx. 4500 IP addresses.

Note that in the source box you enter the number only and it will bring up a list to select from.

Then save the page and force an IP RELOAD.

Hi @FCS001FCS - thanks for the response. I tried to duplicate what you have in your screenshot above. However, when I try to create the list, I don't even get any dropdown options when I enter something in the Source field. I'm guessing this is because I'm missing the ASN data from the download so there is nothing to choose from.

jrey

@tman222

and no error in the extras.log ?

tman222

@jrey said in pfBlockerNG-devel Not Downloading ASN Information:

@tman222

and no error in the extras.log ?

No errors that I can see - in fact no references to ASN at all in my extras.log file.

FCS001FCS

@tman222 said in pfBlockerNG-devel Not Downloading ASN Information:

I try to create the list, I don't even get any dropdown options when I enter something in the Source field. I'm guessing this is because I'm missing the ASN data from the download so there is nothing to choose from.

I don't know what can be wrong, since mine sort of "worked-out-of-the-box", when I did all the relevant entries.

Just confirm that your token is correct, as it should work. Otherwise, I am out of suggestions.

Screenshot 2024-10-05 190253.png

jrey

@tman222

try and reenter the token and save the form - it should trigger an initial download, and then from that point cron updates it once a day.

anything in the directory

/usr/local/share/GeoIP

tman222

Thanks @jrey and @FCS001FCS

I have confirmed that the token works. I've reentered the token as well and forced an Update. Still no ASN data is being downloaded. All I see in /usr/local/share/GeoIP are GeoLite2 files.

Tagging @BBcan177 as well to see if he might have any idea of what could be wrong here.

jrey

@tman222 said in pfBlockerNG-devel Not Downloading ASN Information:

have confirmed that the token works

That's not the point -- reenter it again and save the form - see if it triggers the initial download.

tman222

@jrey said in pfBlockerNG-devel Not Downloading ASN Information:

@tman222 said in pfBlockerNG-devel Not Downloading ASN Information:

have confirmed that the token works

That's not the point -- reenter it again and save the form - see if it triggers the initial download.

Thanks @jrey - I tried that (per my second sentence above), and still no dice. For what it's worth, I just tried configuring ASN settings on a second, separate pfSense machine and ran into the same problem (no initial ASN data download). Neither of these machines had ASN enabled ever before, so I'm not sure if something needs to be bootstrapped, or I have a missing / conflicting setting somewhere.

jrey

@tman222

The initial download will run this

 # Download IPinfo asn databases on first use.
                        if [ ! -f "${pathasncsv}" ]; then
                                printf "Downloading [ IPinfo databases ] [ ${now} ]"
                                /usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php asn_shell
                                printf "... completed"
                        fi

so this line

/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php asn_shell

then do you have files in the GeoIP directory ?

tman222

@jrey said in pfBlockerNG-devel Not Downloading ASN Information:

@tman222

The initial download will run this

 # Download IPinfo asn databases on first use.
                        if [ ! -f "${pathasncsv}" ]; then
                                printf "Downloading [ IPinfo databases ] [ ${now} ]"
                                /usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php asn_shell
                                printf "... completed"
                        fi

so this line

/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php asn_shell

then do you have files in the GeoIP directory ?

Hi @jrey - yes, running the command above manually downloaded the ASN files and they are now in the GeoIP directory.

However, when forcing an Update subsequently it is not attempting to re-download them. Is there anyway I can further test that the downloaded ASN data is actually being used? Thanks again.

jrey

@tman222

how many asn* files did you see in the directory - should be 3

then when you add to the ASN list as shown in the screen above do they now show up in the drop down selection list?

if you use ipv6 there is a change required --- mentioned in one of the other posts related to line 777 and a missing escape character in the grep command.

also ASN with international characters in the name won't save. (if that applies to you) there is a workaround mentioned in one of the current threads.

you might also want to just look at /etc/crontab and check for a line containing this.
just make sure it has been added.

/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php dcc >> /var/log/pfblockerng/extras.log 2>&1

Keep in mind that if the 3 files are there -- that during a reload or update - no further download from the internet is used. -- it just uses the local data --- if that is not populating the alias or whatever you are using - what is in errors.log and pfblockerng.log for the ASN in question ?

jrey

@tman222

this change for ipv6 is referenced here if you need that

https://forum.netgate.com/topic/190240/pfblockerng_devel-commit-reverse/19

tman222

@jrey and @FCS001FCS - thank you both for your help. I believe I now have this resolved and part of the problem was my unfamiliarity with the ASN process in pfBlockerNG. It turns out that the post from @FCS001FCS was right on and the process does need to be bootstrapped somewhat with an ASN IPv4 list to download the files for the first time.

More Details:

If just enable ASN reporting (e.g. 1 hour cache and enter IPinfo token), save, and force an update, nothing happens, and no ASN files are initially downloaded.
If I perform step 1, and then also create a new ASN IPv4 list, add an ASN to it under the Source field (as described above), save & enable that list and then force an update, the IPinfo ASN files are downloaded as part of the forced update and the ASN IPv4 list is populated. It turns out that I could see ASN information all along when creating a new ASN IPv4 list. I incorrectly formatted the text in the Source field the first time preventing the drop down list from showing.
If I do not perform step 1 first, but still create the ASN IPv4 list and force an update as described in step 2, I will get an error during the update that I need to register for an IPinfo token and ASN IPv4 list will not be populated.

Long story short, after enabling the ASN reporting capability, to trigger the initial download of ASN files from IPinfo during a forced update, an IPv4 block list with one or more ASNs should be created as well. So in my view the initial download process is bootstrapped via the existence of an ASN IPv4 list.

A couple follow up questions:

Do the ASN files from IPInfo get updated on a regular basis now?
Should ASN information also show up in the pfBlockerNG reports (e.g. permit, deny, unified logs)?

Thanks again for all your help, I really appreciate it.

tman222

Just wanted to follow up to confirm that the ASN files are downloaded on a regular basis now (verified by checking the extras.log log file today) and that the ASN information also shows up in the pfBlockerNG reports.

This makes me wonder whether just enabling ASN reporting capability and waiting ~24hrs would have eventually downloaded the ASN files (via the Cron job), and if creating the ASN IPv4 list to trigger the initial download is actually required (i.e. to bootstrap the process). In any case, I'm glad it's working now. Thanks again everyone for all your help.

jrey

@tman222

yes if the cron job referenced above is in place. -- however it should also do the initial download when everything is setup. properly.

Barmaleyster

@FCS001FCS

yes it works