pfblockerNG ASN bgpview trouble

jrey

@Patch

it pulled and parsed as expected -- no issue

Screen Shot 2024-08-16 at 8.46.50 PM.png

Patch

@jrey said in pfblockerNG ASN bgpview trouble:

it pulled and parsed as expected -- no issue

Interesting.
I'm running

pfsense v2.7.2
pfBlockerNG v3.2.0_8

At two sites both are not able to pull any ASN data (for multiple asn's) so insert 127.1.7.7

Not sure what is different between my and your system

pfblockerng ASN no entries.jpg

pfblockerng ASN no entries native.jpg

Perhaps is a Cloudflare cache refresh problem https://bgpview.docs.apiary.io/#reference/0/asn/view-asn-details?console=1
pfblockerng ASN no entries Cloudflare.jpg

jrey

@Patch

Ok, so I fired up my test box 2.7.2 and 3.2.0_8 same versions
created an ASN entry for AS40027

works fine.

ssh into your device and run this - the simplest form required for bpg

curl -sS1 https://api.bgpview.io/asn/40027/prefixes

What do you see? Should be a json format dump of the ASN data starting with

{"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0\/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code

Patch

@jrey said in pfblockerNG ASN bgpview trouble:

curl -sS1 https://api.bgpview.io/asn/40027/prefixes

What do you see? Should be a json format dump of the ASN data starting with

{"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code

Hey, that works for me too.
But

Firewall / pfBlockerNG / Log Browser -> Original IP Files -> AS40027_v4.orig -> Trash
Firewall / pfBlockerNG / Update -> Reload, IP ->

[ AS40027_v4 ]			 Downloading update [ 08/17/24 22:48:06 ] .
  Downloading ASN: 40027...... completed ..
  Empty file, Adding '127.1.7.7' to avoid download failure.

Similarly restarting pfsense also clears the log files then

Firewall / pfBlockerNG / Update -> empty file again

jrey

@Patch

Please show me the screen where you have the feed set up.
Can you also include the first part of the response from the curl you did manually "that works for me too"

Patch

@jrey
I tried un-installing nmap v1.4.4_7
un-installing pfBlockerNG then re installing -> no difference
The only other package I have installed is System_Patches v2.2.11_15

@jrey said in pfblockerNG ASN bgpview trouble:

Please show me the screen where you have the feed set up.

pfblockerng ASN Netflix.jpg

@jrey said in pfblockerNG ASN bgpview trouble:

Can you also include the first part of the response from the curl you did manually

{"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0\/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.8.0\/24","ip":"45.57.8.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.9.0\/24","ip":"45.57.9.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.40.0\/23","ip":"45.57.40.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.120.152.0\/22","ip":"45.120.152.0","cidr":22,"rir_name":"APNIC","allocation_status":"unknown"}},{"prefix":"45.57.40.0\/24","ip":"45.57.40.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.41.0\/24","ip":"45.57.41.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.155.40.0\/22","ip":"45.155.40.0","cidr":22,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"45.57.86.0\/23","ip":"45.57.86.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.86.0\/24","ip":"45.57.86.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.87.0\/24","ip":"45.57.87.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.90.0\/23","ip":"45.57.90.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.90.0\/24","ip":"45.57.90.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.91.0\/24","ip":"45.57.91.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"207.45.72.0\/24","ip":"207.45.72.0","cidr":24,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}},{"prefix":"207.45.72.0\/23","ip":"207.45.72.0","cidr":23,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}},{"prefix":"207.45.73.0\/24","ip":"207.45.73.0","cidr":24,"roa_status":"Valid","name":"DVD-NETFLIX","description":"Netflix, Inc","country_code":"US","parent":{"prefix":"207.45.72.0\/22","ip":"207.45.72.0","cidr":22,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"207.45.74.0\/23","ip":"207.45.74.0","cidr":23,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}}],"ipv6_prefixes":[{"prefix":"2a00:86c0:2008::\/48","ip":"2a00:86c0:2008::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2008::\/47","ip":"2a00:86c0:2008::","cidr":47,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2009::\/48","ip":"2a00:86c0:2009::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2040::\/48","ip":"2a00:86c0:2040::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2040::\/47","ip":"2a00:86c0:2040::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2041::\/48","ip":"2a00:86c0:2041::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2042::\/48","ip":"2a00:86c0:2042::","cidr":48,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2042::\/47","ip":"2a00:86c0:2042::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2043::\/48","ip":"2a00:86c0:2043::","cidr":48,"roa_status":"Valid","name":"NET6-2A00-86C-3","description":"NET6 2A00 86C 3","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2052::\/47","ip":"2a00:86c0:2052::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2086::\/48","ip":"2a00:86c0:2086::","cidr":48,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2086::\/47","ip":"2a00:86c0:2086::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2087::\/48","ip":"2a00:86c0:2087::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2090::\/47","ip":"2a00:86c0:2090::","cidr":47,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2090::\/48","ip":"2a00:86c0:2090::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2091::\/48","ip":"2a00:86c0:2091::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}}]},"@meta":{"time_zone":"UTC","api_version":1,"execution_time":"28.06 ms"}}

Which Beyond compare suggests is identical to yours for the data shown.

Patch

I also tried

disabling pfblockerNG ASN cache,
deleting the /var/db/pfblockerng/original/ files
Deleting the /var/db/pfblockerng/native/ files
Rerunning Firewall / pfBlockerNG / Update -> Reload, IP

But still only empty files result

pfblockerng ASN Cache disable.jpg

Bob.Dig

@Patch I tried it myself, making this alias and failed. I see the same as you. Older ASN-aliases aren't affected as far as I can tell. So you are not alone. I guess I will try another ASN next.
Edit: Also no luck, old ones work.

jrey

@Bob-Dig

and yet it works for me (still) (Don't read any of this as directed at you. I'm just tagging you because:

You might recall, as I think you where part of a thread, I'm guessing about a year ago, with the same OMG it is creating 127... Empty files

You have to ask "why does it work for me ?"
They patch I created then still works. However words of
WARNING: That version no longer applies directly on 3.0.2_9 (I had to tweak it a bit because of an underlying file change. On my end that tweak took less than five minutes. Unfortunately I can't just provide my updated patch because I have "fixed/added" several other things that are "not important" to either the developer and/or the public.. (or so I'm told)
WARNING: Not sure that the older version of the patch specifically addresses this issue, which I know I've tweaked it a couple of times over the past year. I'm to lazy to look at, code change log to find out. So, the patch you might still find floating around might only address the specific case at the time and not additional cases I've encountered in daily use since then.

I can clearly prove it on both my 2.7.2 test box or 23.03 production box, simple to do - revert the patch watch it fail as being discussed, apply the patch watch it work. Clearly it is NOT the feed at this point as demonstrated by the direct curl command test I suggested earlier in this thread.

After the last go around and being told (by several) that the patch was "silly", "not needed" and not going to be incorporated because what is there, "works fine" I just moved on, hence the I can't release an updated patch I'm currently using. In part, some of those other changes are direct updating to remote syslog in real time, for example.

When this all started going down again last week, my graylog immediately started informing me of the download issue, (filtered on level 2) (critical enough but not notifications.)

Screen Shot 2024-08-18 at 9.18.42 AM.png

part of that most recent outage was the opportunity for me to simulate the case of the system creating an empty file --- to you know -- "test the emergency broadcast system." I had the OMG (filtered on level 1) failure event email within seconds of creating the "empty file manually" --

Screen Shot 2024-08-18 at 9.21.26 AM.png

Sorry, not sure what else I can do to help. I've given up on trying to create tickets, discuss with the developers, suggest improvements etc Now I just "twist" the "silly" into the code so that it suits my application, requirements and use of the device... I been through a few OS upgrades and other than the small adjustment I had to make to one patch, going to 24.03 and pf 3. _9 I have never had a problem

Moving on..

Bob.Dig

@jrey said in pfblockerNG ASN bgpview trouble:

You might recall, as I think you where part of a thread, I'm guessing about a year ago, with the same OMG it is creating 127... Empty files

Sure, I do remember and am following you for that.

What I didn't remembered, that the patch is still needed. And I can confirm, once again, that your patch is working. I applied it a few seconds ago, thanks again @jrey .

jrey

@Bob-Dig

Cool for the record can you just confirm what versions of things you are currently running ?

Edit: because what I couldn't remember is what specific change I may have made in that patch, caused the specific hiccup when I upgraded to 24.03 and 3.0.2_9 came with it. I just recall having to tweak my current version of the patch to make it apply. (I guessing it might have been something else I've changed in the patch since that earlier one)

Bob.Dig

This post is deleted!

Bob.Dig

24.03-RELEASE, pfBlockerNG 3.2.0_10
2.7.2-RELEASE, pfBlockerNG 3.2.0_8

@jrey When I tested with CE, it failed with one AS for me (AS8881). I then retested with another and this one was working (AS1299)...
I have not tested (AS8881) on Plus though. So there might be a problem still.

jrey

@Bob-Dig

Great thanks - so then the version of the patch I originally provided you still applies to both. good to know (I guess)

Means something I changed since that earlier version is what sent me down the path won't apply path when I upgrade to 24.03 --

Oh darn, I just shut my 2.7.2 virtual network down. Let me fire it back up and look at those two ASnumbers you provided ..

Thanks

Bob.Dig

@jrey Retesting this ASN again, now works everywhere. So I think we are good. Maybe it was just a hiccup.

jrey

@Bob-Dig

Interesting, can you share the log snippet where it failed ?

I tried 2.7.2 and for me it failed on the first attempt but picked it up on the auto retry.

[ AS40027_v4 ]			 Downloading update [ 08/18/24 11:01:12 ] .
  Downloading ASN: 40027.
.. completed (Download Valid)
. completed ..

[ AS8881_v4 ]			 Downloading update .
  Downloading ASN: 8881.
.. Failed to download ASN
.
.. completed (Download Valid)
. completed ..

[ AS1299_v4 ]			 Downloading update [ 08/18/24 11:01:13 ] .
  Downloading ASN: 1299.
.. completed (Download Valid)
. completed ..

Bob.Dig

@jrey said in pfblockerNG ASN bgpview trouble:

Interesting, can you share the log snippet where it failed ?

Where is that exactly, sry still not an experienced pfBlocker User.
Also I thought I hadn't this bevor on that firewall but I guess I am wrong and it was there so it could be an old list...

jrey

@Bob-Dig

In the log here -- you will have to scroll through to find the one that failed (don't need the whole file just the part for the one that failed.

Screen Shot 2024-08-18 at 11.32.47 AM.png

Bob.Dig

@jrey

  Restoring previously downloaded file contents... [ 08/9/24 20:45:57 ]
[PFB_FILTER - 2] Invalid URL (not allowed2) [ AS8881 [ VERSATEL, DE ] ] [ 08/17/24 18:21:15 ]
[PFB_FILTER - 2] Invalid URL (not allowed2) [ AS8881 ] [ 08/17/24 18:21:32 ]

jrey

@Bob-Dig

That's different.

Doesn't look related to the actual "download" of the file. but rather something bad left over from when it didn't download -- so it though it needed to restore the previous one 08/9/24 and then tried to process it.

if the latest download worked (the file in question has IPs listed) and "worked" as you stated above "Retesting this ASN again, now works everywhere. So I think we are good. Maybe it was just a hiccup." would likely be a good caption for the event.

if the [PFB_FILTER - 2] error shows up again, that might be something else to look at, I've never logged one of those and it is a completely different block of code.

Cheers