pfblockerNG ASN bgpview trouble

Aug 14 2024 news article published 7:34PM "GitHub is recovering after major network outage" "the company is rolling back a database infrastructure change"

interesting--- article published shortly after they both went down, and there are others still down...

We wait..

fireodo

@jrey said in pfblockerNG ASN bgpview trouble:

interesting--- article published shortly after they both went down, and there are others still down...

Maybe coincidence but ... anyway a little bit strange ...
Btw. BGP had in the past similar problems as right now ... so its not very surprisingly ...

Edit: Now the service is back online (20:00 CEST)

Patch

@fireodo said in pfblockerNG ASN bgpview trouble:

Now the service is back online (20:00 CEST)

Not completely for me.
Deleting these files
Running Firewall/pfBockerNG/Update -> Update, Run
I still get

[ AS40027_v4 ]			 Downloading update .
  Downloading ASN: 40027...... completed ..
  Empty file, Adding '127.1.7.7' to avoid download failure.

but https://api.bgpview.io/ no longer gives a error page, instead redirecting to https://bgpview.docs.apiary.io/#

jrey

@Patch said in pfblockerNG ASN bgpview trouble:

no longer gives a error page

not sure what you are trying but the actual api that pfblockerNG will hit for the ASN (netflix) you have listed is returning values (visually it looks fine to me, I'll add it to a list and see what happens when it pulls) as I don't use that particular ASN, all of the ones I do use are working as expected

try this is a browser
https://api.bgpview.io/asn/40027/prefixes

guessing it is likely redirecting based on the fact you tried it in a browser and didn't provide the parameters ?

jrey

@Patch

it pulled and parsed as expected -- no issue

Screen Shot 2024-08-16 at 8.46.50 PM.png

Patch

@jrey said in pfblockerNG ASN bgpview trouble:

it pulled and parsed as expected -- no issue

Interesting.
I'm running

pfsense v2.7.2
pfBlockerNG v3.2.0_8

At two sites both are not able to pull any ASN data (for multiple asn's) so insert 127.1.7.7

Not sure what is different between my and your system

pfblockerng ASN no entries.jpg

pfblockerng ASN no entries native.jpg

Perhaps is a Cloudflare cache refresh problem https://bgpview.docs.apiary.io/#reference/0/asn/view-asn-details?console=1
pfblockerng ASN no entries Cloudflare.jpg

jrey

@Patch

Ok, so I fired up my test box 2.7.2 and 3.2.0_8 same versions
created an ASN entry for AS40027

works fine.

ssh into your device and run this - the simplest form required for bpg

curl -sS1 https://api.bgpview.io/asn/40027/prefixes

What do you see? Should be a json format dump of the ASN data starting with

{"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0\/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code

Patch

@jrey said in pfblockerNG ASN bgpview trouble:

curl -sS1 https://api.bgpview.io/asn/40027/prefixes

What do you see? Should be a json format dump of the ASN data starting with

{"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code

Hey, that works for me too.
But

Firewall / pfBlockerNG / Log Browser -> Original IP Files -> AS40027_v4.orig -> Trash
Firewall / pfBlockerNG / Update -> Reload, IP ->

[ AS40027_v4 ]			 Downloading update [ 08/17/24 22:48:06 ] .
  Downloading ASN: 40027...... completed ..
  Empty file, Adding '127.1.7.7' to avoid download failure.

Similarly restarting pfsense also clears the log files then

Firewall / pfBlockerNG / Update -> empty file again

jrey

@Patch

Please show me the screen where you have the feed set up.
Can you also include the first part of the response from the curl you did manually "that works for me too"

Patch

@jrey
I tried un-installing nmap v1.4.4_7
un-installing pfBlockerNG then re installing -> no difference
The only other package I have installed is System_Patches v2.2.11_15

@jrey said in pfblockerNG ASN bgpview trouble:

Please show me the screen where you have the feed set up.

pfblockerng ASN Netflix.jpg

@jrey said in pfblockerNG ASN bgpview trouble:

Can you also include the first part of the response from the curl you did manually

{"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0\/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.8.0\/24","ip":"45.57.8.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.9.0\/24","ip":"45.57.9.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.40.0\/23","ip":"45.57.40.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.120.152.0\/22","ip":"45.120.152.0","cidr":22,"rir_name":"APNIC","allocation_status":"unknown"}},{"prefix":"45.57.40.0\/24","ip":"45.57.40.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.41.0\/24","ip":"45.57.41.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.155.40.0\/22","ip":"45.155.40.0","cidr":22,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"45.57.86.0\/23","ip":"45.57.86.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.86.0\/24","ip":"45.57.86.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.87.0\/24","ip":"45.57.87.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.90.0\/23","ip":"45.57.90.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.90.0\/24","ip":"45.57.90.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.91.0\/24","ip":"45.57.91.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"207.45.72.0\/24","ip":"207.45.72.0","cidr":24,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}},{"prefix":"207.45.72.0\/23","ip":"207.45.72.0","cidr":23,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}},{"prefix":"207.45.73.0\/24","ip":"207.45.73.0","cidr":24,"roa_status":"Valid","name":"DVD-NETFLIX","description":"Netflix, Inc","country_code":"US","parent":{"prefix":"207.45.72.0\/22","ip":"207.45.72.0","cidr":22,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"207.45.74.0\/23","ip":"207.45.74.0","cidr":23,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}}],"ipv6_prefixes":[{"prefix":"2a00:86c0:2008::\/48","ip":"2a00:86c0:2008::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2008::\/47","ip":"2a00:86c0:2008::","cidr":47,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2009::\/48","ip":"2a00:86c0:2009::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2040::\/48","ip":"2a00:86c0:2040::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2040::\/47","ip":"2a00:86c0:2040::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2041::\/48","ip":"2a00:86c0:2041::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2042::\/48","ip":"2a00:86c0:2042::","cidr":48,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2042::\/47","ip":"2a00:86c0:2042::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2043::\/48","ip":"2a00:86c0:2043::","cidr":48,"roa_status":"Valid","name":"NET6-2A00-86C-3","description":"NET6 2A00 86C 3","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2052::\/47","ip":"2a00:86c0:2052::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2086::\/48","ip":"2a00:86c0:2086::","cidr":48,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2086::\/47","ip":"2a00:86c0:2086::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2087::\/48","ip":"2a00:86c0:2087::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2090::\/47","ip":"2a00:86c0:2090::","cidr":47,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2090::\/48","ip":"2a00:86c0:2090::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2091::\/48","ip":"2a00:86c0:2091::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}}]},"@meta":{"time_zone":"UTC","api_version":1,"execution_time":"28.06 ms"}}

Which Beyond compare suggests is identical to yours for the data shown.

Patch

I also tried

disabling pfblockerNG ASN cache,
deleting the /var/db/pfblockerng/original/ files
Deleting the /var/db/pfblockerng/native/ files
Rerunning Firewall / pfBlockerNG / Update -> Reload, IP

But still only empty files result

pfblockerng ASN Cache disable.jpg

Bob.Dig

@Patch I tried it myself, making this alias and failed. I see the same as you. Older ASN-aliases aren't affected as far as I can tell. So you are not alone. I guess I will try another ASN next.
Edit: Also no luck, old ones work.

jrey

@Bob-Dig

and yet it works for me (still) (Don't read any of this as directed at you. I'm just tagging you because:

You might recall, as I think you where part of a thread, I'm guessing about a year ago, with the same OMG it is creating 127... Empty files

You have to ask "why does it work for me ?"
They patch I created then still works. However words of
WARNING: That version no longer applies directly on 3.0.2_9 (I had to tweak it a bit because of an underlying file change. On my end that tweak took less than five minutes. Unfortunately I can't just provide my updated patch because I have "fixed/added" several other things that are "not important" to either the developer and/or the public.. (or so I'm told)
WARNING: Not sure that the older version of the patch specifically addresses this issue, which I know I've tweaked it a couple of times over the past year. I'm to lazy to look at, code change log to find out. So, the patch you might still find floating around might only address the specific case at the time and not additional cases I've encountered in daily use since then.

I can clearly prove it on both my 2.7.2 test box or 23.03 production box, simple to do - revert the patch watch it fail as being discussed, apply the patch watch it work. Clearly it is NOT the feed at this point as demonstrated by the direct curl command test I suggested earlier in this thread.

After the last go around and being told (by several) that the patch was "silly", "not needed" and not going to be incorporated because what is there, "works fine" I just moved on, hence the I can't release an updated patch I'm currently using. In part, some of those other changes are direct updating to remote syslog in real time, for example.

When this all started going down again last week, my graylog immediately started informing me of the download issue, (filtered on level 2) (critical enough but not notifications.)

Screen Shot 2024-08-18 at 9.18.42 AM.png

part of that most recent outage was the opportunity for me to simulate the case of the system creating an empty file --- to you know -- "test the emergency broadcast system." I had the OMG (filtered on level 1) failure event email within seconds of creating the "empty file manually" --

Screen Shot 2024-08-18 at 9.21.26 AM.png

Sorry, not sure what else I can do to help. I've given up on trying to create tickets, discuss with the developers, suggest improvements etc Now I just "twist" the "silly" into the code so that it suits my application, requirements and use of the device... I been through a few OS upgrades and other than the small adjustment I had to make to one patch, going to 24.03 and pf 3. _9 I have never had a problem

Moving on..

Bob.Dig

@jrey said in pfblockerNG ASN bgpview trouble:

You might recall, as I think you where part of a thread, I'm guessing about a year ago, with the same OMG it is creating 127... Empty files

Sure, I do remember and am following you for that.

What I didn't remembered, that the patch is still needed. And I can confirm, once again, that your patch is working. I applied it a few seconds ago, thanks again @jrey .