Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfblockerNG ASN bgpview trouble

    Scheduled Pinned Locked Moved pfBlockerNG
    35 Posts 7 Posters 1.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      Patch @jrey
      last edited by Patch

      @jrey
      I tried un-installing nmap v1.4.4_7
      un-installing pfBlockerNG then re installing -> no difference
      The only other package I have installed is System_Patches v2.2.11_15

      @jrey said in pfblockerNG ASN bgpview trouble:

      Please show me the screen where you have the feed set up.

      pfblockerng ASN Netflix.jpg

      @jrey said in pfblockerNG ASN bgpview trouble:

      Can you also include the first part of the response from the curl you did manually

      {"status":"ok","status_message":"Query was successful","data":{"ipv4_prefixes":[{"prefix":"45.57.8.0\/23","ip":"45.57.8.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.8.0\/24","ip":"45.57.8.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.9.0\/24","ip":"45.57.9.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.40.0\/23","ip":"45.57.40.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.120.152.0\/22","ip":"45.120.152.0","cidr":22,"rir_name":"APNIC","allocation_status":"unknown"}},{"prefix":"45.57.40.0\/24","ip":"45.57.40.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.41.0\/24","ip":"45.57.41.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.155.40.0\/22","ip":"45.155.40.0","cidr":22,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"45.57.86.0\/23","ip":"45.57.86.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.86.0\/24","ip":"45.57.86.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.87.0\/24","ip":"45.57.87.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.90.0\/23","ip":"45.57.90.0","cidr":23,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.90.0\/24","ip":"45.57.90.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"45.57.91.0\/24","ip":"45.57.91.0","cidr":24,"roa_status":"Valid","name":"SS-CDN-4","description":"Netflix Streaming Services Inc.","country_code":"US","parent":{"prefix":"45.57.0.0\/17","ip":"45.57.0.0","cidr":17,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"207.45.72.0\/24","ip":"207.45.72.0","cidr":24,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}},{"prefix":"207.45.72.0\/23","ip":"207.45.72.0","cidr":23,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}},{"prefix":"207.45.73.0\/24","ip":"207.45.73.0","cidr":24,"roa_status":"Valid","name":"DVD-NETFLIX","description":"Netflix, Inc","country_code":"US","parent":{"prefix":"207.45.72.0\/22","ip":"207.45.72.0","cidr":22,"rir_name":"ARIN","allocation_status":"unknown"}},{"prefix":"207.45.74.0\/23","ip":"207.45.74.0","cidr":23,"roa_status":"Valid","name":null,"description":null,"country_code":null,"parent":{"prefix":null,"ip":null,"cidr":null,"rir_name":null,"allocation_status":"unknown"}}],"ipv6_prefixes":[{"prefix":"2a00:86c0:2008::\/48","ip":"2a00:86c0:2008::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2008::\/47","ip":"2a00:86c0:2008::","cidr":47,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2009::\/48","ip":"2a00:86c0:2009::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2040::\/48","ip":"2a00:86c0:2040::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2040::\/47","ip":"2a00:86c0:2040::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2041::\/48","ip":"2a00:86c0:2041::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2042::\/48","ip":"2a00:86c0:2042::","cidr":48,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2042::\/47","ip":"2a00:86c0:2042::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2043::\/48","ip":"2a00:86c0:2043::","cidr":48,"roa_status":"Valid","name":"NET6-2A00-86C-3","description":"NET6 2A00 86C 3","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2052::\/47","ip":"2a00:86c0:2052::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2086::\/48","ip":"2a00:86c0:2086::","cidr":48,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2086::\/47","ip":"2a00:86c0:2086::","cidr":47,"roa_status":"Valid","name":"US-NETFLIX1-20120130","description":"Netflix Inc","country_code":"GB","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2087::\/48","ip":"2a00:86c0:2087::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2090::\/47","ip":"2a00:86c0:2090::","cidr":47,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2090::\/48","ip":"2a00:86c0:2090::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}},{"prefix":"2a00:86c0:2091::\/48","ip":"2a00:86c0:2091::","cidr":48,"roa_status":"Valid","name":null,"description":null,"country_code":"US","parent":{"prefix":"2a00:86c0::\/32","ip":"2a00:86c0::","cidr":32,"rir_name":"RIPE","allocation_status":"unknown"}}]},"@meta":{"time_zone":"UTC","api_version":1,"execution_time":"28.06 ms"}}
      

      Which Beyond compare suggests is identical to yours for the data shown.

      P 1 Reply Last reply Reply Quote 0
      • P
        Patch @Patch
        last edited by Patch

        I also tried

        • disabling pfblockerNG ASN cache,
        • deleting the /var/db/pfblockerng/original/ files
        • Deleting the /var/db/pfblockerng/native/ files
        • Rerunning Firewall / pfBlockerNG / Update -> Reload, IP

        But still only empty files result

        pfblockerng ASN Cache disable.jpg

        Bob.DigB 1 Reply Last reply Reply Quote 0
        • Bob.DigB
          Bob.Dig LAYER 8 @Patch
          last edited by Bob.Dig

          @Patch I tried it myself, making this alias and failed. I see the same as you. Older ASN-aliases aren't affected as far as I can tell. So you are not alone. I guess I will try another ASN next.
          Edit: Also no luck, old ones work.

          J 1 Reply Last reply Reply Quote 0
          • J
            jrey @Bob.Dig
            last edited by

            @Bob-Dig

            and yet it works for me (still) (Don't read any of this as directed at you. I'm just tagging you because:

            You might recall, as I think you where part of a thread, I'm guessing about a year ago, with the same OMG it is creating 127... Empty files

            You have to ask "why does it work for me ?"
            They patch I created then still works. However words of
            WARNING: That version no longer applies directly on 3.0.2_9 (I had to tweak it a bit because of an underlying file change. On my end that tweak took less than five minutes. Unfortunately I can't just provide my updated patch because I have "fixed/added" several other things that are "not important" to either the developer and/or the public.. (or so I'm told)
            WARNING: Not sure that the older version of the patch specifically addresses this issue, which I know I've tweaked it a couple of times over the past year. I'm to lazy to look at, code change log to find out. So, the patch you might still find floating around might only address the specific case at the time and not additional cases I've encountered in daily use since then.

            I can clearly prove it on both my 2.7.2 test box or 23.03 production box, simple to do - revert the patch watch it fail as being discussed, apply the patch watch it work. Clearly it is NOT the feed at this point as demonstrated by the direct curl command test I suggested earlier in this thread.

            After the last go around and being told (by several) that the patch was "silly", "not needed" and not going to be incorporated because what is there, "works fine" I just moved on, hence the I can't release an updated patch I'm currently using. In part, some of those other changes are direct updating to remote syslog in real time, for example.

            When this all started going down again last week, my graylog immediately started informing me of the download issue, (filtered on level 2) (critical enough but not notifications.)

            Screen Shot 2024-08-18 at 9.18.42 AM.png

            part of that most recent outage was the opportunity for me to simulate the case of the system creating an empty file --- to you know -- "test the emergency broadcast system." I had the OMG (filtered on level 1) failure event email within seconds of creating the "empty file manually" --

            Screen Shot 2024-08-18 at 9.21.26 AM.png

            Sorry, not sure what else I can do to help. I've given up on trying to create tickets, discuss with the developers, suggest improvements etc Now I just "twist" the "silly" into the code so that it suits my application, requirements and use of the device... I been through a few OS upgrades and other than the small adjustment I had to make to one patch, going to 24.03 and pf 3. _9 I have never had a problem

            Moving on..

            Bob.DigB 1 Reply Last reply Reply Quote 1
            • Bob.DigB
              Bob.Dig LAYER 8 @jrey
              last edited by Bob.Dig

              @jrey said in pfblockerNG ASN bgpview trouble:

              You might recall, as I think you where part of a thread, I'm guessing about a year ago, with the same OMG it is creating 127... Empty files

              Sure, I do remember and am following you for that. 😀

              What I didn't remembered, that the patch is still needed. And I can confirm, once again, that your patch is working. I applied it a few seconds ago, thanks again @jrey .

              J 1 Reply Last reply Reply Quote 0
              • J
                jrey @Bob.Dig
                last edited by jrey

                @Bob-Dig

                Cool 😏 for the record can you just confirm what versions of things you are currently running ?

                Edit: because what I couldn't remember is what specific change I may have made in that patch, caused the specific hiccup when I upgraded to 24.03 and 3.0.2_9 came with it. I just recall having to tweak my current version of the patch to make it apply. (I guessing it might have been something else I've changed in the patch since that earlier one)

                Bob.DigB 2 Replies Last reply Reply Quote 1
                • Bob.DigB
                  Bob.Dig LAYER 8 @jrey
                  last edited by

                  This post is deleted!
                  J 1 Reply Last reply Reply Quote 0
                  • Bob.DigB
                    Bob.Dig LAYER 8 @jrey
                    last edited by Bob.Dig

                    24.03-RELEASE, pfBlockerNG 3.2.0_10
                    2.7.2-RELEASE, pfBlockerNG 3.2.0_8

                    @jrey When I tested with CE, it failed with one AS for me (AS8881). I then retested with another and this one was working (AS1299)...
                    I have not tested (AS8881) on Plus though. So there might be a problem still.

                    1 Reply Last reply Reply Quote 0
                    • J
                      jrey @Bob.Dig
                      last edited by

                      @Bob-Dig

                      Great thanks - so then the version of the patch I originally provided you still applies to both. good to know (I guess)

                      Means something I changed since that earlier version is what sent me down the path won't apply path when I upgrade to 24.03 --

                      Oh darn, I just shut my 2.7.2 virtual network down. Let me fire it back up and look at those two ASnumbers you provided ..

                      Thanks

                      Bob.DigB 1 Reply Last reply Reply Quote 1
                      • Bob.DigB
                        Bob.Dig LAYER 8 @jrey
                        last edited by Bob.Dig

                        @jrey Retesting this ASN again, now works everywhere. So I think we are good. Maybe it was just a hiccup.

                        J 1 Reply Last reply Reply Quote 0
                        • J
                          jrey @Bob.Dig
                          last edited by

                          @Bob-Dig

                          Interesting, can you share the log snippet where it failed ?

                          I tried 2.7.2 and for me it failed on the first attempt but picked it up on the auto retry.

                          [ AS40027_v4 ]			 Downloading update [ 08/18/24 11:01:12 ] .
                            Downloading ASN: 40027.
                          .. completed (Download Valid)
                          . completed ..
                          
                          [ AS8881_v4 ]			 Downloading update .
                            Downloading ASN: 8881.
                          .. Failed to download ASN
                          .
                          .. completed (Download Valid)
                          . completed ..
                          
                          [ AS1299_v4 ]			 Downloading update [ 08/18/24 11:01:13 ] .
                            Downloading ASN: 1299.
                          .. completed (Download Valid)
                          . completed ..
                          
                          Bob.DigB 1 Reply Last reply Reply Quote 0
                          • Bob.DigB
                            Bob.Dig LAYER 8 @jrey
                            last edited by

                            @jrey said in pfblockerNG ASN bgpview trouble:

                            Interesting, can you share the log snippet where it failed ?

                            Where is that exactly, sry still not an experienced pfBlocker User.
                            Also I thought I hadn't this bevor on that firewall but I guess I am wrong and it was there so it could be an old list...

                            J 1 Reply Last reply Reply Quote 0
                            • J
                              jrey @Bob.Dig
                              last edited by

                              @Bob-Dig

                              In the log here -- you will have to scroll through to find the one that failed (don't need the whole file just the part for the one that failed.

                              Screen Shot 2024-08-18 at 11.32.47 AM.png

                              Bob.DigB 1 Reply Last reply Reply Quote 0
                              • Bob.DigB
                                Bob.Dig LAYER 8 @jrey
                                last edited by Bob.Dig

                                @jrey

                                  Restoring previously downloaded file contents... [ 08/9/24 20:45:57 ]
                                [PFB_FILTER - 2] Invalid URL (not allowed2) [ AS8881 [ VERSATEL, DE ] ] [ 08/17/24 18:21:15 ]
                                [PFB_FILTER - 2] Invalid URL (not allowed2) [ AS8881 ] [ 08/17/24 18:21:32 ]
                                
                                J 1 Reply Last reply Reply Quote 0
                                • J
                                  jrey @Bob.Dig
                                  last edited by

                                  @Bob-Dig

                                  That's different.

                                  Doesn't look related to the actual "download" of the file. but rather something bad left over from when it didn't download -- so it though it needed to restore the previous one 08/9/24 and then tried to process it.

                                  if the latest download worked (the file in question has IPs listed) and "worked" as you stated above "Retesting this ASN again, now works everywhere. So I think we are good. Maybe it was just a hiccup." would likely be a good caption for the event.

                                  if the [PFB_FILTER - 2] error shows up again, that might be something else to look at, I've never logged one of those and it is a completely different block of code.

                                  Cheers

                                  Bob.DigB 1 Reply Last reply Reply Quote 0
                                  • Bob.DigB
                                    Bob.Dig LAYER 8 @jrey
                                    last edited by Bob.Dig

                                    @jrey Sry, reading the log again, the first line I posted is not related to this but something before that.

                                    And yes the file is working fine. Maybe I was to fast klicking reload again or something, it is an install on a VPS.

                                    BBcan177B 1 Reply Last reply Reply Quote 1
                                    • BBcan177B
                                      BBcan177 Moderator @Bob.Dig
                                      last edited by

                                      The problem is with BGPview.io which is now owned by Recorded Future. They are rate limiting their service due to some users who hit their services too much. The current code in pfB contains a User Agent Header which is being blocked en masse by them. I have requested that if they rate limit to block specific Agent strings as the suffix of the Agent String is unique to the user.

                                      I have been emailing their support team for several weeks and they are saying it's low priority. I have also asked for their usage policy which they say they will add to their FYI page but it's low on their to do.

                                      So I assume if it's working for some users, it could be they changed the Agent String. But if that were to happen en masse, it's back to the same result.

                                      I am looking at an alternative ASN source as BGPview.io and previously Hurricane Electric don't want to support Open Source projects. Most likely will try to use ipinfo if their free options are accurate enough.

                                      "Experience is something you don't get until just after you need it."

                                      Website: http://pfBlockerNG.com
                                      Twitter: @BBcan177  #pfBlockerNG
                                      Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                      1 Reply Last reply Reply Quote 9
                                      • M michmoor referenced this topic on
                                      • P
                                        Popolou
                                        last edited by Popolou

                                        Came across the same issue on one of our boxes. For the time being, have disabled the ASN download and copied over the relevant original files to reload into the affected system.

                                        1 Reply Last reply Reply Quote 0
                                        • Bob.DigB Bob.Dig referenced this topic on
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.