• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Adding a large list as an alias under "URL Table (IPs)" fails.

Scheduled Pinned Locked Moved Firewalling
9 Posts 3 Posters 711 Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S
    scilek
    last edited by 13 days ago

    I'm trying to add this list as an alias:
    https://www.usom.gov.tr/url-list.txt

    The list contains 404.168 entries. So I know I have to make this adjustment:
    7f6e54da-56f7-432c-8ee4-28fc681020eb-image.png

    Then I try to create the alias:
    6baaf51c-dc00-427f-bd0f-ed3cbbf513b1-image.png

    But I get this:
    6718cf41-40da-4ba7-be56-84f709395f84-image.png

    I tried editing the system.inc file and increase the timeout values to 600:
    dbfcd664-5da0-426b-9f81-97111970427a-image.png
    f0796efd-154d-4339-aa76-3cba3fd0ce0b-image.png

    But that did not help either.

    Is this a bug or is there something I can do to tackle this issue?

    B J 2 Replies Last reply 13 days ago Reply Quote 0
    • B
      Bob.Dig LAYER 8 @scilek
      last edited by 13 days ago

      @scilek Try pfBlocker

      S 1 Reply Last reply 13 days ago Reply Quote 0
      • S
        scilek @Bob.Dig
        last edited by 13 days ago

        @Bob-Dig
        Is that a recommendation or don't I have a choice?

        B 1 Reply Last reply 13 days ago Reply Quote 0
        • B
          Bob.Dig LAYER 8 @scilek
          last edited by Bob.Dig 13 days ago 13 days ago

          @scilek said in Adding a large list as an alias under "URL Table (IPs)" fails.:

          don't I have a choice?

          It looks like you have no choice. Doing it with your settings in pfSense doesn't work for me either, with pfBlocker, no problem.

          1 Reply Last reply Reply Quote 0
          • J
            johnpoz LAYER 8 Global Moderator @scilek
            last edited by 13 days ago

            @scilek that list your pointing at doesn't contain IPs or subnets - it contains domain names. It has a few ips in it - but most of it is domains.

            Which all would have to be resolved.

            An intelligent man is sometimes forced to be drunk to spend time with his fools
            If you get confused: Listen to the Music Play
            Please don't Chat/PM me for help, unless mod related
            SG-4860 24.11 | Lab VMs 2.8, 24.11

            S B 2 Replies Last reply 13 days ago Reply Quote 1
            • S
              scilek @johnpoz
              last edited by 13 days ago

              @johnpoz Yes, and that takes a lot of time, and I understand that. But I still cannot understand why that should trigger a 504 error. It doesn't have to be like that.

              J 1 Reply Last reply 13 days ago Reply Quote 0
              • B
                Bob.Dig LAYER 8 @johnpoz
                last edited by 13 days ago

                @johnpoz For me it wasn't stopping so I closed the window. in pfBlocker, it took seconds. I only looked briefly, saw many IPs and deleted it. Haven't checked if it was complete.

                1 Reply Last reply Reply Quote 0
                • J
                  johnpoz LAYER 8 Global Moderator @scilek
                  last edited by johnpoz 13 days ago 13 days ago

                  @scilek doesn't matter to be honest - that function is for IPs and subnets.. I don't even know if works at all even if you had only 1 fqdn in it.

                  ips.jpg

                  I don't think I ever got a full download of it just in my browser.. Let alone pfsense having to resolve every one of the you say some 400k entries to an IP(s)..

                  But that alias is not meant for domains, it meant for IPs and subnets..

                  Even if was pure IPs and subnets - the note says large 30k+ records, your way over 10x that..

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  S 1 Reply Last reply 12 days ago Reply Quote 0
                  • S
                    scilek @johnpoz
                    last edited by 12 days ago

                    @johnpoz I think I know why it fails. Once again, I realized that a software man cannot afford to be lazy. I scanned the entries, and realised (to my amazement) that there are many invalid entries, i.e. ones that contain characters that cannot exist in a domain name, the most common one being the humble "/" (slash). It appears someone did a sloppy job there. "Mystery solved, I guess; right?" (https://www.youtube.com/watch?v=dDmqOott-_4)

                    1 Reply Last reply Reply Quote 0
                    9 out of 9
                    • First post
                      9/9
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                      This community forum collects and processes your personal information.
                      consent.not_received