Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    IP-Blocklist

    pfSense Packages
    86
    496
    493.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      ToxIcon
      last edited by

      $ pfctl -s rules | grep  ipblocklist
      pass quick from <ipblocklistw>to any flags S/SA keep state label "IP-Blocklist"
      pass quick inet from 192.168.1.100 to <ipblocklistw>flags S/SA keep state label "IP-Blocklist"
      pass quick on em1 inet6 from fe00::3e3:5yff:fgx44:8c84 to <ipblocklistw>flags S/SA keep state label "IP-Blocklist"
      block drop quick inet from <ipblocklist>to 192.168.1.100 label "IP-Blocklist"
      block drop quick on em1 inet6 from <ipblocklist>to fe00::3e3:5yff:fgx44:8c84 label "IP-Blocklist"
      block drop quick inet from 192.168.1.100 to <ipblocklist>label "IP-Blocklist"
      block drop quick on em1 inet6 from fe00::3e3:5yff:fgx44:8c84 to <ipblocklist>label "IP-Blocklist"
      pass quick from <ipblocklistw>to any flags S/SA keep state label "IP-Blocklist"
      pass quick on em0 inet6 from fe93::6k04:hh:fhg0:5783 to <ipblocklistw>flags S/SA keep state label "IP-Blocklist"
      pass quick inet from 42.200.59.16 to <ipblocklistw>flags S/SA keep state label "IP-Blocklist"
      block drop quick on em0 inet6 from <ipblocklist>to fe93::6k04:hh:fhg0:5783 label "IP-Blocklist"
      block drop quick inet from <ipblocklist>to 192.168.1.100 label "IP-Blocklist"
      block drop quick on em0 inet6 from fe93::6k04:hh:fhg0:5783 to <ipblocklist>label "IP-Blocklist"
      block drop quick inet from 42.200.59.16 to <ipblocklist>label "IP-Blocklist"

      pfctl -T show -t ipblocklist 
      has no output

      $ /usr/local/www/packages/ipblocklist/convert-execute.sh
      0
      1
      2
      3
      4
      5
      269
      270
      271
      272
      273
      274</ipblocklist></ipblocklist></ipblocklist></ipblocklist></ipblocklistw></ipblocklistw></ipblocklistw></ipblocklist></ipblocklist></ipblocklist></ipblocklist></ipblocklistw></ipblocklistw></ipblocklistw>

      1 Reply Last reply Reply Quote 0
      • T
        tommyboy180
        last edited by

        Kill that table and start over. Uncheck the enable checkbox and click save but before you re-enable it make sure that it's gone by running "pfctl -s rules | grep  ipblocklist" again to make sure it's all gone.

        Reload the firewall filter too to regenerate your rules.debug. Then enable the package and try again. (rebooting is easiest or edit a firewall rule without making changes and save, doing Status->Filter Reload does not work for this)

        Also edit /usr/local/www/packages/ipblocklist/interfaces.txt and replace everything in there with "any" on the first line and save the file. I have had some problems by specifying the interface for IP-Blocklist to use and there really is no advantage to it. Only specify interfaces if you absolutely need it and even then it still might work correctly.

        I'm not sure why your output from  /usr/local/www/packages/ipblocklist/convert-execute.sh didn't include the errors and warnings that mine had. You should see that two tables got deleted and it should produce an error on deleting those TEMP files that don't exist.

        I'm not sure how comfortable you are with strangers looking at your system but if you give me SSH access I can figure out what's going on faster or make a snapshot of the box. I know that's a little extreme to troubleshoot a package but I'm really curious why it's not working for you.

        Are you running countryblock by any chance, and if so is that working?

        -Tom Schaefer
        SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

        Please support pfBlocker | File Browser | Strikeback

        1 Reply Last reply Reply Quote 0
        • P
          Pistolero
          last edited by

          Hi tommyboy!

          I am VERY happy to report that I am no longer seeing any issues between IPBlockList, CountryBlock and HAVP! THANK YOU!

          Quick Q: will the URLs from iblocklist work in their regular format (http://list.iblocklist.com/?list=ynkdjqsjyfmilsgbogqf), or do we have to point to the .gz file directly? I ask because I had a bunch of direct links to the gz files, and IPBL showed 103000 blocked networks… I replaced them all with the /?listname links, and now I am showing only 1978 blocked IPs/Networks.

          Also, how can I set up IPBL so it will automatically update the lists every night?

          Thanks!

          1 Reply Last reply Reply Quote 0
          • T
            tommyboy180
            last edited by

            @Pistolero:

            Hi tommyboy!

            I am VERY happy to report that I am no longer seeing any issues between IPBlockList, CountryBlock and HAVP! THANK YOU!

            Quick Q: will the URLs from iblocklist work in their regular format (http://list.iblocklist.com/?list=ynkdjqsjyfmilsgbogqf), or do we have to point to the .gz file directly? I ask because I had a bunch of direct links to the gz files, and IPBL showed 103000 blocked networks… I replaced them all with the /?listname links, and now I am showing only 1978 blocked IPs/Networks.

            Also, how can I set up IPBL so it will automatically update the lists every night?

            Thanks!

            Glad to hear it!
            You do have to directly link to the lists. Perhaps in the future I may find a way around this.
            To have the package update your lists every night you can edit the cron executable which is located at /usr/local/etc/rc.d/IP-Blocklist.sh

            Just comment out lines 3-14 and it should be good. Setup a cron job to run this executable whenever you like. Use the cron job package for that.

            -Tom Schaefer
            SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

            Please support pfBlocker | File Browser | Strikeback

            1 Reply Last reply Reply Quote 0
            • P
              Pistolero
              last edited by

              Thanks for the quick reply!

              What is the best repository to get at the gz files directly? is it this? http://list11.iblocklist.com/files/

              Do you have a preferred source, and also what are your favorite lists?

              @tommyboy180:

              Glad to hear it!
              You do have to directly link to the lists. Perhaps in the future I may find a way around this.
              To have the package update your lists every night you can edit the cron executable which is located at /usr/local/etc/rc.d/IP-Blocklist.sh

              Just comment out lines 3-14 and it should be good. Setup a cron job to run this executable whenever you like. Use the cron job package for that.

              1 Reply Last reply Reply Quote 0
              • T
                tommyboy180
                last edited by

                @Pistolero:

                Thanks for the quick reply!

                What is the best repository to get at the gz files directly? is it this? http://list11.iblocklist.com/files/

                Do you have a preferred source, and also what are your favorite lists?

                @tommyboy180:

                Glad to hear it!
                You do have to directly link to the lists. Perhaps in the future I may find a way around this.
                To have the package update your lists every night you can edit the cron executable which is located at /usr/local/etc/rc.d/IP-Blocklist.sh

                Just comment out lines 3-14 and it should be good. Setup a cron job to run this executable whenever you like. Use the cron job package for that.

                I didn't know that there was a directory listing! I usually download the list and go back to my download history and copy the address. I am working on a way to upload lists but I won't have that out for some time. I usually use iblocklist.com for most of my lists.

                My favorite lists include the following:

                • http://iblocklist.dbnservers.net/files/bt_spyware.gz

                • http://withhorns.com/files/ficutxiwawokxlcyoeye.gz

                • http://iblocklist.dchubad.com/files/ghlzqtqxnzctvvajwwag.gz

                • http://list.iblocklist.com/files/sh_drop.gz

                • http://www.tomschaefer.org/temp/pfsense/IP-Blocklist-ForumSpam.txt

                • http://iblocklist.dchubad.com/files/bt_ads.gz

                • http://www.tomschaefer.org/temp/pfsense/MISC-Block.txt

                The TomSchaefer.org ones are my custom lists that I made.

                -Tom Schaefer
                SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                Please support pfBlocker | File Browser | Strikeback

                1 Reply Last reply Reply Quote 0
                • P
                  Pistolero
                  last edited by

                  Whoa… check this out:

                  
                  [12-31-10 16:12:19]root@/usr/local/www/packages/ipblocklist/lists#  lh
                  total 32776
                  -rw-r--r--  1 root  wheel    88K Dec 31 16:06 ?list=tor
                  -rw-r--r--  1 root  wheel   131K Dec 30 17:00 bt_ads
                  -rw-r--r--  1 root  wheel   969K Dec 30 17:00 bt_badpeers
                  -rw-r--r--  1 root  wheel   6.8K Dec 30 17:00 bt_dshield
                  -rw-r--r--  1 root  wheel    12M Dec 30 17:00 bt_level1
                  -rw-r--r--  1 root  wheel    19K Mar 21  2010 bt_webexploit-forumspam
                  -rw-r--r--  1 root  wheel   1.7K Dec 30 17:00 dcha_faker
                  -rw-r--r--  1 root  wheel   5.4K Dec 30 17:00 dcha_hacker
                  -rw-r--r--  1 root  wheel   111K Dec 30 17:00 dcha_pedophiles
                  -rw-r--r--  1 root  wheel   6.9K Dec 30 17:00 dcha_spammer
                  -rw-r--r--  1 root  wheel   3.8M Dec 31 16:09 ipfw.ipfw
                  -rw-r--r--  1 root  wheel    11K Dec 28 17:00 sh_drop
                  -rw-r--r--  1 root  wheel    11K Dec 19 17:00 tbg_hijacked
                  -rw-r--r--  1 root  wheel    15M Dec 28 17:00 tbg_primarythreats
                  -rw-r--r--  1 root  wheel   6.6K Dec 30 17:00 ynkdjqsjyfmilsgbogqf
                  [12-31-10 16:12:30]root@/usr/local/www/packages/ipblocklist/lists#  cat ?list=tor
                  # List distributed by iblocklist.com
                  
                  The Onion Router:2.36.33.51-2.36.33.51
                  The Onion Router:8.17.81.25-8.17.81.25
                  The Onion Router:8.24.61.246-8.24.61.246
                  ...
                  
                  

                  Looks like the /?list= links DO somewhat work… weird, huh?

                  1 Reply Last reply Reply Quote 0
                  • T
                    tommyboy180
                    last edited by

                    @Pistolero:

                    Whoa… check this out:

                    
                    [12-31-10 16:12:19]root@/usr/local/www/packages/ipblocklist/lists#  lh
                    total 32776
                    -rw-r--r--  1 root  wheel    88K Dec 31 16:06 ?list=tor
                    -rw-r--r--  1 root  wheel   131K Dec 30 17:00 bt_ads
                    -rw-r--r--  1 root  wheel   969K Dec 30 17:00 bt_badpeers
                    -rw-r--r--  1 root  wheel   6.8K Dec 30 17:00 bt_dshield
                    -rw-r--r--  1 root  wheel    12M Dec 30 17:00 bt_level1
                    -rw-r--r--  1 root  wheel    19K Mar 21  2010 bt_webexploit-forumspam
                    -rw-r--r--  1 root  wheel   1.7K Dec 30 17:00 dcha_faker
                    -rw-r--r--  1 root  wheel   5.4K Dec 30 17:00 dcha_hacker
                    -rw-r--r--  1 root  wheel   111K Dec 30 17:00 dcha_pedophiles
                    -rw-r--r--  1 root  wheel   6.9K Dec 30 17:00 dcha_spammer
                    -rw-r--r--  1 root  wheel   3.8M Dec 31 16:09 ipfw.ipfw
                    -rw-r--r--  1 root  wheel    11K Dec 28 17:00 sh_drop
                    -rw-r--r--  1 root  wheel    11K Dec 19 17:00 tbg_hijacked
                    -rw-r--r--  1 root  wheel    15M Dec 28 17:00 tbg_primarythreats
                    -rw-r--r--  1 root  wheel   6.6K Dec 30 17:00 ynkdjqsjyfmilsgbogqf
                    [12-31-10 16:12:30]root@/usr/local/www/packages/ipblocklist/lists#  cat ?list=tor
                    # List distributed by iblocklist.com
                    
                    The Onion Router:2.36.33.51-2.36.33.51
                    The Onion Router:8.17.81.25-8.17.81.25
                    The Onion Router:8.24.61.246-8.24.61.246
                    ...
                    
                    

                    Looks like the /?list= links DO somewhat work… weird, huh?

                    Yeah but inspect the file. Make sure they are real lists with IPs.
                    EDIT. I see you ran cat on one, so I guess you're right. I have found that it doesn't work on some links so just be careful. It's probably a good idea to go direct just to ensure you really are getting the right content.

                    -Tom Schaefer
                    SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                    Please support pfBlocker | File Browser | Strikeback

                    1 Reply Last reply Reply Quote 0
                    • P
                      Pistolero
                      last edited by

                      About the CRON job… are these the lines I have to comment out?

                      
                      #check if ipblocklist running
                      export resultr=`pfctl -s rules | grep -c ipblocklist`
                      
                      #echo $resultr
                      if [ "$resultr" -gt "0" ]; then
                              echo running
                              exit 1
                      else
                              echo not running
                              /usr/bin/logger -s "IP-Blocklist was found not running"
                              echo "IP-Blocklist not running" | /usr/local/bin/php /usr/local/www/packages/ipblocklist/email_send.php
                      fi
                      
                      

                      After they get commented, I make a copy of the script and schedule that one? (I am kinda UNIX n00b :P)

                      So, my script now looks like so:

                      
                      #!/bin/sh
                      
                      ####check if ipblocklist running
                      #export resultr=`pfctl -s rules | grep -c ipblocklist`
                      
                      ####echo $resultr
                      #if [ "$resultr" -gt "0" ]; then
                      #       echo running
                      #       exit 1
                      #else
                      #       echo not running
                      #       /usr/bin/logger -s "IP-Blocklist was found not running"
                      #       echo "IP-Blocklist not running" | /usr/local/bin/php /usr/local/www/packages/ipblocklist/email_send.php
                      #fi
                      
                      ...
                      
                      

                      and I saved that one with a different name, (I don't know if commenting those line on the original script will break the package, so I made a copy and set it to executable)

                      
                      [12-31-10 16:28:13]root@/usr/local/etc/rc.d#  lh | grep IP
                      -rwxr-xr-x  1 root   wheel   4.6K Dec 31 09:27 IP-Blocklist.sh
                      -rwxr-xr-x  1 root   wheel   4.6K Dec 31 16:25 IP-Blocklist_cron.sh
                      
                      
                      1 Reply Last reply Reply Quote 0
                      • T
                        tommyboy180
                        last edited by

                        Yeah. Make a copy, remove each of the lines you quoted and schedule your copy.

                        What lists change every day? I'm curious.

                        -Tom Schaefer
                        SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                        Please support pfBlocker | File Browser | Strikeback

                        1 Reply Last reply Reply Quote 0
                        • P
                          Pistolero
                          last edited by

                          One last question:

                          The CRONTAB format, if I want the lists to be updated every day @ 3:30 AM:

                          
                          30  	*/03  	*  	*  	*  	root  	/usr/local/etc/rc.d/IP-Blocklist_cron.sh 
                          
                          

                          Does this look OK to you, sir?

                          1 Reply Last reply Reply Quote 0
                          • T
                            tommyboy180
                            last edited by

                            I would move the script out of rc.d so it doesn't run on startup. That way you don't have anything conflicting with the other startup script.
                            Other than that everything looks good!

                            -Tom Schaefer
                            SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                            Please support pfBlocker | File Browser | Strikeback

                            1 Reply Last reply Reply Quote 0
                            • T
                              ToxIcon
                              last edited by

                              tommyboy180

                              Kill table and start over

                              Uncheck checkbox click save

                              pfctl -s rules | grep  ipblocklist  no output

                              Reload firewall filter

                              edit /usr/local/www/packages/ipblocklist/interfaces.txt  "any"

                              /usr/local/www/packages/ipblocklist/convert-execute.sh no errors and warnings

                              Current Status = Running

                              You are blocking 0 Networks/IPs

                              yes running countryblock  working

                              1 Reply Last reply Reply Quote 0
                              • T
                                tommyboy180
                                last edited by

                                @ToxIcon:

                                tommyboy180

                                Kill table and start over

                                Uncheck checkbox click save

                                pfctl -s rules | grep  ipblocklist  no output

                                Reload firewall filter

                                edit /usr/local/www/packages/ipblocklist/interfaces.txt   "any"

                                /usr/local/www/packages/ipblocklist/convert-execute.sh no errors and warnings

                                Current Status = Running

                                You are blocking 0 Networks/IPs

                                yes running countryblock  working

                                Without being able to inspect your system it looks like convert-execute.sh isn't running the commands correctly.

                                On countryblock when it runs a program it calls for it without direct path. For example a line might read, "pfctl -t countryblock -T kill"
                                However on ipblocklist when it runs a program it calls for it via its direct path. For example a line might read, "/sbin/pfctl -t ipblocklist -T kill"

                                I think this is why you're not seeing any warnings or output. Attached is a replacement BASH script file that you should try. See if that fixes the issue.

                                Warning For other users downloading this replacement BASH script this may break your package. This is for troubleshooting only.

                                convert-execute.sh.txt

                                -Tom Schaefer
                                SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                                Please support pfBlocker | File Browser | Strikeback

                                1 Reply Last reply Reply Quote 0
                                • R
                                  rclare70
                                  last edited by

                                  Using Multi pal Nat Routes,

                                  WAN
                                  LAN
                                  OPT1 - Running a Hot Public WiFi zone, the previously attached replacement bash script fixed the issues for me.

                                  for the index.html?list=bt_level1, style files, I created a work around for my own use age, if this helps anyone:

                                  I made a Sub directory on my web sever (Cloud hosted) and restricted this access down to my pfSense gateway IP.
                                  created the following bash script.:

                                  cd /var/www/pg2
                                  rm -rf bt_level1.gz && wget http://list.iblocklist.com/?list=bt_level1 && mv index.html?list=bt_level1 bt_level1.gz
                                  rm -rf bt_level2.gz && wget http://list.iblocklist.com/?list=bt_level2 && mv index.html?list=bt_level2 bt_level2.gz
                                  rm -rf bt_level3.gz && wget http://list.iblocklist.com/?list=bt_level3 && mv index.html?list=bt_level3 bt_level3.gz

                                  scheduled this to run on a weekly basis, and then have pfSense query the Remote Web server,

                                  but there is no reason why this can't be changed to function local on the pfSense box.
                                  Have it fetch the index.html?list=bt_level1 file, store it locally on the web-server, rename the file, and then re-query it's own web-sever from the ipblocklist interface, which should end up in the index.html?list=bt_level1 style links working without the need for the intermediate server, however for my needs I prefer to have the mirror pre-fetching occurring in a hosted environment.

                                  If there is any interest I'll happily attempt to create something to run this all locally on the pfSense Box.

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    tommyboy180
                                    last edited by

                                    If only pfsense came with wget to begin with then you wouldn't have to do this. Since fetch doesn't work well with these dynamic links it becomes a problem. Maybe I will consider adding wget after I get file uploads going.

                                    -Tom Schaefer
                                    SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                                    Please support pfBlocker | File Browser | Strikeback

                                    1 Reply Last reply Reply Quote 0
                                    • ?
                                      Guest
                                      last edited by

                                      I came across your forum after reviewing our server logs at Country IP Blocks. We noticed the discussion included Country IP Blocks Access Control Lists in a PG2 format. As a courtesy we have created these lists from our Country IP Blocks Database. The data is in a format that looks like this:

                                      AZERBAIJAN:46.18.64.0-46.18.71.255
                                      …
                                      UNITED STATES:3.0.0.0-3.255.255.255
                                      ...
                                      CHINA:1.12.0.0-1.15.255.255

                                      A reminder, our database is updated daily.

                                      We can output our data in any format needed.

                                      You can get your PG2 formatted list here: http://www.countryipblocks.net/networking/pg2-formatted-acess-control-lists/

                                      Thanks,

                                      Stewart White

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        tommyboy180
                                        last edited by

                                        @countryipblocks:

                                        I came across your forum after reviewing our server logs at Country IP Blocks. We noticed the discussion included Country IP Blocks Access Control Lists in a PG2 format. As a courtesy we have created these lists from our Country IP Blocks Database. The data is in a format that looks like this:

                                        AZERBAIJAN:46.18.64.0-46.18.71.255
                                        …
                                        UNITED STATES:3.0.0.0-3.255.255.255
                                        ...
                                        CHINA:1.12.0.0-1.15.255.255

                                        A reminder, our database is updated daily.

                                        We can output our data in any format needed.

                                        You can get your PG2 formatted list here: http://www.countryipblocks.net/networking/pg2-formatted-acess-control-lists/

                                        Thanks,

                                        Stewart White

                                        Excellent! Thank you for taking the time to post!

                                        -Tom Schaefer
                                        SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                                        Please support pfBlocker | File Browser | Strikeback

                                        1 Reply Last reply Reply Quote 0
                                        • ?
                                          Guest
                                          last edited by

                                          @tommyboy180:

                                          Excellent! Thank you for taking the time to post!

                                          You're welcome. We'll keep the data flowing, just let us know your needs.

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            tommyboy180
                                            last edited by

                                            @countryipblocks:

                                            @tommyboy180:

                                            Excellent! Thank you for taking the time to post!

                                            You're welcome. We'll keep the data flowing, just let us know your needs.

                                            I had a question for you. What do you think about the countryblock package for pfsense? It downloads countrylists from your site directly. A while ago we had a small discussion over the semi automated process and your rules within the FAQ. At the time we determined that it does not break your rules, however now that you're here you could comment and give me a final say.

                                            -Tom Schaefer
                                            SuperMicro 1U 2X Intel pro/1000 Dual Core Intel 2.2 Ghz - 2 Gig RAM

                                            Please support pfBlocker | File Browser | Strikeback

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.