Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Guide to filtering web content (http and https) with pfsense 2.3

    Scheduled Pinned Locked Moved Documentation
    190 Posts 54 Posters 235.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      aGeekhere
      last edited by

      Post a link to the https fake virus test file that you are testing and I will see if it works for me.

      Never Fear, A Geek is Here!

      1 Reply Last reply Reply Quote 0
      • A
        AR15USR
        last edited by

        http://www.eicar.org/85-0-Download.html

        There is one group for http, and one group for https.


        2.6.0-RELEASE

        1 Reply Last reply Reply Quote 0
        • A
          asterix
          last edited by

          There are 4 https links

          https://secure.eicar.org/eicar.com
          https://secure.eicar.org/eicar.com.txt
          https://secure.eicar.org/eicar_com.zip
          https://secure.eicar.org/eicarcom2.zip

          1 Reply Last reply Reply Quote 0
          • A
            asterix
            last edited by

            Would you know how to get the below google safesearch info in pfSense BIND DNS?

            server: include: /var/unbound/forecegoogle.conf

            1 Reply Last reply Reply Quote 0
            • A
              aGeekhere
              last edited by

              Ok done some research squidclamav only supports http not https because it is encrypted.

              would you know how to get the below google safesearch info in pfSense BIND DNS?

              Not sure you will need to ask that in the proxy forum.

              Never Fear, A Geek is Here!

              1 Reply Last reply Reply Quote 0
              • A
                AR15USR
                last edited by

                @aGeekHere:

                Ok done some research squidclamav only supports http not https because it is encrypted.

                I just did a test. Squidclamav will scan https traffic when using Squids MITM option..


                2.6.0-RELEASE

                1 Reply Last reply Reply Quote 0
                • A
                  asterix
                  last edited by

                  @AR15USR:

                  @aGeekHere:

                  Ok done some research squidclamav only supports http not https because it is encrypted.

                  I just did a test. Squidclamav will scan https traffic when using Squids MITM option..

                  And that would need certificates have to be installed on the clients..

                  1 Reply Last reply Reply Quote 0
                  • A
                    AR15USR
                    last edited by

                    @Asterix:

                    @AR15USR:

                    @aGeekHere:

                    Ok done some research squidclamav only supports http not https because it is encrypted.

                    I just did a test. Squidclamav will scan https traffic when using Squids MITM option..

                    And that would need certificates have to be installed on the clients..

                    Correct.


                    2.6.0-RELEASE

                    1 Reply Last reply Reply Quote 0
                    • A
                      aGeekhere
                      last edited by

                      To help stop virus or spywhere we can enable squidguard block list blk_BL_spyware,

                      Never Fear, A Geek is Here!

                      1 Reply Last reply Reply Quote 0
                      • johnpozJ
                        johnpoz LAYER 8 Global Moderator
                        last edited by

                        What is the point of all the safesearch nonsense and redirecting users to only use your dns..  You do understand when a proxy is being used, the proxy does the query not the client..

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        1 Reply Last reply Reply Quote 0
                        • P
                          phunni
                          last edited by

                          While setting up wpad we'r supposed ot enter the following into /usr/local/www/wpad.da:

                          function FindProxyForURL(url, host)
                          {
                              if (isPlainHostName(host) ||
                                  shExpMatch(host, "*.local") ||
                                  isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
                                  return "DIRECT";

                          return "PROXY 192.168.1.1:3128";
                          }

                          I've actually changed my pfsense server ip to 192.168.0.1 - do I need to edit both ip addresses listed above?

                          1 Reply Last reply Reply Quote 0
                          • A
                            AR15USR
                            last edited by

                            @johnpoz:

                            What is the point of all the safesearch nonsense and redirecting users to only use your dns..  You do understand when a proxy is being used, the proxy does the query not the client..

                            I think the point of the safe search is to stop, say your young kids, from Googling porn images. I'm not sure about the dns redirecting.


                            2.6.0-RELEASE

                            1 Reply Last reply Reply Quote 0
                            • A
                              aGeekhere
                              last edited by

                              @AR15USR:

                              @johnpoz:

                              What is the point of all the safesearch nonsense and redirecting users to only use your dns..  You do understand when a proxy is being used, the proxy does the query not the client..

                              I think the point of the safe search is to stop, say your young kids, from Googling porn images. I'm not sure about the dns redirecting.

                              1. What is the point of forcing search engines from using safe search?
                              Answer: To aid in the filtering of adult content, this is most importantly for google images as squidguard does not block them, if you do not want to filter web content then this guide is not designed for you.

                              2. What is the point of redirecting users to use THEIR pfsense router as the DNS server.
                              Answer: There are many advantages not all relating to web filtering however tries and stops the user from bypassing the dns redirect rule.

                              @phunni:

                              While setting up wpad we'r supposed ot enter the following into /usr/local/www/wpad.da:

                              function FindProxyForURL(url, host)
                              {
                                  if (isPlainHostName(host) ||
                                      shExpMatch(host, "*.local") ||
                                      isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
                                      return "DIRECT";

                              return "PROXY 192.168.1.1:3128";
                              }

                              I've actually changed my pfsense server ip to 192.168.0.1 - do I need to edit both ip addresses listed above?

                              Yes, set it to 192.168.0.0

                              Never Fear, A Geek is Here!

                              1 Reply Last reply Reply Quote 0
                              • A
                                AR15USR
                                last edited by

                                I'm constantly getting these entries in the Squid Real Time log:

                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:59488	-	-
                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:55735	-	-
                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:49806	-	-
                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:46365	-	-
                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:38156	-	-
                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:25012	-	-
                                01.07.2016 06:06:34	192.168.1.5	TCP_DENIED/403	127.0.0.1:24866	-	-
                                01.07.2016 06:06:33	192.168.1.5	TCP_DENIED/403	127.0.0.1:14826	-	-
                                01.07.2016 06:06:33	192.168.1.5	TCP_DENIED/403	127.0.0.1:10196	-	-
                                01.07.2016 06:06:33	192.168.1.5	TCP_DENIED/403	127.0.0.1:6263	-	-
                                

                                192.168.1.5 is my local machine. Is this normal?

                                Update: Resolved this in another thread..


                                2.6.0-RELEASE

                                1 Reply Last reply Reply Quote 0
                                • P
                                  phunni
                                  last edited by

                                  To stop users from bypassing your proxy setup a new firewall lan rule and block port 80 and 443

                                  Just to clarify - this is to prevent users from bypassing the proxy altogether, rather than just  bypassing the autoproxy/wpad stuff?

                                  1 Reply Last reply Reply Quote 0
                                  • P
                                    phunni
                                    last edited by

                                    The Squid Guard settings aren't working for me.  I download the blacklist and then go to Common ACL (I've already completed the target categories step on a previous run through), but the "Target Rules" only contains "^whitelist all" and there are only [whitelist] and Default access [all] options there.

                                    On another point, I don't want to have to set up auto proxy for every device connecting to my network - would it work if I use transparent proxy and then explicitly set the https_proxy on the couple of machines I really need locked down?

                                    1 Reply Last reply Reply Quote 0
                                    • A
                                      aGeekhere
                                      last edited by

                                      @phunni:

                                      To stop users from bypassing your proxy setup a new firewall lan rule and block port 80 and 443

                                      Just to clarify - this is to prevent users from bypassing the proxy altogether, rather than just  bypassing the autoproxy/wpad stuff?

                                      If port 80 and 443 are lefted open then the user can simply untic the auto configure proxy (on their PC, Mac, phone etc) and set the setting to go direct. This will not call for the wpad and the user will go direct and not use the proxy.

                                      @phunni:

                                      The Squid Guard settings aren't working for me.  I download the blacklist and then go to Common ACL (I've already completed the target categories step on a previous run through), but the "Target Rules" only contains "^whitelist all" and there are only [whitelist] and Default access [all] options there.

                                      On another point, I don't want to have to set up auto proxy for every device connecting to my network - would it work if I use transparent proxy and then explicitly set the https_proxy on the couple of machines I really need locked down?

                                      1.With squidguard set default access to allow then set the categories you want to deny. Save and then go to squidguard General settings and hit apply. Please read the tip under this section it is very important.
                                      Read this https://doc.pfsense.org/index.php/SquidGuard_package

                                      2. Transperrent proxy does not use a wpad, if you want to use transperrent proxy for http then you need SSL Man In the Middle Filtering for https which must have a certificate installed on evey device.

                                      So you can either set auto configure proxy on all devices or install a certificate on all devices.

                                      You cannot use a transperrent proxy and a wpad at the same time. If you did get a transperrent proxy with SSL mitm working then there is no need for manual proxy mode.

                                      Never Fear, A Geek is Here!

                                      1 Reply Last reply Reply Quote 0
                                      • P
                                        phunni
                                        last edited by

                                        @aGeekHere:

                                        1.With squidguard set default access to allow then set the categories you want to deny. Save and then go to squidguard General settings and hit apply. Please read the tip under this section it is very important.
                                        Read this https://doc.pfsense.org/index.php/SquidGuard_package

                                        That was the problem - there were no other categories than whitelist and default access.  I managed to get the others by enabling "Blackilist" under the general settings of Squid Guard.  Forgive me if I just missed it, but I couldn't see that step in the guide.

                                        1 Reply Last reply Reply Quote 0
                                        • A
                                          aGeekhere
                                          last edited by

                                          I added
                                          In squidguard under General settings
                                          Tic enable
                                          Tic Enable log
                                          Tic Enable log rotation
                                          Tic enable blacklist
                                          Under Blacklist URL add http://www.shallalist.de/Downloads/shallalist.tar.gz
                                          Save
                                          apply (you must always hit apply for any changes you made to squidguard).

                                          Never Fear, A Geek is Here!

                                          1 Reply Last reply Reply Quote 0
                                          • A
                                            aGeekhere
                                            last edited by

                                            I found this post and I am researching to see how well it will work.

                                            https://forum.pfsense.org/index.php?topic=106016.0

                                            The idea is to use the wpad for https content and have a transperrent proxy for http traffic. This could remove the bypass rules for programs with no proxy settings and need to use port 80.

                                            Update
                                            OK it looks to be working fine, now all the traffic that was block on port 80 is now using the transperrent proxy, you will still need a pass rule for port 443 but not for port 80.

                                            So you can use the wpad for http and https filtering (or for just https) and enable the transperrent proxy to catch the leftovers.

                                            Never Fear, A Geek is Here!

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.