Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Guide to filtering web content (http and https) with pfsense 2.3

    Scheduled Pinned Locked Moved Documentation
    190 Posts 54 Posters 223.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      aGeekhere
      last edited by

      hi techbee, yeah after discovering a few new things the guide is a little messy now.

      The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

      when i get time I will try an clean it up a bit.

      Never Fear, A Geek is Here!

      1 Reply Last reply Reply Quote 0
      • V
        vielfede
        last edited by

        @aGeekHere:

        The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

        Indeed I do not think MITM and splice all can work toghether, as stated on squid documentation MITM is associated to the "bump" directive that is something complety different from splice directive.
        With MITM (bump) squid is able to decrypt traffic (and analyse it) meanwhile with splice all you can do is just "web filtering".
        Summarizing
        Directive          Advantages/features                                                                    Disadvantage
        Splice (all)        No needs of certificate installation on Clients + webfiltering              No traffic analysis i.e no AntiVirus
        Bump (MITM)    Traffic analysis i.e noYES AntiVirus    + webfiltering                              Needs to install certs on clients

        Please take a look here and let me know if I missed something. Thanks
        http://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions
        http://marek.helion.pl/install/squid.html

        1 Reply Last reply Reply Quote 1
        • A
          AR15USR
          last edited by

          @vielfede:

          @aGeekHere:

          The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

          Indeed I do not think MITM and splice all can work toghether, as stated on squid documentation MITM is associated to the "bump" directive that is something complety different from splice directive.
          With MITM (bump) squid is able to decrypt traffic (and analyse it) meanwhile with splice all you can do is just "web filtering".
          Summarizing
          Directive          Advantages/features                                                                    Disadvantage
          Splice (all)        No needs of certificate installation on Clients + webfiltering              No traffic analysis i.e no AntiVirus
          Bump (MITM)    Traffic analysis i.e no AntiVirus    + webfiltering                              Needs to install certs on clients

          Please take a look here and let me know if I missed something. Thanks
          http://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions
          http://marek.helion.pl/install/squid.html

          Shouldn't the second one not have the word "no"..


          2.6.0-RELEASE

          1 Reply Last reply Reply Quote 0
          • V
            vielfede
            last edited by

            @AR15USR:

            Shouldn't the second one not have the word "no"..

            Sorry cut & paste mistake, I fixed it.

            1 Reply Last reply Reply Quote 0
            • M
              mahnonsaprei
              last edited by

              Hi at all,

              I'm Marcello from italy. Nice to meet you, i'm a newbie of pfsense.

              I have read the tutorial, but i have one problem.
              When i try to connect to an https website in blacklist, the browser show me a generic error see the attachment: (ERR_TUNNEL_CONNECTION_FAILED) while i want to see a message of pfsense that explain the block. In http it's ok

              I'm looking for in google but i not found the answer.

              Please help me.
              Thank you veeeeery much!!!

              error.PNG
              error.PNG_thumb

              1 Reply Last reply Reply Quote 0
              • A
                aGeekhere
                last edited by

                When i try to connect to an https website in blacklist, the browser show me a generic error see the attachment: (ERR_TUNNEL_CONNECTION_FAILED) while i want to see a message of pfsense that explain the block. In http it's ok

                Known issue with squid, I do not think there is a fix for it.

                Never Fear, A Geek is Here!

                1 Reply Last reply Reply Quote 0
                • M
                  mahnonsaprei
                  last edited by

                  Thank you so much. There aren't alternatives to redirect https via proxy also blank page or other?

                  B.r.

                  1 Reply Last reply Reply Quote 0
                  • T
                    techbee
                    last edited by

                    hello ageekhere,

                    can you update your guide now.

                    1 Reply Last reply Reply Quote 0
                    • A
                      aGeekhere
                      last edited by

                      what part are you stuck on?

                      Never Fear, A Geek is Here!

                      1 Reply Last reply Reply Quote 0
                      • T
                        techbee
                        last edited by

                        1. I dont know what exact firewall rules to add
                        2. I am confused if I need to install unofficial wpad package of marcelloc.
                        3. I am confused if there is a need to have separate webserver of wpad.
                        4. I am confused if I can enable both transparent and ssl mitm filtering on squid.
                        5. I cant nslookup wpad on clients command prompt to resolve to my pfsense box ip.

                        1 Reply Last reply Reply Quote 0
                        • A
                          aGeekhere
                          last edited by

                          The firewall rules that you need are

                          
                          IPv4+6 TCP	*	*	*	80 (HTTP)	*	none	 	port 80 block
                          IPv4+6 TCP	*	*	*	443 (HTTPS)	*	none	 	port 443 block  
                          
                          

                          This blocks access to port 80 and 443, it stops users from bypassing the proxy.

                          Also if you want to force pfsense to be the DNS server use.

                          
                          In Firewall/NAT/Port forward 
                          add a new rule
                          
                          Interface = LAN 
                          Protocol = TCP/UDP
                          Source ports = *
                          Dest address = *
                          Dest ports = 53
                          NAT IP = 127.0.0.1
                          NAT Ports = 53
                          Description = Redirect DNS
                          LAN TCP/UDP * * * 53 127.0.0.1 53 Redirect DNS
                          Save
                          
                          

                          Have a look through a few of these videos https://www.youtube.com/playlist?list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk
                          There are a few really good configs he does there.

                          2. Wait for the package to become stable and available in the package manager, in the mean time you can use the pfsense webserver, put the wpad files in /usr/local/www/ read Part 3. (all that needs to be done is add the wpad files (wpad.da, proxy.pac and wpad.dat) to /usr/local/www/).

                          3. Just use pfsense webserver /usr/local/www/ (I have not used an external webserver)

                          4. This is what I did, I set up a WPAD and transparent proxy with ssl mitm (splice all so you do not have to install cert on every device) and set your device to auto configure. I found that just using WPAD a lot of programs that cannot use the proxy were getting blocked so the transparent proxy catches that and something the transparent proxy ssl has issues (like windows updates) however the WPAD is used for that. Best of both worlds.

                          5 I have had issues as well, if you put http://192.168.1.1/wpad.dat in your web browser and it downloads than it is working.

                          This guide goes a few steps fever by forcing google and bing to be in safe mode (even youtube).

                          So what I would do is get WPAD working then enable the transparent proxy with ssl mitm (create a cert for the router).

                          Hope this helps

                          Never Fear, A Geek is Here!

                          1 Reply Last reply Reply Quote 0
                          • T
                            techbee
                            last edited by

                            thanks for the reply aGeekHere.

                            What is the tcp port that you used on your webconfigurator ?  If I block port 80, then, I will also be blocked opening the pfsense gui.

                            Also, regarding to my question number 4, do you mean that you did not enable the http transparent proxy on port 3128 but only the HTTPS/SSL MITM on port 3129 and setup wpad. did I understood you correctly ?

                            1 Reply Last reply Reply Quote 0
                            • A
                              aGeekhere
                              last edited by

                              What is the tcp port that you used on your webconfigurator ?

                              set pfsense Protocol to http (This is a MUST, it will not work if you do not do this)
                              System: Advanced: Admin Access Protocol http

                              If I block port 80, then, I will also be blocked opening the pfsense gui.

                              No it should not (have the rule above the anti lockout rule);

                              Also, regarding to my question number 4

                              Enable squid proxy 3128
                              Get WPAD working follow guide

                              Enable transparent mode to forward all requests for destination port 80 to the proxy server.

                              Enable SSL filtering
                              Splice all
                              create Certificate (for pfsense).

                              Summery
                              Set Advanced: Admin Access Protocol to http
                              Enable squid proxy
                              Follow guide on setting up WPAD
                              Enable transparent mode
                              Enable SSL filtering Splice all

                              Never Fear, A Geek is Here!

                              1 Reply Last reply Reply Quote 0
                              • T
                                techbee
                                last edited by

                                Yep, done that all.

                                I can download the wpad files from the browser. I can access the http sites but still cannot access the https sites.

                                1 Reply Last reply Reply Quote 0
                                • A
                                  aGeekhere
                                  last edited by

                                  what web browser?

                                  Clear your cache?

                                  What Operating system?

                                  Operating system update work?

                                  Did you set auto configure proxy?

                                  Try Diagnostics States States clear?

                                  What sites, google?

                                  Never Fear, A Geek is Here!

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    techbee
                                    last edited by

                                    @aGeekHere:

                                    what web browser?

                                    Clear your cache?

                                    What Operating system?

                                    Operating system update work?

                                    Did you set auto configure proxy?

                                    Try Diagnostics States States clear?

                                    What sites, google?

                                    '

                                    Hi aGeekHere,

                                    Yes, cache is cleared,
                                    os=win7 pro sp1,
                                    I disabled the update coz I dont need it,
                                    autoconfigure is set,
                                    yes, google and yahoo https sites. facebook and youtube does not load and shows error.

                                    I use splice-all.  Don't I really need to install certs to client device ?

                                    If https site is successfully blocked, what error page do I expect to see ?  Can you show me the screenshot for the error page that will show !

                                    1 Reply Last reply Reply Quote 0
                                    • A
                                      aGeekhere
                                      last edited by

                                      firefox ie or chrome?

                                      I use splice-all.  Don't I really need to install certs to client device ?

                                      not when splice all

                                      If https site is successfully blocked, what error page do I expect to see ?

                                      The default cannot access page of the web browser.

                                      If you enabled squid, transparent mode and SSL filtering Splice all with a cert on pfsense and you still cannot access https sites, then something is broken.
                                      Try reinstall.
                                      If you still have the same issue post screen shots of your setup.

                                      Never Fear, A Geek is Here!

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        techbee
                                        last edited by

                                        @aGeekHere:

                                        firefox ie or chrome?

                                        I use splice-all.  Don't I really need to install certs to client device ?

                                        not when splice all

                                        If https site is successfully blocked, what error page do I expect to see ?

                                        The default cannot access page of the web browser.

                                        If you enabled squid, transparent mode and SSL filtering Splice all with a cert on pfsense and you still cannot access https sites, then something is broken.
                                        Try reinstall.
                                        If you still have the same issue post screen shots of your setup.

                                        I am using chrome. most of the users are chrome.
                                        On the other hand, the firewall rules you shown was to block port 80 and 443 right and redirect dns resolution to pfsense dns server and no more else.  That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right ?

                                        1 Reply Last reply Reply Quote 0
                                        • A
                                          aGeekhere
                                          last edited by

                                          That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right

                                          That is what the transparent proxy is doing, remove that rule, that could be what is causing the issue.

                                          Never Fear, A Geek is Here!

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            techbee
                                            last edited by

                                            @aGeekHere:

                                            That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right

                                            That is what the transparent proxy is doing, remove that rule, that could be what is causing the issue.

                                            Alright

                                            So what I have now on my firewall rules are:
                                            1. antilock rule on port 80 which says (allow all on tcp port 80)
                                            2. block LAN_NET on tcp port 80 and 443
                                            3. redirect rule for dns resolution

                                            would that be ok ?

                                            On the other hand, what is the default cannot access page of the web browser, is it the same like of that of http transparent proxy error page ?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.