Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Guide to filtering web content (http and https) with pfsense 2.3

    Scheduled Pinned Locked Moved Documentation
    190 Posts 54 Posters 236.7k Views 15 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V Offline
      vielfede
      last edited by

      @aGeekHere:

      Update

      You can try setting up MITM by setting the SSL/MITM Mode to splice all, that way you do not need to create a certificate for each device on the network. (you still need to create a main certificate though).

      That works fine for me also setting Client proxy settings manually:
      Proxy address/PORT= SQUID_IP 3128

      And that only! Indeed if i specify different proxy settings for http/https in (client win10=> Internet Settings=>Lan settings=>Advanced=>

      • http=SQUID_IP 3128
      • https=SQUID_IP 3129

      It does not work, as in the contrary I'd excpect….  ::)
      Geek can you explain why?

      1 Reply Last reply Reply Quote 0
      • A Offline
        aGeekhere
        last edited by

        Does automatically detect settings work?

        Never Fear, A Geek is Here!

        1 Reply Last reply Reply Quote 0
        • V Offline
          vielfede
          last edited by

          @aGeekHere:

          Does automatically detect settings work?

          Ehmm sorry, could you explain me better what you mean? As I stated I do not use WPAD, I configure proxy manually on client see picture…

          Thanks.
          PS. Sorry to answer late

          ProxyClientSettings.JPG
          ProxyClientSettings.JPG_thumb

          1 Reply Last reply Reply Quote 0
          • A Offline
            aGeekhere
            last edited by

            if you have the transparent proxy working then you do not need to define the proxy server.

            Never Fear, A Geek is Here!

            1 Reply Last reply Reply Quote 0
            • V Offline
              vielfede
              last edited by

              If I use it in transparent mode, https does not work!
              Better sometimes it works, sometimes it does not! (http works always!)

              If anyone managed to get http+https in splice all + transparent mode work please let me know… ;)

              1 Reply Last reply Reply Quote 0
              • A Offline
                aGeekhere
                last edited by

                I just enabled it mitm, added a cert, set it to splice all and its working.

                Never Fear, A Geek is Here!

                1 Reply Last reply Reply Quote 0
                • T Offline
                  techbee
                  last edited by

                  The guide is update but confusing.  Can you please clean up the guide and have it step by step.  Its somewhat hard to follow if you are not familiar with wpad and related firewall rules.

                  Thanks for the guide.

                  1 Reply Last reply Reply Quote 0
                  • A Offline
                    aGeekhere
                    last edited by

                    hi techbee, yeah after discovering a few new things the guide is a little messy now.

                    The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

                    when i get time I will try an clean it up a bit.

                    Never Fear, A Geek is Here!

                    1 Reply Last reply Reply Quote 0
                    • V Offline
                      vielfede
                      last edited by

                      @aGeekHere:

                      The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

                      Indeed I do not think MITM and splice all can work toghether, as stated on squid documentation MITM is associated to the "bump" directive that is something complety different from splice directive.
                      With MITM (bump) squid is able to decrypt traffic (and analyse it) meanwhile with splice all you can do is just "web filtering".
                      Summarizing
                      Directive          Advantages/features                                                                    Disadvantage
                      Splice (all)        No needs of certificate installation on Clients + webfiltering              No traffic analysis i.e no AntiVirus
                      Bump (MITM)    Traffic analysis i.e noYES AntiVirus    + webfiltering                              Needs to install certs on clients

                      Please take a look here and let me know if I missed something. Thanks
                      http://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions
                      http://marek.helion.pl/install/squid.html

                      1 Reply Last reply Reply Quote 1
                      • A Offline
                        AR15USR
                        last edited by

                        @vielfede:

                        @aGeekHere:

                        The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

                        Indeed I do not think MITM and splice all can work toghether, as stated on squid documentation MITM is associated to the "bump" directive that is something complety different from splice directive.
                        With MITM (bump) squid is able to decrypt traffic (and analyse it) meanwhile with splice all you can do is just "web filtering".
                        Summarizing
                        Directive          Advantages/features                                                                    Disadvantage
                        Splice (all)        No needs of certificate installation on Clients + webfiltering              No traffic analysis i.e no AntiVirus
                        Bump (MITM)    Traffic analysis i.e no AntiVirus    + webfiltering                              Needs to install certs on clients

                        Please take a look here and let me know if I missed something. Thanks
                        http://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions
                        http://marek.helion.pl/install/squid.html

                        Shouldn't the second one not have the word "no"..


                        2.6.0-RELEASE

                        1 Reply Last reply Reply Quote 0
                        • V Offline
                          vielfede
                          last edited by

                          @AR15USR:

                          Shouldn't the second one not have the word "no"..

                          Sorry cut & paste mistake, I fixed it.

                          1 Reply Last reply Reply Quote 0
                          • M Offline
                            mahnonsaprei
                            last edited by

                            Hi at all,

                            I'm Marcello from italy. Nice to meet you, i'm a newbie of pfsense.

                            I have read the tutorial, but i have one problem.
                            When i try to connect to an https website in blacklist, the browser show me a generic error see the attachment: (ERR_TUNNEL_CONNECTION_FAILED) while i want to see a message of pfsense that explain the block. In http it's ok

                            I'm looking for in google but i not found the answer.

                            Please help me.
                            Thank you veeeeery much!!!

                            error.PNG
                            error.PNG_thumb

                            1 Reply Last reply Reply Quote 0
                            • A Offline
                              aGeekhere
                              last edited by

                              When i try to connect to an https website in blacklist, the browser show me a generic error see the attachment: (ERR_TUNNEL_CONNECTION_FAILED) while i want to see a message of pfsense that explain the block. In http it's ok

                              Known issue with squid, I do not think there is a fix for it.

                              Never Fear, A Geek is Here!

                              1 Reply Last reply Reply Quote 0
                              • M Offline
                                mahnonsaprei
                                last edited by

                                Thank you so much. There aren't alternatives to redirect https via proxy also blank page or other?

                                B.r.

                                1 Reply Last reply Reply Quote 0
                                • T Offline
                                  techbee
                                  last edited by

                                  hello ageekhere,

                                  can you update your guide now.

                                  1 Reply Last reply Reply Quote 0
                                  • A Offline
                                    aGeekhere
                                    last edited by

                                    what part are you stuck on?

                                    Never Fear, A Geek is Here!

                                    1 Reply Last reply Reply Quote 0
                                    • T Offline
                                      techbee
                                      last edited by

                                      1. I dont know what exact firewall rules to add
                                      2. I am confused if I need to install unofficial wpad package of marcelloc.
                                      3. I am confused if there is a need to have separate webserver of wpad.
                                      4. I am confused if I can enable both transparent and ssl mitm filtering on squid.
                                      5. I cant nslookup wpad on clients command prompt to resolve to my pfsense box ip.

                                      1 Reply Last reply Reply Quote 0
                                      • A Offline
                                        aGeekhere
                                        last edited by

                                        The firewall rules that you need are

                                        
                                        IPv4+6 TCP	*	*	*	80 (HTTP)	*	none	 	port 80 block
                                        IPv4+6 TCP	*	*	*	443 (HTTPS)	*	none	 	port 443 block  
                                        
                                        

                                        This blocks access to port 80 and 443, it stops users from bypassing the proxy.

                                        Also if you want to force pfsense to be the DNS server use.

                                        
                                        In Firewall/NAT/Port forward 
                                        add a new rule
                                        
                                        Interface = LAN 
                                        Protocol = TCP/UDP
                                        Source ports = *
                                        Dest address = *
                                        Dest ports = 53
                                        NAT IP = 127.0.0.1
                                        NAT Ports = 53
                                        Description = Redirect DNS
                                        LAN TCP/UDP * * * 53 127.0.0.1 53 Redirect DNS
                                        Save
                                        
                                        

                                        Have a look through a few of these videos https://www.youtube.com/playlist?list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk
                                        There are a few really good configs he does there.

                                        2. Wait for the package to become stable and available in the package manager, in the mean time you can use the pfsense webserver, put the wpad files in /usr/local/www/ read Part 3. (all that needs to be done is add the wpad files (wpad.da, proxy.pac and wpad.dat) to /usr/local/www/).

                                        3. Just use pfsense webserver /usr/local/www/ (I have not used an external webserver)

                                        4. This is what I did, I set up a WPAD and transparent proxy with ssl mitm (splice all so you do not have to install cert on every device) and set your device to auto configure. I found that just using WPAD a lot of programs that cannot use the proxy were getting blocked so the transparent proxy catches that and something the transparent proxy ssl has issues (like windows updates) however the WPAD is used for that. Best of both worlds.

                                        5 I have had issues as well, if you put http://192.168.1.1/wpad.dat in your web browser and it downloads than it is working.

                                        This guide goes a few steps fever by forcing google and bing to be in safe mode (even youtube).

                                        So what I would do is get WPAD working then enable the transparent proxy with ssl mitm (create a cert for the router).

                                        Hope this helps

                                        Never Fear, A Geek is Here!

                                        1 Reply Last reply Reply Quote 0
                                        • T Offline
                                          techbee
                                          last edited by

                                          thanks for the reply aGeekHere.

                                          What is the tcp port that you used on your webconfigurator ?  If I block port 80, then, I will also be blocked opening the pfsense gui.

                                          Also, regarding to my question number 4, do you mean that you did not enable the http transparent proxy on port 3128 but only the HTTPS/SSL MITM on port 3129 and setup wpad. did I understood you correctly ?

                                          1 Reply Last reply Reply Quote 0
                                          • A Offline
                                            aGeekhere
                                            last edited by

                                            What is the tcp port that you used on your webconfigurator ?

                                            set pfsense Protocol to http (This is a MUST, it will not work if you do not do this)
                                            System: Advanced: Admin Access Protocol http

                                            If I block port 80, then, I will also be blocked opening the pfsense gui.

                                            No it should not (have the rule above the anti lockout rule);

                                            Also, regarding to my question number 4

                                            Enable squid proxy 3128
                                            Get WPAD working follow guide

                                            Enable transparent mode to forward all requests for destination port 80 to the proxy server.

                                            Enable SSL filtering
                                            Splice all
                                            create Certificate (for pfsense).

                                            Summery
                                            Set Advanced: Admin Access Protocol to http
                                            Enable squid proxy
                                            Follow guide on setting up WPAD
                                            Enable transparent mode
                                            Enable SSL filtering Splice all

                                            Never Fear, A Geek is Here!

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.