Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Error with some website

    Scheduled Pinned Locked Moved General pfSense Questions
    16 Posts 3 Posters 908 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Well it seems very likely parts of the site are being filtered. Amazon have a lot of ads on their pages!

      I assume you have Squidguard installed with ad lists loaded? If you disable filtering does the page load correctly?

      What is logged as blocked when you visit that page?

      R 1 Reply Last reply Reply Quote 0
      • R
        rjcab @stephenw10
        last edited by

        @stephenw10 you mean pfBlockerNG ? Not I don't have.
        Only these packages installed:

        abee16b0-d6ee-4a31-b640-47989df431ea-image.png

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          No I meant Squidguard.

          Are you using blacklists directly in Squid to filter URLs then?

          R 1 Reply Last reply Reply Quote 0
          • R
            rjcab @stephenw10
            last edited by

            @stephenw10
            In the proxy menu I haven't changed anything.

            1b283106-987c-42f6-8f7d-c604fdbaf657-image.png

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator @rjcab
              last edited by

              @rjcab said in Error with some website:

              I suppose proxy filters ads

              Where are you doing that filtering then?

              R 1 Reply Last reply Reply Quote 0
              • R
                rjcab @stephenw10
                last edited by

                @stephenw10

                Well pretty good question. I am newbee on Pfsense

                I assume that in disabling squid it works so the config should be below on one of these tab:

                f2664fc3-ecdd-4154-a2f1-bae2a11737df-image.png

                But I don't see, I have just configured the proxy to catch all visited websites.

                GertjanG 1 Reply Last reply Reply Quote 0
                • GertjanG
                  Gertjan @rjcab
                  last edited by

                  @rjcab said in Error with some website:

                  the proxy to catch all visited websites

                  be aware : there are many 'sites' that can't be 'proxied'. And this list grows every day.
                  To make things worse, it's easy for a web site administrator to forbid his web site being proxied. Use a HSTS flagged certificat, and a web browser can't use a proxy anymore to get that site. There will be a certificat failure.
                  Or the web site, like your amazonaws example, detects that a proxy is used, and they do not allow that : they inform the client, using a very cryptic message, that visiting "amazonaws" only works if the MITM is disabled.

                  Quiet understandably, as bank web sites, medical data web site, and actually any web site, and any visiting client (that is you) doesn't want a MITM as that opens the door to all kind of issues.

                  So, when you decide to use a proxy, you have to baby-sit it every day and collect the web sites that "don't work when handled by the proxy" and add them to the "don't proxy this web site" list. This list will eventually grow to the "all the sites avaible on the internet" and that will be the day MITM has been dealt with.

                  No "help me" PM's please. Use the forum, the community will thank you.
                  Edit : and where are the logs ??

                  R 1 Reply Last reply Reply Quote 0
                  • R
                    rjcab @Gertjan
                    last edited by

                    @Gertjan Thanks, I now understand the approach.
                    I have done this:

                    100e92f2-ecff-4e9a-be00-435c7590f218-image.png

                    It seems that is doen't work, I will try with other websites.

                    GertjanG 1 Reply Last reply Reply Quote 0
                    • GertjanG
                      Gertjan @rjcab
                      last edited by

                      @rjcab said in Error with some website:

                      It seems that is doen't work

                      The "whitelist" accepts URLs like that ? Or host names ?

                      "amazon.fr" probably uses and redirects to something other host name(s). And of course, these other host name(s) can be different tomorrow.

                      To see what happens : (I used Firefox) :
                      Open amazon.fr
                      Goto Menu > More tools > Web Developers Tools
                      Select Network (between Memory and Storage) and hit Crtl-F5

                      You saw the list with host names flying by ? You probably have to add them all.

                      See it like this : 'they', amazon, but also Google, Apple, Microsoft, and actually all the big players, all hired the best network engineers just to make your live, doing MITM, hard.
                      Outsmarting them .... are you sure you want to go down that path ?

                      No "help me" PM's please. Use the forum, the community will thank you.
                      Edit : and where are the logs ??

                      R 1 Reply Last reply Reply Quote 0
                      • R
                        rjcab @Gertjan
                        last edited by

                        @Gertjan many thanks, I will try your advise

                        R 1 Reply Last reply Reply Quote 0
                        • R
                          rjcab @rjcab
                          last edited by

                          Hello,

                          I made some tests and add URL as below

                          2557ac6d-5eaa-4d6b-ae7e-e62a466f777a-image.png

                          When I tried to access with a mobil device it doesn't work as maybe the URL or content is different. I will continue to investigate

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            The whitelist should be domains not URLs. So:

                            amazon.fr
                            leboncoin.fr
                            netflix.com
                            
                            R 1 Reply Last reply Reply Quote 0
                            • R
                              rjcab @stephenw10
                              last edited by

                              @stephenw10 thank you. I tried but still the issue. I think there are more URL with netflix on mobile device to whitelist

                              Also I have tested to connect to my company VPN. It works but once connected I don't have access to Gmail whereas if I disable squid it works. Maybe I have to whitelist the network adress of my company which provide internet services ?

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.