• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

$10 BTC Bounty: Squid Proxy - Whitelist Per Source IP

Scheduled Pinned Locked Moved Cache/Proxy
17 Posts 2 Posters 1.6k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • F
    FoolCoconut
    last edited by Oct 3, 2022, 5:43 PM

    Hello,

    I'm trying to lock down my environment by allowing only access to commonly used domains.

    I have enabled both the Transparent HTTP Proxy and HTTPS/SSL Interception, and also deployed a custom CA on all servers.
    I've been gathering info for some time now, as in what URLs my servers are connecting to, so now the idea would be to create individual whitelists, for individual servers.

    eg:
    All dstdomain ACL's should be pointing to a file for their designated host:

    acl host1 src 192.168.1.1
    acl host1_allowed dstdomain "/usr/local/etc/squid/acls/1"
    

    etc...

    As for the ACLs:
    192.168.1.1 - allow only:
    http://deb.debian.org/* # (implying any url within this http domain)
    http://security.debian.org/*
    https://example.org/* # (implying any url within this https domain)

    192.168.1.2 - allow only:
    http://security.debian.org/*

    192.168.1.3 - allow only:
    https://example.org/* # (implying any url within this https domain)

    192.168.1.4 - allow all (splice all, and log entries)

    .4 queries are to be spliced, while the rest src IPs should be bumped.

    Rules:
    Bounty goes out to the first person to provide a full working solution (with the example IPs above) for everyone to see.
    If a solution is not complete, and another one supplements it, the bounty is split between the commenters.

    FAQ:
    Should you trust me? No.
    Is this 2 minute knowledge transfer session worth your time? From my POV, most definitely yes.

    F 1 Reply Last reply Oct 4, 2022, 12:51 PM Reply Quote 0
    • F
      FoolCoconut @FoolCoconut
      last edited by Oct 4, 2022, 12:51 PM

      Update: Increasing the bounty to $20. Also, here's a block of squid.conf that is close to the solution. I'm losing my mind, all the search engine queries have turned purple.

      acl srv1 src 10.1.1.1
      acl srv2 src 10.1.1.2
      acl srv3 src 10.1.1.3
      acl srv4 src 10.1.1.4
      acl hv1 src 10.2.1.1
      acl hv2 src  10.2.1.2
      acl lb1 src 10.3.1.1
      
      # ACLS
      acl debian-repos dstdomain "/usr/local/etc/squid/hq-whitelist/debian-repos"
      http_access allow debian-repos srv1
      http_access allow debian-repos srv2
      http_access allow debian-repos srv3
      http_access allow debian-repos srv4
      # Hypervisors
      acl xcp-repos dstdomain "/usr/local/etc/squid/hq-whitelist/xcp-repos"
      http_access allow xcp-repos hv1
      http_access allow xcp-repos hv2
      # Load Balancers
      acl load-balancers dstdomain "/usr/local/etc/squid/hq-whitelist/load-balancer"
      http_access allow load-balancers lb1
      

      Anyways, I keep getting ERR_ACCESS_DENIED and have added this block in Custom Options (SSL/MITM) - and selected Custom for SSL/MITM Mode.

      F 1 Reply Last reply Oct 4, 2022, 6:34 PM Reply Quote 0
      • F
        FoolCoconut @FoolCoconut
        last edited by Oct 4, 2022, 6:34 PM

        Nope, still didn't solve this, rip to 8 hours yesterday and at least 6 today.

        With the above configuration servers still have access to all domains.

        I've read through the entire squid documentation.

        How freaking hard can it be to have it have a PER HOST WHITELIST???

        K 1 Reply Last reply Oct 4, 2022, 7:41 PM Reply Quote 0
        • K
          KOM @FoolCoconut
          last edited by Oct 4, 2022, 7:41 PM

          @foolcoconut Squidguard does that with ease. Squid is just a cache. if you want URL filtering then squidguard is required.

          F 1 Reply Last reply Oct 4, 2022, 9:39 PM Reply Quote 0
          • F
            FoolCoconut @KOM
            last edited by Oct 4, 2022, 9:39 PM

            No joy, I've been looking into it for the past 2 hours.

            The bounty is still on, if you have an idea how to make this work you win.

            I can't seem to make a whitelist in SquidGuard either, it's either all or nothing, which makes absolutely no sense.

            All I want is:
            server group 1: allow x.com
            server group 2: allow x.com y.com z.com
            server group 3: allow a.com b.com
            AND DENY EVERYTHING ELSE BY DEFAULT FOR EVERYONE FOREVER PLEASE

            F 1 Reply Last reply Oct 5, 2022, 6:02 AM Reply Quote 0
            • F
              FoolCoconut @FoolCoconut
              last edited by Oct 5, 2022, 6:02 AM

              Think I'm going coocoo a bit here.

              If deny is set in Groups ACL - Default access [all] access deny, nothing passes (curl example.org):

              <div id="content">
              <p>The following error was encountered while trying to retrieve the URL: <a href="https://https/*">https://https/*</a></p>
              
              <blockquote id="error">
              <p><b>Unable to forward this request at this time.</b></p>
              </blockquote>
              
              <p>This request could not be forwarded to the origin server or to any parent caches.</p>
              
              <p>Some possible problems are:</p>
              <ul>
              <li id="network-down">An Internet connection needed to access this domains origin servers may be down.</li>
              <li id="no-peer">All configured parent caches may be currently unreachable.</li>
              <li id="permission-denied">The administrator may not allow this cache to make direct connections to origin servers.</li>
              </ul>
              
              <p>Your cache administrator is <hidden></a>.</p>
              
              <br>
              </div>
              
              <hr>
              <div id="footer">
              <p>Generated Wed, 05 Oct 2022 05:52:46 GMT by baseplate (squid)</p>
              <!-- ERR_CANNOT_FORWARD -->
              </div>
              </body></html>
              

              If allow is set in Groups ACL - Default access [all] access allow, literally everything passes.


              My config:

              917d9a6b-8c6f-45e2-b78a-ce727c37637b-image.png


              bd02f464-de05-4d65-a706-2c0b85ce6ca7-image.png


              5a5cbbf4-efbb-4a04-b907-db62be9367ed-image.png


              5777f963-39cd-460c-a2be-020f2432db7e-image.png

              F 1 Reply Last reply Oct 5, 2022, 6:08 AM Reply Quote 0
              • F
                FoolCoconut @FoolCoconut
                last edited by Oct 5, 2022, 6:08 AM

                Squid works properly by itself without squidguard, relevant config:

                3d617158-4236-4764-9972-ddd8455a9582-image.png

                9b337de6-4506-4101-bda6-f81bccf2c817-image.png

                56d0d5e8-95f6-4430-a4a8-e18fb8256381-image.png

                79538a33-06f8-49ed-be7b-c986d1215bff-image.png

                Custom Options (SSL/MITM):

                # some source IPs that should not be MITM-ed
                acl splice_only src 10.2.3.4
                acl splice_only src 10.3.4.5
                acl splice_only src 10.4.5.6
                
                ssl_bump peek step1
                ssl_bump splice splice_only
                ssl_bump bump all
                
                K 1 Reply Last reply Oct 5, 2022, 1:04 PM Reply Quote 0
                • K
                  KOM @FoolCoconut
                  last edited by Oct 5, 2022, 1:04 PM

                  @foolcoconut You create your groups via Squidguard - Group ACLs. For each group, you edit the Target Rules List to determine where they can go. For you, you would only need a simple whitelist and then block all else.

                  F 1 Reply Last reply Oct 5, 2022, 1:07 PM Reply Quote 0
                  • F
                    FoolCoconut @KOM
                    last edited by Oct 5, 2022, 1:07 PM

                    @kom Exactly what I posted in the screenshots above, a single whitelist/allow (neither work) for a target group, and deny everything else.

                    K 1 Reply Last reply Oct 5, 2022, 1:12 PM Reply Quote 0
                    • K
                      KOM @FoolCoconut
                      last edited by Oct 5, 2022, 1:12 PM

                      @foolcoconut When you make changes to squidguard , are you going back to the General Settings tab, clicking Save and then Apply? You have to apply your changes or nothing works and I've seen many people get burned by this.

                      F 1 Reply Last reply Oct 5, 2022, 1:14 PM Reply Quote 0
                      • F
                        FoolCoconut @KOM
                        last edited by Oct 5, 2022, 1:14 PM

                        @kom Of course, that's why I've been doing this for 3 days 8 hours daily. Because every time both squid and squidguard need about 3 mintues to fully restart.

                        K 1 Reply Last reply Oct 5, 2022, 1:54 PM Reply Quote 0
                        • K
                          KOM @FoolCoconut
                          last edited by Oct 5, 2022, 1:54 PM

                          @foolcoconut I have it working here (per-user) but I don't use transparent proxy as I find that to be a real PITA.

                          F 1 Reply Last reply Oct 5, 2022, 1:58 PM Reply Quote 0
                          • F
                            FoolCoconut @KOM
                            last edited by FoolCoconut Oct 5, 2022, 1:59 PM Oct 5, 2022, 1:58 PM

                            @kom Did you configure it to be exclusively per user, or do you know if it's possible for it to have:

                            group1: user1
                            group2: user1 user2 user3
                            group3: user3 user4

                            etc.

                            And as for the transparent proxy, well god damn it if it doesn't work with it, because it's a great feature.

                            K 1 Reply Last reply Oct 5, 2022, 2:01 PM Reply Quote 0
                            • K
                              KOM @FoolCoconut
                              last edited by Oct 5, 2022, 2:01 PM

                              @foolcoconut I have ACLs for individual users as well as for an IP range but there is no overlap. I don't know how squidguard would handle that. It seems to do thing top-down so it might hit on the first match and ignore the rest.

                              If I was you, I would redo it all and start with a simple single-user example with squid in explicit mode. Once you get that working, you can then start adding your layers and note if/where it breaks.

                              F 1 Reply Last reply Oct 6, 2022, 5:28 PM Reply Quote 0
                              • F
                                FoolCoconut @KOM
                                last edited by FoolCoconut Oct 6, 2022, 5:32 PM Oct 6, 2022, 5:28 PM

                                @kom Well, you didn't technically solve my problem, but you did help lead me in the correct path. I'll send you a DM to further discuss the delivery of a small appreciation token, for your time spent.

                                As a solution, I've implemented SG rules per host and disabled the transparent proxy option in squid, apparently that was causing issues. Everything now works as it would make sense, otherwise I would have literally lost my mind troubleshooting and trying the same thing in 15 different ways over and over.

                                K 1 Reply Last reply Oct 6, 2022, 5:54 PM Reply Quote 0
                                • K
                                  KOM @FoolCoconut
                                  last edited by Oct 6, 2022, 5:54 PM

                                  @foolcoconut Transparent mode seems more convenient but I've found it to be a major pain in the ass. I use explicit mode along with WPAD to help clients find the proxy, and DHCP option 252.

                                  Don't forget to add a block rule on LAN for tcp80,443 to force clients to use the proxy. I also created an exemption list and allow any IPs on that list to go direct.

                                  F 1 Reply Last reply Oct 6, 2022, 5:59 PM Reply Quote 0
                                  • F
                                    FoolCoconut @KOM
                                    last edited by FoolCoconut Oct 6, 2022, 5:59 PM Oct 6, 2022, 5:59 PM

                                    @kom My use case was to restrict internet access to internal servers, allowing only permitted URL/IP combos, distro repos, etc. So no need for WPAD (+ I have no idea how it works)
                                    And yes good of you to mention it because I forgot, also blocked all internet access in pfsense rules.

                                    1 Reply Last reply Reply Quote 0
                                    17 out of 17
                                    • First post
                                      17/17
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                      This community forum collects and processes your personal information.
                                      consent.not_received