$10 BTC Bounty: Squid Proxy - Whitelist Per Source IP

FoolCoconut

Hello,

I'm trying to lock down my environment by allowing only access to commonly used domains.

I have enabled both the Transparent HTTP Proxy and HTTPS/SSL Interception, and also deployed a custom CA on all servers.
I've been gathering info for some time now, as in what URLs my servers are connecting to, so now the idea would be to create individual whitelists, for individual servers.

eg:
All dstdomain ACL's should be pointing to a file for their designated host:

acl host1 src 192.168.1.1
acl host1_allowed dstdomain "/usr/local/etc/squid/acls/1"

etc...

As for the ACLs:
192.168.1.1 - allow only:
http://deb.debian.org/* # (implying any url within this http domain)
http://security.debian.org/*
https://example.org/* # (implying any url within this https domain)

192.168.1.2 - allow only:
http://security.debian.org/*

192.168.1.3 - allow only:
https://example.org/* # (implying any url within this https domain)

192.168.1.4 - allow all (splice all, and log entries)

.4 queries are to be spliced, while the rest src IPs should be bumped.

Rules:
Bounty goes out to the first person to provide a full working solution (with the example IPs above) for everyone to see.
If a solution is not complete, and another one supplements it, the bounty is split between the commenters.

FAQ:
Should you trust me? No.
Is this 2 minute knowledge transfer session worth your time? From my POV, most definitely yes.

FoolCoconut

Update: Increasing the bounty to $20. Also, here's a block of squid.conf that is close to the solution. I'm losing my mind, all the search engine queries have turned purple.

acl srv1 src 10.1.1.1
acl srv2 src 10.1.1.2
acl srv3 src 10.1.1.3
acl srv4 src 10.1.1.4
acl hv1 src 10.2.1.1
acl hv2 src  10.2.1.2
acl lb1 src 10.3.1.1

# ACLS
acl debian-repos dstdomain "/usr/local/etc/squid/hq-whitelist/debian-repos"
http_access allow debian-repos srv1
http_access allow debian-repos srv2
http_access allow debian-repos srv3
http_access allow debian-repos srv4
# Hypervisors
acl xcp-repos dstdomain "/usr/local/etc/squid/hq-whitelist/xcp-repos"
http_access allow xcp-repos hv1
http_access allow xcp-repos hv2
# Load Balancers
acl load-balancers dstdomain "/usr/local/etc/squid/hq-whitelist/load-balancer"
http_access allow load-balancers lb1

Anyways, I keep getting ERR_ACCESS_DENIED and have added this block in Custom Options (SSL/MITM) - and selected Custom for SSL/MITM Mode.

FoolCoconut

Nope, still didn't solve this, rip to 8 hours yesterday and at least 6 today.

With the above configuration servers still have access to all domains.

I've read through the entire squid documentation.

How freaking hard can it be to have it have a PER HOST WHITELIST???

KOM

@foolcoconut Squidguard does that with ease. Squid is just a cache. if you want URL filtering then squidguard is required.

FoolCoconut

No joy, I've been looking into it for the past 2 hours.

The bounty is still on, if you have an idea how to make this work you win.

I can't seem to make a whitelist in SquidGuard either, it's either all or nothing, which makes absolutely no sense.

All I want is:
server group 1: allow x.com
server group 2: allow x.com y.com z.com
server group 3: allow a.com b.com
AND DENY EVERYTHING ELSE BY DEFAULT FOR EVERYONE FOREVER PLEASE

FoolCoconut

Think I'm going coocoo a bit here.

If deny is set in Groups ACL - Default access [all] access deny, nothing passes (curl example.org):

<div id="content">
<p>The following error was encountered while trying to retrieve the URL: <a href="https://https/*">https://https/*</a></p>

<blockquote id="error">
<p><b>Unable to forward this request at this time.</b></p>
</blockquote>

<p>This request could not be forwarded to the origin server or to any parent caches.</p>

<p>Some possible problems are:</p>
<ul>
<li id="network-down">An Internet connection needed to access this domains origin servers may be down.</li>
<li id="no-peer">All configured parent caches may be currently unreachable.</li>
<li id="permission-denied">The administrator may not allow this cache to make direct connections to origin servers.</li>
</ul>

<p>Your cache administrator is <hidden></a>.</p>

<br>
</div>

<hr>
<div id="footer">
<p>Generated Wed, 05 Oct 2022 05:52:46 GMT by baseplate (squid)</p>
<!-- ERR_CANNOT_FORWARD -->
</div>
</body></html>

If allow is set in Groups ACL - Default access [all] access allow, literally everything passes.

My config:

FoolCoconut

Squid works properly by itself without squidguard, relevant config:

Custom Options (SSL/MITM):

# some source IPs that should not be MITM-ed
acl splice_only src 10.2.3.4
acl splice_only src 10.3.4.5
acl splice_only src 10.4.5.6

ssl_bump peek step1
ssl_bump splice splice_only
ssl_bump bump all

KOM

@foolcoconut You create your groups via Squidguard - Group ACLs. For each group, you edit the Target Rules List to determine where they can go. For you, you would only need a simple whitelist and then block all else.

FoolCoconut

@kom Exactly what I posted in the screenshots above, a single whitelist/allow (neither work) for a target group, and deny everything else.

KOM

@foolcoconut When you make changes to squidguard , are you going back to the General Settings tab, clicking Save and then Apply? You have to apply your changes or nothing works and I've seen many people get burned by this.

FoolCoconut

@kom Of course, that's why I've been doing this for 3 days 8 hours daily. Because every time both squid and squidguard need about 3 mintues to fully restart.

KOM

@foolcoconut I have it working here (per-user) but I don't use transparent proxy as I find that to be a real PITA.

FoolCoconut

@kom Did you configure it to be exclusively per user, or do you know if it's possible for it to have:

group1: user1
group2: user1 user2 user3
group3: user3 user4

etc.

And as for the transparent proxy, well god damn it if it doesn't work with it, because it's a great feature.

KOM

@foolcoconut I have ACLs for individual users as well as for an IP range but there is no overlap. I don't know how squidguard would handle that. It seems to do thing top-down so it might hit on the first match and ignore the rest.

If I was you, I would redo it all and start with a simple single-user example with squid in explicit mode. Once you get that working, you can then start adding your layers and note if/where it breaks.

FoolCoconut

@kom Well, you didn't technically solve my problem, but you did help lead me in the correct path. I'll send you a DM to further discuss the delivery of a small appreciation token, for your time spent.

As a solution, I've implemented SG rules per host and disabled the transparent proxy option in squid, apparently that was causing issues. Everything now works as it would make sense, otherwise I would have literally lost my mind troubleshooting and trying the same thing in 15 different ways over and over.

KOM

@foolcoconut Transparent mode seems more convenient but I've found it to be a major pain in the ass. I use explicit mode along with WPAD to help clients find the proxy, and DHCP option 252.

Don't forget to add a block rule on LAN for tcp80,443 to force clients to use the proxy. I also created an exemption list and allow any IPs on that list to go direct.

FoolCoconut

@kom My use case was to restrict internet access to internal servers, allowing only permitted URL/IP combos, distro repos, etc. So no need for WPAD (+ I have no idea how it works)
And yes good of you to mention it because I forgot, also blocked all internet access in pfsense rules.