Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception

JonathanLee

@dkzsys for MITM to work you must install certificates on your devices you run SSL intercept with.

When using both transparent and SSL intercept like you and I are, for transparent to work we must define what urls and or devices require splicing.

Not everything can be SSL intercepted, example being banks, private messages, and most smartphones. Think ethically, what should be spliced? It's powerful. It's unethical to open people's email or text messages so that would be an obvious spliced connection. However, the splice setting still allows you to perform basic URL filtering and blocking. Example, block all the HTTP GET requests for adult websites, or maybe block tictok access, finally you could install timers for YouTube for your kids and have the firewall block YouTube access after 10pm. You just won't inspect everything in those spliced connections or devices. Leading to, even with SSL intercept and your devices running a root certificate authority not all websites allow SSL interception. They could block you too from access.

I like to block sites that use AI tracking. Tracking like this evades privacy and thus it's not legal, by way of the new CCPA and GDPR laws that protect privacy. Most advertising companies ignore the privacy laws right now. They don't care if a user has set their browser to request no tracking. That's when URL blocking comes in.

Another example of devices that you can not do SSL interception with is an iPhone and Amazon fire tablets, you can not have them set to MITM ssl intercept, if you do none of the built in applications work only the browser will. I still install certificates anyway and set those devices as splice only.

For Windows 11 you must install the root certificates on the devices or it will not trust them, however transparent mode works without them as long as wpad or dhcp option 252 lands the device on the Proxy.

Transparent only mode allows basic URL filtering/inspection. Squid even has a YouTube API for age restrictions that works in transparent mode.

SSL intercept allows deep packet inspection and ClamAV works very effectively. It's harder to configure as you have to fine tune it.

Some sites will not allow SSL interception so splice them.

You are required to install certificates or it will show as untrusted for SSL intercept.

This is a security feature as you can only install root certificates on the devices you physically own. So Windows warns you when a certificate is being used that has not been marked trusted. It's warning the end users that traffic is going to a proxy.

Install the certificate and mark it as a trusted root authority to correct the issue you are seeing.

Here is another spliced device example: you bring home your work laptop, you can not legally install a certificate on that device because you don't own it. So, that device would be a splice only setup and use your proxy that way in transparent mode.

This is how most enterprise class firewalls work. They use certificates on the devices that in turn interact with the firewalls.

JonathanLee

@dkzsys if your not using a server for forwarding change this setting to transparent.

I had issues with this setting in the past.

dkzsys

@JonathanLee said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys for MITM to work you must install certificates on your devices you run SSL intercept with.
......
Here is another example, you bring home your work laptop, you can not legally install a certificate on that device so, that device would be a splice only setup.

Great context, Jonathan - adding the dimensions of ethic, commercial implementation and practicality.

My challenge with the SSL intercept is the continuous extension on the splicing as we start using new apps or web services. Ideally, I'd like to have it as set-and-forget and expect it to work most of the time, like 95%, and I am ok with the occasional manual effort. Just like what seems to be working for @michmoor. I am really curious on what the differences are in terms of implementation between his config and ours - the example for the TV charting data is quite revealing.

I'm also going to check out the squidGuard features today; to see how to best utilise it for (ad) blocking and access control. Do you have any recommendation on:

list itemany blacklist,
how best to implement the self-defined block list? (let's say roblox service. I had a quick play with Target Categories - toggling the redirect seems to take quite a long time; either I ran out of patient or it was ineffective. Tried different browsers. Ended up disabling squidGuard.)

JonathanLee

@dkzsys I use Squidguard, it takes time to configure.

But it does allow a URL blacklist that is pretty effective.

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

I love this blacklist as it's reduced for my 2100

I made my Roblox list with help of their website for proxy/school district usage setup also approves my son's Roblox

I agree Mike has his set such that no certificates are required quicker and still effective. Again, I need the SSL intercept for viruses and major issues with my old job. I had a target on my back for years so bad that I couldn't even watch Netflix I would get hit with pings of death non stop and many other issues. I still find residual invasive actors and containers to this day even on my personal devices.

michmoor

@JonathanLee why set the XFF ito transparent?

michmoor

@JonathanLee said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys I use Squidguard, it takes time to configure.

But it does allow a URL blacklist that is pretty effective.

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

I love this blacklist as it's reduced for my 2100

I made my Roblox list with help of their website for proxy/school district usage setup also approves my son's Roblox

I agree Mike has his set such that no certificates are required quicker and still effective. Again, I need the SSL intercept for viruses and major issues with my old job. I had a target on my back for years so bad that I couldn't even watch Netflix I would get hit with pings of death non stop and many other issues. I still find residual invasive actors and containers to this day even on my personal devices

This seems like a very interesting story I must hear one day.

JonathanLee

@michmoor I had a container VM inside .trashes and the v100 spotlight folders that was all the sudden 1tb in size and that was running under within a system integrity environment. I got it removed and recovered my resource in the end, but come on, hide a VM in the .trashes folder. That was just last week. I kept seeing weird snort blocks and I started investigating it, I was very surprised.

JonathanLee

@michmoor the forum all the sudden was seeing my private LAN address getting passed a couple months ago with it set to on.

dkzsys

@JonathanLee said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys if your not using a server for forwarding change this setting to transparent.

I had issues with this setting in the past.

Good one - will try to update the setting.

But it does allow a URL blacklist that is pretty effective.

http://dsi.ut-capitole.fr/blacklists/download/blacklists_for_pfsense_reducted.tar.gz

I love this blacklist as it's reduced for my 2100
I made my Roblox list with help of their website for proxy/school district usage setup also approves my son's Roblox

Thanks - will try them out.

@ michmoor I had a container VM inside .trashes and the v100 spotlight folders that was all the sudden 1tb in size and that was running under within a system integrity environment. I got it removed and recovered my resource in the end, but come on, hide a VM in the .trashes folder. That was just last week. I kept seeing weird snort blocks and I started investigating it, I was very surprised.

Never short of creativity in this world!

periko

@dkzsys I see u have add a lot of custom settings, squid will filter +/- 90% of the traffic.
Now, this apple things maybe is inside that 10% that will reject to work behind a proxy.

What version of pfsense u have?

My recommendation:

Resolve DNS IPv4 First doesn't work, is a obsolete is u are running pfsense 2.6+.
Your alias in the bypass finish them with ';'
Try splice all.
Remove all that custom stuff.
Select all in the "Remote Cert Checks and Certificate Adapt".
Enable WPAD in your interface where u want to run squid.

I don't have apple stuff that I can test, I will check evernote.

Now, what is the problem with 'Evernote'?, I can try that app.

Regards!!!

michmoor

@periko WPAD is for whn you are running an explict proxy right? Otherwise in a Transparent set up what would be the requirement?

dkzsys

@periko said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys I see u have add a lot of custom settings, squid will filter +/- 90% of the traffic.
Now, this apple things maybe is inside that 10% that will reject to work behind a proxy.

What version of pfsense u have?

My recommendation:

Resolve DNS IPv4 First doesn't work, is a obsolete is u are running pfsense 2.6+.
Your alias in the bypass finish them with ';'
Try splice all.
Remove all that custom stuff.
Select all in the "Remote Cert Checks and Certificate Adapt".
Enable WPAD in your interface where u want to run squid.

I don't have apple stuff that I can test, I will check evernote.

Now, what is the problem with 'Evernote'?, I can try that app.

Regards!!!

Thanks for chipping in, Pedro @periko

I am running pfSense Plus 23.05.1-RELEASE (amd64) (built on Wed Jun 28 03:57:27 UTC 2023 FreeBSD 14.0-CURRENT).

Will try out your recommendations.

Re a couple of your points:

~~"Try splice all." - do you mean, in addition to source ip address bypass, splice them all in squid config as well?~~ Ignore this, figured out that you were referring to "SSL/MITM Mode" setting.
"Remove all that custom stuff." They were automatically added after installing suiqdGuard I think. I'll remove the config and uninstall squidGuard for now.

Re Evernote, I had to whitelist "www.evernote.com", otherwise, the app is having sync issue with the server. On macOS app, it's a grey pop-up error box on the note page when I tried to update the note. For a failure scenario testing, it's easier to use this URL "https://www.tradingview.com/markets", as it worked on Mike's, and failed on mine and Jonathan's.

periko

@michmoor It wont' affect at the end the client will need to reach the proxy.

The only difference is that transparent pfsense will forward the traffic to the proxy in no none we need to allow the client reach the proxy, resume both options the client will need to reach the proxy.

Regards!!!

periko

@dkzsys I need to get a apple to test, just curiosity, in a windows box evernote do the same thing?

dkzsys

@periko said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys I need to get a apple to test, just curiosity, in a windows box evernote do the same thing?

Let me try it on a windows VM and let you know. If you are interested in the broken SSL session, you can use "https://www.tradingview.com/markets" to test.

Enable WPAD in your interface where u want to run squid.

A question for your point above, do you mean enabling WPAD on the interface setting on pefsense (if so, do you mind letting me how to do it?), or enabling WPAD or PAD on the client machine (either os system setting, or browser setting)?

dkzsys

@periko said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys I see u have add a lot of custom settings, squid will filter +/- 90% of the traffic.
Now, this apple things maybe is inside that 10% that will reject to work behind a proxy.

What version of pfsense u have?

My recommendation:

Resolve DNS IPv4 First doesn't work, is a obsolete is u are running pfsense 2.6+.
Your alias in the bypass finish them with ';'
Try splice all.
Remove all that custom stuff.
Select all in the "Remote Cert Checks and Certificate Adapt".
Enable WPAD in your interface where u want to run squid.

I don't have apple stuff that I can test, I will check evernote.

Now, what is the problem with 'Evernote'?, I can try that app.

Regards!!!

I have tried a few different config combinations, and the "Splice All" does most of the trick!

Some observations with "Splice All":

"Remote Cert Checks" with only "Do not verify remote certificate"; and "Certificate Adapt" - none
microsoft domains are mainly 409; but apple icloud gateway is getting 500.

1695347891.621      0 10.0.1.11 NONE_NONE/409 4012 CONNECT outlook.office365.com:443 - HIER_NONE/- text/html
1695347891.621      0 10.0.1.11 NONE_NONE/000 0 - error:transaction-end-before-headers - HIER_NONE/- -

1695347967.351     29 10.0.1.11 TCP_TUNNEL/500 7 CONNECT gateway.icloud.com:443 - ORIGINAL_DST/17.248.219.1 -

After enabling all in "Remote Cert Checks" and "Certificate Adapt"
Both microsoft domains and icloud gateway are getting 409.

1695348201.835      0 10.0.1.11 NONE_NONE/409 4003 CONNECT gateway.icloud.com:443 - HIER_NONE/- text/html
1695348201.835      0 10.0.1.11 NONE_NONE/000 0 - error:transaction-end-before-headers - HIER_NONE/- -

It would be easier for me to fix up all the 409s after leaving it running for a bit.

I've checked squid access log and it has sufficient info for my audit purpose.

Thank you for helping out, @periko @michmoor @JonathanLee! It fixed most of my issues for my particular use case with this less intrusive implementation. For Jonathan's use case of content filtering it would still need that extra work for whitelisting.

P.S. @periko no more evernote sync issue either after Splice All!

JonathanLee

@dkzsys
Check this out for WPAD

https://docs.netgate.com/pfsense/en/latest/recipes/http-client-proxy-wpad.html

It's a really good how to overview of WPAD. I have mine set up this way with an addition of DHCP options to automatically send the proxy and DNS information when IP addresses are distributed.

DHCP option 252 and 42

"A WPAD host may be supplied via DHCP numbered option 252 (string value containing the entire URL to the WPAD file) or DNS, which is easy to do with the built-in DNS forwarder" (Netgate docs).

Normally you would use standard ports again my GUI uses port 8080 and it works good for the XBOX One

dkzsys

@JonathanLee said in Squid Proxy "Bypass Proxy for These Destination IPs" Not Working?? Transparent HTTP Proxy Mode + HTTPS/SSL Interception:

@dkzsys
Check this out for WPAD

https://docs.netgate.com/pfsense/en/latest/recipes/http-client-proxy-wpad.html

It's a really good how to overview of WPAD. I have mine set up this way with an addition of DHCP options to automatically send the proxy and DNS information when IP addresses are distributed.

Thanks for the illustration as always @JonathanLee ! Yes - I went through the same page as well:)

For me, the next is to evaluate switching back to AdGuard Home for DNS on RaspberryPi, or use SquidGuard with pfSense DNS. Since I've blocked DNS (53, DoH, DoT) to WAN, and NATing locally, AdGuard would have a similar effect of SquidGuard in terms of service/domain blocking. (and neither approach will prevent kids bypassing with VPN). And at this stage, I don't need time-of-day control. So my criteria would be relatively simple:

#1: no impact to Squid transparent proxy implementation (re earlier discussion in this thread)
Ease of adding/removing domains
Ease and effectiveness of rule toggles.

JonathanLee

@michmoor WPAD auto configures devices to use the proxy. It's for laptops that move from the office and back to home. It's so you an admin never need to set the poxy settings. When a known device jumps on your network the device already knows to use the proxy, versus having to configure it on the device each and every time you change from a lan at the office to your lan at home. It's explicitly for proxy use. It just does it automagically.

dkzsys

After a few days of testing with Transparent proxy + SSL Interception (Splice All), a couple of observations:

For the whitelisted source IP by FQDN (e.g. microsoft, Apple icloud etc), the result is intermittent - sometimes 200 and others 409... I have updated the "Aliases Hostnames Resolve Interval" to 60 sec. but still not resolved. Some input for resolution will be appreciated.
FW Blocking to WAN is no longer 100% effective, to the FQDNs on and off the source bypass list. For example, blocked clients can still search on google.com, play chess on chess.com, etc. See screenshot for the fw block rule below. I will have to reset fw states to block all outgoing traffic to WAN from those clients. @periko - can I get your input on this pls - any other options to effectively block client to WAN without fw state reset?