Transparent Squid proxy for https without SSL Interception

slu

I am not sure this is a good setup, but find a way to have a transparent squid proxy for https without SSL interception:

1. Enable "HTTPS/SSL Interception Enable SSL filtering."
2. Add in Squid -> Advanced features -> Integrations -> ssl_bump none all

This allows you to use a transparent proxy without config on the client side.
Is there any reason why this setup is not per default available?

KOM

I'm not sure that would even work.

https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid

slu

It does, but it is more work to configure (webserver,…).

chidgear

I tried it, the "ssl_bump none all" command invalidates the configuration. The result, It works like the "HTTPS/SSL filtering" checkbox was never checked.
Too good to be true :c

Thanks anyway!

slu

You can't filter but the transparent proxy works without additional configuration.

chidgear

Right, I cannot filter HTTPS, just normal/plain HTTP. :(

Maybe I'm short of vission but, if I cannot filter HTTPS then, why do I need an https proxy?

chris4916

There is something quite often not well understood:
transparent proxy can NOT filter HTTPS content unless one intercepts (and breaks) HTTPS tunnel implementing SSL Bump (AKA Man In The Middle).
Explicit proxy (therefore not transparent ;D) can not filter HTTPS content neither (for the same reasons) but can apply rules bases on the left part of URL (domain) because HTTP CONNECT sends it out of SSL tunnel (meaning clear text).

Thus if you do need to filter content, scan for virus or whatever else related to content, SSL Bump is the only way.
If you only need to prevent access to some domains, SSL Bump is not required.

In some organizations, HTTP proxy is not even used for filtering but to keep log of HTTP(S) activity, often coupled with authentication.

chidgear

Let me see if I get it. In my case, if I need to filter some content (based on the url, not in the actual content of the websites) so the navigation in sites like facebook, youtube, and another ones (Pr0n) is restricted, and the websites uses https for soome reason, it cannot be done with a transparent proxy, because it needs to "break" the package with ssl bump. The explicit proxy cannot see the content without ssl bump either, but it can see the url, (which the transparent proxy doesn't) so I can filter using rules applied to the URL that way.

I got it right?

I was hoping to use WPAD applied to HTTPS traffic, and transparent proxy for HTTP traffic, but right now I got nothing to do the tests.

chris4916

@chidgear:

I got it right?

yes 8)

I was hoping to use WPAD applied to HTTPS traffic, and transparent proxy for HTTP traffic, but right now I got nothing to do the tests.

although one can configure Squid to support both explicit and transparent proxy mode (I mean at same time) implementing something that would forward to proxy for HTTPS and go DIRECT for HTTP is somewhat strange, plus I don't understand the added value. Once you have proxy configured at browser level, why would you want to avoid proxy (explicitly thanks to proxy.pac) in order to use it at the end, but transparently?

chidgear

I heard from someone that it would be a good idea to use a transparent proxy to filter HTTP and a explicit proxy to filter https urls. but, since there are a lot of computers on my working enviroment, then I want to avoid going one by one doing the configurations, then I though that usin WPAD for HTTPS will be nice but, now than I think about it… maybe its weird to configure WPAD for https, but not using it for http...
anyway, thanks a lot :D

chris4916

@chidgear:

I heard from someone that it would be a good idea to use a transparent proxy to filter HTTP and a explicit proxy to filter https urls. but, since there are a lot of computers on my working enviroment, then I want to avoid going one by one doing the configurations, then I though that usin WPAD for HTTPS will be nice but, now than I think about it… maybe its weird to configure WPAD for https, but not using it for http...
anyway, thanks a lot :D

If you are using WPAD, I don't really understand why you would want to still use transparent proxy but…. you can still do it 8)
if, in proxy.pac, you redirect HTTPS flow to your proxy (explicit mode) and HTTP flow to "DIRECT", then it will be intercepted at gateway level and (transparently) redirected ti you proxy.

Cool isn't it? ;D ;D

Still I don't understand what the purpose is (or would be)

Luckas

@chris4916:

Explicit proxy (therefore not transparent ;D) can not filter HTTPS content neither (for the same reasons) but can apply rules bases on the left part of URL (domain) because HTTP CONNECT sends it out of SSL tunnel (meaning clear text).

Hello! Please, tell me, how can i make rules based on the left part of url?

aGeekhere

Still I don't understand what the purpose is (or would be)

When using a wpad or explicit mode some programs do not have proxy setting and want to use port 80. If you have port 80 blocked to stop users from bypassing the proxy then that program will have connection issues, you then need to find that address and allow a pass rule.

Now if you could have both then you will be able to filter https content without mitm and redirect traffic getting block from port 80 to the proxy port. Going to research this a bit more.

Update
Works fine thanks chris4916 for the tip of running both.

chris4916

@Luckas:

@chris4916:

Explicit proxy (therefore not transparent ;D) can not filter HTTPS content neither (for the same reasons) but can apply rules bases on the left part of URL (domain) because HTTP CONNECT sends it out of SSL tunnel (meaning clear text).

Hello! Please, tell me, how can i make rules based on the left part of url?

This is as simple as applying rules for "domain". Domain filtering looks only at the left side of URLs that is used during CONNECT.
This allows to write ACL but you can also, e.g., use Squidguard

chris4916

@aGeekHere:

Update
Works fine thanks chris4916 for the tip of running both.

You're welcome ;D

Sure it works (I try not to write too stupid stuff :-[) but, at least to me, added value is only for the very few devices that would not support WPAD because once you have WPAD configured, almost all devices will go through explicit proxy smoothly.
With transparent proxy, there is also no capability to prompt for authentication then apply any kind of profiling neither efficient log :(

erwintwr

Just feel i want to add to this topic for future reference. After playing around with mutiple options, and having issues wiht wPAD on lots of android devices (did i say i HATE touch screens :/), my network is now stable on the following setup:

squid configured to be explicit (not transparent)
Normal users gets IP from DHCP which contains WPAD details (this is for desktop computers and laptops) connected via LAN

WIFI, i configured 3 different SSID's.
Normal users SSID on vlan 0 (which will thus receive the same DHCP and WPAD settings - This is for Laptops that connect via WIFI
Phone users SSID on VLAN 3 (which have a separate DHCP server / subnet but no WPAD and all ports open) -> this is not passed through a proxy, but just rate limited via captive portal. (MAC address is captured on first connection and remembered)
Guest users SSID on VLAN4. Also a separate DHCP, and access is controllled via Captive portal and tickets that expire

The phone SSID password i keep classified, thus control who connects to iet, and i can monitor the guest SSID for abuse (each VLAN interface is seperate on pFsense, thus activity can be monitored.

Squidguard is active and working, and i am looking at activating PFBLOCKER with DNSBL as well. Network is quite secure i think for our purposes (yes i know some advanced users will download and torrent via their phones, but for now it seems like they are happy).
Squidguard is configured to open all blocked sites(social sites/ youtube etc) during lunch hours and outside working hours.

Hope this helps someone for future designs

mrpush

erwintwr,

Ok so I have squid in place and am using the HTTPS man in the middle filtering and it works fine for PC's on the LAN.

My problem however is that i have a WIFI router hooked up to the network, and any WIFI connections to that router are also filtered but I cant get certificates of phones and it blocks all HTTPS traffic.

How do I get mobile devices to be able to BYPASS all Squid filtering whether it be HTTP or HTTPS?????

I have added the mobile device IPs to the TRANSPARENT proxy setting *(HTTP) to "BYPASS PROXY FOR THESE SOURCE IP's" but I can't do that for the HTTPS man-in the-middle filtering!

Any ideas how to bypass the BOTH of these for mobile devices like iphones?

I don't think an Iphone is going to take a .CRT file (not that I can tell) for a certificate based pass-through.

Can I somehow setup my WIFI router (and it connected devices) to automatically pass all HTTP OR HTTPS through Squid maybe?

Any ideas?

Thanks,

MP

mrpush

Hi,

Ok, I did get them working by passing them "THROUGH" the proxy by manual changing the WIFI Proxy settings on the Iphones.

However, it would still be nice to know how to BYPASS HTTPS traffic (Man in the middle) for these mobile devices. Is it possible in the Squid interface?

Thanks,

MP

tpklge

You say that ssl filtering works normally, but I have problems with sites like google, which although they are not blocked continue without access because of the hsts protocol that google, facebook and other sites use. I have tried to install the certificate generated by pfsense on the machines stations, without success, How did you solve this problem?

zacha

hsts does not hinder you bumping tls traffic, it just forces the client to use tls instead of plain text. you have to have your ca in place on your client devices. I would recommend

1. setting up a ca in pfsense (you don't necessarily have to have the private key on the pfsense box and I recommend againt it, it is you last resort if you private keys of you sub cas are leaked at some point)
2. setting up a sub ca for ssl bumping
3. exporting the ca certificate of the top ca (just the cert)
4. selecting the right ca in the squid config
5. configure bumping as i describe over here https://forum.pfsense.org/index.php?topic=135178.0
6. put on the whitelist what you desire
7. install ca on the client. that should generally be done by your endpoint management solution (active directory gpo, kaspersky endpoint security, you name it). if you want to manually install the ca make sure you put it into the /SYSTEM'S/ Trusted Root Certifaction Authorities else it won't work.
8. here you go (push f12 in your browser to verify your certs are being generated by your bumping ca.