Squid + https

[techtester-m]

@DaddyGo Ok...
Few more if you may :)

As I understand it so far it goes something like this (and please correct me where I'm wrong) -

(1) Client --> Squid interception of PK, aka MITM --> destination.

(2) It will do the above only for what is in its whitelist.

(3) A private self created CA has to be installed on every client that needs to use the Squid proxy, so it could dynamically generate certificates. Using Let's Encrypt or anything of the sort won't work here and also be a big no no.

(4) Everything in the whitelist will be cached (up to X G/MB limit), including files.

(5) A destination that is not on the whitelist will be blocked? If so then I think I prefer DNS blocking.

(6) Is their an option to cache/intercept only certain destinations (whitelist or something) and treat all the others as usual without "squidding" them?

[DaddyGo]

@techtester-m

1. Yes
2. no, these are exceptions as they require special rules and accesses
   (otherwise they do not work)
   note the whole world cannot be whitelisted
3. Yes, with this method... ("Using Let's Encrypt or anything of the sort won't work here and also be a big no no.") yes

4. half true, (including files - it would require awful storage capacity - depending on what file we are talking about)
5. No, the whitelist is necessary, because of the above, I also recommend DNS blocking in SOHO environment...
   (Squid is a big boys game, because of big systems and proxy capabilities)
   6.Yes by bypassing the proxy (ACL, PAC,etc)
   https://wiki.squid-cache.org/ConfigExamples/Authenticate/Bypass

[techtester-m]

@DaddyGo said in Squid + https:

no, these are exceptions as they require special rules and accesses
(otherwise they do not work)

So what do you do with exceptions? Let them bypass squid and go as usual or "fine tuning" them?

@DaddyGo said in Squid + https:

half true, (including files - it would require awful storage capacity - depending on what file we are talking about)

For the sake of learning let's say there's enough storage, even though an organized NAS would be MUCH better for any business/company. Does squid has the option to set a maximum X G/MB for caching? Sort of a circular caching where a new file would push and old one(s) when that maximum X is met.

@DaddyGo said in Squid + https:

No, the whitelist is necessary, because of the above

Perhaps it's only semantics here but the whitelist in this case is what you want Squid to authenticate for you (MITM) and cache and has nothing to do with restrictions, blocking etc.?

Ok...Semantics again. Squid does both access restrictions AND caching?

[Gertjan]

There are still some sites left on the Internet that accept http (NON SSL) requests.
Install Squid, and focus on the classic 'intercept port 80' or 'use a wpad file' and have your browser been 'redirected to a 8080 proxy port'.
This way, you can see how it works in the old days- I guess, for that type of set-up, many examples are available.

When you feel up to it, include 'https' in the addition.

Take note : it's an on-going battle. I wonder how one proxies the sites that have HSTS ( https://fr.wikipedia.org/wiki/HTTP_Strict_Transport_Security - it's yet another anti MITM ) activated. I use HSTS on all my sites.

[DaddyGo]

@Gertjan said in Squid + https:

Take note : it's an on-going battle. I wonder how one proxies the sites that have HSTS ( https://fr.wikipedia.org/wiki/HTTP_Strict_Transport_Security - it's yet another anti MITM ) activated. I use HSTS on all my sites.

exactly, I referred to problems such as those described by @Gertjan
and then there are the mobile OP systems....etc.

never give up, but I think Squid and similar solutions are coming to an end

or "fine tuning" them? - yes otherwise behind a proxy, WhatsApp, etc. will be dead
(because it is not only one encrypted process going on in the background)

like Apple stuff:

*.phobos.itunes-apple.com.akadns.net
*.gateway.push-apple.com.akadns.net
*.ax.itunes.apple.com
*.mesu.apple.com
*.phobos.apple.com
*.albert.gcsis-apple.com.akadns.net
*.ax.init.itunes.apple.com
*.init.itunes.apple.com
*.oscp.apple.com
*.deploy.static.akamaitechnologies.com
*.itunes.apple.com.edgekey.net
*.swcdn.apple.com
*.swdownload.apple.com
*.swquery.apple.com
*.swscan.apple.com
*.appldnld.apple.com
*.suconfig.apple.com
*.serverstatus.apple.com
*.gs.apple.com
*.apple.com
*.updates.cdn-apple.com

@techtester-m "For the sake of learning let's say there's enough storage, even though an organized NAS would be MUCH better for any business/company. Does squid has the option to set a maximum X G/MB for caching? Sort of a circular caching where a new file would push and old one(s) when that maximum X is met."

• just think of a multi-gig Windows update here

and has nothing to do with restrictions, blocking etc.?

don't think of Squid as pfBlockerNG for example
and we didn’t even talk about SquidGuard then

++++edit:
just watch the Squid cache directory...

[techtester-m]

@Gertjan @DaddyGo Ok...as I see it, the headache that comes with Squid/MITM solutions just in order to cache data, makes it not so worth it. A business would prefer solutions like NAS or a local server with let's say Elasticsearch or something like that. If an employee needs a certain file first search the local storage.

For simply caching web pages it's almost redundant and would make sense only in places without a broadband or for very large organizations.

[techtester-m]

@DaddyGo @Gertjan A MITM can intercept PKs easily, but he would need his CA to be installed on the client's side.
But what prevents a MITM to simply forward you the certificate it received from the original destination?

Such an attempt would fail because the client's browser would see the mismatch between the cert and the source address/domain of the MITM, right? Feels like such a basic question but still...

I just wanna make sure I fully understand it.

From what I've read the combination between RSA encryption/PKs & Certificate authentication is as secure as it gets so long the end user hasn't installed any CA other than what is already trusted by the browser.

[DaddyGo]

@techtester-m

nowadays, due to the development of physical hardware, this caching is not really interesting, on small-scale systems

a proxy can help for clusters with hundreds of users if you always have to load the same content

and behind a proxy, a lot of things can be well hidden

but as mentioned https and all its advantages, it becomes a disadvantage here
everyone is try hiding something

[techtester-m]

@DaddyGo said in Squid + https:

and behind a proxy, a lot of things can be well hidden

A VPN server would do the job without MITM stuff :)

[DaddyGo]

@techtester-m

In the past, I worked as a sysadmin for a company, with 2 800-2 900 direct workstations

well then and there the proxy was good, it kept the stupidity of the employees in check

the VPN also obscures everything, just a little differently than, for example, the intranet proxy on a campus network

[techtester-m]

@DaddyGo Is there a good way to prevent http access and force https without installing browser add-ons?

via pfSense, Pi-hole etc. maybe?

Also, things like squid are great when you wanna know the content of whatever your employees do centralized in one place, but I guess there's many software solutions to do that on each device owned by the company.

[DaddyGo]

@techtester-m said in Squid + https:

Is there a good way to prevent http access and force https without installing browser add-ons?
via pfSense, Pi-hole etc. maybe?

slowly wear out the "http" ...
disable port 80, though that would still be crazy, hahaha

I think the pi-hole is forgettable, here is the great Unbound + pfBlockerNG pair

@techtester-m "Also, things like squid are great when you wanna know the content of whatever your employees do centralized in one place, but I guess there's many software solutions to do that on each device owned by the company."

don't think, that...
big companies are the driest jobs, even though I worked as an outside worker...hmmmm

they invest in an IT structure and after that they think it will work forever

I have a friend who works for a very large company (their names are not public, because you would faint if you knew) and still runs Win7 on VM (hypv) enviroment with Citrix
as management regrets the money, for out of date Win7 replacements

[techtester-m]

@DaddyGo said in Squid + https:

disable port 80, though that would still be crazy

That's too much haha. I want to do what "https everywhere" does but on the pfsense or other end equipment and without breaking internal pc/os stuff. On a DNS level basically but on the other hand how do you determine what comes from browsing and what not...

@DaddyGo said in Squid + https:

I think the pi-hole is forgettable, here is the great Unbound + pfBlockerNG pair

DNS is probably one of the only services I wouldn't mind to "outsource" from the firewall. Especially when Pi-hole is much more organized, visual and has a bunch of people who all they do everyday 24/7 is trying improve it.
An entire project dedicated solely to DNS.
pfBlockerNG is at the end of the day a package. I'd still use it as an IP blocker though :)

@DaddyGo said in Squid + https:

their names are not public, because you would faint if you knew

Crazy! With all the money these companies have they still try to save as much as possible and be cheap, even when it's in their best interest to invest it hahaha unbelievable.

[DaddyGo]

@techtester-m

Hmmm HTTP...

with http you you can't start anything this is a webserver question, for example on a web server in the .htaccess file all http must be redirected to https
plus Let'sE certificate and that's about

only web masters are lazy, cca. 10 minutes job it

you don't think so,.. the Pi-hole is just popular and beautiful nothing more
(I apologize to all the Pi-hole fans) - perfect for raspberry pi

Unbound + DoT + DNSSEC + CF DNS servers and pfBlockerNG -devel, it is a professional solution

+++edit:
people tend to overcomplicate everything even though it is simple and everything is included in pfSense (more or less)

[techtester-m]

@DaddyGo said in Squid + https:

Unbound + DoT + DNSSEC + CF DNS servers and pfBlockerNG

That's exactly what I use! But...except for the fact that it's AIO solution + IP blocking what would be another advantage of pfBlocker over Pi-Hole (DNS wise)? Also, I've seen more lists and just info in general about Pi-Hole and as I said it seems much more organized, visual etc.

One last question if you may.
What happens when/if a MITM has its PK signed by a trusted CA? Would it still break because the browser will check from which domain/address the data came from resulting in a mismatch between what the client requested and the source?

[DaddyGo]

@techtester-m

pfBlockerNG does exactly the same thing as Pi-hole, only better
mainly because it does with a professional DNS resolver, which is Unbound
don't be fooled by the look...(GUI, etc) - Pi-hole

professional stuff usually doesn't even have a GUI, just a CLI
just look at Brocade, Juniper, etc. switches or Cisco enterprise devices
pfBlockerNG has a clean design and what is good can be set even by those who don't understand it so much

the browser only sees the Squid internal certificate behind the proxy
he does not even know the existence of the original certificate

[techtester-m]

@DaddyGo said in Squid + https:

professional stuff usually doesn't even have a GUI, just a CLI

I know, but it's not only the GUI part here. It's an entire dedicated solution for one single task - "DNSing", but I'll search for some good lists for pfBlockerNG and see.

@DaddyGo said in Squid + https:

mainly because it does with a professional DNS resolver

I forward the requests anyway, because the 13 DNS root servers don't support DoT (yet).
With Pi-Hole it would be DoH because they don't support DoT yet for some reason.

[techtester-m]

@DaddyGo @Gertjan Do any of you know how does a certificate validation works exactly, in technical details?
I just posted it on security.stackexchange but decided to try and ask you guys as well :)

I've read this post - https://security.stackexchange.com/questions/135401/certificate-validation,
where it says this : "...an attacker can still take the whole signed content and present it to you but won't be able to change any details or the signature won't match."

My questions are these:
(1) How does the browser validates the details of a certificate and see its content like: domain name etc.? Knowing that the cert is encrypted.
(2) If a MITM wanna change anything he would need to decrypt the cert, which is impossible, or simply change what he wants but then fail at the browser because, as mentioned above, the signature won't match?

Thank you,

[DaddyGo]

@techtester-m said in Squid + https:

but I'll search for some good lists for pfBlockerNG and see.

in the "devel" version have a lot of built-in lists, but I also have a lot of collected lists

don't be fooled by the word = DEVEL - this current / actual version

[Gertjan]

@techtester-m said in Squid + https:

Do any of you know how does a certificate validation works exactly, in technical details?

Yep.
Have a look at least 10 videos from this list https://www.youtube.com/results?search_query=https+certificate+how+it+works

Added to that :
In a perfect world, DNS is non spoofable (because you use the Resolver and DNSSEC ;) ), your app will know for sure what the real IP of a site is.

The important things :
In a certificate, you find the dates of validity,
The CA, used to certify that the certicate is valid. An OS, web browser, app, whatever, sues a build in list with known trusted CA's - and take note : you can add your own CA .... you can become a CA for your own network.
In a certificate is embedded the domain name or even a root domain name :