Squid + https
-
@DaddyGo That's the visualization of my question haha...
-
I understand your question, but now you are next...
the best way to learn, if you do everything yourself and just get guidance from colleagues
https://www.howtoforge.com/filtering-https-traffic-with-squid
https://wiki.squid-cache.org/Features/SslBumpof course have to run a test copy of Squid and practice the steps and watch what happens
if you get completely stuck, then it is our job to help
-
@DaddyGo Ok...I've read it. So in order to intercept Squid does exactly this:
Correct?
-
exactly, somehow so
now the test time can come
(if you use it at home, create a test environment)you will run into a lot of problems, mainly with Android, iOS devices, so you’ll see what you need to shape
next dose of curriculum (for example):
http://www.webdnstools.com/articles/squid-proxy-whitelistI give a sample file:
https://www.dropbox.com/sh/pp3m9reh2eikks2/AADIJmyBKZ4cZKOqs3A-eBGva?dl=0 -
@DaddyGo Ok...
Few more if you may :)As I understand it so far it goes something like this (and please correct me where I'm wrong) -
(1) Client --> Squid interception of PK, aka MITM --> destination.
(2) It will do the above only for what is in its whitelist.
(3) A private self created CA has to be installed on every client that needs to use the Squid proxy, so it could dynamically generate certificates. Using Let's Encrypt or anything of the sort won't work here and also be a big no no.
(4) Everything in the whitelist will be cached (up to X G/MB limit), including files.
(5) A destination that is not on the whitelist will be blocked? If so then I think I prefer DNS blocking.
(6) Is their an option to cache/intercept only certain destinations (whitelist or something) and treat all the others as usual without "squidding" them?
-
- Yes
- no, these are exceptions as they require special rules and accesses
(otherwise they do not work)
note the whole world cannot be whitelisted - Yes, with this method... ("Using Let's Encrypt or anything of the sort won't work here and also be a big no no.") yes
- half true, (including files - it would require awful storage capacity - depending on what file we are talking about)
- No, the whitelist is necessary, because of the above, I also recommend DNS blocking in SOHO environment...
(Squid is a big boys game, because of big systems and proxy capabilities)
6.Yes by bypassing the proxy (ACL, PAC,etc)
https://wiki.squid-cache.org/ConfigExamples/Authenticate/Bypass
-
@DaddyGo said in Squid + https:
no, these are exceptions as they require special rules and accesses
(otherwise they do not work)So what do you do with exceptions? Let them bypass squid and go as usual or "fine tuning" them?
@DaddyGo said in Squid + https:
half true, (including files - it would require awful storage capacity - depending on what file we are talking about)
For the sake of learning let's say there's enough storage, even though an organized NAS would be MUCH better for any business/company. Does squid has the option to set a maximum X G/MB for caching? Sort of a circular caching where a new file would push and old one(s) when that maximum X is met.
@DaddyGo said in Squid + https:
No, the whitelist is necessary, because of the above
Perhaps it's only semantics here but the whitelist in this case is what you want Squid to authenticate for you (MITM) and cache and has nothing to do with restrictions, blocking etc.?
Ok...Semantics again. Squid does both access restrictions AND caching? -
There are still some sites left on the Internet that accept http (NON SSL) requests.
Install Squid, and focus on the classic 'intercept port 80' or 'use a wpad file' and have your browser been 'redirected to a 8080 proxy port'.
This way, you can see how it works in the old days- I guess, for that type of set-up, many examples are available.When you feel up to it, include 'https' in the addition.
Take note : it's an on-going battle. I wonder how one proxies the sites that have HSTS ( https://fr.wikipedia.org/wiki/HTTP_Strict_Transport_Security - it's yet another anti MITM ) activated. I use HSTS on all my sites.
-
@Gertjan said in Squid + https:
Take note : it's an on-going battle. I wonder how one proxies the sites that have HSTS ( https://fr.wikipedia.org/wiki/HTTP_Strict_Transport_Security - it's yet another anti MITM ) activated. I use HSTS on all my sites.
exactly, I referred to problems such as those described by @Gertjan
and then there are the mobile OP systems....etc.never give up, but I think Squid and similar solutions are coming to an end
or "fine tuning" them? - yes otherwise behind a proxy, WhatsApp, etc. will be dead
(because it is not only one encrypted process going on in the background)like Apple stuff:
*.phobos.itunes-apple.com.akadns.net
*.gateway.push-apple.com.akadns.net
*.ax.itunes.apple.com
*.mesu.apple.com
*.phobos.apple.com
*.albert.gcsis-apple.com.akadns.net
*.ax.init.itunes.apple.com
*.init.itunes.apple.com
*.oscp.apple.com
*.deploy.static.akamaitechnologies.com
*.itunes.apple.com.edgekey.net
*.swcdn.apple.com
*.swdownload.apple.com
*.swquery.apple.com
*.swscan.apple.com
*.appldnld.apple.com
*.suconfig.apple.com
*.serverstatus.apple.com
*.gs.apple.com
*.apple.com
*.updates.cdn-apple.com@techtester-m "For the sake of learning let's say there's enough storage, even though an organized NAS would be MUCH better for any business/company. Does squid has the option to set a maximum X G/MB for caching? Sort of a circular caching where a new file would push and old one(s) when that maximum X is met."
- just think of a multi-gig Windows update here
and has nothing to do with restrictions, blocking etc.?
don't think of Squid as pfBlockerNG for example
and we didn’t even talk about SquidGuard then
++++edit:
just watch the Squid cache directory...
-
@Gertjan @DaddyGo Ok...as I see it, the headache that comes with Squid/MITM solutions just in order to cache data, makes it not so worth it. A business would prefer solutions like NAS or a local server with let's say Elasticsearch or something like that. If an employee needs a certain file first search the local storage.
For simply caching web pages it's almost redundant and would make sense only in places without a broadband or for very large organizations.
-
@DaddyGo @Gertjan A MITM can intercept PKs easily, but he would need his CA to be installed on the client's side.
But what prevents a MITM to simply forward you the certificate it received from the original destination?Such an attempt would fail because the client's browser would see the mismatch between the cert and the source address/domain of the MITM, right? Feels like such a basic question but still...
I just wanna make sure I fully understand it.
From what I've read the combination between RSA encryption/PKs & Certificate authentication is as secure as it gets so long the end user hasn't installed any CA other than what is already trusted by the browser.
-
nowadays, due to the development of physical hardware, this caching is not really interesting, on small-scale systems
a proxy can help for clusters with hundreds of users if you always have to load the same content
and behind a proxy, a lot of things can be well hidden
but as mentioned https and all its advantages, it becomes a disadvantage here
everyone is try hiding something -
@DaddyGo said in Squid + https:
and behind a proxy, a lot of things can be well hidden
A VPN server would do the job without MITM stuff :)
-
In the past, I worked as a sysadmin for a company, with 2 800-2 900 direct workstations
well then and there the proxy was good, it kept the stupidity of the employees in check
the VPN also obscures everything, just a little differently than, for example, the intranet proxy on a campus network
-
@DaddyGo Is there a good way to prevent http access and force https without installing browser add-ons?
via pfSense, Pi-hole etc. maybe?
Also, things like squid are great when you wanna know the content of whatever your employees do centralized in one place, but I guess there's many software solutions to do that on each device owned by the company.
-
@techtester-m said in Squid + https:
Is there a good way to prevent http access and force https without installing browser add-ons?
via pfSense, Pi-hole etc. maybe?slowly wear out the "http" ...
disable port 80, though that would still be crazy, hahahaI think the pi-hole is forgettable, here is the great Unbound + pfBlockerNG pair
@techtester-m "Also, things like squid are great when you wanna know the content of whatever your employees do centralized in one place, but I guess there's many software solutions to do that on each device owned by the company."
don't think, that...
big companies are the driest jobs, even though I worked as an outside worker...hmmmmthey invest in an IT structure and after that they think it will work forever
I have a friend who works for a very large company (their names are not public, because you would faint if you knew) and still runs Win7 on VM (hypv) enviroment with Citrix
as management regrets the money, for out of date Win7 replacements -
@DaddyGo said in Squid + https:
disable port 80, though that would still be crazy
That's too much haha. I want to do what "https everywhere" does but on the pfsense or other end equipment and without breaking internal pc/os stuff. On a DNS level basically but on the other hand how do you determine what comes from browsing and what not...
@DaddyGo said in Squid + https:
I think the pi-hole is forgettable, here is the great Unbound + pfBlockerNG pair
DNS is probably one of the only services I wouldn't mind to "outsource" from the firewall. Especially when Pi-hole is much more organized, visual and has a bunch of people who all they do everyday 24/7 is trying improve it.
An entire project dedicated solely to DNS.
pfBlockerNG is at the end of the day a package. I'd still use it as an IP blocker though :)@DaddyGo said in Squid + https:
their names are not public, because you would faint if you knew
Crazy! With all the money these companies have they still try to save as much as possible and be cheap, even when it's in their best interest to invest it hahaha unbelievable.
-
Hmmm HTTP...
with http you you can't start anything this is a webserver question, for example on a web server in the .htaccess file all http must be redirected to https
plus Let'sE certificate and that's aboutonly web masters are lazy, cca. 10 minutes job it
you don't think so,.. the Pi-hole is just popular and beautiful nothing more
(I apologize to all the Pi-hole fans) - perfect for raspberry piUnbound + DoT + DNSSEC + CF DNS servers and pfBlockerNG -devel, it is a professional solution
+++edit:
people tend to overcomplicate everything even though it is simple and everything is included in pfSense (more or less) -
@DaddyGo said in Squid + https:
Unbound + DoT + DNSSEC + CF DNS servers and pfBlockerNG
That's exactly what I use! But...except for the fact that it's AIO solution + IP blocking what would be another advantage of pfBlocker over Pi-Hole (DNS wise)? Also, I've seen more lists and just info in general about Pi-Hole and as I said it seems much more organized, visual etc.
One last question if you may.
What happens when/if a MITM has its PK signed by a trusted CA? Would it still break because the browser will check from which domain/address the data came from resulting in a mismatch between what the client requested and the source? -
pfBlockerNG does exactly the same thing as Pi-hole, only better
mainly because it does with a professional DNS resolver, which is Unbound
don't be fooled by the look...(GUI, etc) - Pi-holeprofessional stuff usually doesn't even have a GUI, just a CLI
just look at Brocade, Juniper, etc. switches or Cisco enterprise devices
pfBlockerNG has a clean design and what is good can be set even by those who don't understand it so muchthe browser only sees the Squid internal certificate behind the proxy
he does not even know the existence of the original certificate
