Squid HTTPS/SSL filtering 2017 [Is this it?]
-
it seems squid is changing the domain from the domain typed in to "http"
No, it's not. See https://redmine.pfsense.org/issues/6777#note-2
If Squidguard is blocking Google for you, that's unfortunate, you'll need to fix that issue instead. (No idea how, the package is broken and the code not maintainable.)
This is just extremely strange. What would be best to use as a replacement for Squid? There really does seem no proper fix for this.
-
There is no need to replace Squid, this is not a Squid issue. Anyway, see https://www.freshports.org/www/e2guardian/ (the GUI/pfSense package got never finished - https://github.com/pfsense/pfsense-packages/pull/866, https://forum.pfsense.org/index.php?topic=87526.0). As for squidGuard replacements, there's https://www.freshports.org/www/ufdbguard/ (not available in pfSense and obviously no GUI for that.)
-
I wouldn't bother with MITM with a CA to be honest… If you are using WPAD your browser and SQUID both should be letting HTTPS traffic pass through properly. This happens even for HSTS based websites like Google.com. However this also negates the ability of ClamAV working, which, is useless anyways.
-
I wouldn't bother with MITM with a CA to be honest… If you are using WPAD your browser and SQUID both should be letting HTTPS traffic pass through properly. This happens even for HSTS based websites like Google.com. However this also negates the ability of ClamAV working, which, is useless anyways.
Is it even possible to work like this? From what I understand, I cannot do any HTTPS filtering without MITM as the traffic is encrypted.
There is no need to replace Squid, this is not a Squid issue. Anyway, see https://www.freshports.org/www/e2guardian/ (the GUI/pfSense package got never finished - https://github.com/pfsense/pfsense-packages/pull/866, https://forum.pfsense.org/index.php?topic=87526.0). As for squidGuard replacements, there's https://www.freshports.org/www/ufdbguard/ (not available in pfSense and obviously no GUI for that.)
Thanks for that, I will have a look at e2guardian, from the description, it does look like what I would wanna use but it's quite outdated. However, what I really wanted to know is how others are filtering HTTPS content using Pfsense in this day and age. I'm not an expert, but I know there's many hoops to jump through as HTTPS was designed so that inspection couldn't be done easily (HSTS for example).
I would really appreciate if someone who is already doing this could message, and give me pointers to the system they're using and how they're jumping through the hoops I mentioned above.
-
shrugs I don't use MITM and CA's on my network, but I use WPAD and I seem to get a "block page" any time I hit a blocked secure website, granted it doesn't show the actual block page since you can't intercept, so SQUID just breaks the tunnel.
Blocked Secure Site
Allowed Secure Site
I also have Transparent disabled… If you are wanting to get the block page that you would normally use however, not sure how it would be possible all because of HSTS.
-
shrugs I don't use MITM and CA's on my network, but I use WPAD and I seem to get a "block page" any time I hit a blocked secure website, granted it doesn't show the actual block page since you can't intercept, so SQUID just breaks the tunnel.
Blocked Secure Site
Allowed Secure Site
I also have Transparent disabled… If you are wanting to get the block page that you would normally use however, not sure how it would be possible all because of HSTS.
I'm having that same issue, but I think it maybe linked to chrome. The block page correctly displays on other browers, and devices. It's kinda sad really, I'm sure others have the setup I want but not sure if they're not willing to share. Or haven't seen this thread yet.
Squid is making me want to bash my head on the wall…
EDIT : The page has been fixed on Chrome now, after the latest update. I'm using Chrome beta.
-
This might be related - https://docs.diladele.com/faq/squid/cannot_connect_to_site_using_https.html
In short - browser ignores everything but 200 Connection established from initial CONNECT tunnel setup to remote HTTPS sites thorough proxy (hope this is not too technical) -
This might be related - https://docs.diladele.com/faq/squid/cannot_connect_to_site_using_https.html
In short - browser ignores everything but 200 Connection established from initial CONNECT tunnel setup to remote HTTPS sites thorough proxy (hope this is not too technical)Thank you for the reply @Sichent, so is Diladele the only filtering system available for pfsense that's actually actively updated and maintained? Doktornotor seems to be working extremely hard trying to fix Squid itself, but issues like this just remain. I haven't been able to use anything else decent enough that also does SSL inspection of sorts, I'm stuck relying on SquidGuard which is half broken and OpenDNS.
I'm not sure if others are not doing this, or just don't want to share their methods. But it seems bizarre to me, how can everyone be filtering without HTTPS… As most do just use Squid + Squid Guard.
-
Fixing Squid (pretty much done as of 0.4.33) sadly will not do anything for the SquidGuard package. It's not just buggy, but that code is completely unreadable and something I definitely won't touch. Would hugely benefit from rewriting from scratch.
-
I've got HTTPS filtering running with squid+clamav+squidguard on pfsense.
If you want, i can send you my configs ive changed and try to get it running that way.
If that works for you too, we can make it public in this forum.I still have to exclude some sites. for example some sites with certificate pinning activated…
-
I've got HTTPS filtering running with squid+clamav+squidguard on pfsense.
If you want, i can send you my configs ive changed and try to get it running that way.
If that works for you too, we can make it public in this forum.I still have to exclude some sites. for example some sites with certificate pinning activated…
Sounds great, I'd love to try it. How did you go about excluding sites? Excluding doesn't seem like a good method because more and more websites are jumping on the HSTS bandwagon, Github, Python, and of course Google, just to name a few. Furthermore, Squid guard also doesn't do any kind of content inspection, it just seems to work off a black list, I'm not sure, but I think I can't create different levels of user groups and block certain websites for certain people either.
-
Yes, hello. Can you please share the config to me. I am also having big https problems with squidguard. Some https sites working, others not.
-
https://forum.pfsense.org/index.php?topic=112335.0
-
Thanks for that.
-
But what about actually decrypting and scanning?
-
But what about actually decrypting and scanning?
Do you mean using the pfsense antivirus to scan https files? Or something else?
-
But what about actually decrypting and scanning?
Do you mean using the pfsense antivirus to scan https files? Or something else?
I mean using pfsense to scan and look into SSL/HTTPS traffic to do certain things like identify new websites to block. Currently I am using OpenDNS + SquidGuard and Shalla and it pretty much sucks. If there's a new domain which isn't in those lists, it'll open. E2 Guardian seems like it's perfect for me but it's still got issues as I understand, and doesn't work as well as it should.
-
The best way I found is to force google or bing into safe mode and use squidguard to catch the rest.
-
The best way I found is to force google or bing into safe mode and use squidguard to catch the rest.
That's pretty old and basic. I'm already using those but it's not good enough. What about uncategorized websites, that's where OpenDNS, shalla list and these methods really fail.
-
The best way I found is to force google or bing into safe mode and use squidguard to catch the rest.
That's pretty old and basic. I'm already using those but it's not good enough. What about uncategorized websites, that's where OpenDNS, shalla list and these methods really fail.
Well you cannot find these uncategorized websites using google/bing safe search so someone would have to find a new website (not using safe search) that is not added to shalla list yet and go to that website direct. Well like with all things if there is a will there is a way.