Squid HTTPS/SSL filtering 2017 [Is this it?]

spanguel

I've got HTTPS filtering running with squid+clamav+squidguard on pfsense.
If you want, i can send you my configs ive changed and try to get it running that way.
If that works for you too, we can make it public in this forum.

I still have to exclude some sites. for example some sites with certificate pinning activated…

pfsensation

@spanguel:

I've got HTTPS filtering running with squid+clamav+squidguard on pfsense.
If you want, i can send you my configs ive changed and try to get it running that way.
If that works for you too, we can make it public in this forum.

I still have to exclude some sites. for example some sites with certificate pinning activated…

Sounds great, I'd love to try it. How did you go about excluding sites? Excluding doesn't seem like a good method because more and more websites are jumping on the HSTS bandwagon, Github, Python, and of course Google, just to name a few. Furthermore, Squid guard also doesn't do any kind of content inspection, it just seems to work off a black list, I'm not sure, but I think I can't create different levels of user groups and block certain websites for certain people either.

ha100

Yes, hello. Can you please share the config to me. I am also having big https problems with squidguard. Some https sites working, others not.

aGeekhere

https://forum.pfsense.org/index.php?topic=112335.0

ha100

Thanks for that.

pfsensation

But what about actually decrypting and scanning?

aGeekhere

@pfsensation:

But what about actually decrypting and scanning?

Do you mean using the pfsense antivirus to scan https files? Or something else?

pfsensation

@aGeekHere:

@pfsensation:

But what about actually decrypting and scanning?

Do you mean using the pfsense antivirus to scan https files? Or something else?

I mean using pfsense to scan and look into SSL/HTTPS traffic to do certain things like identify new websites to block. Currently I am using OpenDNS + SquidGuard and Shalla and it pretty much sucks. If there's a new domain which isn't in those lists, it'll open. E2 Guardian seems like it's perfect for me but it's still got issues as I understand, and doesn't work as well as it should.

aGeekhere

The best way I found is to force google or bing into safe mode and use squidguard to catch the rest.

pfsensation

@aGeekHere:

The best way I found is to force google or bing into safe mode and use squidguard to catch the rest.

That's pretty old and basic. I'm already using those but it's not good enough. What about uncategorized websites, that's where OpenDNS, shalla list and these methods really fail.

aGeekhere

@pfsensation:

@aGeekHere:

The best way I found is to force google or bing into safe mode and use squidguard to catch the rest.

That's pretty old and basic. I'm already using those but it's not good enough. What about uncategorized websites, that's where OpenDNS, shalla list and these methods really fail.

Well you cannot find these uncategorized websites using google/bing safe search so someone would have to find a new website (not using safe search) that is not added to shalla list yet and go to that website direct. Well like with all things if there is a will there is a way.

andyschmid

@doktornotor:

@pfsensation:

it seems squid is changing the domain from the domain typed in to "http"

No, it's not. See https://redmine.pfsense.org/issues/6777#note-2

If Squidguard is blocking Google for you, that's unfortunate, you'll need to fix that issue instead. (No idea how, the package is broken and the code not maintainable.)

@doktornotor:

Fixing Squid (pretty much done as of 0.4.33) sadly will not do anything for the SquidGuard package. It's not just buggy, but that code is completely unreadable and something I definitely won't touch. Would hugely benefit from rewriting from scratch.

So this looks like the SquidGuard package shouldn't be used at all which leads me to 2 questions:

With 2.3 a lot of packages got cleaned up that weren't maintained. So if SquidGuard isn't maintained and broken why didn't it get removed as well?
If SquidGuard is a no go and the other two solutions e2guarding and ufDBGuard are not available as packages how do we do content filtering on pfSense?

While I understand all the issues mentioned just keen to understand what's the solution moving forward.

Thanks!

pfsensation

@andyschmid:

@doktornotor:

@pfsensation:

it seems squid is changing the domain from the domain typed in to "http"

No, it's not. See https://redmine.pfsense.org/issues/6777#note-2

If Squidguard is blocking Google for you, that's unfortunate, you'll need to fix that issue instead. (No idea how, the package is broken and the code not maintainable.)

@doktornotor:

Fixing Squid (pretty much done as of 0.4.33) sadly will not do anything for the SquidGuard package. It's not just buggy, but that code is completely unreadable and something I definitely won't touch. Would hugely benefit from rewriting from scratch.

So this looks like the SquidGuard package shouldn't be used at all which leads me to 2 questions:

With 2.3 a lot of packages got cleaned up that weren't maintained. So if SquidGuard isn't maintained and broken why didn't it get removed as well?

If SquidGuard is a no go and the other two solutions e2guarding and ufDBGuard are not available as packages how do we do content filtering on pfSense?

While I understand all the issues mentioned just keen to understand what's the solution moving forward.

Thanks!

It's a big doddle!

E2 Guardian is actually quite fantastic, I've been testing it in a virtual machine. It's way better than SquidGuard for sure, however it still has this issue of not working with chrome as it still uses Squid as a upstream proxy. I've found this website explaining a bit more : http://stackoverflow.com/questions/43665243/chrome-invalid-self-signed-ssl-cert-subject-alternative-name-missing

However, anyone reading this. Remember that E2 Guardian is still unofficial and the GUI isn't perfect. But so far it seems to work half decently, and be able to phrase match - Something I always wanted. It's stupid to rely solely on URL filtering these days, you want your firewall to be able to scan a website and see if it should be blocked or allowed based on whats on the page. This also means that any new URL or proxy that may slip through the nets will also be blocked.

fptelecom

I used PFSENSE for almost two years. It is an AWESOME firewall and router, capable to support about 500 users concurrently whit just a CPU and a couple of good NICs. It's just sad to see one of their "killer apps", the https web filtering falling down defeated and without any hope. Is this the begining of the end for PFSENSE? I hope not, but unfortunaty I need to search another options. Push play baby!!! GOODBYYYYYEEEEE MOON MEN, GOODBYYYYYEEEE GOODBYYYYYEEEE…...... :'(

J_Unit

I had the same problem with the new 2.3.4 install of pfSense. I configured everything exactly how it should be configured but kept getting that https://http* error everyone seems to be getting. Looking through various websites I found nothing that was of any use. A lot of people even said that's the nature of ssl filtering with pfSense. I finally got it working. Literally, all I did was go into the common ACL tab and change the 'Default access [all]' setting from allow to deny. Save and apply the settings. Then go back in and change it back from deny to allow again. Save and apply the settings. Now it works like it should.

sichent

See alternative to SquidGuard and Dansguardian / e2. GUI is own though and harder to install than desired. https://docs.diladele.com/tutorials/filtering_https_traffic_squid_pfsense/index.html