Guide to filtering web content (http and https) with pfsense 2.3

huuur · May 23, 2017, 11:26 AM

@aGeekHere:

Update Youtube safe mode
Click add under Host overrides
Host = www
Domain = youtube.com
IP = 216.239.38.120
Description = youtube
Save
NOTE: Safe search for youtube is not as advanced as google safe search, which results in a lot of safe content be filtered out.

How can we get this working with mobile devices Android and iOS that use the youtube mobile app or m.youtube.com?

I can't express how useful this guide been for me.

For the record I found the below is working for youtube mobile app safe search:


Host = youtube
Domain = googleapis.com
IP =  216.239.38.120
Description = youtube app1


Host = youtubei
Domain = googleapis.com
IP =  216.239.38.120
Description = youtube app2

and maybe..


Host = www
Domain = youtube-nocookie.com
IP =  216.239.38.120
Description = youtube nocookie

in addition to what's mentioned in Reply#60 for mobile browsers.

aGeekhere · May 24, 2017, 9:41 AM

Thanks huuur, that should help others.

Though it would be good if youtube was better at filtering videos.

aGeekhere · Jun 9, 2017, 1:25 AM

Update

You can try setting up MITM by setting the SSL/MITM Mode to splice all, that way you do not need to create a certificate for each device on the network. (you still need to create a main certificate though).

vielfede · Jun 9, 2017, 10:47 AM

@aGeekHere:

Update

You can try setting up MITM by setting the SSL/MITM Mode to splice all, that way you do not need to create a certificate for each device on the network. (you still need to create a main certificate though).

That works fine for me also setting Client proxy settings manually:
Proxy address/PORT= SQUID_IP 3128

And that only! Indeed if i specify different proxy settings for http/https in (client win10=> Internet Settings=>Lan settings=>Advanced=>

http=SQUID_IP 3128
https=SQUID_IP 3129

It does not work, as in the contrary I'd excpect…. ::)
Geek can you explain why?

aGeekhere · Jun 10, 2017, 6:52 AM

Does automatically detect settings work?

vielfede · Jun 14, 2017, 12:22 PM

@aGeekHere:

Does automatically detect settings work?

Ehmm sorry, could you explain me better what you mean? As I stated I do not use WPAD, I configure proxy manually on client see picture…

Thanks.
PS. Sorry to answer late

aGeekhere · Jun 19, 2017, 9:10 AM

if you have the transparent proxy working then you do not need to define the proxy server.

vielfede · Jun 19, 2017, 2:37 PM

If I use it in transparent mode, https does not work!
Better sometimes it works, sometimes it does not! (http works always!)

If anyone managed to get http+https in splice all + transparent mode work please let me know… ;)

aGeekhere · Jun 19, 2017, 11:57 PM

I just enabled it mitm, added a cert, set it to splice all and its working.

techbee · Jun 21, 2017, 2:51 AM

The guide is update but confusing. Can you please clean up the guide and have it step by step. Its somewhat hard to follow if you are not familiar with wpad and related firewall rules.

Thanks for the guide.

aGeekhere · Jun 21, 2017, 7:33 AM

hi techbee, yeah after discovering a few new things the guide is a little messy now.

The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

when i get time I will try an clean it up a bit.

vielfede · Jun 23, 2017, 7:56 AM

@aGeekHere:

The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

Indeed I do not think MITM and splice all can work toghether, as stated on squid documentation MITM is associated to the "bump" directive that is something complety different from splice directive.
With MITM (bump) squid is able to decrypt traffic (and analyse it) meanwhile with splice all you can do is just "web filtering".
Summarizing
Directive Advantages/features Disadvantage
Splice (all) No needs of certificate installation on Clients + webfiltering No traffic analysis i.e no AntiVirus
Bump (MITM) Traffic analysis i.e noYES AntiVirus + webfiltering Needs to install certs on clients

Please take a look here and let me know if I missed something. Thanks
http://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions
http://marek.helion.pl/install/squid.html

AR15USR · Jun 22, 2017, 5:08 PM

@vielfede:

@aGeekHere:

The transparent proxy with MITM (splice all) vs wpad should really be divided into two different choices. When I made the guide I did not know that all you need to get transparent proxy for https working is set splice all and create a cert for the router. So in fact you can now choose either using a wpad or the transparent proxy MITM or both.

Indeed I do not think MITM and splice all can work toghether, as stated on squid documentation MITM is associated to the "bump" directive that is something complety different from splice directive.
With MITM (bump) squid is able to decrypt traffic (and analyse it) meanwhile with splice all you can do is just "web filtering".
Summarizing
Directive Advantages/features Disadvantage
Splice (all) No needs of certificate installation on Clients + webfiltering No traffic analysis i.e no AntiVirus
Bump (MITM) Traffic analysis i.e no AntiVirus + webfiltering Needs to install certs on clients

Please take a look here and let me know if I missed something. Thanks
http://wiki.squid-cache.org/Features/SslPeekAndSplice#Actions
http://marek.helion.pl/install/squid.html

Shouldn't the second one not have the word "no"..

vielfede · Jun 23, 2017, 7:58 AM

@AR15USR:

Shouldn't the second one not have the word "no"..

Sorry cut & paste mistake, I fixed it.

mahnonsaprei · Jun 29, 2017, 9:59 PM

Hi at all,

I'm Marcello from italy. Nice to meet you, i'm a newbie of pfsense.

I have read the tutorial, but i have one problem.
When i try to connect to an https website in blacklist, the browser show me a generic error see the attachment: (ERR_TUNNEL_CONNECTION_FAILED) while i want to see a message of pfsense that explain the block. In http it's ok

I'm looking for in google but i not found the answer.

Please help me.
Thank you veeeeery much!!!

aGeekhere · Jun 30, 2017, 12:10 AM

When i try to connect to an https website in blacklist, the browser show me a generic error see the attachment: (ERR_TUNNEL_CONNECTION_FAILED) while i want to see a message of pfsense that explain the block. In http it's ok

Known issue with squid, I do not think there is a fix for it.

mahnonsaprei · Jun 30, 2017, 2:08 PM

Thank you so much. There aren't alternatives to redirect https via proxy also blank page or other?

B.r.

techbee · Jul 3, 2017, 9:45 AM

hello ageekhere,

can you update your guide now.

aGeekhere · Jul 3, 2017, 1:44 PM

what part are you stuck on?

techbee · Jul 3, 2017, 2:03 PM

1. I dont know what exact firewall rules to add
2. I am confused if I need to install unofficial wpad package of marcelloc.
3. I am confused if there is a need to have separate webserver of wpad.
4. I am confused if I can enable both transparent and ssl mitm filtering on squid.
5. I cant nslookup wpad on clients command prompt to resolve to my pfsense box ip.