Guide to filtering web content (http and https) with pfsense 2.3

[zonda]

Is it still possible to create exceptions to not proxy trusted sites with transparent proxy?
There are some specific sites that will need to bypass the proxy for some of our application to function. WPAD was used initially in our environment but some traveling users had issues with persistent proxy settings so would prefer to only use the transparent method.

[aGeekhere]

@zonda
Sure, the option is under ACLs in squid. If users do not want to use the WPAD (auto config proxy) then they should manually enter in the proxy settings in internet options (if using windows). Do not rely on SSL/MITM Mode splice all for all 443 traffic or else you will get connection errors. That is if you are blocking port 80 and 443 and forcing users to use the proxy.

[Jopeme2000]

Hello. I use your guide http and https to filter traffic but i have problems with youtube videos

Restricts many videos in an erroneous way

Do you know how i can solve it?

[aGeekhere]

@jopeme2000
"NOTE: Safe search for youtube is not as advanced as google safe search, which results in a lot of safe content be filtered out."

Google has to improve it, nothing you can do about it

[roberz]

@aGeekhere
Hi,
A great thank you for this guide. It is the most complete guide for traffic filtering on pfSense I was able to find.
I made traffic filtering work for a customer who wants porn filtering and it works. But doesn't work perfect. It works great on desktop machines while on Android phones it blocks more than required. For example: it also blocks all Skype and Facebook Messenger traffic and also blocks my RSS feed reader to connect to Internet on Android phones.
Can you explain a bit what you thought with this in june 2016 edit:
"I have found that if you have connection issues using auto config for android or other smart phones try manually setting the proxy, now opening port 80 and 443 is not needed."
Thank you.

[aGeekhere]

@roberz
For android phones (depending on with android version) auto config has issues so you have to manually enter in the proxy settings.

If things like Skype and Facebook Messenger are being blocked make sure you also have set up transparent proxy with splice all for SSL Man In the Middle Filtering.

The idea is
First use the WPAD to connect to the proxy (if the device has auto config issues you have to manually add it). If a program does not have proxy settings the transparent proxy and SSL Man In the Middle should catch the rest. The reason why to first use the WPAD is that some programs/software (like windows updates) have connection issues with transparent proxy and SSL Man In the Middle, splice all.

So first use the WPAD as a shortcut so you do not have to configure all devices on your network to use your proxy.
Then for devices that have auto conf issues manually config them.
Finally have the transparent proxy and SSL Man In the Middle splice all to catch any traffic that cannot use the proxy .

Hope this was helpful

[marksantos]

Hi sir, all step are follow, search engine n safe search is working, but some of the images in whitelist website can't load.

[aGeekhere]

Just tested and it loads fine for me. Check your blocklist in your web filter (e.g squidguard).

[marksantos]

the problem is default acl. when i try do deny all, the whitelist website can't load some images, when i try to allow default acl the whitelist website, can load images but the browsing of the whitelist are very slow.

[marksantos]

i'll try the whitelist website can only access and the other can block.

[marksantos]

hi sir, how about gmail, how to allow gmail in whitelist, i tried add gmail.com in whitelist but still block the access.

[aGeekhere]

In squidguard set Default access [all] to allow. Now only block the categories you want. You can use shallalist for categories.

[4o4rh]

dns resolver doesn't start when i include the forcegoogle.conf in the custom options.

if i remove it, it works just fine.

[aGeekhere]

did you make the forecegoogle.conf file?

[4o4rh]

@aGeekhere problem was, i followed the initial instructions by putting the youtube and bing into the host overrides, then copied the whole forcegoogle section without checking the contents.

In has youtube with a different address and bing. Removed those and it works.

[johnchon]

i have follow the topic to do the filter but when i just activate transparent mode only i get this message " 400 Bad Request
The plain HTTP request was sent to HTTPS port
nginx"

May i know what happen for this ?

[aGeekhere]

only using the transparent can cause connection issues, in this guide i have it as a fallback for programs which do not have proxy settings.

First setup WPAD or manually configure devices to use proxy
then configure windows to auto use the proxy in lan settings
then set up the transparent proxy with mitm splace all to catch programs with no proxy settings

[johnchon]

@aGeekhere

1. But at firewall rules already forward all Lan to proxy still need to setup WPAD ?
2. At proxy settings i have Enable Transparent Proxy Settings & SSL Man In the Middle Filtering by spice all.
3. Because i have filter social network and shopping and youtube website.

[aGeekhere]

@johnchon said in Guide to filtering web content (http and https) with pfsense 2.3:

But at firewall rules already forward all Lan to proxy still need to setup WPAD ?

When you only use Transparent you can get connection issues. Try this, configure your device to use the proxy,for windows it is under lan settings (WPAD automates this).

Next make sure your web browser is using the proxy.

Do you still get connection issues?

[johnchon]

@aGeekhere i have both enable tranparent and the SSL filter, when only enable using transparent mode only all website running good. No issues for connection. i'm using firefox for auto detect proxy server.