Guide to filtering web content (http and https) with pfsense 2.3

aGeekhere

What is the tcp port that you used on your webconfigurator ?

set pfsense Protocol to http (This is a MUST, it will not work if you do not do this)
System: Advanced: Admin Access Protocol http

If I block port 80, then, I will also be blocked opening the pfsense gui.

No it should not (have the rule above the anti lockout rule);

Also, regarding to my question number 4

Enable squid proxy 3128
Get WPAD working follow guide

Enable transparent mode to forward all requests for destination port 80 to the proxy server.

Enable SSL filtering
Splice all
create Certificate (for pfsense).

Summery
Set Advanced: Admin Access Protocol to http
Enable squid proxy
Follow guide on setting up WPAD
Enable transparent mode
Enable SSL filtering Splice all

techbee

Yep, done that all.

I can download the wpad files from the browser. I can access the http sites but still cannot access the https sites.

aGeekhere

what web browser?

Clear your cache?

What Operating system?

Operating system update work?

Did you set auto configure proxy?

Try Diagnostics States States clear?

What sites, google?

techbee

@aGeekHere:

what web browser?

Clear your cache?

What Operating system?

Operating system update work?

Did you set auto configure proxy?

Try Diagnostics States States clear?

What sites, google?

'

Hi aGeekHere,

Yes, cache is cleared,
os=win7 pro sp1,
I disabled the update coz I dont need it,
autoconfigure is set,
yes, google and yahoo https sites. facebook and youtube does not load and shows error.

I use splice-all. Don't I really need to install certs to client device ?

If https site is successfully blocked, what error page do I expect to see ? Can you show me the screenshot for the error page that will show !

aGeekhere

firefox ie or chrome?

I use splice-all. Don't I really need to install certs to client device ?

not when splice all

If https site is successfully blocked, what error page do I expect to see ?

The default cannot access page of the web browser.

If you enabled squid, transparent mode and SSL filtering Splice all with a cert on pfsense and you still cannot access https sites, then something is broken.
Try reinstall.
If you still have the same issue post screen shots of your setup.

techbee

@aGeekHere:

firefox ie or chrome?

I use splice-all. Don't I really need to install certs to client device ?

not when splice all

If https site is successfully blocked, what error page do I expect to see ?

The default cannot access page of the web browser.

If you enabled squid, transparent mode and SSL filtering Splice all with a cert on pfsense and you still cannot access https sites, then something is broken.
Try reinstall.
If you still have the same issue post screen shots of your setup.

I am using chrome. most of the users are chrome.
On the other hand, the firewall rules you shown was to block port 80 and 443 right and redirect dns resolution to pfsense dns server and no more else. That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right ?

aGeekhere

That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right

That is what the transparent proxy is doing, remove that rule, that could be what is causing the issue.

techbee

@aGeekHere:

That means, I don't need to redirect traffic from port 80 to 3128 and 443 to 3129 right

That is what the transparent proxy is doing, remove that rule, that could be what is causing the issue.

Alright

So what I have now on my firewall rules are:
1. antilock rule on port 80 which says (allow all on tcp port 80)
2. block LAN_NET on tcp port 80 and 443
3. redirect rule for dns resolution

would that be ok ?

On the other hand, what is the default cannot access page of the web browser, is it the same like of that of http transparent proxy error page ?

techbee

aGeekHere,

My wpad now works. I can download it from chrome browser.
I can ping and nslookup wpad now.
All squidguard group ACL and its target rules are applied to block facebook and youtube for testing purposes for now.
I HAVE NOT INSTALLED THE CA CERT ON CLIENT DEVICES.

I used http and https filtering with splice all option so that I dont need to install the cert to client devices.

I also cleared browser cache and history.

The result is I can browse http but cannot browse https. Https sites like yahoo and google mail works though.

I have screenshot of the error for browsing https site.

error.jpg_thumb

aGeekhere

Is squidguard blocking it by mistake? is it set to deny all?

Try allowing all in squidguard rules to test.

Post screen shots of squidguard.

Try clearing states in pfsense.

Reboot.

techbee

Squidguard is not blocking by mistake.

I tried clearing firewall states and Rebooted, still the same situation.

techbee

I also have this wireshark done where I can see that it made a CONNECT method to facebook, then it has established connection, send client hello but later have RST and there is no SERVER HELLO thus no application data received.

What could I miss here.

Untitled.jpg_thumb

aGeekhere

lets try without the wpad and use just the transparent, set proxy direct does it work then?
If you still cannot access https sites then post you squid config.

techbee

@aGeekHere:

lets try without the wpad and use just the transparent, set proxy direct does it work then?
If you still cannot access https sites then post you squid config.

If I use only the http proxy without the ssl mitm and set browser to auto, then the http proxy filter works.

If you are saying to setup http transparent proxy and ssl mitm filtering and install cert to client device and set browser proxy, then I have not done this yet.

aGeekhere

and install cert to client device and set browser proxy, then I have not done this yet.

Just use splice all for mitm (and use the cert created in pfsense under SSL Man In the Middle Filtering\CA).
Set browser to direct.
This will use transparent proxy for both http and https.

If this works then there is an issue with your WPAD.
If this does not work then something is broken.

techbee

@aGeekHere:

and install cert to client device and set browser proxy, then I have not done this yet.

Just use splice all for mitm (and use the cert created in pfsense under SSL Man In the Middle Filtering\CA).
Set browser to direct.
This will use transparent proxy for both http and https.

If this works then there is an issue with your WPAD.
If this does not work then something is broken.

What do you mean for "Set browser to direct", is this the automatic detect settings ?

On the other hand, my wpad only have the following inside:

function FindProxyForURL(url,host)
{
return "PROXY 192.168.1.1:3128";
}

I am not sure what you example below means so I use the example from pfsense wiki.

function FindProxyForURL(url, host)
{
if (isPlainHostName(host) ||
shExpMatch(host, "*.local") ||
isInNet(dnsResolve(host), "192.168.1.0", "255.255.255.0"))
return "DIRECT";

return "PROXY 192.168.1.1:3128";
}

aGeekhere

yeah turn off automatic detect setting so the traffic goes direct and gets intercepted by the transparent proxy.

techbee

@aGeekHere:

yeah turn off automatic detect setting so the traffic goes direct and gets intercepted by the transparent proxy.

By turn off, you mean uncheck the automatic detect settings and others options as well ?

aGeekhere

yes

techbee

May I ask though, what is the exact version of pfsense that you are using just for comparison. If this will not work, then I will need to reinstall my pfsense box, maybe its broken.

I am using:

2.3.4-RELEASE (amd64)
built on Wed May 03 15:13:29 CDT 2017
FreeBSD 10.3-RELEASE-p19

On the other hand, below is the screen shot of my testing NOT USING WPAD but setup the browser proxy.

Also, I tried direct approach that you said, and the result the same error as before.

What exact pfsense version are you using ?

error1.jpg_thumb