Filtering HTTP success BUT HTTPS is not successful

[techbee]

no update at all, not so helpful. no clear wiki/instruction even.

[techbee]

any help here !

[nikkon]

My problem seems to be kinda of the same.
Most of the https sites are just not working. In google for example if i load a search and try to access any of the results it dies. When i disable ssl filtering everything works.

[aGeekhere]

That should work, did you try restarting?

[techbee]

yes, i have tried restarting pfsense box, still same problem.

[aGeekhere]

8. firewall NAT RULE TO REDIRECT DEST PORT 80 TO 3128
9. firewall LAN RULES TO BLOCK DEST PORT 80 AND 443 on LAN NET SOURCE
  but allowed LAN ADDRESS TO DESTINATION PORTS 80 AND 443

I think your issue may be here
Disable rule 8

[techbee]

I have the rule number 8, because in the article from nguvu.org says

Block surfing on port 80

Once you have the WPAD redirection working and all traffic is flowing over port 3128, if you require you can create a firewall rule to prevent browsing via the usual HTTP port 80 ensuring all browsing is only done via your proxy.

NAT backup

To catch any PCs which aren’t configured with ‘auto configure’ in their settings, its possible to implement a port forward to catch any traffic directed at port 80 through to 3128.

[techbee]

Hey guys,

Anybody here have a working solution. Kindly share your solution. Thanks.

[techbee]

@aGeekHere:

8. firewall NAT RULE TO REDIRECT DEST PORT 80 TO 3128
9. firewall LAN RULES TO BLOCK DEST PORT 80 AND 443 on LAN NET SOURCE
  but allowed LAN ADDRESS TO DESTINATION PORTS 80 AND 443

I think your issue may be here
Disable rule 8

After disabling my step 8, still same problem shown on my screenshot attachment.

[nikkon]

can you do a traffic capture and see what happens?
I'm curious to see it

[aGeekhere]

if you haven already look through here https://forum.pfsense.org/index.php?topic=112335.0

[techbee]

@nikkon:

can you do a traffic capture and see what happens?
I'm curious to see it

No I can't.  I don't do those stuff so I know less on it.

[techbee]

@aGeekHere:

if you haven already look through here https://forum.pfsense.org/index.php?topic=112335.0

Yes, that post of yours is one of my reference and I collate other informations from others as well and I come up with the steps above I enumerated.
I just did not follow the safe modes you included.

Perhaps you can tidy up and update that post of yours and make it more specific, step by step, I will really be helpful reading it.

As I read your post, it did not mention about the following:

-if needed to install the wpad package,
-if need change protocol,
-if to enable both http and https filtering,
-what exact firewall rules to apply, what NAT firewall rules, or what LAN rules,
-if to use splice all or not.

So it would really be helpful if the information is complete.

[pfsensation]

Squid guard is pretty much broken and is walking on its last legs. Try E2Guardian, it's way better, has more options. can actually "scan" websites and see if it's good or bad based on blocked categories. And is better if you want to block something for a group of people, but allow it for others.

[techbee]

@pfsensation:

Squid guard is pretty much broken and is walking on its last legs. Try E2Guardian, it's way better, has more options. can actually "scan" websites and see if it's good or bad based on blocked categories. And is better if you want to block something for a group of people, but allow it for others.

Does it filter both http and https sites?

[pfsensation]

@techbee:

@pfsensation:

Squid guard is pretty much broken and is walking on its last legs. Try E2Guardian, it's way better, has more options. can actually "scan" websites and see if it's good or bad based on blocked categories. And is better if you want to block something for a group of people, but allow it for others.

Does it filter both http and https sites?

Yes, the mode you want to go on for HTTPS sites is up to you. If you choose to go with MITM (man in the middle) option, it will make the proxy actually "see" the content being transferred and block it. For example, images in Google images. This however requires you to install a CA certificate on clients, you can also setup HTTPS filtering without CA, but this doesn't let the filter see any of the content within the page, just the URL. I've got it setup as a hybrid in my home, CA for all kids devices, and family devices, non CA approach for all guest devices. I think this works best, but it's up to you, you could go ahead and install a Captive Portal, and force everyone to install the CA if you wished.

[techbee]

@pfsensation:

@techbee:

@pfsensation:

Squid guard is pretty much broken and is walking on its last legs. Try E2Guardian, it's way better, has more options. can actually "scan" websites and see if it's good or bad based on blocked categories. And is better if you want to block something for a group of people, but allow it for others.

Does it filter both http and https sites?

Yes, the mode you want to go on for HTTPS sites is up to you. If you choose to go with MITM (man in the middle) option, it will make the proxy actually "see" the content being transferred and block it. For example, images in Google images. This however requires you to install a CA certificate on clients, you can also setup HTTPS filtering without CA, but this doesn't let the filter see any of the content within the page, just the URL. I've got it setup as a hybrid in my home, CA for all kids devices, and family devices, non CA approach for all guest devices. I think this works best, but it's up to you, you could go ahead and install a Captive Portal, and force everyone to install the CA if you wished.

I would like to try this but I have further question if I may.

My problem is:

1. Does the users need to configure their browser and set the proxy configuration ?
2. Do I need to uninstall my current squid package and squidguard ?
3. How do I allow and block certain sites for group of users with it ?

I have no background configuring e2guardian.  Can you kindly give me guide how to configure E2guardian.
I need the https filtering withoout the CA and without CA on client devices coz I only need to block the URL.

Thanks in advance.

[pfsensation]

@techbee:

@pfsensation:

@techbee:

@pfsensation:

Squid guard is pretty much broken and is walking on its last legs. Try E2Guardian, it's way better, has more options. can actually "scan" websites and see if it's good or bad based on blocked categories. And is better if you want to block something for a group of people, but allow it for others.

Does it filter both http and https sites?

Yes, the mode you want to go on for HTTPS sites is up to you. If you choose to go with MITM (man in the middle) option, it will make the proxy actually "see" the content being transferred and block it. For example, images in Google images. This however requires you to install a CA certificate on clients, you can also setup HTTPS filtering without CA, but this doesn't let the filter see any of the content within the page, just the URL. I've got it setup as a hybrid in my home, CA for all kids devices, and family devices, non CA approach for all guest devices. I think this works best, but it's up to you, you could go ahead and install a Captive Portal, and force everyone to install the CA if you wished.

I would like to try this but I have further question if I may.

My problem is:

1. Does the users need to configure their browser and set the proxy configuration ?
2. Do I need to uninstall my current squid package and squidguard ?
3. How do I allow and block certain sites for group of users with it ?

I have no background configuring e2guardian.  Can you kindly give me guide how to configure E2guardian.
I need the https filtering withoout the CA and without CA on client devices coz I only need to block the URL.

Thanks in advance.

1. Depends what kinda device, you can use a package that comes with the E2Guardian unofficial repo called WPAD. In essence what that will do is advertise your proxy out to devices on the network, and Windows, IOS, Mac, etc will automatically pick it up. Android however, is a little more finnicky. However, you could just setup a NAT which I've done for port 80 (HTTP), haven't got it perfectly working with 443 (HTTPS) yet, what the NAT will do is catch all traffic coming out your devices on port 80, and redirect it to the proxy IP 8080, for E2Guardian.

2. E2Guardian works perfectly fine with Squid, or you may use TinyProxy. I prefer Squid because then I get to keep the caching benefits etc, while having the filtering I need.

3. Quite simple really. In order to do this, you go to the site URL tab, create a rule in there and throw in all the sites you want to block. Then go to the groups TAB and then create a group with your users, and then assign the rule to that group.

For this to work, you need a way of identifying users. I use IP because it's simple and easy to use, I have all home users on a static IP, that won't change, therefore I have them all assigned to a group.

For example, kids in one group. They have certain restrictions like forced YouTube restricted mode, more harsh filter based on profanity etc. The adults in the house have been assigned to their own group with YouTube restricted off, profanity blocking way more relaxed etc. Then I have the default group which is used for unauthenticated users (guests) that are just assigned any IP from DHCP, they are put there.

Hope that answers all your questions, and as you can see E2Guardian is very configurable, scalable and from my opinion works much much better than any other system out there. Due to how it can block content based off blacklists, phrase checking, PICS rating, etc. Where as SquidGuard solely blocks based off a black list. In this day and age you have loads of new websites and proxies coming out, those black lists alone will be useless when it comes to proper filtering. This is why I recommend phrase list blocking, which can detect phrases on a website, and links on a website and actually block it. In my testing this blocks everything that all other methods of blocking fail  at.

To sum up, here are a few benefits of E2Guardian:

• Scalable (you can get it exactly how you want it)
• Who, What, When, Where type blocking
• Different blocking methods
• Custom block pages (Enjoy my one, which I submitted to E2Guardian Github, looks cool!)
• Administrator bypass, you can allow a group to have a bypass link to bypass blocks and view the content for however long you want (configure it)
• Deep URL analysis, can scan links within sites. For example, Google images
• Can correctly forge SSL certificates and look at HTTPS traffic

[techbee]

pfsensation,

Thanks for summarizing the benefits.

By device, I meant the windows, android, ios, mac devices, etc.  You mentioned "you can use a package that comes with the E2Guardian unofficial repo called WPAD" but I don't see it on the e2guardian options. I am blind what you meant by that.

However, if it is not too much to ask, as I am not aware how to configure e2guardian and related package/firewall, could you possibly provide a step by step on how to set it up so I can follow you.  Or perhaps there is already a e2guardian setup step by step guide that you can point us to.

Although, I can post here and ask one by one the steps to be done while configuring on my own.  But since there is no wiki or setup guide about this on how to configure, then this could be hard and it will take long to configure.  I hope you understand a novice users like me.

[techbee]

no update on this !