Squid + SquidGuard not reliable

[MrIT]

Just found out today: You can test WPAD when using Microsoft Edge (I suppose Chrome works as well?!) on Windows. If deployed and configured successfully, the you can open developer tools and switch to the "Network" tab. On all requests you'll see the proxy's IP adress as remote address for any website.

[JonathanLee]

@michmoor I have one i will share when I get home it's simple.

[michmoor]

@jonathanlee said in Squid + SquidGuard not reliable:

@michmoor I have one i will share when I get home it's simple.

Thanks!
Do you use a separate web server?

[JonathanLee]

@michmoor no my WPAD script is inside the firewall. I followed the Netgate website for reference.

[JonathanLee]

@michmoor

(Image: files named wpad.da, wpad.dat, and proxy.pac are locally stored they are all part of wpad and contain the same script)

(Image: Create and save text files as wpad.da, wpad.dat and proxy.pac)

(Image: Save all the WPAD files into path usr local www, this is so when any time port 80 is accessed it tells the system to use proxy)

(Image: All files should contain the following inside the text files wpad.da wpad.dat and proxy.pac this is the script)

(Image: You must also add a DNS resolver for the name of the firewall mine is home arpa to link in the ip address of the firewall mine is 192 168
1 1)

(Image: DNS resolve after adding)

(Image: You must also create an ACL for the wpad anything that hits port 80 will auto configure to the port of the proxy. For me I have a lan group that is all of my devices on my lan Lan_group * any destination the firewall)

(Image: ACL "Access Control Close line up")

(NAT)

(ACL with port specific and mail server specific rules)

(Image: Alias used for mail acl)

(Last three images explained: Manual NAT mine firewall approves mail and Xbox ports they are IP address specific; my mail ports also are blocked from accessing anything but the current dns resolved mail lists. remember recently when the FBI website had their mail server hacked and it was redirecting this is my idea of what a firewall could do to stop that type of hack)

So if you really want to go crazy you can make manual entries for the NAT to be port specific also. Or just leave it auto configure.

(Image: Squidguard Rules, make sure you add the loopback into the group acl and let it have access to redirectors as it redirects ports and needs to be able to do its job without Squidguard mistaking it.)

(Image: Squid proxy SSL certificate issue, mark you do not validate if you want to use the Clam AV splash screens and you use an SSL certificate that was created with Squid and not the firewall.) This creates problems as the SSL is checked like a normal website so it will normally block the splash screen for some reason)

(Image: Squid Proxy My Custom options, add your custom splice only rules I have Xbox, Amazon, Iphone) This means it will not SSL intercept only look at the get requests and headers for specific websites or devices, Amazon tablet if you have it to SSL intercept it will only work with the Silk browers for example, it needs splice to work with all the applications)

(Image: Make a locally stored no bump text file for really picky websites, as well as ethical needs like banks and private messages. We are only looking for viruses and websites we do not use normally. By need only. Must use regex)

(Image: Now you can see Everything is working now and blocking viruses from Google cloud, SSL intercept and Splice at the same time)

(Image: Squid Proxy make sure you add the loopback into what can access the proxy, as the loopback needs to access clam av)

(Image: Now my firewall runs the show, it protects everything the way it should. Notice the common name is my name, meaning the certificate for this ssl intercept is processed by the firewall)

(Image: Squid guard running like a champ and it can do url filtering on everything and virus checking on the URLS we do not normally see. notice https blocks)

(Image: NAT port forwarding, make sure you use port forwarding for DNS and NTP so the firewall cannot be bypassed with a different DNS)

(Image: SSL Intercept and Splice working. So, when you need to bump SSL it works when you need to splice it does that. Best of both worlds.)

PfSense and this community support team can make it work, just don't give up. Keep reading into it, you can even use the TAC support if you want to make a support ticket

[michmoor]

@jonathanlee wow....Bravo..You got a great functioning Proxy there. Incredible. 1.For custom options, where did you get the syntax?
2.Where did you get the sites that should be spliced(ignored)?
3. Your certificate is loaded on client devices but what do you do with IoT?

Seriously you did a great job here.

[JonathanLee]

@michmoor

1. The custom options are from the Squid website and the regex is used. Some was nice enough to share their no bump file with me, so I made my own with all my own websites that I need to not bump. I think Squid content accelerator software runs a large amount of the internet. I would love to learn more about it. let me know if you have any tips for me.
2. Every device I own have a certificate installed on them for this to work. Xbox does not have a certificate only uses WPAD that was the most confusing one for me. I only wanted to filter http get requests to block specific web URLS with that and it works.

[michmoor]

@jonathanlee I really would like to keep in touch with you. This was a great write up you gave here. The documentation is a bit sparse in getting the interception to work but your screenshots are solid.

[JonathanLee]

@michmoor thanks, it's amazing, I wish I understood more about the code. I did purchase an official version of the Netgate firewall. So, this was not the free version.

[JonathanLee]

@michmoor Thanks,

I sometimes have issues with the splash screen as it gets mixed up with the SSL certificate of the proxy over the pfSense at times,

(Image: Squid stopped a https virus before it got to my system, with splashscreen)

It does work good it's just you have to accept remote ssl with errors to make it CLAM AV work, I wish it would auto accept the SSL of the Squid and remote check the others. That is my main issue right now finding a way to make it say ignore the splashscreen SSL cert and go ahead and check the others. The Splashscreen SSL is an SSL that was made with open ssl inside of Squid proxy)

(NOTE: You must use the SSL certificate that is issued from SQUID proxy via command line and import it into the firewall, not the other way arround, I could never get it to work via the firewall into the proxy)

Meaning Squid is the CA "Certificate authority" And that certificate is used by Pfsense as an imported certificate.

I am going to test the upstream proxy certificate and set it to the pfsense certificate and see if that helps.

[michmoor]

@jonathanlee Hey Jon. Wanted to follow up here. I got WPAD working with the help of an Apache server. Took about 1hr of googling as im not a sysadmin but its working flawlessly.
During this process i discovered that SquidGuard does not work well with Transparent mode. This is the need for WPAD comes into play.
Really grateful for your help here.