Squid + SquidGuard not reliable

MrIT

Hello everyone,

we are running a Netgate 7100 with the latest 22.05 version of pfSense+ at our school. We are deploying multiple wifi networks (faculty, students, guests). Our Netgate device manages their respective VLAN networks. During the school day, approx. 500 devices are connected accross all wifi networks.

To ensure unappropriate content is blocked, we are using Squid and Squidguard, which Squid running in transparent mode on all wifi networks (we also deploy WPAD but not all devices seem to support it properly).

In the past weeks, I notices the following things:

Although Squid is running, transparent mode does not seem to work. When I manually set the proxy, all traffic is handled properly by Squid. It seems that pfSense seens to loose some kind of redirect rule for the transparent mode?
Today I notices that SquidGuard seemed to hung up. The GUI showed that it is running, but every device received a CONNECTION_RESET error when navigating to any website. After clicking "Apply" SquidGuard worked fine (a restart of Squid did not work).

Did anyone else notices this kind of behavior? In the first half of the year we were running the same setup (but with 22.01) and we haven't had any issues at all.

Regards
Marcel

periko

@mrit hello, have u check Squid/SG logs?

Did u change SG settings? by default it handles 16 simultaneous connections, with the amount of users, there u can have a issue.

Important to watch your logs first...

Regards!!!

CZvacko

@periko Right, need adjust SquidGuard - Service options - Rewrite process (all 3 values)

MrIT

@periko I will try to increase the number of simultaneos connections.

But I think, I will spawn another pfSense instance as a virtual machine with much more power than our Netgate 7100 applicance.

JonathanLee

@mrit there is settings on Squidguard and Squid you need to increase.

Screen Shot 2022-10-11 at 11.08.37 PM.png
(Image: Squid Proxy Location)

Screen Shot 2022-10-11 at 11.11.13 PM.png
(Image: Squidguard location)

blucenasilva

Bom dia, na empresa que trabalho tenho cenário parecido com o seu, na verdade se você utilizar squid + squidguard e pretende restringir https em modo transparent com wpad apontando automaticamente o endereço e porta do proxy funciona muito bem com computadores mais smartphones e tablets android e iphones e ipads que usam ios não usam proxy portanto seu trafego https vai passar por fora do proxy e as regras de bloqueio não funcionaram.

Neste cenário o ideal é não usar proxy transparente e fazer com que os dispositivos funcionem nos roteadores wifi com apontamento do proxy, quem tentar passar por fora dele não conseguirá navegar.

Outro ponto é que as vezes dessa forma alguns serviços que usam https podem não funcionar adequadamente sendo necessário testar outros cenários para uma solução definitiva do problema.

michmoor

@mrit I still want to know what the logs show

JonathanLee

@blucenasilva said in Squid + SquidGuard not reliable:

Bom dia, na empresa que trabalho tenho cenário parecido com o seu, na verdade se você utilizar squid + squidguard e pretende restringir https em modo transparent com wpad apontando automaticamente o endereço e porta do proxy funciona muito bem com computadores mais smartphones e tablets android e iphones e ipads que usam ios não usam proxy portanto seu trafego https vai passar por fora do proxy e as regras de bloqueio não funcionaram.

Neste cenário o ideal é não usar proxy transparente e fazer com que os dispositivos funcionem nos roteadores wifi com apontamento do proxy, quem tentar passar por fora dele não conseguirá navegar.

Outro ponto é que as vezes dessa forma alguns serviços que usam https podem não funcionar adequadamente sendo necessário testar outros cenários para uma solução definitiva do problema.
Translate:
automatically the address and port of the proxy works very well with computers plus android smartphones and tablets and iphones and ipads that use ios do not use a proxy so your https traffic will pass outside the proxy and the blocking rules will not work.

In this scenario, the ideal is not to use a transparent proxy and make the devices work on wifi routers with proxy pointing, anyone who tries to bypass it will not be able to navigate.

Another point is that sometimes, in this way, some services that use https may not work properly, and it is necessary to test other scenarios for a definitive solution to the problem.

WPAD use resolves all non proxy systems, it will automatically assign the proxy, also use of NAT will fix any issues with bypassing traffic. NTP nat use to the proxy is a good example. I will add screenshots later

michmoor

@jonathanlee Anyone got a working WPAD script? I would like to toy around with this and see how scalable it is.

MrIT

Sorry for the late update: So I just created a virtual machine (Hyper-V, 6 Cores, 4 GB RAM) which purpose is to host Squid+SquidGuard. Turns out: I haven't had any issues at all with the transparent proxy.

I also tweaked some settings, but I have no clue if that helps with our performance (performance seems to be fine now):

Squid -> SSL Certificate Deamon Children -> 10
SquidGuard -> Rewrite process children -> 32
SquidGuard -> Rewrite process children startup -> 16
SquidGuard -> Rewrite process children idle -> 4

@michmoor Sorry, I cannot provide any logs because it seems to work fine now and Squid/SquidGuard was removed from the original pfSense (bare metal) instance.

@michmore I configure WPAD using DHCP, which seems to work with Windows and iOS users (Android does not seem to be able to use DHCP for WPAD*).

The address points to a local web service in our DMZ. This is my WPAD:

function FindProxyForURL(url,host) {
    # Important so protocols like FTP do not connect through Squid
    if(!url.startsWith("http:") && !url.startsWith("https:")) {
        return "DIRECT";
    }

    # This is used so apps like Spotify work which seem to make
    # outgoing connections on port 443, but TCP (no HTTPS)
    if(shExpMatch(url, "*://" + host + ":*")) {
        return "DIRECT";
    }

    var resolved_ip = dnsResolve(host);

    # If resolved IP address is local, connect directly
    if(isPlainHostName(host) || isInNet(resolved_ip, "10.0.0.0", "255.0.0.0") || isInNet(resolved_ip, "172.16.0.0", "255.240.0.0") || isInNet(resolved_ip, "192.168.0.0", "255.255.0.0") || isInNet(resolved_ip, "127.0.0.0", "255.255.255.0")) {
         return "DIRECT";
    }

    # Connect through Squid (or directly if Squid is not available)
    return "PROXY 10.XX.XX.XX:3128;DIRECT";
}

Regards

(*) I cannot verify this as of now. That's a memory from few months ago when I configured and tested WPAD.

blucenasilva

@jonathanlee

GNU nano 5.9 /usr/local/www/wpad.dat
function FindProxyForURL(url,host)
{
return "PROXY 192.168.240.1:3128";
}

MrIT

Just found out today: You can test WPAD when using Microsoft Edge (I suppose Chrome works as well?!) on Windows. If deployed and configured successfully, the you can open developer tools and switch to the "Network" tab. On all requests you'll see the proxy's IP adress as remote address for any website.

JonathanLee

@michmoor I have one i will share when I get home it's simple.

michmoor

@jonathanlee said in Squid + SquidGuard not reliable:

@michmoor I have one i will share when I get home it's simple.

Thanks!
Do you use a separate web server?

JonathanLee

@michmoor no my WPAD script is inside the firewall. I followed the Netgate website for reference.

JonathanLee

@michmoor

(Image: files named wpad.da, wpad.dat, and proxy.pac are locally stored they are all part of wpad and contain the same script)

(Image: Create and save text files as wpad.da, wpad.dat and proxy.pac)

(Image: Save all the WPAD files into path usr local www, this is so when any time port 80 is accessed it tells the system to use proxy)

(Image: All files should contain the following inside the text files wpad.da wpad.dat and proxy.pac this is the script)

(Image: You must also add a DNS resolver for the name of the firewall mine is home arpa to link in the ip address of the firewall mine is 192 168
1 1)

(Image: DNS resolve after adding)

(Image: You must also create an ACL for the wpad anything that hits port 80 will auto configure to the port of the proxy. For me I have a lan group that is all of my devices on my lan Lan_group * any destination the firewall)

(Image: ACL "Access Control Close line up")

(NAT)

(ACL with port specific and mail server specific rules)

(Image: Alias used for mail acl)

(Last three images explained: Manual NAT mine firewall approves mail and Xbox ports they are IP address specific; my mail ports also are blocked from accessing anything but the current dns resolved mail lists. remember recently when the FBI website had their mail server hacked and it was redirecting this is my idea of what a firewall could do to stop that type of hack)

So if you really want to go crazy you can make manual entries for the NAT to be port specific also. Or just leave it auto configure.

(Image: Squidguard Rules, make sure you add the loopback into the group acl and let it have access to redirectors as it redirects ports and needs to be able to do its job without Squidguard mistaking it.)

(Image: Squid proxy SSL certificate issue, mark you do not validate if you want to use the Clam AV splash screens and you use an SSL certificate that was created with Squid and not the firewall.) This creates problems as the SSL is checked like a normal website so it will normally block the splash screen for some reason)

(Image: Squid Proxy My Custom options, add your custom splice only rules I have Xbox, Amazon, Iphone) This means it will not SSL intercept only look at the get requests and headers for specific websites or devices, Amazon tablet if you have it to SSL intercept it will only work with the Silk browers for example, it needs splice to work with all the applications)

(Image: Make a locally stored no bump text file for really picky websites, as well as ethical needs like banks and private messages. We are only looking for viruses and websites we do not use normally. By need only. Must use regex)

(Image: Now you can see Everything is working now and blocking viruses from Google cloud, SSL intercept and Splice at the same time)

(Image: Squid Proxy make sure you add the loopback into what can access the proxy, as the loopback needs to access clam av)

(Image: Now my firewall runs the show, it protects everything the way it should. Notice the common name is my name, meaning the certificate for this ssl intercept is processed by the firewall)

(Image: Squid guard running like a champ and it can do url filtering on everything and virus checking on the URLS we do not normally see. notice https blocks)

(Image: NAT port forwarding, make sure you use port forwarding for DNS and NTP so the firewall cannot be bypassed with a different DNS)

(Image: SSL Intercept and Splice working. So, when you need to bump SSL it works when you need to splice it does that. Best of both worlds.)

PfSense and this community support team can make it work, just don't give up. Keep reading into it, you can even use the TAC support if you want to make a support ticket

michmoor

@jonathanlee wow....Bravo..You got a great functioning Proxy there. Incredible. 1.For custom options, where did you get the syntax?
2.Where did you get the sites that should be spliced(ignored)?
3. Your certificate is loaded on client devices but what do you do with IoT?

Seriously you did a great job here.

JonathanLee

@michmoor

The custom options are from the Squid website and the regex is used. Some was nice enough to share their no bump file with me, so I made my own with all my own websites that I need to not bump. I think Squid content accelerator software runs a large amount of the internet. I would love to learn more about it. let me know if you have any tips for me.
Every device I own have a certificate installed on them for this to work. Xbox does not have a certificate only uses WPAD that was the most confusing one for me. I only wanted to filter http get requests to block specific web URLS with that and it works.

michmoor

@jonathanlee I really would like to keep in touch with you. This was a great write up you gave here. The documentation is a bit sparse in getting the interception to work but your screenshots are solid.

JonathanLee

@michmoor thanks, it's amazing, I wish I understood more about the code. I did purchase an official version of the Netgate firewall. So, this was not the free version.