Squid and squidGuard - monitor websites and block content

AliasNeo

Hello,
I’m trying to make the following:

A) Monitor the websites our employees visit
B) Block content like pornography, torrent sites, gambling and so on…

To do so I need squid and Lightsquid for the first part and squidGuard for the second part.

What’s with the “Package Dependencies” listed below the Installed Packages? Do I have to install them too? For example, under squid there is squid_radius, squid-3.5.19_1 and so on?

Before I begin – can the B option work without squid and Lightsquid?

Solving A):
I followed the instructions from this site https://turbofuture.com/internet/Intercepting-HTTPS-Traffic-Using-the-Squid-Proxy-in-pfSense
First I tried it without SSL filtering, like on my old pfSense router – and every time I visited a HTTPS site I would get an error “Your connection is not secure. The owner of –LAN IP- has configured their website improperly”. So I made the CA certificate, as explained in the tutorial above, and installed it on a PC and enabled the SSL filtering – but it’s still not working. I get the same error.

Solving B):
I followed the https://doc.pfsense.org/index.php/SquidGuard_package and it isn’t blocking anything. And at this time I do think that something from A has to work before this part works as well. My configuration for this part is in the attachment.

Could someone help me out here? I can add aditional screenshots for solving A.

Thank you!

BlackList.png_thumb

![Common ACL.png_thumb](/public/imported_attachments/1/Common ACL.png_thumb)
![Common ACL.png](/public/imported_attachments/1/Common ACL.png)
![General settings.png_thumb](/public/imported_attachments/1/General settings.png_thumb)
![General settings.png](/public/imported_attachments/1/General settings.png)

AliasNeo

OK, I found this post https://forum.pfsense.org/index.php?topic=112335.0

that could solve my problem since it looks like it’s a common problem that SSL filtering is not working.

Gonna try that and report back.

KOM

The Cache/Proxy forum is here.

What’s with the “Package Dependencies” listed below the Installed Packages? Do I have to install them too?

No, they get installed for you.

can the B option work without squid and Lightsquid?

pfBlocker can do some blocking based on lists but I don't know about reporting. Squid + squidguard + lightsquid is the way to go.

Transparent mode is a royal pain in the ass. I find it much easier to use explicit mode along with WPAD.

Get squid working first before you start playing with squidguard. When you install squidguard, I think everything is blocked by default, so you would have to open up the default ACL and set it to Allow.

AliasNeo

Thank you for the advice.

This part confuses me:

function FindProxyForURL(url,host)
{
return "PROXY 192.168.1.1:3128";
}

Do I need to add the IP adress for each subnet I have on this list?

If you ever saw the post from "aGeekHere" https://forum.pfsense.org/index.php?topic=112335.0
Do you know if I can skip the first part from 3?

KOM

Do I need to add the IP adress for each subnet I have on this list?

You can if you like but you don't need to. I use this:

function FindProxyForURL(url,host)
{
// If the requested website is hosted within the internal network, send direct.
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
        isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
        isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
        isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
        return "DIRECT";
// Else use the proxy
    return "PROXY 10.10.4.1:3128";
}

This handles all private IP space.

AliasNeo

Ok and in your case

return "PROXY 10.10.4.1:3128";

That is just any lan address on your pfsense or?

KOM

It is the LAN IP used by pfSense, and the interface I have squid bound to via Services - Squid Proxy Server - General - Proxy Interface(s).

AliasNeo

There is this little problem and I will understand it all.

I have 4 interfaces, 3 for LAN with multiple "vlan - router on stick" subinterfaces and one WAN

And I access my pfsense from any lan I find my self in. So… the million dollar question for me is - which IP address do I use?

In the Squid Proxy General Proxy Interfaces I plan on CTRL select all interfaces, but which IP do I need for the WPAD file?

So with WPAD I'll get to see all the traffic in the light Squid log files?

Thank you again, a lot. Tomorrow I'm planning on putting this thing in motion.

KOM

In the Squid Proxy General Proxy Interfaces I plan on CTRL select all interfaces, but which IP do I need for the WPAD file?

Don't select all interfaces, just LAN and Loopback. As long as your VLANs can all access the LAN IP, use that.

So with WPAD I'll get to see all the traffic in the light Squid log files?

Yes.

AliasNeo

So even if I have 3 different NICs with 3 interfaces I should make one the "main lan" interface or make 3 different JavaScript codes in the same WPAD file for each NIC?

Hope I didn't cross the Lise with too many questions

KOM

Yes.

Here is how it works. Your LAN client will do a DNS lookup on wpad.YourDomain to get the IP address of your WPAD server which is just an HTTP server (not HTTPS). LAN client then contacts the HTTP server and requests (based on host OS) wpad.dat, proxy.pac or wpad.da. (Best practice is to support all three; Use symbolic links so that you don't have to duplicate the files.) It then parses the content of the wpad file to see where the proxy is and then directs all HTTP/S traffic to the specified proxy address & port. As long as your VLAN clients can access your pfSense LAN IP address and WPAD server, then they can use the proxy without any special magic. You can even use pfSense as the WPAD server if you have it running in HTTP mode.

AliasNeo

I have done most of what you have suggested but my traffic isn’t logged.
If you / or someone else / has some time to look at my config – it would be great.

For start, I want to make it work on one Interface (LAN) which has a few “vlan” subinterfaces.
The LAN interface is the physical interface on which I connect a Switch and it has an IP address of 192.168.130.1
So I made the wpad file like this:


function FindProxyForURL(url,host)
{
// If the requested website is hosted within the internal network, send direct.
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
        isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
        isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
        isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
        return "DIRECT";
// Else use the proxy
    return "PROXY 192.168.130.1:3128";
}

In Firewall/NAT/Port forward
I added a new rule

Interface = LAN
Protocol = TCP/UDP
Source ports = *
Dest address = *
Dest ports = 53
NAT IP = 127.0.0.1
NAT Ports = 53
Description = Redirect DNS
LAN TCP/UDP * * * 53 127.0.0.1 53 Redirect DNS

Do I have to apply this rule to the LAN interface or the subinterface for my VLAN?

In DNS Resolver I add new host override for host wpad, and in DHCP server under Additional BOOTP/DHCP Options
added the 252.

Now when I enter LightSquid I can see that something is logged. But it’s not really working, if I visit the Microsoft webpage the log will only show:
v10.vortex-win.data.microsoft.com:443
I can’t see the web pages, only from time to time /favicon.ico or www.msftconnecttest.com

I can ping my pfSense server via its hostname.mydomain.local and I can access the wpad file via browser, but something is not working. Please help :)

KOM

It looks like it's working to me. What were you expecting to see? Use the console option #8 to get to Shell. Go to /var/squid/logs. Do you have an access.log? Does it have any contents? Doe sit have today' date and time close to current? Run tail -f access.log and then browse somewhere. Does access.log reflect your browse session?

AliasNeo

Well if I visit https://www.microsoft.com/ I thought I would see www.microsoft.com on the list and not only some /favico.ico or v10.vortex-win.data.microsoft.com:443

Gonna return to work now to check the log as you instructed.

AliasNeo

Still no luck - it has content and todays date - but as I said earlier it only shows some strange parts of my browsing sessions - not the actual websites I visit.

It almost feels like it only logs the communication my Windows 10 is making in the background and not the actual traffic I make on the browser!

Edit: I read that the system can get the WPAD info from the dhcp, but browsers only from DNS. Could it be that the logs really only show my windows talking? Can I manually add the IP and port in my browser as Proxy and check?

KOM

Are your clients set to autodetect the proxy settings as shown in the image below using Windows as an example? What you should be doing is blocking off 80,443 on LAN to enforce the use of the proxy. Then there is no guessing. If 80,443 are blocked but you can still use your browser then the proxy is working. The URLs you use are often redirected to other domains. Vortex.blahblah.microsoft.com is definitely an MS Windows 10 telemetry domain.

InternetOptions.png_thumb

AliasNeo

Auto detect is enabled, but if I block those ports do I block them on the subinterface?
Because when I do nothing works I can't browse only ping…

EDIT: I got it working! It's logging everything - I'm not sure yet why but I'll check tomorrow and write back! Thanks again