Guide to configure squid, squidguard, https?



  • I have tried several guides to configure squid guard and dansguardian without success. .

    I managed to get squid guard to filter most porn, but Google images is still unfiltered. Also, I tried to add an exception IP address to bypass the proxy however it would not bypass the filtered sites..

    My goal is to filter the porn from my kids computers using a transparent proxy ( oldest is in high school now and will probably try to bypass it) and block porn in Google images.

    I did try dansguardian but could not get it to work at all..

    If someone could suggest a suitable setup and a guide, I would be grateful  ;D



  • Did you search? Let me rephrase that..  ;)  did you happen to see this guide on configuring dans with squid?

    https://forum.pfsense.org/index.php?topic=47856.0



  • Since a lot of sites have gone HTTPS, you will need to configure Squid to process HTTPS.  Are you doing that or only doing HTTP?



  • I have yet to see a guide for configuring Squid to process HTTPS. I know it involves creating a root CA on your box, have done that for OpenVPN in the past but it doesn't work well as you need to add the certificate to the end clients (I may be incorrect here). I just don't want to do that for my end clients. If it were like transparent mode it would had been way more simpler to deploy



  • If it were like transparent mode it would had been way more simpler to deploy

    It's kind of the opposite, actually.  In transparent mode, you have to worry about certs on the client side to stop MitM warnings.  I ended up switching our config from transparent to standard, configured WPAD and blocked ports 80/443 on LAN.  Now all my clients use the proxy without any problems and I don't have to screw around with certificates on every client.  HTTPS just works.



  • @Asterix:

    Did you search? Let me rephrase that..  ;)  did you happen to see this guide on configuring dans with squid?

    https://forum.pfsense.org/index.php?topic=47856.0

    Yes, I tried this guide without success.. I could not get Dan's to filter anything.



  • @KOM:

    Since a lot of sites have gone HTTPS, you will need to configure Squid to process HTTPS.  Are you doing that or only doing HTTP?

    I will need to do HTTPS as Google images shows too much..



  • @KOM:

    If it were like transparent mode it would had been way more simpler to deploy

    It's kind of the opposite, actually.  In transparent mode, you have to worry about certs on the client side to stop MitM warnings.  I ended up switching our config from transparent to standard, configured WPAD and blocked ports 80/443 on LAN.  Now all my clients use the proxy without any problems and I don't have to screw around with certificates on every client.  HTTPS just works.

    And you have manually setup the proxy on each machine? I would prefer not to go this route..



  • WPAD is the way to go..  There are ways around it but its the easiest to block pages HTTP/HTTPS without setting up MILM and dealing with certs.



  • WPAD is the way to go..

    Yes, and it's simple.  The only thing you have to do on each client is ensure the browser is set to automatic proxy detection, which is already the default on all browsers these days as far as I know.  If you have a Windows AD structure then you can push down that setting through group policy if you wish.  Configure WPAD, block 80/443 on LAN and then troubleshoot any stray users who suddenly have no web access.

    https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid



  • What about non- PC and non-Mac based clients? Like Smart TV, Blu-ray player etc. I use video caching.



  • If their network config supports a proxy then you're set.  Otherwise, here is what I would do.  I already would have blocks on LAN for ports 80 and 443 to force everyone to use the proxy by denying direct access.  For the devices that don't support a proxy, I would add a rule just above the block rules allowing those IP addresses to talk directly on ports 80 and 443.  So the devices go straight out, all other LAN clients get blocked and the only way they can talk is via the proxy.



  • @Cino:

    WPAD is the way to go..  There are ways around it but its the easiest to block pages HTTP/HTTPS without setting up MILM and dealing with certs.

    +1



  • Thanks for all the suggestions… I will look around for a good guide for WPAD configuration.

    :)



  • I gave you the link for one in my previous post  ;D



  • @KOM:

    I gave you the link for one in my previous post  ;D

    Thanks, I did see that however I am not an expert like yourself  ;), I was looking for a comprehensive guide from A to Z to configure WPAD..

    Is this guide correct? :

    http://irj972.co.uk/articles/pfSense-WPAD-PAC-configuration

    Thanks



  • The guide I linked to was much easier for me to understand (I just did this whole exercise two weeks ago!) than the one you provided.  Really, it's a lot easier than you think:

    1.  Login to pfSense via SSH and go to /usr/local/www
    2.  Create wpad.dat and stuff it with:

    function FindProxyForURL(url,host)
    {
    return "PROXY Your.Proxy.IP.Address:3128";
    }

    3.  Copy wpad.dat to wpad.da and proxy.pac to support other auto-discovery methods
    4.  Create a DNS entry for wpad and point it to your pfSense LAN IP
    5.  Create a DHCP Option 252 entry and point it to your pfSense WPAD URL: http://Your_pfSense_LAN_IP:Port/wpad.dat
    6.  Add LAN rules that block port 80 and port 43

    Done.



  • Only problem with that setup, pfSense should be using port 443 for the WebGUI which means port 80 isn't listening anymore.  I've had look results adding v-host and dropping the wpad files into that folder for hosting.



  • pfSense should be using port 443 for the WebGUI which means port 80 isn't listening anymore

    And yet it works.  It probably listens on both but only serves on the specified port.



  • @KOM:

    pfSense should be using port 443 for the WebGUI which means port 80 isn't listening anymore

    And yet it works.  It probably listens on both but only serves on the specified port.

    Port 80 is closed when I enabled https and is free for another process to use.



  • Actually, I'm a moron.  While my test 2.2 box is HTTPS, my 2.1.5 box is HTTP.  Yes, if you have WebGUI set for HTTPS then you will need another web server to host the WPAD files.



  • Couple of questions:

    So if I put the wpad.dat files on a virtual webserver on my network I could leave pfsense GUI on 443?

    and

    If Squid is used in normal mode does it filter HTTPS without having to do the man in the middle thing?  I am trying to decide how to protect my kids as well…

    Thanks
    Chad



  • Chad,  if wpad stays out of pfsense box(or at least out of default web server)  you can leave it 443.

    While using active proxy,  you can filter https sites but not https urls without ssl interception.



  • Thanks marcelloc,

    I have gotten my setup working with wpad and was pretty happy but then found that minecraft would not use the proxy and failed to launch. I then setup a transparent proxy and everything works good but of course the kids can bypass the filter by going https. I then got https filtering working in transparent mode only to find that Minecraft won't buy my CA and therefore won't download it's package from Amazon.

    Is there a way to have traffic out to a specic website not go through the ssl interception?
    Anyone have a good solution?

    Chad



  • When you have your proxy in standard mode (non-transparent), you typically use firewall rules to block access from LAN to ports 80 and 443.  In your case, you want to add a rule above the block rule that specifically allows your Minecraft box to talk on 80/443 or whatever Minecraft uses for its updates.



  • While using transparent proxy,  create a host alias with  hosts/ips (not domains)  you want to bypass the transparent proxy  and then put this alias on squid general tab(bypass proxy to these Destination ips).



  • I thought i tried putting s3.amazonaws.com in the bypass proxy to these destinations without affect.  Is this the right format for the host alias?

    The problem with the firewall rule for standard mode is I would end up bypassing all of my kids computers from being forced through the proxy.  I did find something about launching minecraft via a shortcut to define the proxy so I may go back to trying standard mode if i cannot get the launcher to work with ssl interception.  So far I have installed my cert on all of our phones and machines and the transparent proxy is working really well.  It just seems to be minecraft that is causing problems.

    Thanks for the help.



  • I used GPO to push the cert to all domain computers, the pain was all the BYODs and had to manually import the cert. I just stuck it on the internal webserver and gave all instructions to get to it and import it. Wasn't that bad.

    My problem now is that Squid has problems with https sites that are using TLS 1.2.  Any one know how to fix that in the Squid-dev so I can continue to use MitM?

    Thanks,

    Jim



  • @KOM:

    The guide I linked to was much easier for me to understand (I just did this whole exercise two weeks ago!) than the one you provided.  Really, it's a lot easier than you think:

    1.  Login to pfSense via SSH and go to /usr/local/www
    2.  Create wpad.dat and stuff it with:

    function FindProxyForURL(url,host)
    {
    return "PROXY Your.Proxy.IP.Address:3128";
    }

    3.  Copy wpad.dat to wpad.da and proxy.pac to support other auto-discovery methods
    4.  Create a DNS entry for wpad and point it to your pfSense LAN IP
    5.  Create a DHCP Option 252 entry and point it to your pfSense WPAD URL: http://Your_pfSense_LAN_IP:Port/wpad.dat
    6.  Add LAN rules that block port 80 and port 43

    Done.

    I understand this however, is there a step by step guide? I am still learning BSD and Linux and don't want to make any mistakes..

    Thanks



  • KOM steps are a step by step guide.  The dhcp stuff you can configure on your active directory (Windows)  but you will also find good guides looking for wpad + dhcp configuration on forum, Google and/or youtube.



  • @marcelloc:

    While using transparent proxy,  create a host alias with  hosts/ips (not domains)  you want to bypass the transparent proxy  and then put this alias on squid general tab(bypass proxy to these Destination ips).

    I tried this again and realized I was missing a required host.  It works!  Minecraft needs the following bypassed:
    s3.amazonaws.com
    authserver.mojang.com

    Thanks again



  • The real time tab is really useful for these diagnostics  :)



  • Maybe this will help some for thecableguy.  From KOM guide which I followed:

    I put my wpad.dat on a virtual machine running ubuntu but the same stuff works in pfsense i believe.  There are a lot of blogs and posts that show how to use this stuff as well.

    1.  Login to pfSense via SSH and go to /usr/local/www
    I use Putty on windows to do this.  You have to turn SSH access on in pfsense

    2.  Create wpad.dat and stuff it with:
    you can also do this from inside pfsense using the "edit file" page
    or cd to the directory above and then use nano to create the file (hopefully nano is included in pfsense otherwise google how to use vi)

    #nano wpad.dat

    function FindProxyForURL(url,host)
    {
    return "PROXY Your.Proxy.IP.Address:3128";
    }

    3.  Copy wpad.dat to wpad.da and proxy.pac to support other auto-discovery methods
    #cp wpad.dat wpad.da
    #cp wpad.dat proxy.pac

    4.  Create a DNS entry for wpad and point it to your pfSense LAN IP

    5.  Create a DHCP Option 252 entry and point it to your pfSense WPAD URL: http://Your_pfSense_LAN_IP:Port/wpad.dat

    6.  Add LAN rules that block port 80 and port 43

    Done.



  • I can't get WPAD to work.

    What i did is installing vHosts and there i made this setting:

    Host: wpad.mydomain.net
    IP Address: 192.168.0.1
    Port: 80
    

    I uploaded .dat, .da and .pac to /usr/local/vhosts/wpad.mydomain.net
    I checked lighty-webConfigurator.conf and the mimetypes are there.

    webConfigurator is set to HTTPS and "Disable webConfigurator redirect rule"

    DNS Resolver Host Overrides has:

    Host: wpad
    Domain: mydomain.net
    IP: 192.168.0.1
    

    DHCP Server LAN Additional BOOTP/DHCP Options has:

    255 TEXT http://192.168.0.1:80/wpad.dat
    255 TEXT http://192.168.0.1:80/wpad.da
    255 TEXT http://192.168.0.1:80/proxy.pac
    

    I can download the files in the browser with wpad.mydomain.net/wpad.dat
    It also works if i add the path to the wpad.dat into the browser proxy settings
    but Auto-Proxy does not work.

    I also wonder about blocking port 80. If i do that no side is reachable anymore.

    And just as i wrote that i had the idea to check if 192.168.0.1:80/wpad.dat downloads
    the file in the browser - it doesn't. Maybe thats the problem?



  • A question about the wpad configuration and squid in standard mode with HTTPS.  Can you cache HTTPS pages?  Do you have to do the MitM and install the self signed certificate on the client workstations in order for you to cache HTTPS?



  • Yes and yes.  :)



  • @Cino:

    WPAD is the way to go..  There are ways around it but its the easiest to block pages HTTP/HTTPS without setting up MILM and dealing with certs.

    However, WPAD has some security vulnerabilities that will run Javascript. It doesn't matter if Javascript is blocked in the browser or switched off. You should prevent WPAD from executing someone else's code should the client fail to find wpad.dat at your intended location.

    For example…
    Your Squid proxy is on 192.168.1.1
    Your client device has a hostname of 'pc7'
    DHCP puts it into the domain 'pfsense.co.uk' so that it's FQDN is pc7.pfsense.co.uk

    Then you need these in your DNS forwarder before using WPAD

    wpad.pfsense.co.uk resolve to your proxy, 192.168.1.1
    wpad.co.uk  resolve to your proxy, 192.168.1.1
    wpad.uk resolve to your proxy, 192.168.1.1

    If you have additional search domains pushed by DHCP, say pfsense.org.uk, you will also need these...

    wpad.pfsense.org.uk resolve to your proxy, 192.168.1.1
    wpad.org.uk resolve to your proxy, 192.168.1.1

    Now that we have many new TLDs, we probably should also have...

    wapd resolve to your proxy, 192.168.1.1

    ...If a client device DHCP's outside of your cosy secure network into someone else's, WPAD will go looking for a wpad.dat anywhere down that domain's hierarchy.



  • Some help please :)