Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Guide to filtering web content (http and https) with pfsense 2.3

    Scheduled Pinned Locked Moved
    Documentation
    54
    190
    213.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      aGeekhere
      last edited by

      Guide to filtering web content (http and https) with pfsense 2.3 updated 10 March 2018

      After seeing a lot of new users asking how to set up web filtering with pfsense I decided to create an extensive guide.

      This document is going to be broken down into 3 main parts

      1 Host overrides with DNS resolver
      2 Squid and squidguard filtering  Transparent vs Non Transparent proxy
      3 wpad

      Lets begin
      Enable DNS resolver
      Services/DNS/Resolver/General Settings
      Tic enable
      Save

      Now we are going to create a rule that will force the network to use our route as the DNS server.
      In Firewall/NAT/Port forward
      add a new rule

      Interface = LAN
      Protocol = TCP/UDP
      Source ports = *
      Dest address = *
      Dest ports = 53
      NAT IP = 127.0.0.1
      NAT Ports = 53
      Description = Redirect DNS
      LAN TCP/UDP * * * 53 127.0.0.1 53 Redirect DNS
      Save

      UPDATED
      Check that the new DNS rule is above the Default allow LAN to any rule in Firewall\Rules\LAN
      Now we are going to create some host overrides, the goal for the host overrides is to force google and bing to use there safe search feature.

      Click add under Host overrides
      Host = www
      Domain = bing.com
      IP =  204.79.197.220
      Description = bing
      Save

      Now bing is using safe search

      Update Youtube safe mode
      Click add under Host overrides
      Host = www
      Domain = youtube.com
      IP =  216.239.38.120
      Description = youtube
      Save
      NOTE: Safe search for youtube is not as advanced as google safe search, which results in a lot of safe content be filtered out.

      Now for google, because google has many different domains it would take a very long time to fill them all in, so we are going to create a short cut.

      Ssh into the router
      type 8
      cd /
      cd var/unbound
      vi forcegoogle.conf
      leave blank for now
      save (wq)

      Go to Diagnostics/Edit File
      click browse
      click var
      click unbound
      now you should see a file called forcegoogle.conf, click it

      enter the following

      local-data: "www.google.ad A 216.239.38.120"
local-data: "www.google.ae A 216.239.38.120"
local-data: "www.google.com A 216.239.38.120"
local-data: "www.google.com.af A 216.239.38.120"
local-data: "www.google.com.ag A 216.239.38.120"
local-data: "www.google.com.ai A 216.239.38.120"
local-data: "www.google.al A 216.239.38.120"
local-data: "www.google.am A 216.239.38.120"
local-data: "www.google.co.ao A 216.239.38.120"
local-data: "www.google.com.ar A 216.239.38.120"
local-data: "www.google.as A 216.239.38.120"
local-data: "www.google.at A 216.239.38.120"
local-data: "www.google.com.au A 216.239.38.120"
local-data: "www.google.az A 216.239.38.120"
local-data: "www.google.ba A 216.239.38.120"
local-data: "www.google.com.bd A 216.239.38.120"
local-data: "www.google.be A 216.239.38.120"
local-data: "www.google.bf A 216.239.38.120"
local-data: "www.google.bg A 216.239.38.120"
local-data: "www.google.com.bh A 216.239.38.120"
local-data: "www.google.bi A 216.239.38.120"
local-data: "www.google.bj A 216.239.38.120"
local-data: "www.google.com.bn A 216.239.38.120"
local-data: "www.google.com.bo A 216.239.38.120"
local-data: "www.google.com.br A 216.239.38.120"
local-data: "www.google.bs A 216.239.38.120"
local-data: "www.google.bt A 216.239.38.120"
local-data: "www.google.co.bw A 216.239.38.120"
local-data: "www.google.by A 216.239.38.120"
local-data: "www.google.com.bz A 216.239.38.120"
local-data: "www.google.ca A 216.239.38.120"
local-data: "www.google.cd A 216.239.38.120"
local-data: "www.google.cf A 216.239.38.120"
local-data: "www.google.cg A 216.239.38.120"
local-data: "www.google.ch A 216.239.38.120"
local-data: "www.google.ci A 216.239.38.120"
local-data: "www.google.co.ck A 216.239.38.120"
local-data: "www.google.cl A 216.239.38.120"
local-data: "www.google.cm A 216.239.38.120"
local-data: "www.google.cn A 216.239.38.120"
local-data: "www.google.com.co A 216.239.38.120"
local-data: "www.google.co.cr A 216.239.38.120"
local-data: "www.google.com.cu A 216.239.38.120"
local-data: "www.google.cv A 216.239.38.120"
local-data: "www.google.com.cy A 216.239.38.120"
local-data: "www.google.cz A 216.239.38.120"
local-data: "www.google.de A 216.239.38.120"
local-data: "www.google.dj A 216.239.38.120"
local-data: "www.google.dk A 216.239.38.120"
local-data: "www.google.dm A 216.239.38.120"
local-data: "www.google.com.do A 216.239.38.120"
local-data: "www.google.dz A 216.239.38.120"
local-data: "www.google.com.ec A 216.239.38.120"
local-data: "www.google.ee A 216.239.38.120"
local-data: "www.google.com.eg A 216.239.38.120"
local-data: "www.google.com.et A 216.239.38.120"
local-data: "www.google.fi A 216.239.38.120"
local-data: "www.google.com.fj A 216.239.38.120"
local-data: "www.google.fm A 216.239.38.120"
local-data: "www.google.fr A 216.239.38.120"
local-data: "www.google.ga A 216.239.38.120"
local-data: "www.google.ge A 216.239.38.120"
local-data: "www.google.gg A 216.239.38.120"
local-data: "www.google.com.gh A 216.239.38.120"
local-data: "www.google.com.gi A 216.239.38.120"
local-data: "www.google.gl A 216.239.38.120"
local-data: "www.google.gm A 216.239.38.120"
local-data: "www.google.gp A 216.239.38.120"
local-data: "www.google.gr A 216.239.38.120"
local-data: "www.google.com.gt A 216.239.38.120"
local-data: "www.google.gy A 216.239.38.120"
local-data: "www.google.com.hk A 216.239.38.120"
local-data: "www.google.hn A 216.239.38.120"
local-data: "www.google.hr A 216.239.38.120"
local-data: "www.google.ht A 216.239.38.120"
local-data: "www.google.hu A 216.239.38.120"
local-data: "www.google.co.id A 216.239.38.120"
local-data: "www.google.ie A 216.239.38.120"
local-data: "www.google.co.il A 216.239.38.120"
local-data: "www.google.im A 216.239.38.120"
local-data: "www.google.co.in A 216.239.38.120"
local-data: "www.google.iq A 216.239.38.120"
local-data: "www.google.is A 216.239.38.120"
local-data: "www.google.it A 216.239.38.120"
local-data: "www.google.je A 216.239.38.120"
local-data: "www.google.com.jm A 216.239.38.120"
local-data: "www.google.jo A 216.239.38.120"
local-data: "www.google.co.jp A 216.239.38.120"
local-data: "www.google.co.ke A 216.239.38.120"
local-data: "www.google.com.kh A 216.239.38.120"
local-data: "www.google.ki A 216.239.38.120"
local-data: "www.google.kg A 216.239.38.120"
local-data: "www.google.co.kr A 216.239.38.120"
local-data: "www.google.com.kw A 216.239.38.120"
local-data: "www.google.kz A 216.239.38.120"
local-data: "www.google.la A 216.239.38.120"
local-data: "www.google.com.lb A 216.239.38.120"
local-data: "www.google.li A 216.239.38.120"
local-data: "www.google.lk A 216.239.38.120"
local-data: "www.google.co.ls A 216.239.38.120"
local-data: "www.google.lt A 216.239.38.120"
local-data: "www.google.lu A 216.239.38.120"
local-data: "www.google.lv A 216.239.38.120"
local-data: "www.google.com.ly A 216.239.38.120"
local-data: "www.google.co.ma A 216.239.38.120"
local-data: "www.google.md A 216.239.38.120"
local-data: "www.google.me A 216.239.38.120"
local-data: "www.google.mg A 216.239.38.120"
local-data: "www.google.mk A 216.239.38.120"
local-data: "www.google.ml A 216.239.38.120"
local-data: "www.google.com.mm A 216.239.38.120"
local-data: "www.google.mn A 216.239.38.120"
local-data: "www.google.ms A 216.239.38.120"
local-data: "www.google.com.mt A 216.239.38.120"
local-data: "www.google.mu A 216.239.38.120"
local-data: "www.google.mv A 216.239.38.120"
local-data: "www.google.mw A 216.239.38.120"
local-data: "www.google.com.mx A 216.239.38.120"
local-data: "www.google.com.my A 216.239.38.120"
local-data: "www.google.co.mz A 216.239.38.120"
local-data: "www.google.com.na A 216.239.38.120"
local-data: "www.google.com.nf A 216.239.38.120"
local-data: "www.google.com.ng A 216.239.38.120"
local-data: "www.google.com.ni A 216.239.38.120"
local-data: "www.google.ne A 216.239.38.120"
local-data: "www.google.nl A 216.239.38.120"
local-data: "www.google.no A 216.239.38.120"
local-data: "www.google.com.np A 216.239.38.120"
local-data: "www.google.nr A 216.239.38.120"
local-data: "www.google.nu A 216.239.38.120"
local-data: "www.google.co.nz A 216.239.38.120"
local-data: "www.google.com.om A 216.239.38.120"
local-data: "www.google.com.pa A 216.239.38.120"
local-data: "www.google.com.pe A 216.239.38.120"
local-data: "www.google.com.pg A 216.239.38.120"
local-data: "www.google.com.ph A 216.239.38.120"
local-data: "www.google.com.pk A 216.239.38.120"
local-data: "www.google.pl A 216.239.38.120"
local-data: "www.google.pn A 216.239.38.120"
local-data: "www.google.com.pr A 216.239.38.120"
local-data: "www.google.ps A 216.239.38.120"
local-data: "www.google.pt A 216.239.38.120"
local-data: "www.google.com.py A 216.239.38.120"
local-data: "www.google.com.qa A 216.239.38.120"
local-data: "www.google.ro A 216.239.38.120"
local-data: "www.google.ru A 216.239.38.120"
local-data: "www.google.rw A 216.239.38.120"
local-data: "www.google.com.sa A 216.239.38.120"
local-data: "www.google.com.sb A 216.239.38.120"
local-data: "www.google.sc A 216.239.38.120"
local-data: "www.google.se A 216.239.38.120"
local-data: "www.google.com.sg A 216.239.38.120"
local-data: "www.google.sh A 216.239.38.120"
local-data: "www.google.si A 216.239.38.120"
local-data: "www.google.sk A 216.239.38.120"
local-data: "www.google.com.sl A 216.239.38.120"
local-data: "www.google.sn A 216.239.38.120"
local-data: "www.google.so A 216.239.38.120"
local-data: "www.google.sm A 216.239.38.120"
local-data: "www.google.sr A 216.239.38.120"
local-data: "www.google.st A 216.239.38.120"
local-data: "www.google.com.sv A 216.239.38.120"
local-data: "www.google.td A 216.239.38.120"
local-data: "www.google.tg A 216.239.38.120"
local-data: "www.google.co.th A 216.239.38.120"
local-data: "www.google.com.tj A 216.239.38.120"
local-data: "www.google.tk A 216.239.38.120"
local-data: "www.google.tl A 216.239.38.120"
local-data: "www.google.tm A 216.239.38.120"
local-data: "www.google.tn A 216.239.38.120"
local-data: "www.google.to A 216.239.38.120"
local-data: "www.google.com.tr A 216.239.38.120"
local-data: "www.google.tt A 216.239.38.120"
local-data: "www.google.com.tw A 216.239.38.120"
local-data: "www.google.co.tz A 216.239.38.120"
local-data: "www.google.com.ua A 216.239.38.120"
local-data: "www.google.co.ug A 216.239.38.120"
local-data: "www.google.co.uk A 216.239.38.120"
local-data: "www.google.com.uy A 216.239.38.120"
local-data: "www.google.co.uz A 216.239.38.120"
local-data: "www.google.com.vc A 216.239.38.120"
local-data: "www.google.co.ve A 216.239.38.120"
local-data: "www.google.vg A 216.239.38.120"
local-data: "www.google.co.vi A 216.239.38.120"
local-data: "www.google.com.vn A 216.239.38.120"
local-data: "www.google.vu A 216.239.38.120"
local-data: "www.google.ws A 216.239.38.120"
local-data: "www.google.rs A 216.239.38.120"
local-data: "www.google.co.za A 216.239.38.120"
local-data: "www.google.co.zm A 216.239.38.120"
local-data: "www.google.co.zw A 216.239.38.120"
local-data: "www.google.cat A 216.239.38.120"

      save

      Go to Services/DNS/Resolver/General Settings
      in custom option enter

      server:
include: /var/unbound/forcegoogle.conf

      save
      now google should be using safe mode.

      Part 2
      Install squid and squidguard in System/PackageManager/Available Packages

      Now we are going to talk about transparent proxy vs non transparent proxy.
      https://doc.pfsense.org/index.php/Setup_Squid_as_a_Transparent_Proxy

      Transparent proxy for http is very easy to set up, you just enable Transparent HTTP Proxy in squid (and install the blacklist in squidguard but I will get to that later). Now all traffic should be going to your proxy server on port 3128. However, if you want to filter https then this is where it gets complicated, you have to enable SSL Man In the Middle Filtering and create Certificates and even after that you may get connection errors and all sorts of issues.

      UPDATE
      You can try setting up MITM by setting the SSL/MITM Mode to splice all, that way you do not need to create a certificate for each device on the network. (you still need to create a main certificate though)

      So in this guide we are going to use a Non Transparent with wpad which will filter http and https content.
      Update
      I found that we can use both a transperrent proxy for port 80 and a wpad for 443 https content (UPDATE or you can use splice all in MITM), the wpad will be setup to use port 80 and 443. The transperrent proxy is going to catch every thing that the wpad misses, enable transperrent proxy in squid once you have the wpad setup.

      First we are going to setup squidguard
      Update
      In squidguard under General settings
      Tic enable
      Tic Enable log
      Tic Enable log rotation
      Tic enable blacklist
      Under Blacklist URL add http://www.shallalist.de/Downloads/shallalist.tar.gz
      Save
      apply (you must always hit apply for any changes you made to squidguard).

      In Package/Proxy filter SquidGuard: General settings/General settings
      click blacklist
      enter http://www.shallalist.de/Downloads/shallalist.tar.gz
      download
      wait to finish

      Now we are going to create a new target category.
      click Target categories (Do not skip this step).
      This will be a white list.
      add
      name whitelist
      description whitelist

      Because google and bing are the only search engines (as of writing) that can force safes search we are going to block all other search engines except google and bing, white list google and bing
      Domain list

      NOTE NOT ALL ADDED YET FOR GOOGLE
      Trying to fix google domains like play.google.com accounts.google.com mail.google.com and sites like www.google.com/contacts from getting blocked       Fixed

      
google.ac google.ad google.ae google.al google.am google.as google.at google.az google.ba google.be google.bf google.bg google.bi google.bj google.bs google.bt google.by google.ca google.cat google.cd google.cf google.cg google.ch google.ci google.cl google.cm google.cn google.co.ao google.co.bw google.co.ck google.co.cr google.co.hu google.co.id google.co.il google.co.in google.co.je google.co.jp google.co.ke google.co.kr google.co.ls google.com google.co.ma google.com.af google.com.ag google.com.ai google.com.ar google.com.au google.com.bd google.com.bh google.com.bn google.com.bo google.com.br google.com.bz google.com.co google.com.cu google.com.cy google.com.do google.com.ec google.com.eg google.com.et google.com.fj google.com.gh google.com.gi google.com.gr google.com.gt google.com.hk google.com.jm google.com.kh google.com.kw google.com.lb google.com.ly google.com.mm google.com.mt google.com.mx google.com.my google.com.na google.com.nf google.com.ng google.com.ni google.com.np google.com.om google.com.pa google.com.pe google.com.pg google.com.ph google.com.pk google.com.pr google.com.py google.com.qa google.com.sa google.com.sb google.com.sg google.com.sl google.com.sv google.com.tj google.com.tr google.com.tw google.com.ua google.com.uy google.com.vc google.com.vn google.co.mz google.co.nz google.co.th google.co.tz google.co.ug google.co.uk google.co.uz google.co.ve google.co.vi google.co.za google.co.zm google.co.zw google.cv google.cz google.de google-directory.co.uk google.dj google.dk google.dm google.dz google.ee google.es google.fi google.fm google.fr google.ga google.ge google.gg google.gl google.gm google.gp google.gr google.gy google.hn google.hr google.ht google.hu google.ie google.im google.iq google.is google.it google.je google.jo google.kg google.ki google.kz google.la google.li google.lk google.lt google.lu google.lv google.md google.me google.mg google.mk google.ml google.mn google.ms google.mu google.mv google.mw google.ne google.nl google.no google.nr google.nu google.off.ai googlepirate.com google.pl google.pn google.ps google.pt google.ro google.rs google.ru google.rw google.sc google.se google.sh google.si google.sk google.sm google.sn google.so google.sr google.st google.td google.tg google.tk google.tl google.tm google.tn google.to google.tt google.uz google.vg google.vu google.ws bing.com

      save

      click Common ACL
      click the plus button
      target categories whitelist access whitelist
      [blk_BL_searchengines] access deny
      Default access [all] allow

      To block ads (including on android and ios)
      [blk_BL_adv] access deny

      To block proxy sites
      [blk_BL_anonvpn] access deny
      Read though all the other categories and deny the ones you want

      next click Do not allow IP-Addresses in URL (If this causes issues deselect it)
      use safe search engines no longer works however you can click it as well.
      Save
      click General settings
      click Apply
      click Save

      If you want you can do a quick test by setting up your pc to use the proxy and see how thing are working.

      Part 3
      Now we are going to set up a wpad read more here about wpad https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid
      ssh in to pfsense
      8
      cd /
      create the wpad.da file
      vi /usr/local/www/wpad.da
      wq

      Create two new symbolic link files

      ln -s /usr/local/www/wpad.da /usr/local/www/wpad.dat
ln -s /usr/local/www/wpad.da /usr/local/www/proxy.pac

      Then go Diagnostics /Edit File
      click browse
      user
      local
      www
      click wpad.da
      add

      function FindProxyForURL(url, host) 
{ 
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
        return "DIRECT";

    return "PROXY 192.168.1.1:3128";
}

      save

      If you connect to a VPN you need to go direct for the VPN instead of the proxy, Remember you need to add the correct network class for the VPN  either A, B or C

      function FindProxyForURL(url, host) 
{ 
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "192.168.1.0",  "255.255.255.0"))
        return "DIRECT";

        if (isInNet(dnsResolve(host), "1.0.0.0",  "255.0.0.0" ))
        { return "DIRECT"; }

    return "PROXY 192.168.1.1:3128";
}

      save
      Go to Configure DNS Resolver add new host overrides
      Host: wpad
      Domain: mylocaldomain.local
      IP Address: 192.168.1.1
      Description: WPAD Autoconfigure Host
      save
      Next go to Services: DHCP server under Additional BOOTP/DHCP Options
      add

      number: 252 type: string value: "http://192.168.1.1/wpad.dat"
number: 252 type: string value: "http://192.168.1.1/wpad.da"
number: 252 type: string value: "http://192.168.1.1/proxy.pac"

      save

      set pfsense Protocol to http (This is a MUST, it will not work if you do not do this)
      System: Advanced: Admin Access Protocol http

      To stop users from bypassing your proxy setup two new firewall lan rule and block port 80 and 443
      IPv4 TCP * * * 80 * none
      IPv4 TCP * * * 443 * none
      Save

      Set your system to automatically detect settings (for windows it is in internet options connections lan settings).

      You also have to set up the proxy setting for each program that cant connect (firefox, graphics drive software, vlc etc)

      If you have programs that cannot connect and have no proxy setting you need to setup a firewall aliases
      Firewall/Aliases/IP
      and add the destination server ip (use wire shark to help find the blocked Ips or in your firewall block rule enable Log packets that are handled by this rule, use http://ip-lookup.net/index.php to check what it is and add to the Aliases. If it is part of a domain add the domain)
      now create a new firewall lan rule
      IPv4 TCP * * * passAliases 80* pass rule.
      IPv4 TCP * * * passAliases 443 * pass rule.

      Save

      A note on smart phones (android, IOS, etc)
      With android (not sure on other smart phones OS) you can not set it so that all the apps on the device use the proxy (not without rooting and other hacks), web browsers (google) will work fine using the proxy (if set in wireless connection options) but not apps or things like google play, so unless there is an option to use proxy for all apps on the device the most practical option here is just to allow smart phones to use port 80 and 443.

      UPDATE 24 JUNE 2016
      I have found that if you have connection issues using auto config for android or other smart phones try manually setting the proxy, now opening port 80 and 443 is not needed.

      Now we should have pfsense all set up for web filtering. I hope this has been helpful and thanks to everyone on the forum who has help me in creating this guide.

      Just a note for any specific issues with squid, squidguard or dns please create a new topic in the correct areas of the forum and link it here if needed

      Never Fear, A Geek is Here!

      High_VoltageH 1 Reply Last reply Reply Quote 1
      • S
        stilez
        last edited by

        One step you missed:

        make a symbolic link between the file
        Then go Diagnostics /Edit File

        You didn't say which file (and which target) to make the symbolic link, or the command you use for it. Some people might need to know.

        1 Reply Last reply Reply Quote 0
        • A
          aGeekhere
          last edited by

          Thanks, will update it soon.

          Done

          Never Fear, A Geek is Here!

          1 Reply Last reply Reply Quote 0
          • S
            stilez
            last edited by

            Is "click wpad.da" and other "wpad.da" a typo?
            If it's correct, it might be worth commenting after it that you do mean "da" not "dat", because having two files called wpad.da and wpad.dat might not be noticed, looks like a typo, is confusing, etc :)

            1 Reply Last reply Reply Quote 0
            • A
              aGeekhere
              last edited by

              Hi, there are 3 wpad files
              wpad.dat
              wpad.da
              proxy.pac
              https://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol

              I made the wpad.da the main file you edit and made a symbolic link for wpad.dat and proxy.pac (so all you need to do is just edit the wpad.da file).
              If this is still confusing me know and I will update the guide.

              Never Fear, A Geek is Here!

              1 Reply Last reply Reply Quote 1
              • A
                agixdota
                last edited by

                thanks for the guide  ;)

                1 Reply Last reply Reply Quote 0
                • D
                  Dogfish
                  last edited by

                  Thank you for the guide!

                  Some guides have a step to assign MIME file types eg: ".wpad"    =>  "application/x-ns-proxy-autoconfig",

                  Is this not necessary if using the webconfig http file server?

                  1 Reply Last reply Reply Quote 0
                  • A
                    aGeekhere
                    last edited by

                    Some guides have a step to assign MIME file types eg: ".wpad"    =>  "application/x-ns-proxy-autoconfig",

                    Is this not necessary if using the webconfig http file server?

                    I do not believe it is needed.

                    Never Fear, A Geek is Here!

                    1 Reply Last reply Reply Quote 0
                    • A
                      AR15USR
                      last edited by

                      Can you please expand on the "use our route as the DNS server." part? Your list of settings is not very complete compared to the options available in the WebUI. Maybe be a little more specific, I'm counting 13 or so available line items available.

                      edit: Also, regarding the new firewall rule, which level on the Firewall list hierarchy do we here to put this new rule? If we put it at the top will we not block ourselves out?

                      Also, how/where does the certs for https inspection get created/used?

                      Thanks

                      2.6.0-RELEASE

                      1 Reply Last reply Reply Quote 0
                      • A
                        aGeekhere
                        last edited by

                        Can you please expand on the "use our route as the DNS server." part?

                        Sure, since we enabled the DNS resolver to do Host overrides to force safe search on a few search engines I also set up a rule that will force the local network to use the pfsense router as the DNS server. The pfsense router will cache DNS addresses and use them instead of calling a DNS server, you can also change the cache size.
                        Read more here https://doc.pfsense.org/index.php/Unbound_DNS_Resolver

                        Your list of settings is not very complete compared to the options available in the WebUI. Maybe be a little more specific, I'm counting 13 or so available line items available.

                        Depending on your setup this could change, this is what I use
                        Tic Enable DNS resolver
                        Listen Port default which is 53
                        Network Interfaces all
                        Outgoing Network Interfaces all
                        System Domain Local Zone Type transperrent
                        Tic Enable DNSSEC Support
                        Untic Enable Forwarding Mode
                        Tic Register DHCP leases in the DNS Resolver
                        Tic Register DHCP static mappings in the DNS Resolver

                        For advance settings
                        Tic
                        Tic
                        Tic
                        Tic
                        Tic
                        Set cache size (I set 100MB)
                        10
                        10
                        4096
                        513
                        200
                        86400
                        0
                        15 min
                        20000
                        Disable
                        1
                        Untic
                        Untic

                        Also, regarding the new firewall rule, which level on the Firewall list hierarchy do we here to put this new rule? If we put it at the top will we not block ourselves out?

                        It should be just under the anti block out rule, so the second rule

                        Also, how/where does the certs for https inspection get created/used?

                        WPAD does not use certs, as there is no man in the middle attack

                        Hope this helps

                        Never Fear, A Geek is Here!

                        1 Reply Last reply Reply Quote 0
                        • A
                          AR15USR
                          last edited by

                          @aGeekHere:

                          Thanks aGeek

                          Actually I was referring to the "Firewall/NAT/Port forward" settings but I got it figured out anyhow.

                          WPAD does not use certs, as there is no man in the middle attack

                          Hope this helps

                          After using your guide, can I enable the MITM section in Squid, step up the CA and certs and it should then inspect https?

                          Also, under System/General/DNS Server Settings should we have/not have any DNS servers listed? I'm guessing we need some listed as this is where pfsense will get its DNS cache from?

                          2.6.0-RELEASE

                          1 Reply Last reply Reply Quote 0
                          • A
                            aGeekhere
                            last edited by

                            After using your guide, can I enable the MITM section in Squid, step up the CA and certs and it should then inspect https?

                            No, the wpad is doing that, the reason why we are using a wpad is so we do not have to use certs. So no transperrent proxy or man in the middle. Do not enable these options!

                            Also, under System/General/DNS Server Settings should we have/not have any DNS servers listed? I'm guessing we need some listed as this is where pfsense will get its DNS cache from?

                            You should include your isp dns (there is an option for that) and add the fastest DNS servers in your area, fill the list up, the DNS cache will get build up as you search the net, it does not just download every DNS address from the DNS server, you would run out of ram if it did.

                            I hope that I have cleared a few things up, have you got your setup working?

                            Never Fear, A Geek is Here!

                            1 Reply Last reply Reply Quote 0
                            • A
                              AR15USR
                              last edited by

                              @aGeekHere:

                              After using your guide, can I enable the MITM section in Squid, step up the CA and certs and it should then inspect https?

                              No, the wpad is doing that, the reason why we are using a wpad is so we do not have to use certs. So no transperrent proxy or man in the middle. Do not enable these options!

                              Also, under System/General/DNS Server Settings should we have/not have any DNS servers listed? I'm guessing we need some listed as this is where pfsense will get its DNS cache from?

                              You should include your isp dns (there is an option for that) and add the fastest DNS servers in your area, fill the list up, the DNS cache will get build up as you search the net, it does not just download every DNS address from the DNS server, you would run out of ram if it did.

                              I hope that I have cleared a few things up, have you got your setup working?

                              no, not working. As soon as I enable the firewall rule there is no access to the internet. With the firewall rule disabled I have access but the eicar test files do not get blocked like they did when in transparent mode indicating the proxy is not being used.

                              2.6.0-RELEASE

                              1 Reply Last reply Reply Quote 0
                              • A
                                asterix
                                last edited by

                                Can't get HTTPS scanning to work. See https sites passing through squid but clamd won't scan the files. Works fine on HTTP and it catches the virus files. Using proxy setting in browser since wpad isn't giving the results I am expecting.

                                1 Reply Last reply Reply Quote 0
                                • A
                                  aGeekhere
                                  last edited by

                                  no, not working. As soon as I enable the firewall rule there is no access to the internet. With the firewall rule disabled I have access but the eicar test files do not get blocked like they did when in transparent mode indicating the proxy is not being used.

                                  Is that the port 80 and 443 block rule? If so than that is correct, now set your device to auto configure proxy. In windows go to global internet settings and there is an option for that, and for each program you have set its proxy settings. For programs with no proxy settings create a pass rule.

                                  Can't get HTTPS scanning to work. See https sites passing through squid but clamd won't scan the files. Works fine on HTTP and it catches the virus files. Using proxy setting in browser since wpad isn't giving the results I am expecting.

                                  Hi, first try above post, second I have not tried clamd scanning because I have found it to have issues, best to ask this question in the proxy forum.

                                  Using proxy setting in browser since wpad isn't giving the results I am expecting

                                  You have to tell your browser to use system settings and in global internet settings set proxy to auto configure.

                                  Never Fear, A Geek is Here!

                                  1 Reply Last reply Reply Quote 0
                                  • A
                                    AR15USR
                                    last edited by

                                    @Asterix:

                                    Can't get HTTPS scanning to work. See https sites passing through squid but clamd won't scan the files. Works fine on HTTP and it catches the virus files. Using proxy setting in browser since wpad isn't giving the results I am expecting.

                                    Same issue here now.

                                    I enabled the 80-443 block rule, unchecked the 'Transparent" option in Squid and I can only get access if I manually enter the wpad.dat location into my local computer(s) settings. Auto discovery does not work. I'm on all Apple computers/devices here btw. Same issue with ClamAV, it scans http but not https as poster above.

                                    2.6.0-RELEASE

                                    1 Reply Last reply Reply Quote 0
                                    • A
                                      asterix
                                      last edited by

                                      Auto config works for me. What I meant in the original post was that the WPAD direction info I was writing was not proper hence was using straight proxy settings for the time being. Will be experimenting with the WPAD file at a later time.

                                      I seriously doubt HTTPS scanning (not filtering) with clamd is working. I have followed the configuration directions to the T and yet the only thing which does not work is https clamd scans for viruses. I believe its a Squid issue.

                                      1 Reply Last reply Reply Quote 0
                                      • A
                                        aGeekhere
                                        last edited by

                                        Post a link to the https fake virus test file that you are testing and I will see if it works for me.

                                        Never Fear, A Geek is Here!

                                        1 Reply Last reply Reply Quote 0
                                        • A
                                          AR15USR
                                          last edited by

                                          http://www.eicar.org/85-0-Download.html

                                          There is one group for http, and one group for https.

                                          2.6.0-RELEASE

                                          1 Reply Last reply Reply Quote 0
                                          • A
                                            asterix
                                            last edited by

                                            There are 4 https links

                                            https://secure.eicar.org/eicar.com
                                            https://secure.eicar.org/eicar.com.txt
                                            https://secure.eicar.org/eicar_com.zip
                                            https://secure.eicar.org/eicarcom2.zip

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post