Squid 3, Squidguard, SSL interception, Clam, WPAD + app store and windows update

dwood

I've been working on getting a functional pfsense box with Squid 3, Squidguard filtering, Lightsquid reporting, Clam AV, SSL interception and autoproxy with WPAD working. Doktornotor has done some great work getting CLAM AV detection working on both http and https sites in Squid3. These are locked down installations, so the SSL interception is necessary to allow full filtering and reporting.

Windows updates and the iOS app store turned out to be challenges, so I found a working setup to get these working.

The tips below assume the following setup:
pfsense 2.2.4
Squid3 0.4.0 set to port 3128 with SSL interception enabled, and transparent mode off.
Squidguard-devel 1.5.7
Lightsquid 2.41

Here's my overview of how to do it. The setup of the packages is nothing unusual and specific to our needs. The WPAD setup and proxy auto configuration though is key to getting exceptions so that windows updates, itunes, and iOS app store will work with a fully proxied, SSL interception setup.

1. You'll need a working WPAD setup for the workstations, iphones etc to auto configure. This is what works so far for me. I more or less used the pfsense guide: https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid

function FindProxyForURL(url, host) {

// If the hostname matches, send direct.
    if (dnsDomainIs(host, ".apple.com"))
       return "DIRECT";
	if (dnsDomainIs(host, ".mzstatic.com"))
       return "DIRECT";
       if (dnsDomainIs(host, ".windows.com"))
       return "DIRECT";
       if (dnsDomainIs(host, ".windowsupdate.com"))
       return "DIRECT";
       if (dnsDomainIs(host, ".microsoft.com"))
        return "DIRECT";

// FTP
if (url.substring(0,4)=="ftp:") return "DIRECT";

// If the requested website is hosted within the internal network, or any Apple sites, send direct.
    if (isPlainHostName(host) ||
        shExpMatch(host, "*.local") ||
        isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
	isInNet(dnsResolve(host), "192.168.0.0",  "255.255.0.0") ||
        isInNet(dnsResolve(host), "17.0.0.0",  "255.0.0.0") ||
        isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
        return "DIRECT";

// If the IP address of the local machine is within a defined
// subnet, send to a specific proxy.
    if (isInNet(myIpAddress(), "192.168.4.0", "255.255.255.0"))
        return "PROXY 192.168.4.1:3128";
    if (isInNet(myIpAddress(), "192.168.2.0", "255.255.255.0"))
        return "PROXY 192.168.2.1:3128";
    if (isInNet(myIpAddress(), "192.168.0.0", "255.255.255.0"))
        return "PROXY 192.168.0.1:3128";
if (isInNet(myIpAddress(), "192.168.5.0", "255.255.255.0"))
        return "PROXY 192.168.0.1:3128";
}

2. Debugging your proxy.pac file is something that can be a bit difficult, however there is a great tool to do this using Chrome:

http://artica-proxy.com/how-to-debug-proxy-pac-with-google-chrome/

You basically paste the link below into chrome, and toggle the events and proxy options. Any errors will display, as will the effective proxy file that results:

chrome://net-internals/#events

2. Place code using notepad or similar into proxy.pac, wapd.da, wapd.dat , then copy these to \usr\local\www using file manager in pfsense. For now, my pfsense box is in http mode, so the proxy files are served from the pfsense box.

3. Set up DNS Forwarder in pfsense (advanced options) with HOST override. "Home" is my domain specified in pfsense general setup. IP is my pfsense IP.

HOST: wpad
domain: home
IP: 192.168.0.1
Desc: WPAD autoconfigure host

Test by typing "nslookup wpad" at command prompt to ensure no errors.

4. In pfsense /System/Advanced check "disable DNS rebinding checks". Not sure if this is required or not, but I had issues with it enabled.

5. Set up DHCP Additional BOOTP/DHCP options. The IP listed is my pfsense IP…change to yours.

252 Text http://192.168.0.1/proxy.pac
252 Text http://192.168.0.1/wpad.da
252 Text http://192.168.0.1/wpad.dat

6. Generate CA certificate on pfsense and copy to /usr/local/www . I copied the certificate here because to install it on iOS you're supposed to email it to the phone. I found several mail clients blocked the .crt file, and zip is a pita on iOS. By browsing directly to the .crt file (192.168.0.1/yourcertificate.crt) on the LAN connected iphone, the certificate can be installed there much easier.

Install certificate to CA local root authority on all workstations to allow SSL with no errors on IE and Chrome.

Firefox has it's own certs so need to install there separately.

In windows, set the windows proxy option to auto. Chrome and IE will use this. Configure Firefox to auto detect proxy settings via options, advanced.

One of the issues I'm having is that as soon as I set up a LAN rule to block ports 80 and 443 (force proxy use), the DIRECT options in WPAD setup don't work for iOS app store and windows updates. I need LAN rules to replicate the exceptions in WPAD, but wildcards like .apple.com which work with WPAD, don't work with Aliases and rules. Any tips on this would be great. Basically I want to allow 17.0.0.0/8 , and the domains listed in my WPAD setup like *.apple.com, *.windowsupdate.com etc. via LAN rules so that the workstations have direct access. Having these exceptions in SQUID does not work...apple doesn't play with proxies. It seems like WPAD and direct access is a better way to deal with the problem.

Any tips on the wildcards via Aliases would be appreciated. Hope this helps others with similar challenges.

santidog

I've also been getting a setup similar to yours up and running on my home network and have also run into similar issues with ssl and the app store/dropbox. I have seen it reported that dropbox is using ssl pinning which is probably why the dynamic certs being created are being rejected. To get around this and caching, I've been playing with these settings in the Custom ACLS (Before Auth) under advanced features for squid:


acl nobump dstdomain .dropbox.com .apple.com
ssl_bump none nobump

acl alwaysdirect dstdomain .apple.com
always_direct allow alwaysdirect
cache deny alwaysdirect

The first 2 are to avoid ssl bumping on those matching labels. This has let the app store on iOS and the dropbox client to connect without error. I believe that by enabling this you will loose the ability to do any path matching in squidguard on secure urls, but since you are direct connecting in your wpad config this is probably fine.
The last 3 lines are to avoid having any results cached. I'm getting mixed results with those, need to read more into the current documentation to make sure they are doing what I would expect.