ClamAV HTTPS scanning

asterix

Did a fresh install of 2.3.1 and installed Squid. Configured exclusive mode (non-transparent). Using WPAD for clients. Checked and confirmed https and https traffic flow thru Squid.

Did a eicar virus download test for ClamAV and I found just http traffic being scanned. It's not scanning https.

Is there any specific setting to get https scanning working?

exograpix

It won't work in wpad configuration as there is no MITM to decry pt the https url.

asterix

Urgh. Have tried using MITM last year but ran into a lot of issues with sites not working correctly. Plus I have a lot of Android and apple devices on the network.. not to mention SmartTVs, settop boxs which I believe wont work with MITM.

Is there any any good documentation on getting the certificate part setup correctly?

exograpix

MITM is curse in disguise, it will never work well with certain sites, personally I feel antivirus on firewall is useless and resource hog. I suggest stick with WPAD only with https antivirus. handle these infected files on the local level. Most of the viruses as propagated through pen drives and emails, for which pfsense is not the suitable product.

Nachtfalke

Hi,

squid + antivirus is working without any problems using WPAD. Theres no reason why it should not work. What I noticed is that you sometimes have to restart c-icap and clamav services in the pfsense services tab. At first you should make sure that it is blocking eicar unzipped and with http.

Can you see the eicar files in squid proxy –> Realtime tab? If yes then perhaps just disable Antivirus again, click save. givt it some time (2-3min) to disable and restart all services and then try again with enabling Antivirus.

Further Antivirus has an option to NOT scan content again which is in your squid proxy cache because we assume that everything which went into the cache was scanned before. So you cann try with squidclient to purge the domain out of the cache. Clear your complete cache or add the eicar domain to the "do not cache" option in squid.

@Android topic:
Had many problems with android devices and proxy. You can configure proxy and proxy-pac file per WLAN but the problem is that not every app is working with the proxy environment or is not recognizing the proxy. So I had to configure many "DIRECT" URLs in my WPAD file. After some testing and with different smartphones in my family and so many different apps I configured that my android phone do not have to use a proxy.

Regards

asterix

@Nachtfalke:

Hi,

squid + antivirus is working without any problems using WPAD. Theres no reason why it should not work. What I noticed is that you sometimes have to restart c-icap and clamav services in the pfsense services tab. At first you should make sure that it is blocking eicar unzipped and with http.

Can you see the eicar files in squid proxy –> Realtime tab? If yes then perhaps just disable Antivirus again, click save. givt it some time (2-3min) to disable and restart all services and then try again with enabling Antivirus.

Further Antivirus has an option to NOT scan content again which is in your squid proxy cache because we assume that everything which went into the cache was scanned before. So you cann try with squidclient to purge the domain out of the cache. Clear your complete cache or add the eicar domain to the "do not cache" option in squid.

@Android topic:
Had many problems with android devices and proxy. You can configure proxy and proxy-pac file per WLAN but the problem is that not every app is working with the proxy environment or is not recognizing the proxy. So I had to configure many "DIRECT" URLs in my WPAD file. After some testing and with different smartphones in my family and so many different apps I configured that my android phone do not have to use a proxy.

Regards

Squid + ClamAV works fine on HTTP. It blocks eicar files. But it fails on "HTTPS". It will not scan files downloaded through HTTPS.

Would you mid sharing your custom PAC file. I too need to configure the Android Tabs and phones to not use proxy and even the iOS devices. Problem is that I have separate subnets. On a side note, I have problems with pfBlocker not working due to this Squid + PAC configuration.

Nachtfalke

Hi,

my proxy.pac looks like this:


function FindProxyForURL(url, host) {

    // Normalize the URL for pattern macthing
    url = url.toLowerCase();
    host = host.toLowerCase();

    // diese Ziel-IPs, Domains oder Hosts gehen DIRECT
    if (dnsDomainIs(host, "ap.bundesbank.de") ||
        dnsDomainIs(host, "on4u3.buhl.de") ||
        dnsDomainIs(host, "*.oberndorf.ca") ||
        dnsDomainIs(host, "data-cdn.mbamupdates.com") ||
        dnsDomainIs(host, "*.data-cdn.mbamupdates.com") ||
        dnsDomainIs(host, "download.windowsupdate.com") ||
        dnsDomainIs(host, "*.download.windowsupdate.com") ||
        dnsDomainIs(host, ".windowsupdate.com") ||
        dnsDomainIs(host, "*.windowsupdate.com") ||
        shExpMatch(host, "*.local") ||
        shExpMatch(host, "*.localhost") ||
        shExpMatch(host, "10.123.123.123"))
    {
    return "DIRECT";
    }

    // If IP of the requested host falls within any of the ranges specified, send direct.
    if (isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
        isInNet(dnsResolve(host), "172.16.0.0",  "255.240.0.0") ||
        isInNet(dnsResolve(host), "192.168.0.0", "255.255.0.0") ||
        isInNet(dnsResolve(host), "169.254.0.0", "255.255.0.0") ||
        isInNet(dnsResolve(host), "240.0.0.0", "240.0.0.0") ||
        isInNet(dnsResolve(host), "127.0.0.0", "255.0.0.0"))
    {
    return "DIRECT";
    }

    // Fuer alle anderen Requests den Proxy zurueckgeben
    return "PROXY 192.168.10.1:3128";
}

The first big chapter contains the domains or IPs I would like the clients go DIRECT. So in the past I added there several domains which were needed by Android. But as I said I removed that from PAC file because it was to much work for me. Instead I made sure that my Android phones get always the same IP address via DHCP and for these source-IPs I added a firewall rule which allows DIRECT traffic from the phone to the internet. So no need to configure something on Android and no need to maintain the proxy.pac for android phones. They simple just don't use the proxy.pac.

The second big chapter in the PAC file ist just to make sure that all local IP ranges can be reached DIRECT and not via the proxy.

@pfblocker-NG
Is working with squid and PAC file but you have to allow:
1. Firewall rule that allows LAN clients to reach 127.0.0.1
2. Firewall rule that allows LAN clients to reach the DNSBL VIP
3. NAT rules with Source: any, Destination 127.0.0.1, NAT-IP: DNSBL Port 3128 redirect to http DNSBL Port, Port 3129 redirect to https DNSBL port
4. NAT rules with Source: any, Destination pfsense LAN IP, NAT-IP: DNSBL Port 80 redirect to http DNSBL Port, Port 443 redirect to https DNSBL port

Further make sure that your squid is using the pfsense DNS first so that Unbound/DNSBL has a chance to resolve and block/redirect.

It was working for me, but I had performance issues with DNSBL / Unbound and why using DNSBL if I could use the same DNSBL domain lists in squidguard to block ads and websites. So now I am using squid+squidguard to get the same result as with DNSBL (without squid).

fw_rules_dnsbl.png_thumb

asterix

Thank you for the details.

For the PAC file.. how do I include my local subnets.. 10.1.x.x, 10.2.x.x (mask 255.255.255.0) to be reached DIRECT and not via proxy.

Should I use this way?

if (isInNet(dnsResolve(host), "10.1.0.0", "255.255.255.0") ||
isInNet(dnsResolve(host), "10.2.0.0", "255.255.255.0") ||
isInNet(dnsResolve(host), "10.3.0.0", "255.255.255.0") ||
isInNet(dnsResolve(host), "127.0.0.0", "255.0.0.0"))

Nachtfalke

Yes, this is a possible way. Or you can add entries like these:

shExpMatch(host, "10...") ||
shExpMatch(host, "192.168..*")