Squid3-dev: Transparent Proxy + Invalid Bypass DNS entry = ALL Domains Bypassed

[longhorn]

Problem: Entering invalid DNS names into Squid's Bypass Destination IP list  ("Bypass proxy for these destination IP's"), when Transparent Proxy Server box is checked, results in Squid ignoring transparent proxy for MOST traffic, and - consequently - Squid stops filtering/caching nearly all HTTP traffic.

Platform: pfsense 2.1.5 + Squid3-dev 3.3.10 using package 2.2.8

Tested 29 October 2014 thru 29 November 2014

I also use Diladele HTTPS filtering. Could be this is only a problem with Squid3-dev + Diladele. If you don't have Diladele but do have Squid3-dev, please test steps below and add comment if you see same issue or not.

Steps:
    - start pfsense and goto Squid config (Services > Proxy server)
    - select your interfaces to filter, and proxy port as 3128 (default)
    - make sure Transparent Proxy is turned on
    - enter one or more invalid domain names into the Bypass Destination IP line for Squid settings
    - click Save button at bottom of form
    - Click "Real time" button at top of form to monitor real-time traffic
    - browse to any website in another browser window
    - notice nothing happens in squid log
    - go back to "General" tab in squid and change the Bypass Destination IP to a valid domain name (e.g. CNN.com)
    - click Save button on bottom of form
    - click "Real time" tab on top of form
    - browse to any other HTTP site in another browser window
    - notice that Squid is caching and logging your HTTP activity as normal

Comments/Notes:

One thing that is odd… Squid continues to report heartbeat.dm.origin.com/pulse? in the log, but every other HTTP stream is ignored. I have not looked into this further, but noted as FYI.

See attached screen shots

Tested while independently monitoring Squid log and Diladele in real time. I use Diladele for HTTPS filtering and also filtering out ads, privacy cookies, etc. Have not tried completely removing Diladele and then re-testing. However, AFAIK Diladele does not modify Squid3-dev, it only piggybacks on it.

Suspect this is bug in Squid3-dev. Seems to me that invalid domain name should not stop all (or nearly all) filtering and caching, but that appears to be the case from what I have observed.

My thoughts to developer:
  1. Why does failed DNS query result in dropping (nearly) all caching/filtering?
  2. Is this a timeout problem for Squid (waiting for DNS translation)?
  3. Does Squid send DNS translation to another process and then waiting for response, ignores filtering traffic during wait time? (btw, I waited over 10 minutes and it never started up the filtering again while invalid domain name present in Bypass list)
  4. Why not check DNS entries in the Bypass field when user commits changes via Save button on General tab? Problem could still occur, but it would prevent all issues unless a site dropped out of DNS for some reason (e.g. failure of owner to renew DNS registration).
  5. if waiting for DNS translations, why not time out Squid after a period of time and ignore just that rejected DNS name in Bypass list, but continue Bypassing remainder of list, and continue filtering all other traffic?

Thoughts from anyone else??? Am I missing something obvious here? I looked around via Google and pfSense site search and did not find anyone reporting similar problem.

[wcrowder]

Squid3-dev is at Squid ver. 3.3.13, Squid is at ver. 3.4.9 about to go to 3.5? Separate box or VM? Port forward? wpad?

Diladele looks cool, and looks like it's actively developed, going to look at it. Thanks.

(Edited to add the last line.)

[longhorn]

@wcrowder:

Squid3-dev is at Squid ver. 3.3.13, Squid is at ver. 3.4.9 about to go to 3.5?

Thanks for your reply.

Are you sure about that for Squid3-dev? Where are you finding these version numbers for Squid3-dev? Remember, I'm using pfsense 2.1.5, and that is still running on FreeBSD 8.

When I look at the package info, pfSense seems to indicate that I'm on the most recent version (production version, not beta) and package. There is no option to upgrade when I look at the available packages via pfSense.

Do you know the location of an official squid3-dev repository? I haven't been able to find one, so I've got to go with whatever I see in pfSense packages.

Separate box or VM? Port forward? wpad?

Dedicated box. No port forwarding. I don't know what a 'wpad' is.

Diladele looks cool, and looks like it's actively developed, going to look at it. Thanks.

(Edited to add the last line.)

No problemo. It seemed like the best option as it allows SSL filtering and some decent ad removal features. I believe there is a 60-day  free trial period. After that you have to pay, but for personal use it's cheap ($1 / month).