Squid as transparent allowing only HTTPS sites.

Gurpreet

Hello all,
I have installed Squid3 in pfSense 2.2 (in VirtualBox). But when i use Squid in transparent mode, it only allows HTTPS site. Any idea why is this behaving like this?

Many Thanks!

heper

probably squid is not funtioning AT ALL.
generally https sites don't go through squid-transparent unless you have tweaked it considerably…. thats why https works and http doesnt.

we'd need more info to find out why squid isn't working.

KOM

SSH in and check /var/squid/logs/access.log. That will tell you if squid is working or not for HTTP and HTTPS, assuming you've tried to go to each kind of site.

Gurpreet

bellow is my access.log recent logs. I just noticed that it not loging the HTTPS activity. i lso visited https://google.co.in and https://yahoo.com. i was avvle to visit but you can see there is not log for them. HTTP sites just keep loading. Their is no squid error for that. Couple of minutes latter it will show that link can't be found.

1426505737.364 2219 192.168.3.10 TCP_MISS/302 604 GET http://go.microsoft.com/fwlink/? - ORIGINAL_DST/134.170.189.4 text/html
1426505738.904 1408 192.168.3.10 TCP_MISS/200 14864 GET http://runonce.msn.com/runonce3.aspx - ORIGINAL_DST/23.99.81.176 text/html
1426505739.837 267 192.168.3.10 TCP_MISS/304 233 GET http://runonce.msn.com/wt_v3.js - ORIGINAL_DST/23.99.81.176 -
1426505858.789 1258 192.168.3.10 TCP_MISS/301 1866 GET http://yahoo.com/ - ORIGINAL_DST/98.139.183.24 text/html

condortek

Hi all, I think have same issue, by the way Gurpreet, https is not filtered by the squid because is an encrypted connection, that is why you get only https connections working. Squid is not working or similar.

In my case:

Version 2.2.1-RELEASE (i386)
Squid and sarg packages installed.
Squid service status say running
"Allow subnets" is correctly configured in Access control section.
Squid work when I configure clients to use proxy manually (access.log show activity).
Cannot navigate (http) when I check boxes "Transparent proxy" and "Allow users on interface"(only https is working). Clients not configured with proxy.
When I check only "Transparent proxy" nothing happen(no content filtered, no activity in access.log)

Apparently firewall is not managing correctly the necesary rules for squid in transparent mode. Have anybody an idea that what can be?
Thank you for your help

KOM

You should all be using x64 build of pfSense 2.2.1, not i386 unless you're restricted by hardware. I think I remember there being some issues with the current 32-bit Squid3. As for HTTPS, what settings do you have for Squid, specifically transparent or standard mode and their related settings?

Gurpreet

Right condortek.
i later realized that why HTTPS is not being logged. As heper said, my squid is not functioning at all. It only has to filter HTTP and it's not even doing that. It just makes look like there is not internet.

And the problem is with transparent mode only(but still logging). I haven't enables SSL mode. All works fine if i manually give the proxy settings to the client.

I have to use i386 build as my hardware is not allowing to install x64.

kesawi

I've started having a similar problem in the last few hours with Squid 2.7. HTTPS traffic is fine, and HTTP traffic works if the browser points to the squid proxy rather than running in transparent mode. Was using release 2.2, upgrade to 2.2.1, noticed problem and have currently downgraded to 2.2 hoping that would fix the problem. I haven't touched anything else Example error message received:


ERROR
The requested URL could not be retrieved

While trying to process the request:

GET / HTTP/1.1
Host: theonion.com
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
X-ClickOnceSupport: ( .NET CLR 3.5.30729; .NET4.0E)
Connection: keep-alive
Cache-Control: max-age=0

The following error was encountered:

    Invalid Request 

Some aspect of the HTTP Request is invalid. Possible problems:

    Missing or unknown request method
    Missing URL
    Missing HTTP Identifier (HTTP/1.0)
    Request is too large
    Content-Length missing for POST or PUT requests
    Illegal character in hostname; underscores are not allowed 

Your cache administrator is webmaster.
Generated Wed, 18 Mar 2015 15:14:38 GMT by XXXX.XXXX.COM (squid/2.7.STABLE9)

condortek

Hi, my squid configuration is default, with log activated, and Allowed subnets configured(192.168.1.0/24).
By the way my system support 64 bits as this command say: grep -w "LM" /var/log/dmesg.boot && echo "Got 64bit"

Thanks a lot.

Jambro1964

I have Squid3 installed and running and it is using transparent proxy and will proxy both HTTP and HTTPS.
Here are the main settings in the General setup:
Proxy interface is always LAN
Proxy port is left for 3128
check the Allow users on interface
I also check the Resolv(e) dns v4 first
Transparent proxy settings:
check Transparent HTTP Proxy, Transparent proxy interface is still LAN
SSL man in the middle Filtering:
check HTTPS/SSL interception
SSL Intercept interface is LAN
Leave SSL Proxy port blank
Pick your Certificate Authority that you have made and make sure you export it and install it to all the computers what will use this proxy, I used GPO to push it out.
I bumped the children up to 10
I am still playing with these two new settings but what seems to work is
select the "Do not verify remote certificate" don't click on any of the others, this interface sucks and you have a hard time removing the others if you click on them.
I also have logging turned on but pretty much kept the defaults
Click save and try it out.

Jim Ambrose

Jambro1964

The problem I still have is most HTTPS sites will bump but I am having issues with GMAIL in Chrome as one user but the other user in Chrome works fine. It would be nice is there was a clear cache in the Proxy setup page. I have it clear it out when it rolls the logs.
I am hoping that this new version will work with sites that are using TLS 1.2 certs that the 2.1.5 version couldn't connect with.
Hope you all get it working!

Jim Ambrose

Supermule

I have the same issue with http sites. Had to delete Squid to get normal browsing back online…

chris4916

@condortek:

https is not filtered by the squid because is an encrypted connection, that is why you get only https connections working.

There is something perhaps worth to be highlighted:
HTTP proxy in transparent or explicit (i.e. non-transparent) mode have very different behaviour for what concerns HTTPS.

Running HTTP proxy in transparent mode will not intercept HTTPS, meaning HTTPS flow will be ignored by Squid and go directly through FW without any control
Running HTTP proxy in explicit mode will force HTTPS to through proxy. It doesn't mean one can implement content filtering(*) but even if this encrypted, access rules and domain filtering defined at proxy level apply.

On top of this, transparent proxy doesn't allow authentication, therefore profiling (meaning here filtering rules based on account)

This is why implementing transparent proxy is most of the time not a good idea :)

(*) content filtering could be however achieved even with HTTPS while implementing man-in-the-middle like mechanisms. Squid could achieve this :-X

Gurpreet

i think this issue is related to NAT. Not sure.

Does there any NAT rule gets created automatically when we enable the transparent mode? If yes, then its not happening in our case. In the system logs i read " SQUID not statred. not insatlling NAT rules." (not exactly the same log, i forgot what it was and its not there now). I can see squid running there but C-ICAP not starting.

Any clues?

condortek

People, good morning, reinstalling with version 2.2.1, 64 still getting same issue.
After frustrated days I was reinstall with version 2.1.5, 64bits and squid is working like a charm. regards