Squid3 transparent proxy not serving pages

lmitchel · Dec 6, 2015, 6:00 PM

Fresh install of pfsense 2.2.5-RELEASE (i386).
One LAN port (fxp0), one WAN port (em0)
Installed Squid3 and rebooted. (I also tried this with Squid with the same results).

Manual proxy works if I set the proxy port in the browser.

If I set to transparent proxy and unset the proxy port in the browser, I get the error message "This page can't be displayed" and /var/squid/logs/access.log does not show any new requests.

using pfctl -sn | grep 3128, I get:
rdr on fxp0 inet proto tcp from any to ! (fxp0) port = http -> 127.0.0.1 port 3128

Squid3 settings:
Proxy Interfaces (LAN, loopback) or just LAN gives the same result.
Allow users on interface set
Disable ICMP set
Transparent proxy set
Proxy interface - LAN
logs enabled and set to rotate
X-Forwarded Header Mode set to "transparent"
local Caches set

This is similar to: https://forum.pfsense.org/index.php?topic=100061.0 but no solution has been given.

I have followed the guide: https://techknight.eu/2015/04/17/pfsense-setup-and-configure-squid3-transparent-proxy/
and https://doc.pfsense.org/index.php/Setup_Squid_as_a_Transparent_Proxy (for squid 2)

What else can I check / do?

Thank you.

KOM · Dec 6, 2015, 11:58 PM

After you install squid, it seems to need to be restarted before it starts working. Either bounce the box or restart the squid service. Anything in /var/squid/logs/cache.log? Or the System log in WebGUI?

lmitchel · Dec 7, 2015, 4:20 AM

I have restarted the box after turning on transparent proxy with no different results. There is nothing in the firewall logs after a reboot to show that I am doing anything as long as it is in transparent mode.

System log shows:
Dec 6 21:04:29 php-fpm[245]: /rc.start_packages: [squid] Starting service…
Dec 6 21:04:29 squid[68075]: Squid Parent: will start 1 kids
Dec 6 21:04:29 squid[68075]: Squid Parent: (squid-1) process 68263 started
Dec 6 21:04:29 php-fpm[245]: /rc.start_packages: [squid] Starting a proxy monitor script

/var/squid/logs.cache.log shows:
Page faults with physical i/o: 0
2015/12/06 10:06:32 kid1| Starting Squid Cache version 3.4.10 for i386-portbld-freebsd10.1…
2015/12/06 21:02:51 kid1| Shutdown: NTLM authentication.
2015/12/06 21:02:51 kid1| Shutdown: Negotiate authentication.
2015/12/06 21:02:51 kid1| Shutdown: Digest authentication.
2015/12/06 21:02:51 kid1| Shutdown: Basic authentication.
CPU Usage: 4.750 seconds = 3.661 user + 1.089 sys
Maximum Resident Size: 101312 KB
Page faults with physical i/o: 11
2015/12/06 21:04:29 kid1| Starting Squid Cache version 3.4.10 for i386-portbld-freebsd10.1...

I tried removing and re-installing Squid3 several times (with the keep settings box unchecked). It's like it is not even getting to the firewall.

KOM · Dec 7, 2015, 3:06 PM

I remember that i386 squid had some issues many months ago and I don't know if they got addressed or not. That might be your problem. Are you able to run x64?

Netizen1 · Dec 7, 2015, 4:15 PM

On your test PC, did you set your default gateway to pfsense's LAN ip address?

lmitchel · Dec 7, 2015, 6:18 PM

@Netizen1:

On your test PC, did you set your default gateway to pfsense's LAN ip address?

Yes, the gateway is via DHCP and I checked.

@KOM:

I remember that i386 squid had some issues many months ago and I don't know if they got addressed or not. That might be your problem. Are you able to run x64?

Yes, I will report back. Originally this was to be on an older machine but I have since put in a new one.

KOM · Dec 7, 2015, 6:35 PM

32-bit builds will go away soon enough so you really should move to 64-bit if you can.

lmitchel · Dec 8, 2015, 1:28 AM

I have installed and setup the amd64 version. I am having the same problems with the transparent proxy. Manually set up in browser is fine.

I have the same settings as before. What else can I check? It seems the requests never hit the cache or firewall. I will keep looking as well.

Thank you.

lmitchel · Dec 8, 2015, 4:53 AM

I ended up unticking the transparent proxy and manually adding a LAN rule to do the same thing:
rdr on fxp0 inet proto tcp from any to 192.168.2.0 port = http -> 127.0.0.1 port 3128

This does not work. The rule shows but does not redirect to http to 3128.

I wonder if there is a problem with the built in rule:
rdr on fxp0 inet proto tcp from any to ! (fxp0) port = http -> 127.0.0.1 port 3128

In the meantime, if anyone has an idea of why the transparent proxy isn't working within Squid3 please let me know.

Thank you.

doktornotor · Dec 8, 2015, 9:01 AM

@lmitchel:

In the meantime, if anyone has an idea of why the transparent proxy isn't working within Squid3 please let me know.

It's working just fine for pretty much everyone but you… Do a traffic capture, this doesn't go anywhere with the amount of info available here.

KOM · Dec 8, 2015, 1:36 PM

Transparent proxy is a PITA when it comes to HTTPS, so perhaps this is a blessing in disguise. Consider keeping squid in explicit mode and use WPAD to help your clients find it automatically.

https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid

doktornotor · Dec 8, 2015, 1:37 PM

@KOM:

Transparent proxy is a PITA when it comes to HTTPS, so perhaps this is a blessing in disguise.

;D 8) Not a fan of transparent proxies either.

lmitchel · Dec 9, 2015, 4:42 AM

Thanks for the suggestion KOM. I used the wpad link you sent and am dropping the transparent proxy. I guess I am stuck in the past - had that set up using centos but lost the hard drive and was trying to recreate it on pfsense. This is a better solution. I will monitor it and make certain that it works as expected.

Thanks for the help doktornotor as well - when I have time I will try the sniffer.