Squid stripping domain from URL with port forwarding

smar · May 17, 2014, 7:36 AM

Hi

I have a pfsense 2.1.3 (and tried on 2.1.2) with squid3-dev (3.3.10). The proxy works absolutely fine if I enable transparent http proxy checkbox in the proxy server settings. The only change on the proxy side is that ACL has been set to allow the local subnet.

However, I am struggling to get it to work with NAT/port forward rules for http traffic instead of the built-in transparent proxy mode. The reason for using NAT/port forwarding is because I want to be able to control which devices are forwarded and which bypass the proxy through firewall rules (so the table of bypass devices can be changed on demand via a password override page).

The NAT rule I have is:

IF: LAN, Protocol: TCP,
Source: Not 'unfiltered', Source Port range: Any,
Destination: Any, Destination port range: HTTP,
Redirect target IP: 127.0.0.1, Redirect target port: 3128

There is also a corresponding Firewall LAN rule automatically created.

Although the forwarding seems to be happening, for some reason the domain name is being stripped when the uri gets to squid. Thus if we try to go to "http://www.bbc.co.uk/news", the proxy log shows:

1400310306.419 0 172.16.10.1 NONE/400 3556 GET /news - HIER_NONE/- text/html

And the browser shows:

The following error was encountered while trying to retrieve the URL: /news
Invalid URL
Some aspect of the requested URL is incorrect.
Some possible problems are:
Missing or incorrect access protocol (should be http:// or similar)
Missing hostname
Illegal double-escape in the URL-Path
Illegal character in hostname; underscores are not allowed.

As you can see, the "http://www.bbc.co.uk" part is being stripped. This happens for any site we try to access. I've tried with completely new builds of pfsense and squid, and also using different browsers, but all give the same error.

If instead of port forwarding, I change the browser's proxy configuration to directly point to squid, all is again well (but this again doesn't serve my purpose).

As a further experiment, I installed Dansguardian, and used the NAT rule to redirect to Dansguardian, and set squid as Dansguardian's parent proxy. This again worked fine.

The problem ONLY seems to be if I port forward directly to squid.

Can anyone please help. I have been struggling for a number of days with this, trying every permutation I can think of!

Many thanks.

elemay · Jun 24, 2014, 4:08 PM

Hi,

same here.

I used squid3-dev and squid3 (squid3-dev didn't work at all :P )

I have a squid running on my LAN if and natting port 80 to 3128 from wlan to lan.

Same happens to me.

The boxes in the lan have the proxy configured with wpad/pac, but my wlan should just be forwarded to squid (as there are different types of devices in that one).

Whats wrong here?

Thanks for all your help!

elemay

Cino · Oct 25, 2014, 9:52 PM

Did you ever solve your issue? I was doing some testing last week and noticed this too. If I create manual NAT rule to redirect http traffic to squid, it drops the domain. I have a vlan that I've setup for kids to use. It using dansguardian then to squid. I have a manual NAT rule redirecting to Dans with no issues, if I change the port to point to squid; domains get dropped..

strange….

iorx · Oct 26, 2014, 8:23 AM

Here is a "me too!"

I thought I was running nuts. I can get a simple port forwarding working.

Going to try adding my 2c on the subject.

LAN is 192.168.1.1/24 (em0_vlan10)
GUESTLAN is 192.168.2.1/24 (em0_vlan10)

It is my GUESTLAN which I try to forward all 80 into 3128 on squid. Can't get it to work no matter what! :-) Can't see that the rule is even created with "pfctl -sr"?!

My port forward look like this:

Set to PASS, so no linked rules in /Firewall/Rules/GUESTLAN

Nothing show up if I list the rules with "pfctl -sr"

Checking Transparent proxy in squid make thees rules show up at the bottom of the list:
pass in quick on em0_vlan10 proto tcp from any to ! (em0_vlan10) port = http flags S/SA keep state
pass in quick on em0_vlan10 proto tcp from any to ! (em0_vlan10) port = 3128 flags S/SA keep state
pass in quick on em0_vlan40 proto tcp from any to ! (em0_vlan40) port = http flags S/SA keep state
pass in quick on em0_vlan40 proto tcp from any to ! (em0_vlan40) port = 3128 flags S/SA keep state

Something with the manual creation of a port forward seems to be broken here.

Brgs,

–-
[141026] Add info (missed that…)
Versions:
squid3 Network 3.1.20 pkg 2.1.1
pfsense 2.1.5x64

Cino · Oct 26, 2014, 1:34 AM

I have to mess around with it again. I have a feeling it could be a squid configuration issue.

What interfaces is squid running on? For me I have it running on LAN and loopback. My NAT rule, I'm using loopback.. Now I can use either the LAN or loopback IP but makes sense to use the loopback. Are you at least seeing clients connect in your log and the domain is stripped off?

I use wpad.. So all my clients are connecting Client IP to LAN IP:3128… I'm thinking transparent mode may add something to squids config that we are missing when manually adding the NAT rule.

aGeekhere · Oct 26, 2014, 2:37 AM

Hi all
for squid3-dev transparent http and https filtering read through this for setup

https://forum.pfsense.org/index.php?topic=73640.0
https://forum.pfsense.org/index.php?topic=79389.0

Let me know how you went.

Cino · Oct 26, 2014, 11:39 AM

Thanks aGeekHere… At least for me, I have squid up and running with no issues. Using SquidGuard and DanGuardian. The issue is if I setup a manual NAT, the domain.com gets strip off we I'm seeing an error in the browser '/index.html' can't be found. Lucky for me, I don't use squid this way so its really a non-issue but I was just testing different features one night..

anas_xrt · Nov 8, 2014, 10:20 AM

"Same here as well"

I am using version 2.1.5-Release (i386) and want to forward the port (HTTP) to the External Squid server (8182) on another interface (DMZ) for my LAN interface.

I see the log on Squid was stripping domain from URL and can't brows the internet.

The port forward rule was simple on Lan interface as following.

If LAN Proto TCP Scr.addr ***** Src. ports ***** Dest. addr ***** Dest. Ports 80(HTTP) NAT IP 172.16.11.1 NAT Ports 8182

I try all NAT reflections but the result all same.

Anyone interesting to fix this problem, please let me know, so I can help you provide all information that you would need.

Cino · Nov 8, 2014, 12:16 PM

I've only tried this with 3.3.10. Has anyone tested/tried this with Squid 2.7?

anas_xrt · Nov 16, 2014, 7:14 AM

I have work around by install Squid package (Stable) for Pfsense. Then I use the transparent to intercept on the interface and I put remote cache by address of my external squid server.

This is work but, it should not be the way it should.

Note… I try by use Squid3 (beta) package. but it will just broke connection when I have run it for a day. I don't know what is the problem, just suddenly not forward the request to remote cache. eventually I remove the package.

grover76 · Mar 5, 2015, 2:14 PM

I had this same issue with squid 2.7.9. This worked for me:

Set squid proxy to listen on port 3129 (or any port you choose, the GUI wouldn't allow me to leave it blank)
Add custom option: http_port 3128 transparent

Port forward on LAN:
Traffic TCP Src * Srcport * Dest * Destport HTTP(80) TargetIP pfsensebox IP Targetport 3128

My guess is that on the GUI without the transparent box checked, squid was not operating transparently on port 3128 until specifically defined to do so.

Unfortunately my ultimate goal was to use this rule to apply limiters to the traffic but apparently there is a bug with limiters and squid in transparent mode that I can't seem to get around!