Problem with transparent squid on transparent firewall (mostly solved)
What I'm trying to do is set up a squid proxy computer. I found documentation online for setting up pfSense as a transparent firewall and followed that (transparent firewall HOWTO.pdf). Then, I installed squid and set it up in transparent mode, attaching to port 3128 on the lan. Network information is as follows:
WAN addr: 192.168.1.11
LAN addr: 192.168.1.12
DHCP is handled by the gateway machine.
In transparent mode, I was not able to reach any websites. If I took squid out of transparent mode, I was able to connect to websites. I have also thought of changing the lan address to be 192.168.2.11 as maybe being on the different class address might make a difference. This did not work.
Is what I am trying to do even possible?
I am using pfSense 1.2.2
Okay, On a first guess, I tried using the proxy in non-transparent mode, and that worked perfectly. So, all data flows perfectly until I make the proxy transparent.
Also, correct me if I'm wrong about this but, the only difference between transparent and non-transparent mode is that it sets up a redirect rule in the firewall to direct all port 80 traffic into the squid? Otherwise, the actual squid setup is the same, correct?
Have you changed your webGUI port to HTTPS/443? Otherwise all local traffic that gets redirected on port 80 will hit the GUI and not the transparent proxy you're trying to use. Also, having WAN/LAN on different subnets is almost always required for proper operation - though I can't say I have ever played with 'transparent firewall'.
Thanks for the response.
WebGUI is on port 443. And, according to the PDF I was reading, they used:
Now, also according to the PDF: (in the configuring lan interface)
Bridge the LAN-Interface with the WAN-Interface and disable the FTP Helper.
The IP you enter here will be ignored when you activate the bridge mode.
You better should not use the same IP on both interfaces, because it can cause BSD-internal problems.
The management IP given in the WAN-settings will be assigned to the bridge interface, which will be created when activating the bridge.
Finally found the PDF link. It's the one posted at: http://pfsense.trendchiller.com/transparent_firewall.pdf
So, if I could get this to work without using separate subnets, that would be great.
Another interesting point is that once I tell it to do transparent, there are no entries in the log. So, where is the traffic being redirected to? Is there a way to put squid into a "verbose" mode so it will tell me if it ever gets any connection?
I found "an" answer. Maybe not the best one, but it's somewhat solved. Here's the topic:
The "answer" seems to be to set the gateway on the client machines to be the address of the firewall. As soon as I did that, it worked perfectly.
But, that opens up another question to me… Why? Is that because of how our version of squid is compiled? Or, is that a squid problem. It seems to me that the proxy should not care what your gateway is. That's not relevant, is it? It should only be interested in keeping your connection open, opening up a link to your destination page, and relaying the information back. In which case, how you got to the proxy makes no difference. right?
I'm still not thrilled with this answer and I would like to find a way around this problem. So, if anyone still has ideas about it, I would love to know!
This behavior seems as expected to me. If you are intending to use squid as a transparent firewall, then the clients must direct their traffic to that box. You can do this one of two way: Set the proxy in the browser/client options (not transparent) or set the proxy box as the gateway. How else would a client know to direct their internet traffic to the proxy? The transparent firewall/proxy that you set up cannot sniff the entire network for traffic on port 80 and 'hijack' it.
Without knowing your full network needs and configuration my first suggestion would be to use pfSense as your primary gateway anyway. Seems like this would work nicely for you.
Well, I'll diagram a situation where this is not (in my mind) expected:
Internet <- commercial router/DHCP server <- (WAN) transparent firewall/proxy (LAN) <- switch <- clients
Internet: cable modem (DHCP assigned address)
commercial router internal address: 192.168.1.1
transparent firewall/proxy: 192.168.1.10
In this case, the logical gateway would be the commercial router (192.168.1.1), not the proxy machine (192.168.1.10).
Now, the proxy machine has the ability to redirect the traffic into it's proxy program as "transparent" to the user. And, can make that redirection based on the interface the traffic comes in on (LAN, OPT1, etc.) and port, not caring about the source or destination address.
The transparent proxy machine is not exactly "transparent" if the clients have to have it as the gateway. Now, I'll grant that this is not much of a thing to deal with. And, most of the time, the admin can just change the DHCP server to use the proxy server as a gateway. The actual users would never know since most of them rarely ever look at network information anyway. But, to me, it seems like the proxy should not care about the gateway on the clients.
I'm not trying to point fingers or put down pfSense. It's a good system. This just seemed to be unexpected to me.
Is this setup all running on the same switch or is the pfsense box physically separating the commercial router/dhcp from the clients?
Each arrow represents a physical separation. And, the pfSense box is acting somewhat like a bridge to a separated internal network (branching off of the router). The physical WAN side of the pfSense is connected to the rest of the internal network. So, essentially, the pfSense is an intelligent bridging device between the separate network on the LAN and the rest of the network on the WAN.