HOWTO: pfSense 1.2.x Traffic Shaping with Squid Transparent Proxy

iBeej

Hello pfSense Community!

I have spent a great deal, configuring my pfSense firewall in a corporate Virtualized VMware ESX 3.5 Hypervisor with all sorts of bells and whistles. I am a heavy user of pfSense and I have spent a huge amount of time tweaking it. I know there are many of you who have expressed interest using Squid in Transparent mode (as a single deployment) and would like to shape HTTP traffic, but realized it wasn't possible. This is because Squid bypasses the shaper by routing to 127.0.0.1:80. Therefore, all of your packets will land in the qLANDef queue, rather than the (wizard default) qOthersDownH queue.

I spent some time digging through the filter code to find out where the rules were created, and managed (very easily I might add) to "hack" a work-around for those who are THAT interested. Please keep in mind, i'm not a developer. I'm just a loyal user and Network Engineer for a medium-size business. Modifying any of the core system scripts in pfSense can VERY EASILY render your entire firewall inoperable if you screw something up. (I did it twice, rofl. Thank god for VMware snapshots!) So, for version 1.2.x…

USE YOUR OWN DISCRETION! TRY THIS AT YOUR OWN RISK!

pfSense is a modular scripted platform. Literally everything you do in the webgui translates to a script, writing a configuration file or another script, which in turn reloads/runs during a filter reload. So the huge majority of files that reside on the system will constantly be rewritten. pfSense stores it's filter generation scripts in multiple locations. The first, and primary location is in /etc/inc and you will find filter.inc there. Inside filter.inc, there is a check for the SQUID package, and if it exists, it loads the SQUID filter file from /usr/local/pkg/squid.inc! This is the file we are going to modify.

Make sure you have SQUID installed, and you have ENABLED the transparent proxy before continuing.

1.) SSH in to your pfSense platform.
We are going to manually modify the squid.inc file using the command line. Keep in mind, this is NOT supported by the pfSense developers. So any changes you make to these files, you're on your own. Also, any official pfSense UPGRADE or Package Reinstallation will wipe any custom modifications you perform inside the system. So technically, these changes are only as permanent as you allow them to be.

2.) Modify the squid.inc file
cd /usr/local/pkg
vi squid.inc

While in the 'vi' editor hit the forward slash '/' key. Type '127.0.0.1' and hit enter. Hitting the 'n' key will cycle through the next instance. This will search through the file looking for every instance of 127.0.0.1. squid.inc is responsible for creating the squid config and rules for routing traffic through 127.0.0.1:80, instead of the LOCAL LAN interface. This effectively bypasses the Traffic Shaper, and we don't want that. At the time of this writing, you will find 3 instances of '127.0.0.1' in this file. The first one is for the squid config file. The 2nd one is a default ACL. And the last one is the actual redirect rule for /tmp/rules.debug.

Change every instance of '127.0.0.1' to the local interface IP. For me, this is our gateway at 10.1.1.1. I know there are probably more "fancy" ways of changing this value. Using the iface variables in the scripts, etc. But this is a hack fix, and I personally WANT my gateway address hard coded. Feel free to experiment here if you want, just make sure the configuration files and rules are generated properly!! DO NOT modify port 80 next to '127.0.0.1' if you see it. We are going to continue using it as the default. And it's natural to look at, when viewing the state table.

3.) Change the webgui port from the web management site.
Navigate to 'System/General Setup' and change the port there. Make sure the port is set to something other than 80/443. I put mine on 1337. Nerdy, I know.. but I love it.

4.) Verify your LAN firewall rules.
It is critical you make sure your LAN rules allow YOU to access the webgui on the port you specify. So if you haven't already allowed ANY on your local subnet, you better add a rule to the webgui port, or you will get locked out of it in the next step.

5.) Disable Web-GUI Anti Lockout rule
Navigate to 'System/Advanced' and check the "Disable Web-GUI Anti Lockout rule". When this rule is enabled, the traffic shaper catches it, and routes traffic in the wrong queues again. Disabling it effectively ensures we will be moving all HTTP traffic in to qOthers. Again, VERIFY your LAN firewall rules. Make sure you can access the webgui port. NOTE: If you DO get locked out, you can easily reset the Anti-Lockout rule and gain access by re-assigning your LAN interface at the console.

6.) Restart the SQUID service and do a FILTER RELOAD from the 'Status/Filter Reload' menu. Wait for the filter to regenerate the new rules and then test your configuration. Give it a few minutes. There was a slight delay before web traffic started flowing again. (For me anyway)

These changes will effectively route packets from SQUID 3128 to GATEWAY 80, which the Traffic Shaper can catch! If all went well, your Squid Transparent Proxy should be functioning as it should, and you should see HTTP packets flowing through the qOthers queue!!! NOTE: This doesn't mean you wont see traffic flow through qLanDef!!! Check your rules, especially if you used the wizard. Remember, qLanDef is the default queue, so unhandled traffic will end up in that queue.

On my traffic shaping rules for HTTP, I used 'ANY' (*) for source. I believe LAN Subnet will work, but I wanted to make sure that everything would hit the qOthers queue. But other than that, I can't think of anything else. It sure is nice pulling web traffic in to the proper queue with Squid running transparent. I am a very happy camper.

Anyway, if you made it this far and you were daring enough to mess around with your config, congratulations!!! I wish you all the best of luck in your experience with pfSense. I hope I didn't miss anything, and you can always feel free to chime in with added information, tips, or questions.

And again, please don't hound the developers if you attempt to make these changes. And remember to fix your files if you do any kind of upgrade or package reinstall!!

Good luck and Farewell!

Hugovsky

Thanks for the howto!

keasley

Will this work on Multiple WAN setup? & the How to pls.

iBeej

I have not setup or tested this in a Multi-WAN configuration. I'm sure there would be more configuration with your routing/rules which again be custom modifications to the base system. I would be interested in hearing more of your stories, for those who have attempted this. I will be setting up a multi-wan for testing purposes in a couple months to test some redundancy features of our WAN, and I will have to post an update with my results.

alejandrolor

Hello
This could work in two interface lan (lan and Opt1) ? Sorry may english is not good

Esto podria trabajar en 2 interfaces lan (lan y opt1) pregunto por que yo tengo una red wireless en opt1
Gracias

Still

Tested and worked pretty well on 1.2 and 1.2.2, many thanks!!
Sticky!

wonslung

i wanted to let you know this works great in 1.2.3 but it didn't work at all until a reboot…very impressed

ldillon

I'm having a similar problem (transparent Squid proxy with traffic shaping enabled (not traffic shaping within Squid, but under Firewall -> Traffic Shaper)) but that the Traffic Shaper is throttling Squid cache hits.

We have the cache tuned to cache large files and most of the hits are things like Windows updates and Ubuntu updates.

Please don't tell me to set up a WUS and Ubuntu mirror ;-)

jigpe

I have installed squid and it works with traffic shaping. How to limit the download/upload if the user will use ftp or ssh then ftp using cli?

jigp
Davao City

hdavy2002

Works beautifully. Thanks

jigpe

Thanks for the tutorials. Still fighting for ftp/ssh/http dl/up.. :(

jigp
Davao City

Meulator

Hey there,

something i didn't get… in squid.inc, do i have to change every "127.0.0.1" by "my.pfsense.lan.ip" ? Or only 3 times?

Cause i tried by changing all, changing the default https port & firewall rule, rebooted the pfsense box... and then i could not get the capitve portal page...

javerleo

Thanks iBeej.
This hack is what I've been looking for a long time ago. It worked fine on pfsense 1.2 but now squidguard filter rules seem not to be working.
Some clues?

i_robot73

This seemed to work for me:

add

function squid_resync_general() {
	global $g, $config, $valid_acls;

	$WAN_IP = "0.0.0.0.0";
	foreach (explode(",", $ifaces) as $i => $iface) {
		$real_ifaces[] = squid_get_real_interface_address($iface);
		if($real_ifaces[$i][0]) {
			$WAN_IP = "{$real_ifaces[$i][0]}";
		}
	}

Change things like:

if (($settings['transparent_proxy'] == 'on')) {
		$conf .= "http_port 127.0.0.1:80 transparent\n";
	}

to

if (($settings['transparent_proxy'] == 'on')) $conf .= "http_port {$WAN_IP}:80 transparent\n";

lifeboy

You said this worked for you, but:

@i_robot73:

function squid_resync_general() {
	global $g, $config, $valid_acls;

	$WAN_IP = "0.0.0.0.0";
	foreach (explode(",", $ifaces) as $i => $iface) {
		$real_ifaces[] = squid_get_real_interface_address($iface);
		if($real_ifaces[$i][0]) {
			$WAN_IP = "{$real_ifaces[$i][0]}";
		}
	}

Why do you use $WAN_IP here? I thought we are looking to let squid send traffic to LAN_IP?

I have rules on the LAN (to provide loadbalancing/failover), which are also not triggered when running a transparent proxy. So changing to WAN_IP above will not be the same as the hack that was described at the top of this post.

The you say:

@i_robot73:

Change things like:

if (($settings['transparent_proxy'] == 'on')) {
		$conf .= "http_port 127.0.0.1:80 transparent\n";
	}

to

if (($settings['transparent_proxy'] == 'on')) $conf .= "http_port {$WAN_IP}:80 transparent\n";

Does this mean that the references below need to be changed as well?

acl localhost src 127.0.0.1/255.255.255.255

and

foreach ($ifaces as $iface) {
$rules .= "rdr on $iface proto tcp from any to !($iface) port 80 -> 127.0.0.1 port 80\n";
}
/* Handle PPPOE case /
if($config['pppoe']['mode'] == "server" && $config['pppoe']['localip']) {
$rules .= "rdr on $PPPOE_ALIAS proto tcp from any to 127.0.0.1 port 80 -> 127.0.0.1 port 80\n";
}
/ Handle PPTP case */
if($config['pptpd']['mode'] == "server" && $config['pptpd']['localip']) {
$rules .= "rdr on $PPTP_ALIAS proto tcp from any to !127.0.0.1 port 80 -> 172.0.0.1 port 80\n";

Please comment, since this seems a little mixed up to me.

dondos

I also tried this hack. I changed every instance of 127.0.0.1 with 192.168.10.1 (my LAN ip address):


Search "192.168.10.1" (9 hits in 1 files)
	Line 603: 		$conf .= "http_port 192.168.10.1:80 transparent\n";
	Line 766: acl localhost src 192.168.10.1/255.255.255.255
	Line 1285: 				$rules .= "rdr on $iface proto tcp from any to !($iface) port 80 -> 192.168.10.1 port 80\n";
	Line 1289: 				$rules .= "rdr on $PPPOE_ALIAS proto tcp from any to !192.168.10.1 port 80 -> 192.168.10.1 port 80\n";
	Line 1289: 				$rules .= "rdr on $PPPOE_ALIAS proto tcp from any to !192.168.10.1 port 80 -> 192.168.10.1 port 80\n";
	Line 1293: 				$rules .= "rdr on $PPTP_ALIAS proto tcp from any to !192.168.10.1 port 80 -> 192.168.10.1 port 80\n";
	Line 1293: 				$rules .= "rdr on $PPTP_ALIAS proto tcp from any to !192.168.10.1 port 80 -> 192.168.10.1 port 80\n";
	Line 1306: 				$rules .= "pass in quick on $PPPOE_ALIAS proto tcp from any to !192.168.10.1 port $port flags S/SA keep state\n";			
	Line 1309: 				$rules .= "pass in quick on $PPTP_ALIAS proto tcp from any to !192.168.10.1 port $port flags S/SA keep state\n";

but I ended up with a state table table full with almost 60.000 connections:

I think I´ve done something wrong…

Hulk-pingu_3223.png_thumb

cylent

i have 1.2.3-release and this REALLY slowed my net and webgui management to a crawl!!!

either it doesnt work with 1.2.3 or i did something wrong.

pls advise

rdlugosz

So I'm using 1.2.3 with the transparent proxy enabled (and verified on b/c it's showing hits in the lightsquid logs). I am NOT seeing the behavior described in the first post. On my system, I see P2P traffic in the P2P queues and web traffic seems to go into the qOthersDownH queue.

Also curious is that I do not see any packets in my queues related to my VoIP adapter now that I've re-run the wizard and gave it the IP Address of the adapter. One of the first rules is that anything on that IP routes to the qVOIP queues… I saw some traffic being registered there when it was just set up to route the SIP port packets to those queues, but for whatever reason it doesn't show up when the whole IP is sent there.

Anyway, my main point in posting is to say that I'm NOT seeing the behavior of web traffic going to the default queues due to the proxy. It's possible that I've got something configured incorrectly, but it appears that the shaper is doing its job. Any thoughts?

moonspud

Does this work with penalty ip shapping?

udhaya

Hello iBeej,

I tried as you said, I am able to penalize the download and not the upload. Is there anything I need to follow. Please suggest me with more ideas. I am new to this.

Ver: 1.2.3-Release