Squid3 - New GUI with sync, normal and reverse proxy

ccesario · Apr 15, 2012, 3:39 PM

Hi guys,

I'm testing new squid3 package, and after install it, I'm having a lot errors in http connections, squid show me a lot 'TCP_MISS/503'. This happen often in forms posts, so I need re-send form ou press F5.
I tested exhaustively the squid-2.7.9_1 + squidGuard and problem no happen. So I too tested exhaustively the squid3 + SquidGuard, and I give this problem.

All squid versions have the same config. And this problem only occurs in 'Transparent Mode'

Somebdoy can please test it and report the results?!

Thanks

mhab12 · Apr 15, 2012, 3:46 PM

Just in case others were seeing performance issues, I saw my bandwidth drop to <5mbps after installing Squid3, however changing from AUFS to diskd brought the bandwidth backup up to approximately 60mbps where it should be.

Donny · Apr 15, 2012, 4:21 PM

@marcelloc:

@Donny:

Next step I just want to be sure, I will try to clean install pfSense again in my testing machine, after that > First install: Dansguardian > Second install: Squid 3.

Yes :)

After clean install pfSense, I try first to install Dansguardian. I got the same result as I told you before. Dansguardian does not appear on services menu. So I wait a few minute and then try to refresh pfSense WebGUI and not thing changed. The last final "fantasy" I reboot pfSense and it does not appear again. (The final "fantasy" I just only make a joke because today is Sunday, you should be relax.). Then the way I have to do before I am going to install Squid3 is reinstall Dansguardian and finally Dansquardian is appear.

The next step I am going to install Squid3

Just let you know, Marcelloc.

marcelloc · Apr 15, 2012, 4:41 PM

Before using disk cache, I suggest you to enable softupdates on /usr and /var. The performance difference is huge.

Donny · Apr 15, 2012, 5:48 PM

Now Squid3 and Dansguardian is working. I don't find any error yet. The next step I will trying to configure firewall, NAT with HTTP and HTTPS for how Squid3 and Dansguardian work together.

harish · Apr 15, 2012, 6:21 PM

error is gone but could not start squidguard, i rechecked with reinstalling the squidguard, but fails to start.

Nachtfalke · Apr 15, 2012, 9:04 PM

@harish:

error is gone but could not start squidguard, i rechecked with reinstalling the squidguard, but fails to start.

As far as I can say that at the moment the "Integrations" box isn't working. So put the commands squidguard creates manually in "custom options":

redirect_program /usr/local/bin/squidGuard -c /usr/local/etc/squidGuard/squidGuard.conf
redirector_bypass on
redirect_children 8

marcelloc · Apr 15, 2012, 10:58 PM

Just updated squid3 package to version 2.0.2 to fix integration erros.

Please update,test and feedback :)

harish · Apr 16, 2012, 5:42 AM

yes now its working after custom option.

marcelloc · Apr 16, 2012, 7:25 AM

Hi all,

After looking for some options o squid-wiki, I've included dynamic update options to cache tab on pkg v 2.0.3

Nachtfalke · Apr 16, 2012, 9:47 AM

@marcelloc:

Hi all,

After looking for some options o squid-wiki, I've included dynamic update options to cache tab on pkg v 2.0.3

Setting the refresh_pattern to -1 is not a really good solution because it always downloads the file even if the user aborted it. This causes that squid downloads most of the time on its own which causes more traffic usage for squid as it saves. it is better to set some values according to the update size:


Finish transfer if less than x KB remaining: 102400
Abort transfer if more than x KB remaining: 102400
Finish transfer if more than x % finished: 60

These are the same values you can set in squid - traffic mangt.
What is happening if I enable squid windows update and set different values on the mngt tab ?

What do you use as refresh pattern for the windows updates ? I am using these for squid2


refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;

Thanks :-)

Cino · Apr 16, 2012, 2:12 PM

@Marcelloc Nice work man!!! I do have a request/wish for this… Would it be possible to setup the GUI to have squid-reserve run as a separate process? This would allow it to have its own options and the log file could be separate. I created a separate conf file and added some code to the squid.inc so it would start with squid processes on my box. Basically where it starts/stop the service and creates the squid.sh file, i added another like to include my squid-reverse.conf.

just a thought when you have "free" time...

marcelloc · Apr 16, 2012, 2:31 PM

@Nachtfalke:

Setting the refresh_pattern to -1 is not a really good solution because it always downloads the file even if the user aborted it. This causes that squid downloads most of the time on its own which causes more traffic usage for squid as it saves. it is better to set some values according to the update size:
Finish transfer if less than x KB remaining: 102400
Abort transfer if more than x KB remaining: 102400
Finish transfer if more than x % finished: 60
These are the same values you can set in squid - traffic mangt.
What is happening if I enable squid windows update and set different values on the mngt tab ?

Nothing, I just force range_offset_limit -1 when updates are set, all traffic mgmt are configured by users.

@Nachtfalke:

What do you use as refresh pattern for the windows updates ? I am using these for squid2


refresh_pattern -i .*microsoft\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;
refresh_pattern -i .*windowsupdate\.com/.*\.(cab|exe|msi|msp) 259200 100% 259200 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-private;

Just the suggested by wiki

refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i my.windowsupdate.website.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

marcelloc · Apr 16, 2012, 2:41 PM

@Cino:

@Marcelloc Nice work man!!! I do have a request/wish for this… Would it be possible to setup the GUI to have squid-reserve run as a separate process? This would allow it to have its own options and the log file could be separate. I created a separate conf file and added some code to the squid.inc so it would start with squid processes on my box. Basically where it starts/stop the service and creates the squid.sh file, i added another like to include my squid-reverse.conf.

just a thought when you have "free" time...

Hi cino,

I'ts a good idea but I have no idea how services tab could identify these two squid processes?

Cino · Apr 16, 2012, 2:49 PM

@marcelloc:

Hi cino,

I'ts a good idea but I have no idea how services tab could identify these two squid processes?

Good point! here is the output of mine… Keep in mind when I have squid.inc, i put the full path for path conf files... if there is a shutdown, reconfigure; i included the full path to the conf in the syantx


[2.1-DEVELOPMENT][root@]/root(1): ps -aux | grep squid
root    7806  0.0  0.2 10420  7120  ??  Is    7:48AM   0:00.00 /usr/local/sbin/squid -f /usr/local/etc/squid/squid-reverse.conf
proxy   7895  0.0  0.4 17596 11036  ??  S     7:48AM   0:02.72 (squid) -f /usr/local/etc/squid/squid-reverse.conf (squid)
root    7953  0.0  0.2 10420  7136  ??  Is    7:48AM   0:00.00 /usr/local/sbin/squid -f /usr/local/etc/squid/squid.conf
proxy   8397  0.0  0.8 35376 24892  ??  S     7:48AM   3:52.19 (squid) -f /usr/local/etc/squid/squid.conf (squid)
proxy  46782  0.0  0.3 54556  8496  ??  S     7:48AM   0:03.85 (squidGuard) -c /usr/local/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  47028  0.0  0.3 54556  8496  ??  I     7:48AM   0:00.84 (squidGuard) -c /usr/local/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  47362  0.0  0.3 54556  8496  ??  I     7:48AM   0:00.39 (squidGuard) -c /usr/local/etc/squidGuard/squidGuard.conf (squidGuard)
root   28706  0.0  0.0  3524  1256   0  S+   10:49AM   0:00.01 grep squid

marcelloc · Apr 16, 2012, 3:06 PM

@Cino:

I'ts a good idea but I have no idea how services tab could identify these two squid processes?
Good point! here is the output of mine… Keep in mind when I have squid.inc, i put the full path for path conf files... if there is a shutdown, reconfigure; i included the full path to the conf in the syantx

Ok. Let's try to config it.

I did a lot of changes on squid.inc for this package. Can you try to reapply you patch on current config or show me what you did?

asterix · Apr 16, 2012, 3:21 PM

Getting this error. Did a clean pfSense install. SquidGuard won't start either.. as Squid fails to start.

Apr 16 11:22:56 php: /pkg_mgr_install.php: The command '/usr/local/sbin/squid' returned exit code '1', the output was 'FATAL: Bungled squid.conf line 7: http_port 127.0.0.1:3128 intercept Squid Cache (Version 2.7.STABLE9): Terminated abnormally.'
Apr 16 11:22:56 squid[34066]: Bungled squid.conf line 7: http_port 127.0.0.1:3128 intercept

marcelloc · Apr 16, 2012, 3:24 PM

@asterix:

Getting this error. Did a clean pfSense install. SquidGuard won't start either.. as Squid fails to start.

Apr 16 11:22:56 php: /pkg_mgr_install.php: The command '/usr/local/sbin/squid' returned exit code '1', the output was 'FATAL: Bungled squid.conf line 7: http_port 127.0.0.1:3128 intercept Squid Cache (Version 2.7.STABLE9): Terminated abnormally.'
Apr 16 11:22:56 squid[34066]: Bungled squid.conf line 7: http_port 127.0.0.1:3128 intercept

You running squid is Version 2.7.STABLE9.

What version of pfsense are you using?

Take a look on first posts of this thread to see package install sequence.

att,
Marcello Coutinho

Cino · Apr 16, 2012, 4:01 PM

@marcelloc:

@Cino:

I'ts a good idea but I have no idea how services tab could identify these two squid processes?
Good point! here is the output of mine… Keep in mind when I have squid.inc, i put the full path for path conf files... if there is a shutdown, reconfigure; i included the full path to the conf in the syantx

Ok. Let's try to config it.

I did a lot of changes on squid.inc for this package. Can you try to reapply you patch on current config or show me what you did?

i sent you a pm

asterix · Apr 16, 2012, 5:42 PM

@marcelloc:

@asterix:

Getting this error. Did a clean pfSense install. SquidGuard won't start either.. as Squid fails to start.

Apr 16 11:22:56 php: /pkg_mgr_install.php: The command '/usr/local/sbin/squid' returned exit code '1', the output was 'FATAL: Bungled squid.conf line 7: http_port 127.0.0.1:3128 intercept Squid Cache (Version 2.7.STABLE9): Terminated abnormally.'
Apr 16 11:22:56 squid[34066]: Bungled squid.conf line 7: http_port 127.0.0.1:3128 intercept

You running squid is Version 2.7.STABLE9.

What version of pfsense are you using?

Take a look on first posts of this thread to see package install sequence.

att,
Marcello Coutinho

I clean installed this version
2.0.1-RELEASE (amd64)
built on Mon Dec 12 18:16:13 EST 2011

Then went into packages and installed squid3 first. Same settings I have been using for over a year. Nothing has changed. I reinstalled pfSense again and again tried with your latest package.. same issue.