Squid - rotate nightmare

bellera

http://wiki.squid-cache.org/SquidFaq/SquidLogs#swap.state

If you remove this file while squid IS running, you can easily recreate it. The safest way is to simply shutdown the running process:

% squid -k shutdown

This will disrupt service, but at least you will have your swap log back. Alternatively, you can tell squid to rotate its log files. This also causes a clean swap log to be written.

% squid -k rotate

A clean swap log to be written –> At 00:00 I see swap.state at 0 bytes. And squid works well…

http://wiki.squid-cache.org/SquidFaq/OperatingSquid#I_want_to_restart_Squid_with_a_clean_cache

Squid-2.6 and later contain mechanisms which will automatically detect dirty information in both the cache directories and swap.state file. When squid starts up it runs these validation and security checks. The objects which fail for any reason are automatically purged from the cache.

The above mechanisms can be triggered manually to force squid into a full cache_dir scan and re-load all objects from disk by simply shuttign down Squid and deleting the swap.state journal from each cache_dir before restarting.

NP: Deleting the swap.state before shutting down will cause Squid to generate new ones and fail to do the re-scan you wanted.

The NP is very interesting. squid will not spend time rebuilding the entire swap.state

marcelloc

@bellera:

http://wiki.squid-cache.org/SquidFaq/OperatingSquid#I_want_to_restart_Squid_with_a_clean_cache

If you remove the swap.state file while squid is not running, it will have to completely rescan your cache folder to rebuild it once squid is started back up. This can be a lengthy and time consuming process. It may be better to remove the contents of the existing cache folder, and rebuild the structure again by running:

I've updated the package to include a new option to admin chose if cache is clean on log rotates o clean if hd reaches 90% of disk space.

marcelloc

I'll bump package version after these tests, please reinstall the package, test and feedback.

bellera

@marcelloc:

I've updated the package to include a new option to admin chose if cache is clean on log rotates o clean if hd reaches 90% of disk space.

It seems a good idea.

I'll bump package version after these tests, please reinstall the package, test and feedback.

Ok, but I need some days. I'm very busy with this big change at my main networking site.

marcelloc

@bellera:

Ok, but I need some days. I'm very busy with this big change at my main networking site.

You can change cron execution time to test it faster :)

bellera

rotate of cache.log and swap.state correct!

[2.1-RELEASE][admin@pfsense.localdomain]/root(36): ./see_squid_logs.sh 
-rw-r-----  1 proxy  proxy  21839 Apr  5 16:03 /var/squid/logs/cache.log
-rw-rw-rw-  1 proxy  proxy  338844 Apr  5 16:27 /var/squid/cache/swap.state
[2.1-RELEASE][admin@pfsense.localdomain]/root(37): ./see_squid_logs.sh
-rw-r-----  1 proxy  proxy      0 Apr  5 16:55 /var/squid/logs/cache.log
-rw-r-----  1 proxy  proxy  21839 Apr  5 16:03 /var/squid/logs/cache.log.0
-rw-rw-rw-  1 proxy  proxy  337212 Apr  5 16:55 /var/squid/cache/swap.state
-rw-r-----  1 proxy  proxy       0 Apr  5 16:55 /var/squid/cache/swap.state.last-clean

but

SSL Bump not working with 2.2.2 package.

Reported at https://forum.pfsense.org/index.php?topic=62256.msg407762#msg407762

bellera

@exograpix asked for explaining the new option about rotate cron.

squid3-dev 3.3.10 pkg 2.2.1

0  	0  	*  	*  	*  	root  	/bin/rm /var/squid/cache/swap.state; /usr/pbi/squid-amd64/sbin/squid -k rotate -f /usr/pbi/squid-amd64/etc/squid/squid.conf
*/15  	*  	*  	*  	*  	root  	/usr/local/pkg/swapstate_check.php

squid3-dev 3.3.10 pkg 2.2.2 version default

0  	0  	*  	*  	*  	root  	/usr/pbi/squid-amd64/sbin/squid -k rotate -f /usr/pbi/squid-amd64/etc/squid/squid.conf
*/15  	*  	*  	*  	*  	root  	/usr/local/pkg/swapstate_check.php

squid3-dev 3.3.10 pkg 2.2.2 version with reseting the cache –--> Proxy server: Cache management: Clear cache on log rotate

0  	0  	*  	*  	*  	root  	/usr/local/pkg/swapstate_check.php clean; /usr/pbi/squid-amd64/sbin/squid -k rotate -f /usr/pbi/squid-amd64/etc/squid/squid.conf
*/15  	*  	*  	*  	*  	root  	/usr/local/pkg/swapstate_check.php

So, if you activate the new option you will reset your cache every night.

At 2.2.1 cache.log file wasn't rotate because squid.conf didn't have any debug_options rotate=N (squid3 syntax) line. And rm for swap.state is not necessary, as discussed in this thread.