Should squid empty its cache directory on restart?
-
Just noticed after restarting one of my pfsense servers that the squid cache directory is now empty, is this expected behavior or can it be made persistent (its main use is for widnows updates).
config:
# This file is automatically generated by pfSense # Do not edit manually ! http_port 192.168.10.254:3128 http_port 127.0.0.1:3128 intercept icp_port 7 dns_v4_first off pid_filename /var/run/squid.pid cache_effective_user proxy cache_effective_group proxy error_default_language en icon_directory /usr/pbi/squid-amd64/etc/squid/icons visible_hostname localhost cache_mgr admin@localhost access_log /var/squid/logs/access.log cache_log /var/squid/logs/cache.log cache_store_log none sslcrtd_children 0 logfile_rotate 0 shutdown_lifetime 3 seconds # Allow local network(s) on interface(s) acl localnet src 192.168.10.0/24 uri_whitespace strip acl dynamic urlpath_regex cgi-bin \? cache deny dynamic cache_mem 2048 MB maximum_object_size_in_memory 128 KB memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA cache_dir ufs /var/squid/cache 240000 16 256 minimum_object_size 0 KB maximum_object_size 5120000 KB offline_mode offcache_swap_low 90 cache_swap_high 95 # No redirector configured #Remote proxies # Setup some default acls acl allsrc src all acl localhost src 127.0.0.1/32 acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 3128 1025-65535 acl sslports port 443 563 acl manager proto cache_object acl purge method PURGE acl connect method CONNECT # Define protocols used for redirects acl HTTP proto HTTP acl HTTPS proto HTTPS http_access allow manager localhost # Allow external cache managers acl ext_manager src 192.168.10.254 http_access allow manager ext_manager http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !safeports http_access deny CONNECT !sslports # Always allow localhost connections http_access allow localhost request_body_max_size 0 KB delay_pools 1 delay_class 1 2 delay_parameters 1 -1/-1 -1/-1 delay_initial_bucket_level 100 delay_access 1 allow allsrc # Reverse Proxy settings # Package Integration never_direct allow all cache_peer 127.0.0.1 parent 3125 0 name=havp no-query no-digest no-netdb-exchange default # Custom options acl aclname snmp_community public snmp_access allow aclname # Setup allowed acls # Allow local network(s) on interface(s) http_access allow localnet # Default block all to be sure http_access deny allsrccache.log snippet:
2015/01/13 00:32:51| Preparing for shutdown after 130895 requests 2015/01/13 00:32:51| Waiting 3 seconds for active connections to finish 2015/01/13 00:32:51| FD 32 Closing HTTP connection 2015/01/13 00:32:51| FD 33 Closing HTTP connection 2015/01/13 00:32:57| Starting Squid Cache version 3.1.22 for amd64-portbld-freebsd8.3... 2015/01/13 00:32:57| Process ID 98530 2015/01/13 00:32:57| With 11095 file descriptors available 2015/01/13 00:32:57| Initializing IP Cache... 2015/01/13 00:32:57| DNS Socket created at [::], FD 14 2015/01/13 00:32:57| DNS Socket created at 0.0.0.0, FD 15 2015/01/13 00:32:57| Adding domain cnlcomputers from /etc/resolv.conf 2015/01/13 00:32:57| Adding nameserver 8.8.8.8 from /etc/resolv.conf 2015/01/13 00:32:57| Adding nameserver 8.8.4.4 from /etc/resolv.conf 2015/01/13 00:32:57| User-Agent logging is disabled. 2015/01/13 00:32:57| Referer logging is disabled. 2015/01/13 00:32:57| Unlinkd pipe opened on FD 22 2015/01/13 00:32:57| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2015/01/13 00:32:57| Store logging disabled 2015/01/13 00:32:57| Swap maxSize 245760000 + 2097152 KB, estimated 19065934 objects 2015/01/13 00:32:57| Target number of buckets: 953296 2015/01/13 00:32:57| Using 1048576 Store buckets 2015/01/13 00:32:57| Max Mem size: 2097152 KB 2015/01/13 00:32:57| Max Swap size: 245760000 KB 2015/01/13 00:32:57| Rebuilding storage in /var/squid/cache (DIRTY) 2015/01/13 00:32:57| Using Least Load store dir selection 2015/01/13 00:32:57| Current Directory is /etc 2015/01/13 00:32:57| Loaded Icons. 2015/01/13 00:32:57| helperOpenServers: Starting 0/0 'ssl_crtd' processes 2015/01/13 00:32:57| helperOpenServers: No 'ssl_crtd' processes needed. 2015/01/13 00:32:57| Accepting HTTP connections at 192.168.10.254:3128, FD 24. 2015/01/13 00:32:57| Accepting intercepted HTTP connections at 127.0.0.1:3128, FD 25. 2015/01/13 00:32:57| Accepting ICP messages at [::]:7, FD 26. 2015/01/13 00:32:57| HTCP Disabled. 2015/01/13 00:32:57| Configuring Parent 127.0.0.1/3125/0 2015/01/13 00:32:57| Ready to serve requests. 2015/01/13 00:47:57| Done scanning /var/squid/cache swaplog (0 entries) 2015/01/13 00:47:57| Finished rebuilding storage from disk. 2015/01/13 00:47:57| 207762 Entries scanned 2015/01/13 00:47:57| 0 Invalid entries. 2015/01/13 00:47:57| 0 With invalid flags. 2015/01/13 00:47:57| 207762 Objects loaded. 2015/01/13 00:47:57| 0 Objects expired. 2015/01/13 00:47:57| 0 Objects cancelled. 2015/01/13 00:47:57| 0 Duplicate URLs purged. 2015/01/13 00:47:57| 0 Swapfile clashes avoided. 2015/01/13 00:47:57| Took 900.03 seconds (230.84 objects/sec). 2015/01/13 00:47:57| Beginning Validation Procedure 2015/01/13 00:47:57| Completed Validation Procedure 2015/01/13 00:47:57| Validated 415549 Entries 2015/01/13 00:47:57| store_swap_size = 47803814 2015/01/13 00:47:57| storeLateRelease: released 0 objects 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 00:47:57| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/13 00:47:57| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/13 01:28:49| logfileOpen: /var/log/squid/netdb.state: (2) No such file or directory 2015/01/13 01:28:49| netdbSaveState: /var/log/squid/netdb.state: (2) No such file or directory 2015/01/14 14:15:43| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/14 14:15:43| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/14 14:15:43| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/14 14:15:43| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/14 14:15:43| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/14 14:15:43| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/14 14:15:43| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/14 14:15:43| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/14 14:15:43| comm_old_accept: FD 25: (53) Software caused connection abort 2015/01/14 14:15:43| httpAccept: FD 25: accept failure: (53) Software caused connection abort 2015/01/14 16:06:05| Reconfiguring Squid Cache (version 3.1.22)... 2015/01/14 16:06:05| FD 24 Closing HTTP connection 2015/01/14 16:06:05| FD 25 Closing HTTP connection 2015/01/14 16:06:05| FD 26 Closing ICP connection 2015/01/14 16:06:05| Processing Configuration File: /usr/pbi/squid-amd64/etc/squid/squid.conf (depth 0) 2015/01/14 16:06:05| Starting Authentication on port 127.0.0.1:3128 2015/01/14 16:06:05| Disabling Authentication on port 127.0.0.1:3128 (interception enabled) 2015/01/14 16:06:05| Disabling IPv6 on port 127.0.0.1:3128 (interception enabled) 2015/01/14 16:06:05| Initializing https proxy context 2015/01/14 16:06:05| Store logging disabled 2015/01/14 16:06:05| User-Agent logging is disabled. 2015/01/14 16:06:05| Referer logging is disabled. 2015/01/14 16:06:05| DNS Socket created at [::], FD 15 2015/01/14 16:06:05| DNS Socket created at 0.0.0.0, FD 16 2015/01/14 16:06:05| Adding domain cnlcomputers from /etc/resolv.conf 2015/01/14 16:06:05| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2015/01/14 16:06:05| Adding nameserver 8.8.8.8 from /etc/resolv.conf 2015/01/14 16:06:05| Adding nameserver 8.8.4.4 from /etc/resolv.conf 2015/01/14 16:06:05| helperOpenServers: Starting 0/0 'ssl_crtd' processes 2015/01/14 16:06:05| helperOpenServers: No 'ssl_crtd' processes needed. 2015/01/14 16:06:05| Accepting HTTP connections at 192.168.10.254:3128, FD 18. 2015/01/14 16:06:05| Accepting intercepted HTTP connections at 127.0.0.1:3128, FD 24. 2015/01/14 16:06:05| Accepting ICP messages at [::]:7, FD 25. 2015/01/14 16:06:05| HTCP Disabled. 2015/01/14 16:06:05| Configuring Parent 127.0.0.1/3125/0 2015/01/14 16:06:05| Loaded Icons. 2015/01/14 16:06:05| Ready to serve requests. 2015/01/14 16:11:53| Reconfiguring Squid Cache (version 3.1.22)... 2015/01/14 16:11:53| FD 18 Closing HTTP connection 2015/01/14 16:11:53| FD 24 Closing HTTP connection 2015/01/14 16:11:53| FD 25 Closing ICP connection 2015/01/14 16:11:53| Processing Configuration File: /usr/pbi/squid-amd64/etc/squid/squid.conf (depth 0) 2015/01/14 16:11:53| Starting Authentication on port 127.0.0.1:3128 2015/01/14 16:11:53| Disabling Authentication on port 127.0.0.1:3128 (interception enabled) 2015/01/14 16:11:53| Disabling IPv6 on port 127.0.0.1:3128 (interception enabled) 2015/01/14 16:11:53| Initializing https proxy context 2015/01/14 16:11:53| Store logging disabled 2015/01/14 16:11:53| User-Agent logging is disabled. 2015/01/14 16:11:53| Referer logging is disabled. 2015/01/14 16:11:53| DNS Socket created at [::], FD 15 2015/01/14 16:11:53| DNS Socket created at 0.0.0.0, FD 16 2015/01/14 16:11:53| Adding domain cnlcomputers from /etc/resolv.conf 2015/01/14 16:11:53| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2015/01/14 16:11:53| Adding nameserver 8.8.8.8 from /etc/resolv.conf 2015/01/14 16:11:53| Adding nameserver 8.8.4.4 from /etc/resolv.conf 2015/01/14 16:11:53| helperOpenServers: Starting 0/0 'ssl_crtd' processes 2015/01/14 16:11:53| helperOpenServers: No 'ssl_crtd' processes needed. 2015/01/14 16:11:53| Accepting HTTP connections at 192.168.10.254:3128, FD 18. 2015/01/14 16:11:53| Accepting intercepted HTTP connections at 127.0.0.1:3128, FD 24. 2015/01/14 16:11:53| Accepting ICP messages at [::]:7, FD 25. 2015/01/14 16:11:53| HTCP Disabled. 2015/01/14 16:11:53| Configuring Parent 127.0.0.1/3125/0 2015/01/14 16:11:53| Loaded Icons. 2015/01/14 16:11:53| Ready to serve requests. 2015/01/14 16:19:10| Preparing for shutdown after 75753 requests 2015/01/14 16:19:10| Waiting 3 seconds for active connections to finish 2015/01/14 16:19:10| FD 18 Closing HTTP connection 2015/01/14 16:19:10| FD 24 Closing HTTP connection 2015/01/14 16:19:22| Starting Squid Cache version 3.1.22 for amd64-portbld-freebsd8.3... 2015/01/14 16:19:22| Process ID 76037 2015/01/14 16:19:22| With 11095 file descriptors available 2015/01/14 16:19:22| Initializing IP Cache... 2015/01/14 16:19:22| DNS Socket created at [::], FD 11 2015/01/14 16:19:22| DNS Socket created at 0.0.0.0, FD 12 2015/01/14 16:19:22| Adding domain cnlcomputers from /etc/resolv.conf 2015/01/14 16:19:22| Adding nameserver 127.0.0.1 from /etc/resolv.conf 2015/01/14 16:19:22| Adding nameserver 8.8.8.8 from /etc/resolv.conf 2015/01/14 16:19:22| Adding nameserver 8.8.4.4 from /etc/resolv.conf 2015/01/14 16:19:22| User-Agent logging is disabled. 2015/01/14 16:19:22| Referer logging is disabled. 2015/01/14 16:19:22| Unlinkd pipe opened on FD 18 2015/01/14 16:19:22| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec 2015/01/14 16:19:22| Store logging disabled 2015/01/14 16:19:22| Swap maxSize 245760000 + 2097152 KB, estimated 19065934 objects 2015/01/14 16:19:22| Target number of buckets: 953296 2015/01/14 16:19:22| Using 1048576 Store buckets 2015/01/14 16:19:22| Max Mem size: 2097152 KB 2015/01/14 16:19:22| Max Swap size: 245760000 KB 2015/01/14 16:19:22| Old swap file detected... 2015/01/14 16:19:22| Rebuilding storage in /var/squid/cache (DIRTY) 2015/01/14 16:19:22| Using Least Load store dir selection 2015/01/14 16:19:22| Current Directory is /usr/local/www 2015/01/14 16:19:22| Loaded Icons. 2015/01/14 16:19:22| helperOpenServers: Starting 0/0 'ssl_crtd' processes 2015/01/14 16:19:22| helperOpenServers: No 'ssl_crtd' processes needed. 2015/01/14 16:19:22| Accepting HTTP connections at 192.168.10.254:3128, FD 23. 2015/01/14 16:19:22| Accepting intercepted HTTP connections at 127.0.0.1:3128, FD 24. 2015/01/14 16:19:22| Accepting ICP messages at [::]:7, FD 25. 2015/01/14 16:19:22| HTCP Disabled. 2015/01/14 16:19:22| Configuring Parent 127.0.0.1/3125/0 2015/01/14 16:19:22| Ready to serve requests. 2015/01/14 16:19:22| Done reading /var/squid/cache swaplog (49 entries) 2015/01/14 16:19:22| Finished rebuilding storage from disk. 2015/01/14 16:19:22| 5 Entries scanned 2015/01/14 16:19:22| 43 Invalid entries. 2015/01/14 16:19:22| 0 With invalid flags. 2015/01/14 16:19:22| 5 Objects loaded. 2015/01/14 16:19:22| 0 Objects expired. 2015/01/14 16:19:22| 0 Objects cancelled. 2015/01/14 16:19:22| 0 Duplicate URLs purged. 2015/01/14 16:19:22| 0 Swapfile clashes avoided. 2015/01/14 16:19:22| Took 0.03 seconds (192.26 objects/sec). 2015/01/14 16:19:22| Beginning Validation Procedure 2015/01/14 16:19:22| Completed Validation Procedure 2015/01/14 16:19:22| Validated 35 Entries 2015/01/14 16:19:22| store_swap_size = 54 2015/01/14 16:19:23| storeLateRelease: released 0 objects 2015/01/14 16:20:46| Preparing for shutdown after 11 requests 2015/01/14 16:20:46| Waiting 3 seconds for active connections to finishThanks
-
You have nothing under /var/squid/cache?? I haven't noticed this myself.
-
When I restart the persistent cache on disk stays untouched. Only thing lost, obviously, is the cached objects in RAM.
-
I enabled cache manager only to then see 5mb used under the /cache directory when it should be gigabytes with all the windows cab and PSF files, I had been getting cache hits so I know they were there. Also have a look at the cache.log it went from tens of thousands of objects to just tens.
-
Hi,
I have had this issue for quite some time, I spent few days going through squid confg and start-up files butfound nothing suspicious. I then started thinking maybe it is pfSense fault not squid, I changed squid cache directory from the default /var/squid/cache to /home/squid_cache and voila!! All my cached downloads are now persistent after reboot.
After changing the cache directory in the GUI, I restarted squid from an ssh terminal "/usr/local/etc/rc.d/squid.sh stop" then "/usr/local/etc/rc.d/squid.sh start". Squid then created the new cache directory and restarted properly. I checked the old cache directory and it was still there (squid did not delete), however, after reboot the old cache directory in /var was deleted!!
I am guessing this is what's happening: user Start the machine –-> ... --> pfSense for some reason deletes /var/squid/cache --> ... --> squid starts and find no cache directory --> squid creates a new empty directory as specified in its confg file --> user ends up with an empty cache.
Give this a try, I hop it helps.
-
Do you have any packages installed like Sarg or Lightsquid?
-
Squid is the only package I have installed, a box that will do DHCP, firewall and DNS & web caching is what I was after when I built this.
By moving the cache out of /var, squid is finally caching (into a persistent cache). I have had this running for like a week now caching updates for different machines, I got it rebooted few times and the cache is ~4GB and keeps growing :)
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.