Squid - "null" storage module



  • Hi,

    Would it be possible to add squid "null" storage module option to "Cache Management" page of squid package ? I have achieved this by modifying squid_cache.xml, but it would be nice to have it "out of the box". I think, with the arrival of 1.2.3 and the ability to install packages on embedded installations, pfsense has become quite powerful platform for web filtering and especially embedded installations would benefit from this.


  • Rebel Alliance Developer Netgate

    You can use custom options to add things like this, but it may be something that can be added to the package eventually.


  • Rebel Alliance Developer Netgate

    Out of curiosity, what changes did you make to the .xml file to enable this?

    You may also want to use this in custom options:

    cache deny all
    cache_dir null /tmp



  • Well, originally I have only modified squid_cache.xml, but since squid complained I came up with this:

    squid.inc.patch:

    
    --- C:\temp\original\squid.inc	2009-12-17 00:10:50.000000000 +0100
    +++ C:\temp\modified\squid.inc	2009-12-17 01:22:39.671246300 +0100
    @@ -691,7 +691,20 @@
     	$memory_policy = ($settings['memory_replacement_policy'] ? $settings['memory_replacement_policy'] : 'heap GDSF');
     	$offline_mode = ($settings['enable_offline'] == 'on' ? 'on' : 'off');
    
    -	$conf = <<<eod<br>+	if ($disk_cache_system == 'null') {
    +		$conf = <<<eod1<br>+cache_mem $memory_cache_size MB
    +maximum_object_size_in_memory 32 KB
    +memory_replacement_policy $memory_policy
    +cache_replacement_policy $cache_policy
    +cache_dir $disk_cache_system $cachedir
    +minimum_object_size $min_objsize KB
    +maximum_object_size $max_objsize KB
    +offline_mode $offline_mode
    +
    +EOD1;
    +	} else {
    +		$conf = << <eod2<br>cache_mem $memory_cache_size MB
     maximum_object_size_in_memory 32 KB
     memory_replacement_policy $memory_policy
    @@ -701,7 +714,8 @@
     maximum_object_size $max_objsize KB
     offline_mode $offline_mode
    
    -EOD;
    +EOD2;
    +	};
    
     	if (!empty($settings['cache_swap_low'])) $conf .= "cache_swap_low {$settings['cache_swap_low']}\n";
     	if (!empty($settings['cache_swap_high'])) $conf .= "cache_swap_high {$settings['cache_swap_high']}\n";</eod2<br></eod1<br></eod<br> 
    

    squid_cache.xml.patch:

    
    --- C:\temp\original\squid_cache.xml	2009-12-17 00:10:51.000000000 +0100
    +++ C:\temp\modified\squid_cache.xml	2009-12-17 01:23:33.717429500 +0100
    @@ -92,13 +92,14 @@
     		 <field><fielddescr>Hard disk cache system</fielddescr>
     			<fieldname>harddisk_cache_system</fieldname>
    -			<description>This specifies the kind of storage system to use. <p> <b> ufs </b> is the old well-known Squid storage format that has always been there. <p> <b> aufs </b> uses POSIX-threads to avoid blocking the main Squid process on disk-I/O. (Formerly known as async-io.) <p> <b> diskd </b> uses a separate process to avoid blocking the main Squid process on disk-I/O.</description>
    +			<description>This specifies the kind of storage system to use. <p> <b> ufs </b> is the old well-known Squid storage format that has always been there. <p> <b> aufs </b> uses POSIX-threads to avoid blocking the main Squid process on disk-I/O. (Formerly known as async-io.) <p> <b> diskd </b> uses a separate process to avoid blocking the main Squid process on disk-I/O. <p> <b> null </b> completely disables all disk caching.</description>
     			<type>select</type>
     			<default_value>aufs</default_value>
     			 <options><option><name>ufs</name><value>ufs</value></option>
     				<option><name>aufs</name><value>aufs</value></option>
     				<option><name>diskd</name><value>diskd</value></option>
    +				<option><name>null</name><value>null</value></option></options></field> 		
    
    

    With this, I am able to configure null storage through web configuration and squid starts without complaints. But after a while squid process exits and I am unable to start it again. However, if I go to squid configuration and save config, squid starts and is running for a while until it exits again.

    squid system log:

    
    squid[963]: Squid Parent: child process 966 started
    Dec 21 09:32:03 	squid[963]: Squid Parent: child process 966 exited with status 0
    Dec 21 09:32:04 	squid[1006]: Squid Parent: child process 1009 started
    Dec 21 09:32:11 	squid[1006]: Squid Parent: child process 1009 exited due to signal 9
    Dec 21 09:32:17 	squid[1044]: Squid Parent: child process 1046 started
    Dec 21 09:32:23 	squid[1044]: Squid Parent: child process 1046 exited with status 0
    Dec 21 09:32:25 	squid[1075]: Squid Parent: child process 1078 started
    Dec 21 09:32:31 	squid[1075]: Squid Parent: child process 1078 exited with status 0
    Dec 21 09:32:32 	squid[1106]: Squid Parent: child process 1108 started
    Dec 21 09:32:38 	squid[1106]: Squid Parent: child process 1108 exited with status 0
    Dec 21 09:32:40 	squid[1137]: Squid Parent: child process 1139 started
    Dec 21 09:32:46 	squid[1137]: Squid Parent: child process 1139 exited with status 0
    Dec 21 09:32:47 	squid[1168]: Squid Parent: child process 1170 started
    Dec 21 09:32:53 	squid[1168]: Squid Parent: child process 1170 exited due to signal 9
    
    


  • This may be a stupid question, but what is the difference between this 'null' mode and setting the max disk cache/max mem cache to zero?


  • Rebel Alliance Developer Netgate

    mhab12,

    Here is what the Squid FAQ says, though apparently since we use squid 2.7.x the null storage module isn't needed anymore:

    http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid#Can_I_make_Squid_proxy_only.2C_without_caching_anything.3F



  • @mhab12:

    I tried setting max disk cache to 0, but it always defaults to 100MB in squid.conf and I need it to be configurable through web interface.

    @jimp:

    My end goal is not to completely disable all caching, but only disk caching so I am still able to use memory cache.
    I'm going to modify squid.inc so that it never includes cache_dir directive to squid.conf, but I suspect that php scripts are causing trouble when they try to check, create and set correct permissions on the cache directory and the end result is squid exiting.


  • Rebel Alliance Developer Netgate

    @wagebox:

    @mhab12:
    I tried setting max disk cache to 0, but it always defaults to 100MB in squid.conf and I need it to be configurable through web interface.

    This is likely a bug. I helped someone fix this the other day, somehow their config had a stray "<config>" tag under the cache management section of the config.xml file and removing it (and other duplicate config entries that happened because of it) cleared up the problem.

    The broken config looked like this:

             <installedpackages><squidcache><config><config>[lots of settings]</config> 
                            <config>[lots of settings from the next save]</config> 
                            <config>[lots of settings from the next save]</config> 
                            [...repeat a bunch...]</config></squidcache></installedpackages> 
    ```</config>

Log in to reply