Squid and SquidGuard does not start after reboot

lsoltero

The way I see it the rm in rc is bad… the reason for this is that there are many programs that set /tmp/filter_dirty expecting to have some process later on detect it and then sync up the filters. The issue is that the rm in rc can happen at any time while subsystems are booting up. This is bad... The rc script should 'wait' until the system is fully booted before removing this file. But it doesn't. It just removes it willy-nilly at some random time without considering the effect on the boot process.

This is your classic multi-processing synchronization problem. You have one process removing a resource that another program needs without checking first.

From everything I have looked at it seems that having rc remove the file is unnecessary. Think about it... reloading process set /tmp/filter_dirty and remove it after the synchronization is complete. So... why should removing it once in rc matter? On my system, for example, currently the file does not exist even though the rm in rc is commented out. This is because the file is unlinked after the filter synchronization was completed. The house keeping in rc is unnecessary.

IMHO 1 of two things needs to be done.
1. rm needs to be removed from rc
or
2. some synchronization mechanism needs to be added to rc so that rc does not attempt to remove this file until after all processes have been started up.

Please check again with the powers that be... I believe that removing the rm in rc will not negatively impact the operation of the system for anyone and therefore should be done since there are now documented examples where in some instances it breaks the boot process. No telling what other users are having weird unexplained behavior because of this. In our case it was squid. but for some other user it might be.... Also, keep in mind that this thread has been open since Oct 2009... so this is a long standing problem. Not something that just cropped up.

summing up... having rm -f /tmp/filter_dirty in rc causes inconsistent behavior in pfSense and it is therefore detrimental to the project.

--luis

PS... here is another thread where boot up problems are probably caused by removal of /tmp/filter_dirty in /etc/rc
http://forum.pfsense.org/index.php?topic=19381.0#lastPost
this one was reported in Sept 2009.

jimp

Taking the rm out is the solution, but since that is a system file and not a package file, I'm hesitant to remove that as part of the package installation - that's what I was referring to.

/tmp/filter_dirty is removed automatically by check_reload_status after it reloads the filter – nothing needs to remove that file manually.

lsoltero

@jimp:

Taking the rm out is the solution, but since that is a system file and not a package file, I'm hesitant to remove that as part of the package installation - that's what I was referring to.

/tmp/filter_dirty is removed automatically by check_reload_status after it reloads the filter – nothing needs to remove that file manually.

Yes… i agree with you 100% this should not be removed by the package....

This needs to be patched in the distribution.

Who takes care of that?

--luis

jimp

Well 1.2.3 won't see any more updates to the base system. I can remove that line on 2.0, though (it's still there), after running it by a few others.

lsoltero

OK. thanks…

Its a pity that there are no more updates to 1.2.3...

I will add this to our list of things to do when building a new distribution...

I will let the folks at the applianceshop know as well.

Take care and thanks for your help.

--luis

jimp

It might be possible to get it committed into the 1.2.3 tree so a rebranded release for a customer could pick it up, failing that, an overlay for the rc file could be setup for a custom build.

Or I could just make a little patch package that removes it, too, but not bundle it with squid (since it doesn't affect everyone)

lsoltero

certainly getting the fix into the tree seems like a reasonable thing to do…

Squid users having problems hopefully will find this form thread and will then know that to fix the problem the need to edit /etc/rc
go to line 322 and delete it.

i.e.

make this

Remove stale files that have already been processed by bootup

scripts

rm -f /tmp/filter_dirty
rm -f /tmp/rc.linkup

look like this

Remove stale files that have already been processed by bootup

scripts

rm -f /tmp/rc.linkup

--luis

jimp

I just added a package to apply a simple patch that removes the line in question.

Give it a spin and see if it helps. It's called "Patch rc to leave filter_dirty" (for lack of a better name :-)

lsoltero

it failed!

Remove stale files that have already been processed by bootup

scripts

rm -f /tmp/filter_dirty
rm -f /tmp/rc.linkup
nohup /usr/bin/nice -n20 /usr/local/sbin/check_reload_status &

Here is the list of installed packages on my system

Patch rc to leave filter_dirty System No info, check the forum 0.1 Patch to stop /etc/rc from removing /tmp/filter_dirty on boot. Fixes boot issues with some packages on certain platforms.

squid Network No info, check the forum Current: 2.7.9
Installed: 2.7.8_1 High performance web proxy cache.

squidGuard Network Management No info, check the forum 1.3-2 High perfomance web proxy URL filter. Required proxy Squid package.

Let me know when you want me to try it again.

–luis

jimp

Apparently I messed up the path in the patch. I fixed it, should be OK now.

lsoltero

Yup… it works now

Remove stale files that have already been processed by bootup

scripts

rm -f /tmp/rc.linkup
nohup /usr/bin/nice -n20 /usr/local/sbin/check_reload_status &

thanks for your help.

--luis

lsoltero

Did you break squid trying to fix this issue per chance?

Here is my current situation.

1. brand new 1.2.3-RELEASE installation with a few configuration chances as required to get the router working on a LAN/WAN
2. installed the filter_dirty package
3. installed squid
4. turn on transparent proxy

squid will not run… proxy_monitor fires it up and it shuts down right away... On reboot the only mesg I see when syncing packages is the rc filter_dirty fix and squid that is it...

ps aux | grep squid
shows that squid is not running... proxy_monitor is running and continually tries to start squid.

if you try to go to any website the pf rule redirects you to the pfsense box which asks for a username and password. You enter the pfsense admin username and password and then it shows you the main admin website.

So... again the proxy is not running.

any ideas?

Here is a list of the packages that are installed.
Patch rc to leave filter_dirty System No info, check the forum 0.1 Patch to stop /etc/rc from removing /tmp/filter_dirty on boot. Fixes boot issues with some packages on certain platforms.

squid Network No info, check the forum 2.7.9 High performance web proxy cache.

--luis

jimp

Well I updated squid to 2.7.9 separately from trying to fix this, but I thought I had tested it on nanobsd at the time. That was late last week though, and I don't recall the specifics.

Is there nothing in the system log about squid at all?

lsoltero

OK found the problem…

in a clean installation /var/squid is not setup with the correct permissions.

/var/squid is owned by root:wheel with permission 755

many of the files such as access.log and owned by squid:squid with permission 640 when they should be owned by proxy.

chown -R proxy /var/squid

solves the problem.

So... this issue has nothing to do with the filter_dirty patch... However, some time in the recent past some changes must have been made to the squid package to break the permissions in /var/squid...

--luis

jimp

ok, I committed a fix for that. It should be in the package repo in about 2 minutes.

lsoltero

OK… it all seems to work now... squid, lightsquid, and squidGuard all install and play nicely.

One very small thing...

i notice that now by default squid install its log files into /var/squid/logs [note the (s)]. When you install and configure lightsquid the package automatically chances the logging directory for squid to /var/squid/log [note the absence of (s)]. This is not really a problem because it doen't break anything but the access files in the original /var/squid/logs folder never get rotated or deleted. They stay there forever… This occupies resources if the log file happens to be big before you turn on lightSquid.

Thanks again for your help.

--luis

lsoltero

I just thought we were done…

all the subsystems are working fine... however, i can't modify any of the cache settings for squid.

when I try to set the hard disk cache size, memory cache size, and maximum object size under proxy server->cache management the web page resets the value after you hit save. I have confirmed that the values are not being updated in squid.conf

cache_mem 8 MB
maximum_object_size_in_memory 32 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir aufs /var/squid/cache 100 16 256
minimum_object_size 0 KB
maximum_object_size 10 KB
offline_mode off

so what causes this?

oh man.... take a look at the entry for squidcache in config.xlm

<squidcache><config><config><harddisk_cache_size>25600</harddisk_cache_size>
<harddisk_cache_system>aufs</harddisk_cache_system>
<harddisk_cache_location>/var/squid/cache</harddisk_cache_location>
<memory_cache_size>128</memory_cache_size>
<minimum_object_size>0</minimum_object_size>
<maximum_object_size>2465792</maximum_object_size>
<level1_subdirs>16</level1_subdirs>
<memory_replacement_policy>heap GDSF</memory_replacement_policy>
<cache_replacement_policy>heap LFUDA</cache_replacement_policy>
<cache_swap_low>90</cache_swap_low>
<cache_swap_high>95</cache_swap_high>
<donotcache><enable_offline></enable_offline></donotcache></config>
<config><harddisk_cache_size>25600</harddisk_cache_size>
<harddisk_cache_system>aufs</harddisk_cache_system>
<harddisk_cache_location>/var/squid/cache</harddisk_cache_location>
<memory_cache_size>128</memory_cache_size>
<minimum_object_size>0</minimum_object_size>
<maximum_object_size>4194304</maximum_object_size>
<level1_subdirs>16</level1_subdirs>
<memory_replacement_policy>heap GDSF</memory_replacement_policy>
<cache_replacement_policy>heap LFUDA</cache_replacement_policy>
<cache_swap_low>90</cache_swap_low>
<cache_swap_high>95</cache_swap_high>
<donotcache><enable_offline></enable_offline></donotcache></config>
<config><harddisk_cache_size>15360</harddisk_cache_size>
<harddisk_cache_system>aufs</harddisk_cache_system>
<harddisk_cache_location>/var/squid/cache</harddisk_cache_location>
<memory_cache_size>128</memory_cache_size>
<minimum_object_size>0</minimum_object_size>
<maximum_object_size>1048576</maximum_object_size>
<level1_subdirs>16</level1_subdirs>
<memory_replacement_policy>heap GDSF</memory_replacement_policy>
<cache_replacement_policy>heap LFUDA</cache_replacement_policy>
<cache_swap_low>90</cache_swap_low>
<cache_swap_high>95</cache_swap_high>
<donotcache><enable_offline></enable_offline></donotcache></config>
<config><harddisk_cache_size>1000</harddisk_cache_size>
<harddisk_cache_system>aufs</harddisk_cache_system>
<harddisk_cache_location>/var/squid/cache</harddisk_cache_location>
<memory_cache_size>80</memory_cache_size>
<minimum_object_size>0</minimum_object_size>
<maximum_object_size>4</maximum_object_size>
<level1_subdirs>16</level1_subdirs>
<memory_replacement_policy>heap GDSF</memory_replacement_policy>
<cache_replacement_policy>heap LFUDA</cache_replacement_policy>
<cache_swap_low>90</cache_swap_low>
<cache_swap_high>95</cache_swap_high>
<donotcache><enable_offline></enable_offline></donotcache></config>
<config><harddisk_cache_size>25600</harddisk_cache_size>
<harddisk_cache_system>aufs</harddisk_cache_system>
<harddisk_cache_location>/var/squid/cache</harddisk_cache_location>
<memory_cache_size>128</memory_cache_size>
<minimum_object_size>0</minimum_object_size>
<maximum_object_size>4194304</maximum_object_size>
<level1_subdirs>16</level1_subdirs>
<memory_replacement_policy>heap GDSF</memory_replacement_policy>
<cache_replacement_policy>heap LFUDA</cache_replacement_policy>
<cache_swap_low>90</cache_swap_low>
<cache_swap_high>95</cache_swap_high>
<donotcache><enable_offline></enable_offline></donotcache></config></config></squidcache>

I would say there are a few too many entries here...what would cause this? It seems that every time I hit add a new config gets written but the old one is not removed!

the

<squidcache><config>extra <config>right at the head of this configuration seems particularly disturbing...

I will edit the config.xml manually to see if I can get this to work....

--luis</config></config></squidcache>

lsoltero

OK… that worked... here is squid.conf

cache_mem 128 MB
maximum_object_size_in_memory 32 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir aufs /var/squid/cache 25600 16 256
minimum_object_size 0 KB
maximum_object_size 2465792 KB
offline_mode off
cache_swap_low 90
cache_swap_high 95

and here is the entry in config.xml
<squidcache><config><harddisk_cache_size>25600</harddisk_cache_size>
<harddisk_cache_system>aufs</harddisk_cache_system>
<harddisk_cache_location>/var/squid/cache</harddisk_cache_location>
<memory_cache_size>128</memory_cache_size>
<minimum_object_size>0</minimum_object_size>
<maximum_object_size>2465792</maximum_object_size>
<level1_subdirs>16</level1_subdirs>
<memory_replacement_policy>heap GDSF</memory_replacement_policy>
<cache_replacement_policy>heap LFUDA</cache_replacement_policy>
<cache_swap_low>90</cache_swap_low>
<cache_swap_high>95</cache_swap_high>
<donotcache><enable_offline></enable_offline></donotcache></config></squidcache>

so something went awol with webinterface/xml editor.

any ideas?

--luis

jimp

Yeah that is a little odd. I committed a change that makes squid's default log dir match. That seemed like the better choice.

lsoltero

I take it that this is a bug in the webconfigurator and not the squid package… I will search the forum to see if anyone else has reported this issue.

--luis