Squid - diskd bug (ipcs and ipcrm not available)
-
Tested with squid3-dev 3.3.10 pkg 2.2.1
I'm using squid2 with diskd (on FreeBSD machines) since year 2009 without problems.
However, I had to modify /usr/local/etc/rc.d/squid script to nicely clean diskd squid processes:
Modified code for squid stop
squid_stop() { echo "Stopping ${name}." ${command} ${squid_flags} -k shutdown run_rc_command poll # http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8 sleep 5 ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\n", $1, $2}' | /bin/sh killall -u squid 2>/dev/null }Original code for squid stop
squid_stop() { echo "Stopping ${name}." ${command} ${squid_flags} -k shutdown run_rc_command poll }Now, I'm migrating to squid into pfSense…
I wanted to do the same with /usr/pbi/squid-amd64/etc/rc.d/squid but ipcs and ipcrm aren't available in pfSense.
http://www.freebsd.org/cgi/man.cgi?query=ipcs
http://www.freebsd.org/cgi/man.cgi?query=ipcrmHow to see unclean diskd processes?
ps aux | grep diskd proxy 740 0.0 0.0 11620 2180 ?? IN Fri07PM 0:00.01 diskd 99774468 99774469 99774470 proxy 12825 0.0 0.0 11620 2316 ?? IN Sat11AM 0:00.07 diskd 8102916 8102917 8102918 proxy 14947 0.0 0.0 11620 2244 ?? IN Fri08PM 0:00.01 diskd 13276164 13276165 13276166 proxy 20939 0.0 0.0 11620 2180 ?? IN Fri07PM 0:00.01 diskd 18546692 18546693 18546694 proxy 53058 0.0 0.0 11620 2264 ?? S 12:24PM 0:00.34 diskd 52115460 52115461 52115462In the example, squid last stop/start has been at Sunday, 12:24PM. We can see many zombie diskd processes.
-
ipcs and ipcrm removed by design.
https://github.com/billm/pfsense-tools-centipede-slbd/blob/master/builder_scripts/remove.list.iso.7
What to do?
-
Download livefs.iso from
ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/
mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.
or http://www.filewatcher.com/m/ipcrm.12888-0.html
64 bit
fetch -o /usr/bin/ipcs ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcs chmod 555 /usr/bin/ipcs fetch -o /usr/bin/ipcrm ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcrm chmod 555 /usr/bin/ipcrm rehashNow, here is my pfSense running squid with diskd problems
ipcs Message Queues: T ID KEY MODE OWNER GROUP q 131072 99774468 --rwa------ proxy proxy q 131073 99774469 --rwa------ proxy proxy q 65538 18546692 --rwa------ proxy proxy q 65539 18546693 --rwa------ proxy proxy q 65540 13276164 --rwa------ proxy proxy q 65541 13276165 --rwa------ proxy proxy q 131078 8102916 --rwa------ proxy proxy q 131079 8102917 --rwa------ proxy proxy q 65544 52115460 --rwa------ proxy proxy q 65545 52115461 --rwa------ proxy proxy Shared Memory: T ID KEY MODE OWNER GROUP m 131072 99774470 --rw------- proxy proxy m 65537 18546694 --rw------- proxy proxy m 65538 13276166 --rw------- proxy proxy m 131075 8102918 --rw------- proxy proxy m 65540 52115462 --rw------- proxy proxy Semaphores: T ID KEY MODE OWNER GROUP -
Looking at /usr/local/pkg/squid.inc the code writes /usr/local/etc/rc.d/squid.sh
I patched /usr/local/pkg/squid.inc
cp -p squid.inc squid.inc-2014-04-30 vi squid.incOriginal code
{$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var} # Just to be sure... sleep 5 killall -9 squid 2>/dev/null killall pinger 2>/dev/nullPatched code
{$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var} # Just to be sure... sleep 5 # http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8 ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\\n", $1, $2}' | /bin/sh killall -9 squid 2>/dev/null killall pinger 2>/dev/nullI modified also /usr/local/etc/rc.d/squid.sh in order to have the changes and stop/start squid. Everything ok!
proxy processes
ps aux | grep ^proxy proxy 84290 0.0 0.4 48996 18532 ?? SN 8:20PM 0:03.12 (squid-1) -f /usr/pbi/squid-amd64/etc/squid/squid.conf (squid) proxy 84619 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 84890 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 85084 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 85292 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 85630 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 85844 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 85945 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 86112 0.0 0.2 64640 11008 ?? IN 8:20PM 0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 86450 0.0 0.0 11252 2124 ?? IN 8:20PM 0:00.01 (unlinkd) (unlinkd) proxy 86470 0.0 0.0 11620 2244 ?? IN 8:20PM 0:00.01 diskd 86312964 86312965 86312966Correct messages queues & shared memory
ipcs Message Queues: T ID KEY MODE OWNER GROUP q 196608 86312964 --rwa------ proxy proxy q 196609 86312965 --rwa------ proxy proxy Shared Memory: T ID KEY MODE OWNER GROUP m 131081 86312966 --rw------- proxy proxy Semaphores: T ID KEY MODE OWNER GROUP -
Corrected /usr/local/pkg/squid.inc patched code
It was
ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s****n", $1, $2}' | /bin/sh
Must be
ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s**\**n", $1, $2}' | /bin/sh
Also corrected at previous message.
-
Package updated to include install instructions and checks
-
Fantastic! Thanks!
-
For Squid3 (3.4.10_2 0.2.5), on pfsense 2.2 you need ipcs and ipcrm from a 10.1 base.
The diskd text should reflect that, now it still refers to 8.3.diskd uses a separate process to avoid blocking the main Squid process on disk-I/O. To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.diskd uses a separate process to avoid blocking the main Squid process on disk-I/O. To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.
