Squid - diskd bug (ipcs and ipcrm not available)

[bellera]

Tested with squid3-dev 3.3.10 pkg 2.2.1

I'm using squid2 with diskd (on FreeBSD machines) since year 2009 without problems.

However, I had to modify /usr/local/etc/rc.d/squid script to nicely clean diskd squid processes:

Modified code for squid stop

squid_stop() {
	echo "Stopping ${name}."
	${command} ${squid_flags} -k shutdown
	run_rc_command poll
	# http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
	sleep 5
	ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\n", $1, $2}' | /bin/sh
	killall -u squid 2>/dev/null
}

Original code for squid stop

squid_stop() {
	echo "Stopping ${name}."
	${command} ${squid_flags} -k shutdown
	run_rc_command poll
}

Now, I'm migrating to squid into pfSense…

I wanted to do the same with /usr/pbi/squid-amd64/etc/rc.d/squid but ipcs and ipcrm aren't available in pfSense.

http://www.freebsd.org/cgi/man.cgi?query=ipcs
http://www.freebsd.org/cgi/man.cgi?query=ipcrm

How to see unclean diskd processes?

ps aux | grep diskd
proxy    740  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 99774468 99774469 99774470
proxy  12825  0.0  0.0 11620  2316  ??  IN   Sat11AM   0:00.07 diskd 8102916 8102917 8102918
proxy  14947  0.0  0.0 11620  2244  ??  IN   Fri08PM   0:00.01 diskd 13276164 13276165 13276166
proxy  20939  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 18546692 18546693 18546694
proxy  53058  0.0  0.0 11620  2264  ??  S    12:24PM   0:00.34 diskd 52115460 52115461 52115462

In the example, squid last stop/start has been at Sunday, 12:24PM. We can see many zombie diskd processes.

[bellera]

ipcs and ipcrm removed by design.

https://github.com/billm/pfsense-tools-centipede-slbd/blob/master/builder_scripts/remove.list.iso.7

What to do?

[bellera]

Download livefs.iso from

ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/

mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.

or http://www.filewatcher.com/m/ipcrm.12888-0.html

64 bit

fetch -o /usr/bin/ipcs ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcs
chmod 555 /usr/bin/ipcs
fetch -o /usr/bin/ipcrm ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcrm
chmod 555 /usr/bin/ipcrm
rehash

Now, here is my pfSense running squid with diskd problems

ipcs
Message Queues:
T           ID          KEY MODE        OWNER    GROUP   
q       131072     99774468 --rwa------ proxy    proxy   
q       131073     99774469 --rwa------ proxy    proxy   
q        65538     18546692 --rwa------ proxy    proxy   
q        65539     18546693 --rwa------ proxy    proxy   
q        65540     13276164 --rwa------ proxy    proxy   
q        65541     13276165 --rwa------ proxy    proxy   
q       131078      8102916 --rwa------ proxy    proxy   
q       131079      8102917 --rwa------ proxy    proxy   
q        65544     52115460 --rwa------ proxy    proxy   
q        65545     52115461 --rwa------ proxy    proxy   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP   
m       131072     99774470 --rw------- proxy    proxy   
m        65537     18546694 --rw------- proxy    proxy   
m        65538     13276166 --rw------- proxy    proxy   
m       131075      8102918 --rw------- proxy    proxy   
m        65540     52115462 --rw------- proxy    proxy   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP

[bellera]

Looking at /usr/local/pkg/squid.inc the code writes /usr/local/etc/rc.d/squid.sh

I patched /usr/local/pkg/squid.inc

cp -p squid.inc squid.inc-2014-04-30
vi squid.inc

Original code

{$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
# Just to be sure...
sleep 5
killall -9 squid 2>/dev/null
killall pinger 2>/dev/null

Patched code

{$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
# Just to be sure...
sleep 5

# http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\\n", $1, $2}' | /bin/sh

killall -9 squid 2>/dev/null
killall pinger 2>/dev/null

I modified also /usr/local/etc/rc.d/squid.sh in order to have the changes and stop/start squid. Everything ok!

proxy processes

ps aux | grep ^proxy
proxy  84290  0.0  0.4 48996 18532  ??  SN    8:20PM   0:03.12 (squid-1) -f /usr/pbi/squid-amd64/etc/squid/squid.conf (squid)
proxy  84619  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  84890  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85084  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85292  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85630  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85844  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85945  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  86112  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  86450  0.0  0.0 11252  2124  ??  IN    8:20PM   0:00.01 (unlinkd) (unlinkd)
proxy  86470  0.0  0.0 11620  2244  ??  IN    8:20PM   0:00.01 diskd 86312964 86312965 86312966

Correct messages queues & shared memory

ipcs
Message Queues:
T           ID          KEY MODE        OWNER    GROUP   
q       196608     86312964 --rwa------ proxy    proxy   
q       196609     86312965 --rwa------ proxy    proxy   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP   
m       131081     86312966 --rw------- proxy    proxy   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP

[bellera]

Corrected /usr/local/pkg/squid.inc patched code

It was

ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s****n", $1, $2}' | /bin/sh

Must be

ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s**\**n", $1, $2}' | /bin/sh

Also corrected at previous message.

[marcelloc]

Package updated to include install instructions and checks

[bellera]

Fantastic! Thanks!

[Tikimotel]

For Squid3 (3.4.10_2 0.2.5), on pfsense 2.2 you need ipcs and ipcrm from a 10.1 base.
The diskd text should reflect that, now it still refers to 8.3.

diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables. 

diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.