3. Squid - diskd bug (ipcs and ipcrm not available)

Squid - diskd bug (ipcs and ipcrm not available)


  • Tested with squid3-dev 3.3.10 pkg 2.2.1

    I'm using squid2 with diskd (on FreeBSD machines) since year 2009 without problems.

    However, I had to modify /usr/local/etc/rc.d/squid script to nicely clean diskd squid processes:

    Modified code for squid stop

    squid_stop() {
	echo "Stopping ${name}."
	${command} ${squid_flags} -k shutdown
	run_rc_command poll
	# http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
	sleep 5
	ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\n", $1, $2}' | /bin/sh
	killall -u squid 2>/dev/null
}

    Original code for squid stop

    squid_stop() {
	echo "Stopping ${name}."
	${command} ${squid_flags} -k shutdown
	run_rc_command poll
}

    Now, I'm migrating to squid into pfSense…

    I wanted to do the same with /usr/pbi/squid-amd64/etc/rc.d/squid but ipcs and ipcrm aren't available in pfSense.

    http://www.freebsd.org/cgi/man.cgi?query=ipcs
    http://www.freebsd.org/cgi/man.cgi?query=ipcrm

    How to see unclean diskd processes?

    ps aux | grep diskd
proxy    740  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 99774468 99774469 99774470
proxy  12825  0.0  0.0 11620  2316  ??  IN   Sat11AM   0:00.07 diskd 8102916 8102917 8102918
proxy  14947  0.0  0.0 11620  2244  ??  IN   Fri08PM   0:00.01 diskd 13276164 13276165 13276166
proxy  20939  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 18546692 18546693 18546694
proxy  53058  0.0  0.0 11620  2264  ??  S    12:24PM   0:00.34 diskd 52115460 52115461 52115462

    In the example, squid last stop/start has been at Sunday, 12:24PM. We can see many zombie diskd processes.

    0

  • ipcs and ipcrm removed by design.

    https://github.com/billm/pfsense-tools-centipede-slbd/blob/master/builder_scripts/remove.list.iso.7

    What to do?

    0

  • Download livefs.iso from

    ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/

    mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.

    or http://www.filewatcher.com/m/ipcrm.12888-0.html

    64 bit

    fetch -o /usr/bin/ipcs ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcs
chmod 555 /usr/bin/ipcs
fetch -o /usr/bin/ipcrm ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcrm
chmod 555 /usr/bin/ipcrm
rehash

    Now, here is my pfSense running squid with diskd problems

    ipcs
Message Queues:
T           ID          KEY MODE        OWNER    GROUP   
q       131072     99774468 --rwa------ proxy    proxy   
q       131073     99774469 --rwa------ proxy    proxy   
q        65538     18546692 --rwa------ proxy    proxy   
q        65539     18546693 --rwa------ proxy    proxy   
q        65540     13276164 --rwa------ proxy    proxy   
q        65541     13276165 --rwa------ proxy    proxy   
q       131078      8102916 --rwa------ proxy    proxy   
q       131079      8102917 --rwa------ proxy    proxy   
q        65544     52115460 --rwa------ proxy    proxy   
q        65545     52115461 --rwa------ proxy    proxy   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP   
m       131072     99774470 --rw------- proxy    proxy   
m        65537     18546694 --rw------- proxy    proxy   
m        65538     13276166 --rw------- proxy    proxy   
m       131075      8102918 --rw------- proxy    proxy   
m        65540     52115462 --rw------- proxy    proxy   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP
    0

  • Looking at /usr/local/pkg/squid.inc the code writes /usr/local/etc/rc.d/squid.sh

    I patched /usr/local/pkg/squid.inc

    cp -p squid.inc squid.inc-2014-04-30
vi squid.inc

    Original code

    {$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
# Just to be sure...
sleep 5
killall -9 squid 2>/dev/null
killall pinger 2>/dev/null

    Patched code

    {$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
# Just to be sure...
sleep 5

# http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\\n", $1, $2}' | /bin/sh

killall -9 squid 2>/dev/null
killall pinger 2>/dev/null

    I modified also /usr/local/etc/rc.d/squid.sh in order to have the changes and stop/start squid. Everything ok!

    proxy processes

    ps aux | grep ^proxy
proxy  84290  0.0  0.4 48996 18532  ??  SN    8:20PM   0:03.12 (squid-1) -f /usr/pbi/squid-amd64/etc/squid/squid.conf (squid)
proxy  84619  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  84890  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85084  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85292  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85630  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85844  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  85945  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  86112  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
proxy  86450  0.0  0.0 11252  2124  ??  IN    8:20PM   0:00.01 (unlinkd) (unlinkd)
proxy  86470  0.0  0.0 11620  2244  ??  IN    8:20PM   0:00.01 diskd 86312964 86312965 86312966

    Correct messages queues & shared memory

    ipcs
Message Queues:
T           ID          KEY MODE        OWNER    GROUP   
q       196608     86312964 --rwa------ proxy    proxy   
q       196609     86312965 --rwa------ proxy    proxy   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP   
m       131081     86312966 --rw------- proxy    proxy   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP
    0

  • Corrected /usr/local/pkg/squid.inc patched code

    It was

    ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s****n", $1, $2}' | /bin/sh

    Must be

    ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s**\**n", $1, $2}' | /bin/sh

    Also corrected at previous message.

    0

  • Package updated to include install instructions and checks

    0

  • Fantastic! Thanks!

    0
  • T

    For Squid3 (3.4.10_2 0.2.5), on pfsense 2.2 you need ipcs and ipcrm from a 10.1 base.
    The diskd text should reflect that, now it still refers to 8.3.

    diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables. 

    diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.
    0
Log in to reply
 

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive for past announcements.

© Copyright 2002 - 2019 Rubicon Communications, LLC | Privacy Policy