Squid - diskd bug (ipcs and ipcrm not available)



  • Tested with squid3-dev 3.3.10 pkg 2.2.1

    I'm using squid2 with diskd (on FreeBSD machines) since year 2009 without problems.

    However, I had to modify /usr/local/etc/rc.d/squid script to nicely clean diskd squid processes:

    Modified code for squid stop

    squid_stop() {
    	echo "Stopping ${name}."
    	${command} ${squid_flags} -k shutdown
    	run_rc_command poll
    	# http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
    	sleep 5
    	ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\n", $1, $2}' | /bin/sh
    	killall -u squid 2>/dev/null
    }
    

    Original code for squid stop

    squid_stop() {
    	echo "Stopping ${name}."
    	${command} ${squid_flags} -k shutdown
    	run_rc_command poll
    }
    

    Now, I'm migrating to squid into pfSense…

    I wanted to do the same with /usr/pbi/squid-amd64/etc/rc.d/squid but ipcs and ipcrm aren't available in pfSense.

    http://www.freebsd.org/cgi/man.cgi?query=ipcs
    http://www.freebsd.org/cgi/man.cgi?query=ipcrm

    How to see unclean diskd processes?

    ps aux | grep diskd
    proxy    740  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 99774468 99774469 99774470
    proxy  12825  0.0  0.0 11620  2316  ??  IN   Sat11AM   0:00.07 diskd 8102916 8102917 8102918
    proxy  14947  0.0  0.0 11620  2244  ??  IN   Fri08PM   0:00.01 diskd 13276164 13276165 13276166
    proxy  20939  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 18546692 18546693 18546694
    proxy  53058  0.0  0.0 11620  2264  ??  S    12:24PM   0:00.34 diskd 52115460 52115461 52115462
    

    In the example, squid last stop/start has been at Sunday, 12:24PM. We can see many zombie diskd processes.





  • Download livefs.iso from

    ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/

    mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.

    or http://www.filewatcher.com/m/ipcrm.12888-0.html

    64 bit

    fetch -o /usr/bin/ipcs ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcs
    chmod 555 /usr/bin/ipcs
    fetch -o /usr/bin/ipcrm ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcrm
    chmod 555 /usr/bin/ipcrm
    rehash
    

    Now, here is my pfSense running squid with diskd problems

    ipcs
    Message Queues:
    T           ID          KEY MODE        OWNER    GROUP   
    q       131072     99774468 --rwa------ proxy    proxy   
    q       131073     99774469 --rwa------ proxy    proxy   
    q        65538     18546692 --rwa------ proxy    proxy   
    q        65539     18546693 --rwa------ proxy    proxy   
    q        65540     13276164 --rwa------ proxy    proxy   
    q        65541     13276165 --rwa------ proxy    proxy   
    q       131078      8102916 --rwa------ proxy    proxy   
    q       131079      8102917 --rwa------ proxy    proxy   
    q        65544     52115460 --rwa------ proxy    proxy   
    q        65545     52115461 --rwa------ proxy    proxy   
    
    Shared Memory:
    T           ID          KEY MODE        OWNER    GROUP   
    m       131072     99774470 --rw------- proxy    proxy   
    m        65537     18546694 --rw------- proxy    proxy   
    m        65538     13276166 --rw------- proxy    proxy   
    m       131075      8102918 --rw------- proxy    proxy   
    m        65540     52115462 --rw------- proxy    proxy   
    
    Semaphores:
    T           ID          KEY MODE        OWNER    GROUP
    


  • Looking at /usr/local/pkg/squid.inc the code writes /usr/local/etc/rc.d/squid.sh

    I patched /usr/local/pkg/squid.inc

    cp -p squid.inc squid.inc-2014-04-30
    vi squid.inc
    

    Original code

    {$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
    # Just to be sure...
    sleep 5
    killall -9 squid 2>/dev/null
    killall pinger 2>/dev/null
    

    Patched code

    {$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
    # Just to be sure...
    sleep 5
    
    # http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
    ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\\n", $1, $2}' | /bin/sh
    
    killall -9 squid 2>/dev/null
    killall pinger 2>/dev/null
    

    I modified also /usr/local/etc/rc.d/squid.sh in order to have the changes and stop/start squid. Everything ok!

    proxy processes

    ps aux | grep ^proxy
    proxy  84290  0.0  0.4 48996 18532  ??  SN    8:20PM   0:03.12 (squid-1) -f /usr/pbi/squid-amd64/etc/squid/squid.conf (squid)
    proxy  84619  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  84890  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  85084  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  85292  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  85630  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  85844  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  85945  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  86112  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
    proxy  86450  0.0  0.0 11252  2124  ??  IN    8:20PM   0:00.01 (unlinkd) (unlinkd)
    proxy  86470  0.0  0.0 11620  2244  ??  IN    8:20PM   0:00.01 diskd 86312964 86312965 86312966
    

    Correct messages queues & shared memory

    ipcs
    Message Queues:
    T           ID          KEY MODE        OWNER    GROUP   
    q       196608     86312964 --rwa------ proxy    proxy   
    q       196609     86312965 --rwa------ proxy    proxy   
    
    Shared Memory:
    T           ID          KEY MODE        OWNER    GROUP   
    m       131081     86312966 --rw------- proxy    proxy   
    
    Semaphores:
    T           ID          KEY MODE        OWNER    GROUP
    


  • Corrected /usr/local/pkg/squid.inc patched code

    It was

    ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s****n", $1, $2}' | /bin/sh

    Must be

    ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s**\**n", $1, $2}' | /bin/sh

    Also corrected at previous message.



  • Package updated to include install instructions and checks



  • Fantastic! Thanks!



  • For Squid3 (3.4.10_2 0.2.5), on pfsense 2.2 you need ipcs and ipcrm from a 10.1 base.
    The diskd text should reflect that, now it still refers to 8.3.

    diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
    To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables. 
    
    diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
    To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables. 
    

 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy