Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid - diskd bug (ipcs and ipcrm not available)

    Scheduled Pinned Locked Moved pfSense Packages
    8 Posts 3 Posters 3.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • belleraB
      bellera
      last edited by

      Tested with squid3-dev 3.3.10 pkg 2.2.1

      I'm using squid2 with diskd (on FreeBSD machines) since year 2009 without problems.

      However, I had to modify /usr/local/etc/rc.d/squid script to nicely clean diskd squid processes:

      Modified code for squid stop

      squid_stop() {
      	echo "Stopping ${name}."
      	${command} ${squid_flags} -k shutdown
      	run_rc_command poll
      	# http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
      	sleep 5
      	ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\n", $1, $2}' | /bin/sh
      	killall -u squid 2>/dev/null
      }
      

      Original code for squid stop

      squid_stop() {
      	echo "Stopping ${name}."
      	${command} ${squid_flags} -k shutdown
      	run_rc_command poll
      }
      

      Now, I'm migrating to squid into pfSense…

      I wanted to do the same with /usr/pbi/squid-amd64/etc/rc.d/squid but ipcs and ipcrm aren't available in pfSense.

      http://www.freebsd.org/cgi/man.cgi?query=ipcs
      http://www.freebsd.org/cgi/man.cgi?query=ipcrm

      How to see unclean diskd processes?

      ps aux | grep diskd
      proxy    740  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 99774468 99774469 99774470
      proxy  12825  0.0  0.0 11620  2316  ??  IN   Sat11AM   0:00.07 diskd 8102916 8102917 8102918
      proxy  14947  0.0  0.0 11620  2244  ??  IN   Fri08PM   0:00.01 diskd 13276164 13276165 13276166
      proxy  20939  0.0  0.0 11620  2180  ??  IN   Fri07PM   0:00.01 diskd 18546692 18546693 18546694
      proxy  53058  0.0  0.0 11620  2264  ??  S    12:24PM   0:00.34 diskd 52115460 52115461 52115462
      

      In the example, squid last stop/start has been at Sunday, 12:24PM. We can see many zombie diskd processes.

      1 Reply Last reply Reply Quote 0
      • belleraB
        bellera
        last edited by

        ipcs and ipcrm removed by design.

        https://github.com/billm/pfsense-tools-centipede-slbd/blob/master/builder_scripts/remove.list.iso.7

        What to do?

        1 Reply Last reply Reply Quote 0
        • belleraB
          bellera
          last edited by

          Download livefs.iso from

          ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/

          mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables.

          or http://www.filewatcher.com/m/ipcrm.12888-0.html

          64 bit

          fetch -o /usr/bin/ipcs ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcs
          chmod 555 /usr/bin/ipcs
          fetch -o /usr/bin/ipcrm ftp://dns.ruweb.net/FreeBSD/8.3-RELEASE-amd64-livefs/usr/bin/ipcrm
          chmod 555 /usr/bin/ipcrm
          rehash
          

          Now, here is my pfSense running squid with diskd problems

          ipcs
          Message Queues:
          T           ID          KEY MODE        OWNER    GROUP   
          q       131072     99774468 --rwa------ proxy    proxy   
          q       131073     99774469 --rwa------ proxy    proxy   
          q        65538     18546692 --rwa------ proxy    proxy   
          q        65539     18546693 --rwa------ proxy    proxy   
          q        65540     13276164 --rwa------ proxy    proxy   
          q        65541     13276165 --rwa------ proxy    proxy   
          q       131078      8102916 --rwa------ proxy    proxy   
          q       131079      8102917 --rwa------ proxy    proxy   
          q        65544     52115460 --rwa------ proxy    proxy   
          q        65545     52115461 --rwa------ proxy    proxy   
          
          Shared Memory:
          T           ID          KEY MODE        OWNER    GROUP   
          m       131072     99774470 --rw------- proxy    proxy   
          m        65537     18546694 --rw------- proxy    proxy   
          m        65538     13276166 --rw------- proxy    proxy   
          m       131075      8102918 --rw------- proxy    proxy   
          m        65540     52115462 --rw------- proxy    proxy   
          
          Semaphores:
          T           ID          KEY MODE        OWNER    GROUP
          
          1 Reply Last reply Reply Quote 0
          • belleraB
            bellera
            last edited by

            Looking at /usr/local/pkg/squid.inc the code writes /usr/local/etc/rc.d/squid.sh

            I patched /usr/local/pkg/squid.inc

            cp -p squid.inc squid.inc-2014-04-30
            vi squid.inc
            

            Original code

            {$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
            # Just to be sure...
            sleep 5
            killall -9 squid 2>/dev/null
            killall pinger 2>/dev/null
            

            Patched code

            {$squid_local_base}/sbin/squid -k shutdown -f {$squid_conffile_var}
            # Just to be sure...
            sleep 5
            
            # http://man.chinaunix.net/newsoft/squid/Squid_FAQ/FAQ-22.html#ss22.8
            ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s\\n", $1, $2}' | /bin/sh
            
            killall -9 squid 2>/dev/null
            killall pinger 2>/dev/null
            

            I modified also /usr/local/etc/rc.d/squid.sh in order to have the changes and stop/start squid. Everything ok!

            proxy processes

            ps aux | grep ^proxy
            proxy  84290  0.0  0.4 48996 18532  ??  SN    8:20PM   0:03.12 (squid-1) -f /usr/pbi/squid-amd64/etc/squid/squid.conf (squid)
            proxy  84619  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  84890  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  85084  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  85292  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  85630  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  85844  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  85945  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  86112  0.0  0.2 64640 11008  ??  IN    8:20PM   0:00.04 (squidGuard) -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf (squidGuard)
            proxy  86450  0.0  0.0 11252  2124  ??  IN    8:20PM   0:00.01 (unlinkd) (unlinkd)
            proxy  86470  0.0  0.0 11620  2244  ??  IN    8:20PM   0:00.01 diskd 86312964 86312965 86312966
            

            Correct messages queues & shared memory

            ipcs
            Message Queues:
            T           ID          KEY MODE        OWNER    GROUP   
            q       196608     86312964 --rwa------ proxy    proxy   
            q       196609     86312965 --rwa------ proxy    proxy   
            
            Shared Memory:
            T           ID          KEY MODE        OWNER    GROUP   
            m       131081     86312966 --rw------- proxy    proxy   
            
            Semaphores:
            T           ID          KEY MODE        OWNER    GROUP
            
            1 Reply Last reply Reply Quote 0
            • belleraB
              bellera
              last edited by

              Corrected /usr/local/pkg/squid.inc patched code

              It was

              ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s****n", $1, $2}' | /bin/sh

              Must be

              ipcs | grep '^[mq]' | awk '{printf "ipcrm -%s %s**\**n", $1, $2}' | /bin/sh

              Also corrected at previous message.

              1 Reply Last reply Reply Quote 0
              • marcellocM
                marcelloc
                last edited by

                Package updated to include install instructions and checks

                Treinamentos de Elite: http://sys-squad.com

                Help a community developer! ;D

                1 Reply Last reply Reply Quote 0
                • belleraB
                  bellera
                  last edited by

                  Fantastic! Thanks!

                  1 Reply Last reply Reply Quote 0
                  • T
                    Tikimotel
                    last edited by

                    For Squid3 (3.4.10_2 0.2.5), on pfsense 2.2 you need ipcs and ipcrm from a 10.1 base.
                    The diskd text should reflect that, now it still refers to 8.3.

                    diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
                    To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/8.3/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables. 
                    
                    diskd uses a separate process to avoid blocking the main Squid process on disk-I/O.
                    To use ipcs and ipcrm on squid, Download livefs.iso from ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/ mount it and copy /usr/bin/ipcs and /usr/bin/ipcrm to your system and set them as executables. 
                    
                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.