Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    php-fpm: pool nginx (php-fpm) eats all CPU

    Scheduled Pinned Locked Moved General pfSense Questions
    21 Posts 2 Posters 3.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • kiokomanK
      kiokoman LAYER 8
      last edited by

      packages installed ?
      screenshot of diagnostic / system activity
      you can check what's going on with
      truss -p PID-of-php-fpm
      and
      lsof -p PID-of-php-fpm

      example

      [2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: ps aux | grep php-fpm
root     340   0.0  0.3 103964 25728  -  Ss   02:14      0:01.56 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm)
root   11873   0.0  0.5 108896 40584  -  I    02:14      0:04.72 php-fpm: pool nginx (php-fpm)
root   92839   0.0  0.5 106124 39228  -  I    02:14      0:08.22 php-fpm: pool nginx (php-fpm)
root   96219   0.0  0.0  11192  2496  1  S+   13:05      0:00.00 grep php-fpm
[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: lsof -p 11873
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF    NODE NAME
php-fpm 11873 root  cwd   VDIR              0,114     1536       2 /var/run
php-fpm 11873 root  rtd   VDIR              0,109     1024       2 /
php-fpm 11873 root    0u  VCHR               0,44      0t0      44 /dev/null
php-fpm 11873 root    1u  VCHR               0,44      0t0      44 /dev/null
php-fpm 11873 root    2u  VCHR               0,44      0t0      44 /dev/null
php-fpm 11873 root    3u  unix 0xfffff8003207aa38      0t0         ->(none)
php-fpm 11873 root    4u  IPv4 0xfffff800320380a0      0t0     UDP *:*
php-fpm 11873 root    5u  IPv6 0xfffff80032038080      0t0     UDP *:*
php-fpm 11873 root    6u  IPv4 0xfffff8003200c1e8      0t0     RAW *:*
php-fpm 11873 root    7u  sock                         0t0         no further information on family 0x20
php-fpm 11873 root    8u  unix 0xfffff80007f7f6d0      0t0         ->0xfffff80007f7f368
php-fpm 11873 root    9u  unix 0xfffff80007f7f368      0t0         ->0xfffff80007f7f6d0
php-fpm 11873 root   10u  VREG              0,109        0 3049730 / (/dev/ufsid/5e7f4db1653191cd)
php-fpm 11873 root   11                                            no more information; ty=0 file may be closing
php-fpm 11873 root   12u  unix 0xfffff80007ff26d0      0t0         /var/run/php-fpm.socket
php-fpm 11873 root   13u  unix 0xfffff8003207e368      0t0         ->0xfffff80064c48a38

      ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
      Please do not use chat/PM to ask for help
      we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
      Don't forget to Upvote with the 👍 button for any post you find to be helpful.

      T 1 Reply Last reply Reply Quote 0
      • T
        Topogigio
        last edited by

        @kiokoman said in php-fpm: pool nginx (php-fpm) eats all CPU:

        truss -p PID-of-php-fpm

        packages

        • frr (used for OSPF on VPNs)
        • haproxy (not used, it's not enabled currently, it was uses in the past)
        • openVM tools
        • openvpn-client-export
        • zabbix-agent5

        I will take other data ASAP when I will get php high CPU load.

        Meantime, thanks

        1 Reply Last reply Reply Quote 0
        • kiokomanK
          kiokoman LAYER 8
          last edited by kiokoman

          also, check if there is something useful under
          Status / System Logs /System / General
          and IPsec logs if you have trouble with that section

          ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
          Please do not use chat/PM to ask for help
          we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
          Don't forget to Upvote with the 👍 button for any post you find to be helpful.

          T 1 Reply Last reply Reply Quote 0
          • T
            Topogigio @kiokoman
            last edited by

            @kiokoman in logs I found a lot of this related to php-fpm:

            Nov 6 13:51:06 	php-fpm 	32031 	/rc.newipsecdns: Gateway, none 'available' for inet6, use the first one configured. ''

            some of

            php-fpm 	32031 	/rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.

            that I cannot explain, nothing has changed IP

            and sometimes it restarts tunnels (not all at same moment), eg:

            Nov 6 13:06:15 	rc.gateway_alarm 	84397 	>>> Gateway alarm: F011XXX11_VTIV4 (Addr:10.77.36.86 Alarm:0 RTT:19.540ms RTTsd:14.562ms Loss:20%)
Nov 6 13:06:15 	check_reload_status 		updating dyndns F011XXXX11_VTIV4
Nov 6 13:06:15 	check_reload_status 		Restarting ipsec tunnels
Nov 6 13:06:15 	check_reload_status 		Restarting OpenVPN tunnels/interfaces
Nov 6 13:06:15 	check_reload_status 		Reloading filter

            diags:

            ps aux | grep php-fpm
root    56408 27.0  4.3 97048 42796  -  R    20:21       8:55.57 php-fpm: pool nginx (php-fpm)
root    32031 25.0  4.5 99292 44356  -  S    11:38       7:51.54 php-fpm: pool nginx (php-fpm)
root    76285  2.0  4.3 97048 42552  -  S    12:27       5:28.28 php-fpm: pool nginx (php-fpm)
root    42487  1.0  4.5 99292 44864  -  S    12:51       4:57.44 php-fpm: pool nginx (php-fpm)
root      340  0.0  2.6 94868 25468  -  Ss    2Sep20     2:28.70 php-fpm: master process (/usr/local/lib/php-f
root    22932  0.0  4.2 97048 41500  -  S    13:21       4:47.64 php-fpm: pool nginx (php-fpm)
root    40549  0.0  4.3 97048 42768  -  S    14:09       0:09.41 php-fpm: pool nginx (php-fpm)
root    64085  0.0  4.3 97048 42632  -  S    02:01       9:06.35 php-fpm: pool nginx (php-fpm)
root    64301  0.0  4.5 99228 44776  -  S    10:50       9:04.06 php-fpm: pool nginx (php-fpm)
myuser      7648  0.0  0.0   416   324  0  R+   14:18       0:00.00 grep php-fpm

            but then:

            truss -p 56408
truss: can not attach to target process: Operation not permitted

            and

            [2.4.5-RELEASE][]/home/myuser: lsof -p 56408
lsof: Command not found.
            1 Reply Last reply Reply Quote 0
            • T
              Topogigio @kiokoman
              last edited by

              @kiokoman said in php-fpm: pool nginx (php-fpm) eats all CPU:

              screenshot of diagnostic / system activity

              act.png

              1 Reply Last reply Reply Quote 0
              • kiokomanK
                kiokoman LAYER 8
                last edited by kiokoman

                you can install lsof with

                pkg install lsof

                the process consuming wcpu is PID 64085 and 64301
                check that

                are you using ipv6 with ipsec ? dual stack? ikev2 ?

                ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                Please do not use chat/PM to ask for help
                we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                T 1 Reply Last reply Reply Quote 0
                • T
                  Topogigio @kiokoman
                  last edited by

                  @kiokoman I'm not using IPv6 (but not explicitly disabled nothing about it, if there are defaults).
                  All 25 IPSEC VTI site-to-site are IKEV2.

                  lsof installed thanks. ASAP I will find php with high CPU I'll debug

                  1 Reply Last reply Reply Quote 0
                  • kiokomanK
                    kiokoman LAYER 8
                    last edited by

                    @Topogigio said in php-fpm: pool nginx (php-fpm) eats all CPU:

                    Gateway, none 'available' for inet6, use the first one configured. ''

                    this string come from /etc/inc/gwlb.inc

                    if (isset($gateways_arr[$gwdefault])) {
                // the configured gateway is a regular one. (not a gwgroup) use it as is..
                $set_dfltgwname = $gwdefault;
        } elseif (empty($gwdefault)) {
                // 'automatic' mode, pick the first one thats 'up' or 'unmonitored' which is always considered up
                $gateways_arr = order_gateways_as_configured($gateways_arr);

                    you can try to set System / Routing / Gateways
                    set it statically instead of automatic

                    ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                    Please do not use chat/PM to ask for help
                    we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                    Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                    T 1 Reply Last reply Reply Quote 0
                    • T
                      Topogigio @kiokoman
                      last edited by

                      @kiokoman assigned to "none", thanks

                      1 Reply Last reply Reply Quote 0
                      • kiokomanK
                        kiokoman LAYER 8
                        last edited by

                        risolto? or do you still have high CPU usage?

                        ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                        Please do not use chat/PM to ask for help
                        we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                        Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                        T 2 Replies Last reply Reply Quote 0
                        • T
                          Topogigio @kiokoman
                          last edited by

                          @kiokoman currenlty is ok, and during last 24 hours status/monitoring does not report any down, nor my Zabbix high CPU usage.

                          But is't Saturday, so not a "normal" day, I need to check this next week.

                          No more logs related to IPv6 gateway anyway.

                          Meantime thanks

                          1 Reply Last reply Reply Quote 0
                          • T
                            Topogigio @kiokoman
                            last edited by

                            @kiokoman Today it started again.

                            [2.4.5-RELEASE][admin@gw01]/root: lsof -p 52202
lsof: WARNING: access /root/.lsof_gw01: No such file or directory
lsof: WARNING: created device cache file: /root/.lsof_gw01
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
php-fpm 52202 root  cwd   VDIR               0,82     4608 481536 /etc
php-fpm 52202 root  rtd   VDIR               0,82     1024      2 /
php-fpm 52202 root    0u  VCHR               0,17      0t0     17 /dev/null
php-fpm 52202 root    1u  VCHR               0,17      0t0     17 /dev/null
php-fpm 52202 root    2u  VCHR               0,17      0t0     17 /dev/null
php-fpm 52202 root    3u  unix 0xfffff80004d6ea38      0t0        ->(none)
php-fpm 52202 root    4u  IPv4 0xfffff80004eb4040      0t0    UDP *:*
php-fpm 52202 root    5u  IPv6 0xfffff80004eb4020      0t0    UDP *:*
php-fpm 52202 root    6u  IPv4 0xfffff80004ebc740      0t0    RAW *:*
php-fpm 52202 root    7u  sock                         0t0        no further information on family 0x20
php-fpm 52202 root    8u  unix 0xfffff80004d6da38      0t0        ->0xfffff80004d6d6d0
php-fpm 52202 root    9u  unix 0xfffff80004d6d6d0      0t0        ->0xfffff80004d6da38
php-fpm 52202 root   10r  VCHR                0,8      0t0      8 /dev/random
php-fpm 52202 root   11ur VREG               0,82        0 321026 / (/dev/ufsid/5f4fb11ba7c2e0ef)
php-fpm 52202 root   12u  unix 0xfffff8002edd6368      0t0        /var/run/php-fpm.socket
php-fpm 52202 root   13u  unix 0xfffff80004d6ca38      0t0        /var/run/php-fpm.socket
php-fpm 52202 root   14u  unix 0xfffff80009880368      0t0        ->0xfffff80009883368
php-fpm 52202 root   15w  VREG               0,82        0 321041 / (/dev/ufsid/5f4fb11ba7c2e0ef)
php-fpm 52202 root   16w  VREG               0,82        0 321049 / (/dev/ufsid/5f4fb11ba7c2e0ef)

                            top:

                            [2.4.5-RELEASE][admin@gw01]/root: top
last pid: 37556;  load averages: 12.57,  8.89,  6.36                                                                                                          up 67+16:23:49  09:35:51
88 processes:  2 running, 86 sleeping
CPU: 57.8% user,  0.0% nice, 19.1% system, 23.0% interrupt,  0.0% idle
Mem: 36M Active, 215M Inact, 291M Wired, 100M Buf, 399M Free
Swap: 768M Total, 768M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
52202 root          1  92    0 97044K 41148K RUN      5:21  59.65% php-fpm
                            1 Reply Last reply Reply Quote 0
                            • T
                              Topogigio
                              last edited by Topogigio

                              also truss:

                              [2.4.5-RELEASE][admin@gw01]/root: truss -p 52202
accept(13,{ AF_UNIX "" },0x7ffffffee958)         = 12 (0xc)
poll({ 12/POLLIN },1,5000)                       = 1 (0x1)
getrusage(RUSAGE_SELF,{ u=391.610731,s=113.553168,in=0,out=2 }) = 0 (0x0)
getrusage(RUSAGE_CHILDREN,{ u=52.231295,s=67.120794,in=0,out=0 }) = 0 (0x0)
read(12,"\^A\^A\0\^A\0\b\0\0",8)                 = 8 (0x8)
read(12,"\0\^A\0\0\0\0\0\0",8)                   = 8 (0x8)
read(12,"\^A\^D\0\^A\0\M-C\0\0",8)               = 8 (0x8)
read(12,"\^Q\vGATEWAY_INTERFACEFastCGI/1."...,195) = 195 (0xc3)
read(12,"\^A\^D\0\^A\0\0\0\0",8)                 = 8 (0x8)
lstat("/etc/rc.expireaccounts",{ mode=-rwxr-xr-x ,inode=481667,size=1630,blksize=32768 }) = 0 (0x0)
lstat("/etc",{ mode=drwxr-xr-x ,inode=481536,size=4608,blksize=32768 }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGPROF,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGHUP,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGINT,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGQUIT,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGTERM,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGUSR1,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGUSR2,0x0,{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
setitimer(2,{ 0.000000, 1800.000000 },0x0)       = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGPROF,{ 0x80170f200 SA_SIGINFO ss_t },{ 0x80170f200 SA_SIGINFO ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)                 = 0 (0x0)
sigprocmask(SIG_UNBLOCK,{ SIGPROF },0x0)         = 0 (0x0)
__getcwd("/var/run",4095)                        = 0 (0x0)
chdir("/etc")                                    = 0 (0x0)
setitimer(2,{ 0.000000, 900.000000 },0x0)        = 0 (0x0)
fcntl(11,F_SETLK,0x7fffffffd230)                 = 0 (0x0)
stat("/etc/rc.expireaccounts",{ mode=-rwxr-xr-x ,inode=481667,size=1630,blksize=32768 }) = 0 (0x0)
__getcwd("/etc",1024)                            = 0 (0x0)
lstat("/etc/version",{ mode=-rw-r--r-- ,inode=481605,size=14,blksize=32768 }) = 0 (0x0)
openat(AT_FDCWD,"/etc/version",O_RDONLY,00)      = 15 (0xf)
fstat(15,{ mode=-rw-r--r-- ,inode=481605,size=14,blksize=32768 }) = 0 (0x0)
lseek(15,0x0,SEEK_CUR)                           = 0 (0x0)
fstat(15,{ mode=-rw-r--r-- ,inode=481605,size=14,blksize=32768 }) = 0 (0x0)
read(15,"2.4.5-RELEASE\n",8192)                  = 14 (0xe)
read(15,0x802a71000,8192)                        = 0 (0x0)
read(15,0x802a71000,8192)                        = 0 (0x0)
close(15)                                        = 0 (0x0)
access("/etc/version.patch",F_OK)                = 0 (0x0)
lstat("/etc/version.patch",{ mode=-rw-r--r-- ,inode=481594,size=2,blksize=32768 }) = 0 (0x0)
openat(AT_FDCWD,"/etc/version.patch",O_RDONLY,00) = 15 (0xf)
fstat(15,{ mode=-rw-r--r-- ,inode=481594,size=2,blksize=32768 }) = 0 (0x0)
lseek(15,0x0,SEEK_CUR)                           = 0 (0x0)
fstat(15,{ mode=-rw-r--r-- ,inode=481594,size=2,blksize=32768 }) = 0 (0x0)
read(15,"1\n",8192)                              = 2 (0x2)
read(15,0x802a78000,8192)                        = 0 (0x0)
read(15,0x802a78000,8192)                        = 0 (0x0)
close(15)                                        = 0 (0x0)
access("/etc/default-config-flavor",F_OK)        = 0 (0x0)
lstat("/etc/default-config-flavor",{ mode=-rw-r--r-- ,inode=481692,size=1,blksize=32768 }) = 0 (0x0)
openat(AT_FDCWD,"/etc/default-config-flavor",O_RDONLY,00) = 15 (0xf)
fstat(15,{ mode=-rw-r--r-- ,inode=481692,size=1,blksize=32768 }) = 0 (0x0)
lseek(15,0x0,SEEK_CUR)                           = 0 (0x0)
fstat(15,{ mode=-rw-r--r-- ,inode=481692,size=1,blksize=32768 }) = 0 (0x0)
read(15,"\n",8192)                               = 1 (0x1)
read(15,0x802a78000,8192)                        = 0 (0x0)
read(15,0x802a78000,8192)                        = 0 (0x0)
close(15)                                        = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffcbb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffccb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffcdb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffceb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffcfb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
access("/etc/inc/globals_override.inc",F_OK)     ERR#2 'No such file or directory'
access("/cf/conf/enableserial_force",F_OK)       ERR#2 'No such file or directory'
__sysctl(0x7fffffffcb68,0x2,0x7fffffffcbb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffccb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffcdb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffceb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffffffcb68,0x2,0x7fffffffcfb0,0x7fffffffcb60,0x0,0x0) = 0 (0x0)
access("/cf/conf/use_xmlreader",F_OK)            ERR#2 'No such file or directory'
access("/debugging",F_OK)                        ERR#2 'No such file or directory'
access("/var/run/booting",F_OK)                  ERR#2 'No such file or directory'
access("/tmp/config.lock",F_OK)                  = 0 (0x0)
lstat("/tmp/config.lock",{ mode=-rw-rw-rw- ,inode=321028,size=0,blksize=32768 }) = 0 (0x0)
lstat("/tmp",{ mode=drwxrwxrwt ,inode=321024,size=2048,blksize=32768 }) = 0 (0x0)
openat(AT_FDCWD,"/tmp/config.lock",O_WRONLY|O_CREAT|O_TRUNC,0666) = 15 (0xf)
fstat(15,{ mode=-rw-rw-rw- ,inode=321028,size=0,blksize=32768 }) = 0 (0x0)
lseek(15,0x0,SEEK_CUR)                           = 0 (0x0)
flock(15,LOCK_SH)                                = 0 (0x0)
access("/conf/config.xml",F_OK)                  = 0 (0x0)
stat("/conf/config.xml",{ mode=-rw-r--r-- ,inode=1284133,size=228319,blksize=32768 }) = 0 (0x0)
access("/var/run/booting",F_OK)                  ERR#2 'No such file or directory'
access("/conf/config.xml",F_OK)                  = 0 (0x0)
access("/var/run/booting",F_OK)                  ERR#2 'No such file or directory'
access("/tmp/config.cache",F_OK)                 = 0 (0x0)
lstat("/tmp/config.cache",{ mode=-rw-r--r-- ,inode=321092,size=199160,blksize=32768 }) = 0 (0x0)
openat(AT_FDCWD,"/tmp/config.cache",O_RDONLY,00) = 16 (0x10)
fstat(16,{ mode=-rw-r--r-- ,inode=321092,size=199160,blksize=32768 }) = 0 (0x0)
lseek(16,0x0,SEEK_CUR)                           = 0 (0x0)
fstat(16,{ mode=-rw-r--r-- ,inode=321092,size=199160,blksize=32768 }) = 0 (0x0)
read(16,"a:33:{s:7:"version";s:4:"19.1";s"...,8192) = 8192 (0x2000)
read(16,"VPNCX02_1_F01_1_LANs";s:7:"gate"...,8192) = 8192 (0x2000)
read(16,"p";s:6:"target";s:7:"R51_LAN";s:"...,8192) = 8192 (0x2000)
read(16,"";s:5:"descr";s:14:"R58 HTTP-HTT"...,8192) = 8192 (0x2000)
read(16,"created";a:2:{s:4:"time";s:10:"1"...,8192) = 8192 (0x2000)
read(16,"te_Subnets";s:3:"not";s:0:"";s:4"...,8192) = 8192 (0x2000)
read(16,"89.186.81.246 (Local Database)";"...,8192) = 8192 (0x2000)
read(16,"ss";s:8:"NG02_LAN";s:4:"port";s:"...,8192) = 8192 (0x2000)
read(16,"";s:8:"protocol";s:4:"icmp";s:8:"...,8192) = 8192 (0x2000)
read(16,"x-src-states";s:0:"";s:12:"state"...,8192) = 8192 (0x2000)
read(16,"s:10:"encryption";a:1:{s:4:"item"...,8192) = 8192 (0x2000)
read(16,"3:"256";}s:14:"hash-algorithm";s"...,8192) = 8192 (0x2000)
read(16,"s:21:"authentication_method";s:1"...,8192) = 8192 (0x2000)
read(16,"alid";a:2:{s:4:"type";s:7:"addre"...,8192) = 8192 (0x2000)
read(16,"5:"reqid";s:2:"20";s:7:"localid""...,8192) = 8192 (0x2000)
read(16,"00 8001 8002";s:5:"descr";s:0:"""...,8192) = 8192 (0x2000)
read(16,"added Mon, 07 Sep 2020 15:38:25 "...,8192) = 8192 (0x2000)
read(16,"ded Mon, 07 Sep 2020 15:38:25 +0"...,8192) = 8192 (0x2000)
read(16,""detail";s:8:"MacroLAN";}i:64;a:"...,8192) = 8192 (0x2000)
read(16,"Rc0ZBREJhTVRnd05nWURWUVFLRXk5d1p"...,8192) = 8192 (0x2000)
read(16,"Z1FETUJUU3k1alhGVzZxNDNGMExJSjlT"...,8192) = 8192 (0x2000)
read(16,"3:"HAProxy Stats";s:11:"tooltipt"...,8192) = 8192 (0x2000)
read(16,"6:"action";s:6:"permit";s:6:"sou"...,8192) = 8192 (0x2000)
read(16,":0:"";s:11:"stats_realm";s:0:"";"...,8192) = 8192 (0x2000)
read(16,"lISUVjTjBGKytnSVNOWHB0ekRQVHF6WU"...,8192) = 2552 (0x9f8)
read(16,0x802ae4000,8192)                        = 0 (0x0)
read(16,0x802ae4000,8192)                        = 0 (0x0)
close(16)                                        = 0 (0x0)
access("/var/run/booting",F_OK)                  ERR#2 'No such file or directory'
flock(15,LOCK_UN)                                = 0 (0x0)
close(15)                                        = 0 (0x0)
stat("/usr/local/pkg/parse_config",0x7fffffffd068) ERR#2 'No such file or directory'
write(12,"\^A\^F\0\^A\0\^B\^F\0001\n\0\0\0"...,16) = 16 (0x10)
write(12,"\^A\^F\0\^A\0\^R\^F\0User baz ex"...,32) = 32 (0x20)
write(12,"\^A\^F\0\^A\0\^B\^F\0001\n\0\0\0"...,16) = 16 (0x10)
write(12,"\^A\^F\0\^A\0\^Q\a\0User sl expi"...,32) = 32 (0x20)
chdir("/var/run")                                = 0 (0x0)
getrusage(RUSAGE_SELF,{ u=391.613602,s=113.758816,in=0,out=2 }) = 0 (0x0)
getrusage(RUSAGE_CHILDREN,{ u=52.231295,s=67.120794,in=0,out=0 }) = 0 (0x0)
setitimer(2,{ 0.000000, 0.000000 },0x0)          = 0 (0x0)
fcntl(11,F_SETLK,0x7fffffffe890)                 = 0 (0x0)
write(12,"\^A\^C\0\^A\0\b\0\0\0\0\0\0\0sl ",16)  = 16 (0x10)
shutdown(12,SHUT_WR)                             ERR#57 'Socket is not connected'
recvfrom(12,"\^A\^E\0\^A\0\0\0\0",8,0,NULL,0x0)  = 8 (0x8)
recvfrom(12,0x7fffffffa848,8,0,NULL,0x0)         = 0 (0x0)
close(12)                                        = 0 (0x0)
setitimer(2,{ 0.000000, 0.000000 },0x0)          = 0 (0x0)
                              1 Reply Last reply Reply Quote 0
                              • kiokomanK
                                kiokoman LAYER 8
                                last edited by

                                it is checking for an expired account and after that I see haproxy stats
                                maybe try to temporarily disable haproxy stats

                                Leave this setting empty to remove the "HAProxyLocalStats" item from the stats page and save a little on recources.

                                ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                Please do not use chat/PM to ask for help
                                we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                T 1 Reply Last reply Reply Quote 0
                                • T
                                  Topogigio @kiokoman
                                  last edited by

                                  @kiokoman it seems strange: HA proxy is disabled. The "Enable HAProxy" flag is disabled.

                                  Now I've uninstalled it to be sure: "pfSense-pkg-haproxy removal successfully completed."
                                  Still php-fpn processes are eating cpu.

                                  Also I cannot find expired accounts :( How can I find id?

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    Topogigio
                                    last edited by Topogigio

                                    I did run truss an other time, and now output is a 40MB of log (full of private data).. :(

                                    errors I see:
                                    access("/tmp/ipsec16000_defaultgw",F_OK) ERR#2 'No such file or directory'
                                    access("/tmp/ipsec17000_defaultgw",F_OK) ERR#2 'No such file or directory'
                                    access("/tmp/ipsec18000_defaultgw",F_OK) ERR#2 'No such file or directory'
                                    access("/tmp/ipsec26000_defaultgw",F_OK) ERR#2 'No such file or directory'

                                    etc..

                                    fcntl(18,F_GETFD,) ERR#9 'Bad file descriptor'

                                    1 Reply Last reply Reply Quote 0
                                    • kiokomanK
                                      kiokoman LAYER 8
                                      last edited by kiokoman

                                      it was checking for an expired account, it does not mean that there are any...
                                      I think it's under system / user account

                                      nothing useful inside Status / System Logs / IPsec ?

                                      hopefully, someone will chime in on this I have no idea, I don't use that many IPsec but I don't think that only 25 IPsec is a problem

                                      maybe that "no such file" are the culprit
                                      go inside any ipsec and just press "save" and see if they are generated

                                      ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                      Please do not use chat/PM to ask for help
                                      we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                      Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                      T 1 Reply Last reply Reply Quote 0
                                      • T
                                        Topogigio @kiokoman
                                        last edited by

                                        @kiokoman I tried to check it from the OS: files exist, and seems recreated all at the same time (now), eg

                                        11 Nov 9 11:31 ipsec10000_router

                                        they are all 11 bytes long.

                                        I cannot paste full output because Askimet blocks me :(

                                        1 Reply Last reply Reply Quote 0
                                        • kiokomanK
                                          kiokoman LAYER 8
                                          last edited by

                                          if you can, disable ipsec tunnel one by one until you find what is causing the problem

                                          ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                          Please do not use chat/PM to ask for help
                                          we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                          Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                          T 1 Reply Last reply Reply Quote 0
                                          • T
                                            Topogigio @kiokoman
                                            last edited by

                                            @kiokoman it's a production node so it's hard :( And to disable a VTI requires to unassign the interface, and so on, I cannot simply disable the P1.

                                            Meantime I've found a small workaround. I noticed in logs many events related to "change of dynamic IP address" related to my IPSEC tunnels (please note that I work only with static IPs). This triggered some kind of refresh of configuration, and php started to consume all CPU during that refresh.
                                            So I disable monitoring on all tunnels, and this mitigate the problem because it seems that pfSense does not reload configuration many times every day as before. Still the problem is on, so if I manually save changes and reload config it starts to eat CPU

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.