• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Disk usage /var/run/ over 100% - big php-fpm.core

Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
14 Posts 7 Posters 5.8k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S
    stm110
    last edited by May 14, 2016, 10:12 AM

    Hi!

    After the upgrade to 2.3 I get disk usage in /var/run/ over 100%
    and when I look in the folder it´s php-fpm.core that is about 3.4 MB in size.
    After restarting the system it´s back to normal for a while before it happens again.

    It seems that the php can be configured differently, but I´m not sure how and where.
    I found a general solution for BSD saying that you could change the settings
    for pm from dynamic to on-demand.

    Anyway, I´m a bit stuck here, and yes I´ve searched for an answer, but nothing
    comes close to my problem.

    So I need a bit of encouragement from you kind souls, please
    give me some hints or tricks I can try.

    Thanks in advance!

    Sven

    1 Reply Last reply Reply Quote 0
    • J
      JorgeOliveira
      last edited by May 14, 2016, 1:49 PM

      Hi,

      I'd appreciate if you could clarify which version you are running. On your post you mention 2.3 but this forum is for 2.3.1. Are you running 2.3 (original version from April 12th), 2.3_1 (2.3 Update 1) or 2.3.1 (Development snapshots) ?

      By the looks of what you are telling, this seems to be a serious problem, unfortunately with the little detail you given, no tips can be given to workaround/fix it.

      It would help to know the specs of the hardware on which you are running pfSense. Additionally, this php-fpm.core, may have insights on what is happening to cause its dumping.

      Hopefully one of the developers can reply, analyze this in detail for more clues, and shed some light on this.

      Thanks.

      Regards,
      Jorge M. Oliveira

      My views have absolutely no warranty express or implied. Always do your own research.

      1 Reply Last reply Reply Quote 0
      • S
        stm110
        last edited by May 14, 2016, 9:43 PM

        Yes, of course.

        It´s the 2.3 update 1 version.
        Sorry if I´m in the wrong topic?

        It´s running on Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz and there was no problem with the 2.2 version.

        I think I see a connection with Squid 0.4.16_2, that seems to be stopping frequently.

        I have sent an error report a few times in order for the developers to know what is going on.

        Do you need a log entry to get more data?

        I don´t like to not being able to solve this on my own, but now I really frustrated.
        I like this version, but of course I can downgrade, but thats a defeat…

        Sven

        1 Reply Last reply Reply Quote 0
        • X
          xedd
          last edited by May 17, 2016, 8:59 PM

          I encountered something similar upon having 2.3 running for a few days. At one point, Unbound would start going crazy servicing DNS requests, and WAN interface would drop off the network. All my internal systems were constantly hammering the firewall trying to renew DHCP as well.

          At one point it dawned on me that /var/run was full. At this point, rebooting seemed to fix it but not for long. I eventually moved /var/run to run in a RAM disk (configured under System > Advanced > Miscellaneous) by checking "Use RAM Disks" and then I proceeded to increase /var RAM Disk Size setting to 600MiB.

          At the time, I too had Squid proxy running with Clam Anti-virus as well (main reason for Squid) but I have since removed this package and everything has been stable since, and I have been monitoring that partition to see if it is growing, and it does not appear to be any longer.

          1 Reply Last reply Reply Quote 0
          • K
            Krisbe
            last edited by Jun 1, 2016, 6:09 AM

            Hi all

            Since a few days, I'm having the same problem with my system. The disk space of /var/run is running out of space …

            2.3.1-RELEASE-p1 (amd64)
            built on Wed May 25 14:53:06 CDT 2016
            FreeBSD 10.3-RELEASE-p3

            Disk usage ( /var/run ) 108% of 3.4MiB - ufs in RAM

            [2.3.1-RELEASE][admin@---]/var/run: ls -lh
            total 3468
            drwxrwxr-x  2 root  operator   512B May 30 21:11 .snap
            srw-rw-rw-  1 root  wheel        0B May 30 21:11 check_reload_status
            -rw-------  1 root  wheel        5B May 30 21:11 cron.pid
            -rw-------  1 root  wheel        3B May 30 21:11 devd.pid
            srw-rw-rw-  1 root  wheel        0B May 30 21:11 devd.pipe
            srw-rw-rw-  1 root  wheel        0B May 30 21:11 devd.seqpacket.pipe
            ----------  1 root  wheel        5B May 31 21:48 dhcpleases.pid
            -rw-r--r--  1 root  wheel        5B May 31 21:48 dnsmasq.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 dpinger_GW_WAN_192.168.254.2_192.168.254.1.pid
            srw-rw-rw-  1 root  wheel        0B May 30 21:11 dpinger_GW_WAN_192.168.254.2_192.168.254.1.sock
            -rw-r--r--  1 root  wheel        6B May 30 21:11 expire_accounts.pid
            -rw-r--r--  1 root  wheel        4B May 31 22:15 filter_reload_status
            -rw-r--r--  1 root  wheel        6B May 30 21:11 filterdns.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 filterlog.pid
            -r--r--r--  1 root  wheel      215B May 30 21:11 ld-elf.so.hints
            -r--r--r--  1 root  wheel      139B May 30 21:11 ld-elf32.so.hints
            srw-rw-rw-  1 root  wheel        0B May 30 21:11 log
            srw-------  1 root  wheel        0B May 30 21:11 logpriv
            -rw-r--r--  1 root  wheel        6B May 30 21:11 miniupnpd.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 nginx-webConfigurator.pid
            -rw-r--r--  1 root  wheel        5B May 30 21:11 ntpd.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 openvpn_server1.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 openvpn_server2.pid
            -rw-------  1 root  wheel      3.3M May 30 21:53 php-fpm.core
            -rw-r--r--  1 root  wheel        3B May 30 21:11 php-fpm.pid
            srw-------  1 root  wheel        0B May 30 21:11 php-fpm.socket
            -rw-r--r--  1 root  wheel        6B May 30 21:11 ping_hosts.pid
            -rw-------  1 root  wheel        5B May 30 21:12 powerd.pid
            -rw-------  1 root  wheel        6B May 31 12:05 snort_igb040327.pid
            -rw-r--r--  1 root  wheel        6B May 31 21:48 sshd.pid
            -rw-------  1 root  wheel        4B May 30 21:12 syslog.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 update_alias_url_data.pid
            -rw-r--r--  1 root  wheel        6B May 30 21:11 updaterrd.sh.pid
            -rw-r--r--  1 root  wheel        0B May 30 21:11 utmp
            -rw-r--r--  1 root  wheel      394B May 31 21:48 utx.active
            -rw-r--r--  1 root  wheel        6B May 30 21:11 xinetd.pid
            

            I already rebooted the system, but without the hoping result. Before 2.3.1-RELEASE-p1 I never had this problem.

            ![2016-05-31 22_12_41-fw01.lowet-willems.be - Status_ Dashboard.jpg](/public/imported_attachments/1/2016-05-31 22_12_41-fw01.lowet-willems.be - Status_ Dashboard.jpg)
            ![2016-05-31 22_12_41-fw01.lowet-willems.be - Status_ Dashboard.jpg_thumb](/public/imported_attachments/1/2016-05-31 22_12_41-fw01.lowet-willems.be - Status_ Dashboard.jpg_thumb)

            1 Reply Last reply Reply Quote 0
            • N
              nkv2311
              last edited by Jun 21, 2016, 2:10 AM

              I'm seeing the same thing on a fresh install built around 7PM CDT 6/20/2016. Not upgraded.
              Version 2.3.1-RELEASE-p5 (amd64) built on Thu Jun 16 12:53:15 CDT 2016

              Reboot seemed to fix the issue, but I got curious and came here to see if any one else was reporting this.
              Now I'm concerned it will come back.

              It's running on a Dell 9020 Corei5 with 8G RAM, 250G HDD, 4 port Intel NIC.  Pretty basic system.

              There are no addition packages installed on this box yet.  Configuring Interfaces, Captive Portal, DHCP Server, and Firewall Rules are the only things I have touched on this box.
              I noticed the high disk usage after I had finished making firewall rules for the 5 vlans I have.  I was dragging the rules around to get them in the order I needed them.
              When I returned to the main page disk usage /var/run was at 108%  This was at around 8PM CDT. So it happened in less than an hour for me, with no traffic on the box besides the one test PC I was using to configure everything.

              1 Reply Last reply Reply Quote 0
              • J
                jimp Rebel Alliance Developer Netgate
                last edited by Jun 21, 2016, 1:23 PM

                Add a tunable to set kern.corefile=/root/%N.core and see if the behavior improves. That won't stop whatever made PHP quit in the first place, but it will prevent it from overrunning /var/run/

                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                Need help fast? Netgate Global Support!

                Do not Chat/PM for help!

                1 Reply Last reply Reply Quote 0
                • N
                  nkv2311
                  last edited by Jun 22, 2016, 3:50 PM Jun 22, 2016, 3:45 PM

                  Forgive my ignorance here, but I'm not exactly sure how you intended me to create this tunable.  If I make a new one should I name it kern.corefile with a value of /root/%N.core?  I assumed that is what you suggested, but when I tried it I was given an error for using unacceptable characters.  If you get a chance, could you please elaborate.
                  Also just to make sure, will I need to reboot to have the tunable take effect?
                  And just to give as much info as I can, I'm also seeing this same message in my log.  https://forum.pfsense.org/index.php?topic=111274.15

                  1 Reply Last reply Reply Quote 0
                  • J
                    jimp Rebel Alliance Developer Netgate
                    last edited by Jun 22, 2016, 4:29 PM

                    We just made that the default value for the next update/release, so you won't have to worry about it much longer. I forgot the tunables page might not allow % there, normally what you said is what it should do.

                    You can apply that change manually using the system patches package:
                    https://github.com/pfsense/pfsense/commit/4a304fbf40eaa1a5a77ae1360f87914989c1b8ef

                    You don't have to reboot, just run this from the shell or Diag > Command in the shell exec box:

                    sysctl kern.corefile=/root/%N.core
                    

                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    1 Reply Last reply Reply Quote 0
                    • J
                      jimp Rebel Alliance Developer Netgate
                      last edited by Jun 22, 2016, 4:34 PM

                      I also pushed a change to allow % and / in the sysctl values since they are valid for that oid (and presumably others)

                      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                      Need help fast? Netgate Global Support!

                      Do not Chat/PM for help!

                      1 Reply Last reply Reply Quote 0
                      • J
                        jpvonhemel
                        last edited by Jul 23, 2016, 12:11 PM

                        I entered the following code a few weeks ago and I have since rebooted the SG 2440.

                        [2.3.1-RELEASE][admin@pfSense.home]/root: sysctl kern.corefile=/root/%N.core
                        kern.corefile: %N.core -> /root/%N.core
                        [2.3.1-RELEASE][admin@pfSense.home]/root:
                        

                        I thought this had resolved, but this morning I noticed I have 106% of 3.4MiB under the Disk Usage (/var/run) section of the dashboard.

                        My version is:

                        2.3.1-RELEASE-p5 (amd64)
                        built on Wed Jun 15 13:58:09 CDT 2016
                        FreeBSD 10.3-RELEASE-p3

                        The system is on the latest version.

                        Any additional suggestions to help me with this problem?

                        Thank you very much.

                        1 Reply Last reply Reply Quote 0
                        • J
                          jimp Rebel Alliance Developer Netgate
                          last edited by Jul 23, 2016, 1:06 PM

                          That setting would not stick across reboots. If you rebooted since you added it, it would not be active.

                          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                          Need help fast? Netgate Global Support!

                          Do not Chat/PM for help!

                          1 Reply Last reply Reply Quote 0
                          • J
                            jpvonhemel
                            last edited by Jul 24, 2016, 2:28 AM

                            So I reentered the code, but I notice my Disk Usage is still 106%.  Here is a copy of me entering the code followed by a look at the /var/run directory.    I'm not sure what I need to do to fix this.

                            Thanks,
                            Jerold

                            
                            [2.3.1-RELEASE][admin@pfSense.home]/var/run: sysctl kern.corefile=/root/%N.core
                            kern.corefile: /root/%N.core -> /root/%N.core
                            [2.3.1-RELEASE][admin@pfSense.home]/var/run: ls -l
                            total 3412
                            drwxrwxr-x  2 root   operator      512 Jul 17 06:24 .snap
                            srw-rw-rw-  1 root   wheel           0 Jul 17 06:24 check_reload_status
                            -rw-------  1 root   wheel           5 Jul 17 06:24 cron.pid
                            -rw-------  1 root   wheel           3 Jul 17 06:24 devd.pid
                            srw-rw-rw-  1 root   wheel           0 Jul 17 06:24 devd.pipe
                            srw-rw-rw-  1 root   wheel           0 Jul 17 06:24 devd.seqpacket.pipe
                            -rw-------  1 root   wheel           5 Jul 19 14:57 dhclient.igb0.pid
                            -rw-r--r--  1 root   wheel           6 Jul 19 14:57 dhcp6c_igb0.pid
                            ----------  1 root   wheel           6 Jul 19 14:57 dhcpleases.pid
                            -rw-r--r--  1 root   wheel           6 Jul 23 14:56 dpinger_WAN_DHCP6_fe80::12dd:b1ff:fea5:220e%igb0_fe80::201:xxx:xxx:a846%igb0.pid
                            srw-rw-rw-  1 root   wheel           0 Jul 23 14:56 dpinger_WAN_DHCP6_fe80::12dd:b1ff:fea5:220e%igb0_fe80::201:xxx:xxx:a846%igb0.sock
                            -rw-r--r--  1 root   wheel           6 Jul 23 14:56 dpinger_WAN_DHCP_71.196.xxx.xxx_71.196.xxx.xxx.pid
                            srw-rw-rw-  1 root   wheel           0 Jul 23 14:56 dpinger_WAN_DHCP_71.196.xxx.xxx_71.196.xxx.xxx.sock
                            -rw-r--r--  1 root   wheel           6 Jul 17 06:24 expire_accounts.pid
                            -rw-r--r--  1 root   wheel           4 Jul 23 20:23 filter_reload_status
                            -rw-r--r--  1 root   wheel           5 Jul 17 06:24 filterlog.pid
                            -r--r--r--  1 root   wheel         194 Jul 17 06:24 ld-elf.so.hints
                            -r--r--r--  1 root   wheel         139 Jul 17 06:24 ld-elf32.so.hints
                            srw-rw-rw-  1 root   wheel           0 Jul 17 06:24 log
                            srw-------  1 root   wheel           0 Jul 17 06:24 logpriv
                            -rw-r--r--  1 root   wheel           6 Jul 17 06:24 miniupnpd.pid
                            -rw-r--r--  1 root   wheel           6 Jul 17 06:24 nginx-webConfigurator.pid
                            -rw-r--r--  1 root   wheel           5 Jul 17 06:24 ntpd.pid
                            -rw-r--r--  1 root   wheel           6 Jul 19 14:57 openvpn_server1.pid
                            -rw-------  1 root   wheel     3342336 Jul 19 14:55 php-fpm.core
                            -rw-r--r--  1 root   wheel           3 Jul 17 06:24 php-fpm.pid
                            srw-------  1 root   wheel           0 Jul 17 06:24 php-fpm.socket
                            -rw-r--r--  1 root   wheel           6 Jul 17 06:24 ping_hosts.pid
                            -rw-------  1 root   wheel           5 Jul 17 06:25 powerd.pid
                            drwxr-xr-x  2 squid  wheel         512 Jul 17 06:24 squid
                            -rw-r--r--  1 root   wheel           5 Jul 17 06:24 sshd.pid
                            -rw-------  1 root   wheel           5 Jul 17 06:25 syslog.pid
                            -rw-r--r--  1 root   wheel           6 Jul 19 14:57 unbound.pid
                            -rw-r--r--  1 root   wheel           6 Jul 17 06:24 update_alias_url_data.pid
                            -rw-r--r--  1 root   wheel           6 Jul 19 14:57 updaterrd.sh.pid
                            -rw-r--r--  1 root   wheel           0 Jul 17 06:24 utmp
                            -rw-r--r--  1 root   wheel         394 Jul 23 06:06 utx.active
                            -rw-------  1 root   wheel           5 Jul 17 06:25 watchdogd.pid
                            -rw-r--r--  1 root   wheel           6 Jul 17 06:24 xinetd.pid
                            
                            1 Reply Last reply Reply Quote 0
                            • J
                              JorgeOliveira
                              last edited by Jul 24, 2016, 10:43 AM

                              @jpvonhemel:

                              So I reentered the code, but I notice my Disk Usage is still 106%. Here is a copy of me entering the code followed by a look at the /var/run directory. I'm not sure what I need to do to fix this.

                              Run on shell:

                              rm /var/run/php-fpm.core
                              

                              My views have absolutely no warranty express or implied. Always do your own research.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                [[user:consent.lead]]
                                [[user:consent.not_received]]