Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    swap_pager_getswapspace Failed

    Scheduled Pinned Locked Moved General pfSense Questions
    14 Posts 3 Posters 5.8k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bmeeksB Offline
      bmeeks
      last edited by bmeeks

      Do you have any packages running such as Snort, Suricata or pfBlockerNG-devel? All of these can use up RAM space while performing certain tasks. It could be that is causing the swap issue.

      There is also the known bug with pcscd and its memory leak documented here: https://redmine.pfsense.org/issues/12095.

      As you may know, swap space is called upon whenever your firewall is out of free RAM. At that point, some of the contents of RAM are written to a temporary area of the disk to make room. The contents that get written to disk are processes that are currently "sleeping" and not actively executing. Then, when those tasks wake up and get ready to execute, their data is read back into RAM from the swap area. But this is extremely inefficient and will slow the firewall down to a crawl.

      The answer to your question is that "yes", you can increase swap space by manually adding another disk and creating a FreeBSD swap partition on it, and then adding that to the available swap inventory. But that is not the solution you need. You need to find out what is causing your firewall to use swap at all. A properly functioning pfSense firewall should almost never use swap space, and certainly never use enough to actually run out of space.

      hugoeyngH 1 Reply Last reply Reply Quote 3
      • hugoeyngH Offline
        hugoeyng @bmeeks
        last edited by

        @bmeeks thank you. Your answer is very clear for me and I totally agree that increasing swap area is not the solution.

        The solution, as you said, is to find the what is the cause of this behavior and not trying to treat the effect.

        As you can see in the image, there are 4GB RAM and, usually, 20-35% in use. So, there is no razon to the system using swap.

        e4b1b5bf-1e79-4210-aa38-7328e2d02645-image.png

        How could I check what process were running in the moment that the error message was logged.

        I love pfSense!

        Hugo Eyng
        Datamais Sistemas

        bmeeksB 1 Reply Last reply Reply Quote 0
        • bmeeksB Offline
          bmeeks @hugoeyng
          last edited by bmeeks

          @hugoeyng said in swap_pager_getswapspace Failed:

          @bmeeks thank you. Your answer is very clear for me and I totally agree that increasing swap area is not the solution.

          The solution, as you said, is to find the what is the cause of this behavior and not trying to treat the effect.

          As you can see in the image, there are 4GB RAM and, usually, 20-35% in use. So, there is no razon to the system using swap.

          e4b1b5bf-1e79-4210-aa38-7328e2d02645-image.png

          How could I check what process were running in the moment that the error message was logged.

          Well, you did not say if you had any installed packages running. It is possible, if you do, that one of them is exhausting RAM during some phase of its operation. One candidate would be the IDS/IPS packages (Snort or Suricata). I created and maintain those packages, so I know that during rule updates, for example, their RAM consumption can temporarily increase quite a bit. I believe the same may be true for some operating modes of pfBlockerNG-devel when using the new Python mode with unbound.

          In the case of the IDS/IPS packages, the RAM usage is temporary, so unless you happened to be logged into the firewall at the exact instant the problem occurred, you would not see it.

          Your second post mentioned the issue happening at the same time each day. That indicates to me a cron task is at fault. So the first step would be to see what is running at that time. You can install the Cron package to provide an easy-to-navigate GUI interface for managing cron tasks. Or, if you are good at the FreeBSD command-line, you can use the various shell tools to do the same thing.

          Listing all of the currently running processes might reveal something, too. Try running this from a shell prompt:

          ps -ax
          

          or

          top -aSH
          

          It is a bit puzzling to me that your swap usage appears to be 80% when you are showing only about 22% of RAM currently utilized.

          hugoeyngH 1 Reply Last reply Reply Quote 1
          • hugoeyngH Offline
            hugoeyng @bmeeks
            last edited by hugoeyng

            @bmeeks said in swap_pager_getswapspace Failed:

            @hugoeyng said in swap_pager_getswapspace Failed:

            @bmeeks thank you. Your answer is very clear for me and I totally agree that increasing swap area is not the solution.

            The solution, as you said, is to find the what is the cause of this behavior and not trying to treat the effect.

            As you can see in the image, there are 4GB RAM and, usually, 20-35% in use. So, there is no razon to the system using swap.

            e4b1b5bf-1e79-4210-aa38-7328e2d02645-image.png

            How could I check what process were running in the moment that the error message was logged.

            Well, you did not say if you had any installed packages running. It is possible, if you do, that one of them is exhausting RAM during some phase of its operation. One candidate would be the IDS/IPS packages (Snort or Suricata). I created and maintain those packages, so I know that during rule updates, for example, their RAM consumption can temporarily increase quite a bit. I believe the same may be true for some operating modes of pfBlockerNG-devel when using the new Python mode with unbound.

            In the case of the IDS/IPS packages, the RAM usage is temporary, so unless you happened to be logged into the firewall at the exact instant the problem occurred, you would not see it.

            Your second post mentioned the issue happening at the same time each day. That indicates to me a cron task is at fault. So the first step would be to see what is running at that time. You can install the Cron package to provide an easy-to-navigate GUI interface for managing cron tasks. Or, if you are good at the FreeBSD command-line, you can use the various shell tools to do the same thing.

            Listing all of the currently running processes might reveal something, too. Try running this from a shell prompt:

            ps -ax
            

            or

            top -aSH
            

            It is a bit puzzling to me that your swap usage appears to be 80% when you are showing only about 22% of RAM currently utilized.

            No Snort, no Suricata installed.

            I will post some images:

            top -aSH -o size

            70f00937-799e-48b7-b899-6070aea18ead-image.png

            Installed packages:

            cc2d2e6d-7c08-40eb-b49f-52f107e54bca-image.png

            Cron:

            8d9ac4a8-61ed-46ec-89f5-e6b0cae493f1-image.png

            ps -ax
            he.txt

            Services runnig:

            bb75a2e2-f838-4aea-93dd-43e6521b6670-image.png

            I love pfSense!

            Hugo Eyng
            Datamais Sistemas

            1 Reply Last reply Reply Quote 0
            • bmeeksB Offline
              bmeeks
              last edited by bmeeks

              A few things I notice in those screenshots.

              First, you have three instances of pcscd running. If you do not have any installed wireless cards in the firewall, I recommend stopping that service. Embedded within the link I provided earlier is a link to a pfSense patch that will disable the service and prevent it from restarting unless an actual wireless card is detected.

              Secondly, clamav appears to be using quite a few resources, including RAM. Unless you have a full MITM (man-in-the-middle) encryption interception scheme configured so the files are decrypted on the firewall, scanning web and email files passing through the firewall is pointless as they are encrypted for the most part these days (via https web sites using SSL and email traffic travelling almost exclusively via TLS). So I would disable anti-virus on the firewall and instead put that resource on the endpoints (workstations and servers) in your network.

              And finally, squid is taking up a fair amount of RAM. I think together all of these packages are periodically consuming all of the RAM and thus triggering the use of swap space. And the 4 GB of configured swap is not enough based on the error you receive.

              As for the hourly trigger, I see only one cron task configured to run at 17 minutes past the hour, and that is the clamav update routine. That process might be the one that puts you over the top with respect to running out of RAM and also swap.

              hugoeyngH 1 Reply Last reply Reply Quote 1
              • hugoeyngH Offline
                hugoeyng @bmeeks
                last edited by

                @bmeeks said in swap_pager_getswapspace Failed:

                A few things I notice in those screenshots.

                First, you have three instances of pcscd running. If you do not have any installed wireless cards in the firewall, I recommend stopping that service. Embedded within the link I provided earlier is a link to a pfSense patch that will disable the service and prevent it from restarting unless an actual wireless card is detected.

                Secondly, clamav appears to be using quite a few resources, including RAM. Unless you have a full MITM (man-in-the-middle) encryption interception scheme configured so the files are decrypted on the firewall, scanning web and email files passing through the firewall is pointless as they are encrypted for the most part these days (via https web sites using SSL and email traffic travelling almost exclusively via TLS). So I would disable anti-virus on the firewall and instead put that resource on the endpoints (workstations and servers) in your network.

                And finally, squid is taking up a fair amount of RAM. I think together all of these packages are periodically consuming all of the RAM and thus triggering the use of swap space. And the 4 GB of configured swap is not enough based on the error you receive.

                As for the hourly trigger, I see only one cron task configured to run at 17 minutes past the hour, and that is the clamav update routine. That process might be the one that puts you over the top with respect to running out of RAM and also swap.

                I have no idea about why, when and who instaled pcscd.
                I will try to understand https://redmine.pfsense.org/issues/12095

                About squid I think that the below values can be critical (or became critical now for som razon).

                6a4506b3-7ee6-494c-a1a9-397a0736b8fa-image.png

                I love pfSense!

                Hugo Eyng
                Datamais Sistemas

                1 Reply Last reply Reply Quote 0
                • bmeeksB Offline
                  bmeeks
                  last edited by

                  pcscd is a native pfSense component (actually, it's part of FreeBSD). It is not something you installed. It comes with pfSense. But it currently has a memory leak problem that causes it to consume RAM and not return it to the operating system properly.

                  As for your squid configuration, you've told it to use 2 GB of RAM. The default for that parameter is 64 MB. That's a huge difference. Your selected value is many times larger than the suggested default.

                  I don't use squid, so I can't offer any guidance beyond simply saying that configuring a value that is so much larger than the default is potentially not optimum.

                  hugoeyngH 1 Reply Last reply Reply Quote 1
                  • hugoeyngH Offline
                    hugoeyng @bmeeks
                    last edited by

                    @bmeeks

                    As I said before, I will try https://redmine.pfsense.org/issues/12095

                    About Squid I will reduce the amount of RAM, even I think that could not be the cause once it defined so since long time ago. But ...

                    About clamav, it is my existencial dilema. Keeping or not to keep installed.

                    This morning the error didn´t happened.

                    Thank you for all your answers. You helped a lot showing options and view points.

                    Best, regards.

                    I love pfSense!

                    Hugo Eyng
                    Datamais Sistemas

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S Online
                      stephenw10 Netgate Administrator
                      last edited by

                      It's almost certainly the leak in pcscd. You should apply that patch and reboot when you can.

                      You should pretty much never see swap use in pfSense. If you are seeing it regularly being used something is misconfigured / needs more tuning.

                      Steve

                      hugoeyngH 1 Reply Last reply Reply Quote 1
                      • hugoeyngH Offline
                        hugoeyng @stephenw10
                        last edited by

                        @stephenw10 I restarted pfSense and the issue has gone. I did it around 3 hours ago.

                        0cbcd3f3-7170-429d-b8e5-69fbb6f9245f-image.png

                        I will monitor to find out if it was a "exception" or if it is a recurrent issue.

                        Thank you for your answer.

                        I love pfSense!

                        Hugo Eyng
                        Datamais Sistemas

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S Online
                          stephenw10 Netgate Administrator
                          last edited by

                          If you saw it once you will probably see it again. You shouldn't see any SWAP use until the RAM is used so it will likely take some days. I would apply that patch, it will be in the next release anyway.

                          Steve

                          hugoeyngH 1 Reply Last reply Reply Quote 1
                          • hugoeyngH Offline
                            hugoeyng @stephenw10
                            last edited by

                            @stephenw10 Thank you Steve.

                            I will apply the patch.

                            I love pfSense!

                            Hugo Eyng
                            Datamais Sistemas

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.