• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

pfsense 2.5.2 slowly leaking memory

Scheduled Pinned Locked Moved General pfSense Questions
44 Posts 11 Posters 8.2k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • G
    Gertjan @mr.rosh
    last edited by Sep 1, 2021, 10:49 AM

    @mr-rosh said in pfsense 2.5.2 slowly leaking memory:

    However, i'm on
    Version 2.5.2-RELEASE (amd64)

    We all are.
    ( if @netnerdy was mentioning another version, that answer would be "upgrade" )

    Read "pcscd PC/SC Smart Card Daemon" to get rid :

    e3fdf05a-59a3-4823-a111-4bd6c5eacb8a-image.png

    so it won't auto start if your pfSense reboots.

    No "help me" PM's please. Use the forum, the community will thank you.
    Edit : and where are the logs ??

    1 Reply Last reply Reply Quote 0
    • S
      stephenw10 Netgate Administrator
      last edited by Sep 1, 2021, 11:26 AM

      Yes, the pcscd process is the most likely suspect. Start by disabling it:
      https://redmine.pfsense.org/issues/11933#note-7

      Steve

      1 Reply Last reply Reply Quote 0
      • N
        netnerdy
        last edited by Sep 1, 2021, 3:30 PM

        I think explaining the graph I posted in the beginning will help.

        The big mountains you see there are from the time when I didn't have pcscd fix. I would reboot pfsense and pcscd would eventually eat away my free memory. Like I mentioned in my first post, I have the full pcscd fix, so it doesn't start on boot anymore. You can see the positive effect it had on memory. I haven't rebooted pfsense for 7 days now. But it seems something else is chipping away at the memory.

        @psp you most likely have a mem leak problem as well. Not sure if it's same root cause.

        My services are below.

        f82b8428-905b-4fcf-a80e-cde35b5cd8ca-image.png

        I wonder if it's haproxy or vmware guest services. Perhaps it's wpa_supplicant (i do att bypass).

        1 Reply Last reply Reply Quote 0
        • S
          stephenw10 Netgate Administrator
          last edited by Sep 1, 2021, 3:43 PM

          Is it actually still growing steadily? Hard to say from that graph.

          Doesn't look like it's wired memory so what's using it? Check ps -auxwwd.

          Also if it never actually exhausts it that may not be a problem. Unused RAM helps nothing.

          Steve

          N 1 Reply Last reply Sep 1, 2021, 4:40 PM Reply Quote 0
          • N
            netnerdy @stephenw10
            last edited by Sep 1, 2021, 4:40 PM

            @stephenw10

            I would have expected a better response from Netgate: "If it never exhausts it that may not be a problem" is a frankly irresponsible way to categorize memory issues for a company that sells hardware that is supposed to run on devices with limited cpu and memory 24/7. Perhaps this kind of attitude allowed to slip a terrible memory leak like pcscd into production.

            Also what is the expectation here? Am I supposed to wait until all my memory is exhausted so my router goes down and suffer the consequences? I would rather be proactive than to be reactive.

            Sorry if this sounds too harsh, but please take this as constructive criticism as I really am not trying to troll you or Netgate.

            M S 2 Replies Last reply Sep 1, 2021, 5:03 PM Reply Quote 0
            • M
              mer @netnerdy
              last edited by Sep 1, 2021, 5:03 PM

              @netnerdy
              I'm just a user of Netgate products but since the dawn of embedded devices there have been arguments around unused RAM.
              One school is RAM should be 100% used the other is basically "you should always have 20% free RAM just in case".

              If you have unused RAM you are not near capacity.
              Is this device using ZFS or standard UFS? That makes a difference.

              One also needs to understand the different types of memory: wired is basically kernel owned can't really be freed quickly. Cache and Free are self explanatory, Inactive is typically "was used but isn't now and should be freed next".

              The position "if it never exhausts it may not be a problem" is technically accurate. Think garbage collection or lazy releasing of memory; so "it's allocated right now but in secs it's freed".

              1 Reply Last reply Reply Quote 0
              • C
                Cool_Corona
                last edited by Sep 1, 2021, 5:32 PM

                82365ff8-71a2-431a-b412-17ecf683bf68-billede.png

                1 Reply Last reply Reply Quote 0
                • S
                  stephenw10 Netgate Administrator @netnerdy
                  last edited by stephenw10 Sep 1, 2021, 5:58 PM Sep 1, 2021, 5:53 PM

                  @netnerdy said in pfsense 2.5.2 slowly leaking memory:

                  Also what is the expectation here? Am I supposed to wait until all my memory is exhausted so my router goes down and suffer the consequences? I would rather be proactive than to be reactive.

                  I'm asking if it's actually a leak or just something using memory because your graph in the first screenshot looks to be leveling off at ~30% which generally wouldn't worry me.
                  The actual used memory value will vary wildly with how much RAM you have installed and what packages you're running. 30% of 64GB with no packages would be unexpected. 😉

                  If it is a linear usage rise that characterises a memory leak then that's obviously a problem.
                  In which case check what is using the memory.

                  Steve

                  N 1 Reply Last reply Sep 2, 2021, 9:56 PM Reply Quote 0
                  • N
                    netnerdy @stephenw10
                    last edited by Sep 2, 2021, 9:56 PM

                    @stephenw10

                    Sorry if misunderstood, thx for the clarification. Unfortunately, I had to reboot my router for another reason so I'll need to wait a week to answer your question.

                    I started paying attention to mem usage just because of the pcscd leak. When I saw the second drop I was worried. I would be totally content with the explanation of "this kind of mem usage trend is by design", which seems to be case. I'll revive this if I see anything fishy. thx!

                    1 Reply Last reply Reply Quote 0
                    • S
                      stephenw10 Netgate Administrator
                      last edited by Sep 2, 2021, 11:51 PM

                      Mmm, hard to say without more data. If is is still leaking somewhere it's small. It could also be something that simply doesn't free memory unless it has to and since you still have >30% free it doesn't.
                      Keep monitoring it. Check the process list.

                      Steve

                      M 1 Reply Last reply Sep 22, 2021, 1:49 AM Reply Quote 0
                      • M
                        murzik @stephenw10
                        last edited by murzik Sep 22, 2021, 1:50 AM Sep 22, 2021, 1:49 AM

                        There is definitely a memory leak in 2.5.2. Have to reboot about once a week, otherwise system just crash, because it runs out of memory (have 8GB) and swap.

                        C 1 Reply Last reply Sep 22, 2021, 3:27 PM Reply Quote 0
                        • J
                          johnpoz LAYER 8 Global Moderator @netnerdy
                          last edited by johnpoz Sep 22, 2021, 10:05 AM Sep 22, 2021, 4:04 AM

                          @netnerdy said in pfsense 2.5.2 slowly leaking memory:

                          when you do a monthly view with 1 hour resolution?

                          Here is my sg4860 running 21.05.1, been up 46 days. Graph 1 month, 1 hour resolution

                          graph.jpg

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                          1 Reply Last reply Reply Quote 0
                          • C
                            Cool_Corona
                            last edited by Cool_Corona Sep 22, 2021, 4:33 AM Sep 22, 2021, 4:29 AM

                            This post is deleted!
                            1 Reply Last reply Reply Quote 0
                            • N
                              netnerdy
                              last edited by Sep 22, 2021, 5:45 AM

                              Turns out the biggest memory hog was pfblocker unbound mode. When I switched to python mode I see significant memory savings.

                              74741cbf-0c79-447b-bf26-ffd3bcffb59f-image.png

                              1 Reply Last reply Reply Quote 0
                              • C
                                Cool_Corona
                                last edited by Sep 22, 2021, 8:00 AM

                                Huge difference in systems all running the same config.

                                1e3f9b28-f62d-4e46-935b-38e31583530e-billede.png

                                3d631020-d0a6-41f4-b0b3-582d731f62cc-billede.png

                                ef2fdb52-9a5a-4e3d-838b-7bca143e630d-billede.png

                                c2f585f0-327e-4aaf-90de-ddff581e2cb7-billede.png

                                08abd6da-e277-463e-b0da-996ac6542987-billede.png

                                M 1 Reply Last reply Sep 22, 2021, 5:19 PM Reply Quote 0
                                • C
                                  Cybermaze @murzik
                                  last edited by Sep 22, 2021, 3:27 PM

                                  @murzik I agree, 2.5.2 is acting weird.

                                  On a system with 4GB of ram and 2GB of swap, after 70 days, I notice unusual behavior.

                                  Ram and swap usage is at 60%, however, I have never ever seen swap usage before, even after months of uptime.

                                  My setup is really basic, no ressource hungry addons of any kind.

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    mer @Cool_Corona
                                    last edited by Sep 22, 2021, 5:19 PM

                                    @cool_corona you say "same config"; is it same hardware, same set of packages, same functionality in your network?
                                    Just curious because your graphs are "interesting"

                                    C 1 Reply Last reply Sep 23, 2021, 9:43 AM Reply Quote 0
                                    • C
                                      Cool_Corona @mer
                                      last edited by Sep 23, 2021, 9:43 AM

                                      @mer Exactly the same.

                                      M 1 Reply Last reply Sep 23, 2021, 4:01 PM Reply Quote 1
                                      • M
                                        murzik @Cool_Corona
                                        last edited by murzik Sep 23, 2021, 4:06 PM Sep 23, 2021, 4:01 PM

                                        Just for one day
                                        8871cf51-d934-48da-b696-23e882b4c8df-image.png
                                        6c1d39ee-fff1-4f33-bcd7-241d09777dec-image.png

                                        After reboot
                                        7bebc9e4-86b3-42c7-8fbd-aecedacc91b1-image.png
                                        20b4451e-a816-4aa1-be64-6eba7080599f-image.png

                                        S 1 Reply Last reply Sep 23, 2021, 5:14 PM Reply Quote 0
                                        • S
                                          SteveITS Galactic Empire @murzik
                                          last edited by SteveITS Sep 23, 2021, 6:13 PM Sep 23, 2021, 5:14 PM

                                          @murzik And just to be clear, you've patched for the pcscd issue and/or that service is stopped (Status/Services)? We do have one 2.5.2 and its graphs are literally flat today except at 6:30 this morning when free memory increased 1%. That's when Suricata updates.

                                          Do you have pfBlocker's DNSBL active, or any other packages?

                                          Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
                                          When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
                                          Upvote 👍 helpful posts!

                                          J 1 Reply Last reply Sep 23, 2021, 5:54 PM Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received