Wire memory slowly increasing



  • I have been seeing my wire memory usage slowly increase more than usual over the past month. I'm trying to understand why and I'm worried it's not gonna stop till it starts overflowing into the swap territory. I'm now at about 60% wire memory (Close to 75% total) and I have 8 GB of memory. That's a lot of memory usage!

    The network usage is pretty minimal in our office. The traffic on average is around 550 kbps. Average packets per second is around 500. I haven't notice any jumps or changes in those numbers during this past month when the wire memory is increasing. Looking at the 3 month trend, the wire memory is typically creeping up but this past month it's going up at a much faster rate. Any thoughts on why that's happening?

    I have two Intel EXPI9301CT gigabit desktop adapters installed.
    The rest of my system info is attached. I also attached the a 3 month memory graph and system activity from shell.

    Packages running:
    Squid
    ClamAV
    Suricata in legacy mode
    Pfblocker
    Syslog
    NUT
    OpenVPN
    ![System info.JPG](/public/imported_attachments/1/System info.JPG)
    ![System info.JPG_thumb](/public/imported_attachments/1/System info.JPG_thumb)
    ![Memory usage.JPG](/public/imported_attachments/1/Memory usage.JPG)
    ![Memory usage.JPG_thumb](/public/imported_attachments/1/Memory usage.JPG_thumb)
    ![system activity.JPG](/public/imported_attachments/1/system activity.JPG)
    ![system activity.JPG_thumb](/public/imported_attachments/1/system activity.JPG_thumb)



  • What is "wire memory"?  The only thing that comes to mind is an old memory technology that stored data on ferrous wire. Or perhaps ferrite core memory, in which the cores were woven into a core plane.  I have some of that here.



  • You are running several heavy packages, and squid is notorious for eating RAM.

    http://www.comfsm.fm/computing/squid/FAQ-8.html

    Shell in and run:

    squidclient -h YOUR_LAN_IP -p 3128 mgr:info
    

    and check the Resource usage for squid and Memory accounted for sections.



  • Thanks Kom, that is very useful info on squid.

    That command didn't seem to give me anything helpful. Below is what it returned. I used the IP of the LAN interface of the pfsense box.

    In any case, I saw one of the recommendation in that link was to lower the cache_mem aka memory cache size. I had it set to 1024 MB since I have a lot of memory. Maybe I was too ambitious with that so I lowered it back to the default 64 MB. I also lowered the Maximum object size in RAM back to the default 256 kB. I had that set to 1024 kB. I also cleared the local cache, but it hasn't made any impact on memory usage, at least not any immediate impact. If anything, wire memory went up a tad. I guess I'll keep an eye on it. Maybe it needs a reboot?

    HTTP/1.1 403 Forbidden
    Server: squid
    Mime-Version: 1.0
    Date: Wed, 02 May 2018 17:34:06 GMT
    Content-Type: text/html;charset=utf-8
    Content-Length: 3470
    X-Squid-Error: ERR_ACCESS_DENIED 0
    Vary: Accept-Language
    Content-Language: en
    X-Cache: MISS from pfsense
    X-Cache-Lookup: NONE from pfsense:3128
    Via: 1.1 pfsense (squid)
    Connection: close
    
    <title>ERROR: The requested URL could not be retrieved</title>
    
    # ERROR
    
    ## The requested URL could not be retrieved
    
    * * *
    
    The following error was encountered while trying to retrieve the URL: [cache_object://172.16.21.2/info](cache_object://172.16.21.2/info)
    
    > **Access Denied.**
    
    Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.
    
    Your cache administrator is [raffi@telebyteusa.com](mailto:raffi@telebyteusa.com?subject=CacheErrorInfo%20-%20ERR_ACCESS_DENIED&body=CacheHost%3A%20pfsense%0D%0AErrPage%3A%20ERR_ACCESS_DENIED%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Wed,%2002%20May%202018%2017%3A34%3A06%20GMT%0D%0A%0D%0AClientIP%3A%20172.16.21.2%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2Finfo%20HTTP%2F1.0%0AUser-Agent%3A%20squidclient%2F3.5.27%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%20close%0D%0AHost%3A%20172.16.21.2%0D%0A%0D%0A%0D%0A).
    
    * * *
    
    Generated Wed, 02 May 2018 17:34:06 GMT by pfsense (squid)
    
    


  • This is a common error.

    You need to go to Services - Squid proxy server - Local Cache - Squid Cache General Settings - External Cache Managers.

    Set it to the LAN IP squid listens on as well as 127.0.0.1 like this:

    10.10.4.1;127.0.0.1

    Replace 10.10.4.1 with your LAN IP.  Save it and then try the squidclient command again.



  • Awesome! That did the trick. This looks more useful, but I don't know what it really means. The section on analyzing the cache manager output doesn't seem to apply to the version I'm using. Here is the new output. I think the total memory accounted for looks low. Only 11 MB.

    HTTP/1.1 200 OK
    Server: squid
    Mime-Version: 1.0
    Date: Wed, 02 May 2018 19:13:59 GMT
    Content-Type: text/plain;charset=utf-8
    Expires: Wed, 02 May 2018 19:13:59 GMT
    Last-Modified: Wed, 02 May 2018 19:13:59 GMT
    X-Cache: MISS from pfsense
    X-Cache-Lookup: MISS from pfsense:3128
    Via: 1.1 pfsense (squid)
    Connection: close
    
    Squid Object Cache: Version 3.5.27
    Build Info:
    Service Name: squid
    Start Time:     Wed, 02 May 2018 17:31:05 GMT
    Current Time:   Wed, 02 May 2018 19:13:59 GMT
    Connection information for squid:
            Number of clients accessing cache:      16
            Number of HTTP requests received:       3247
            Number of ICP messages received:        0
            Number of ICP messages sent:    0
            Number of queued ICP replies:   0
            Number of HTCP messages received:       0
            Number of HTCP messages sent:   0
            Request failure ratio:   0.00
            Average HTTP requests per minute since start:   31.6
            Average ICP messages per minute since start:    0.0
            Select loop called: 517484 times, 11.931 ms avg
    Cache information for squid:
            Hits as % of all requests:      5min: 0.4%, 60min: 2.3%
            Hits as % of bytes sent:        5min: 0.2%, 60min: 0.3%
            Memory hits as % of hit requests:       5min: 100.0%, 60min: 37.0%
            Disk hits as % of hit requests: 5min: 0.0%, 60min: 2.2%
            Storage Swap size:      5714 KB
            Storage Swap capacity:   0.1% used, 99.9% free
            Storage Mem size:       2680 KB
            Storage Mem capacity:    4.1% used, 95.9% free
            Mean Object Size:       55.48 KB
            Requests given to unlinkd:      6
    Median Service Times (seconds)  5 min    60 min:
            HTTP Requests (All):  54.42904 57.44813
            Cache Misses:          0.05951  0.06640
            Cache Hits:            0.00000  0.00091
            Near Hits:             0.00000  0.00000
            Not-Modified Replies:  0.00000  0.00091
            DNS Lookups:           0.02336  0.02562
            ICP Queries:           0.00000  0.00000
    Resource usage for squid:
            UP Time:        6173.953 seconds
            CPU Time:       16.255 seconds
            CPU Usage:      0.26%
            CPU Usage, 5 minute avg:        0.21%
            CPU Usage, 60 minute avg:       0.22%
            Maximum Resident Size: 266352 KB
            Page faults with physical i/o: 9
    Memory accounted for:
            Total accounted:        11164 KB
            memPoolAlloc calls:    726383
            memPoolFree calls:     748846
    File descriptor usage for squid:
            Maximum number of file descriptors:   233370
            Largest file desc currently in use:    215
            Number of file desc currently in use:  179
            Files queued for open:                   0
            Available number of file descriptors: 233191
            Reserved number of file descriptors:   100
            Store Disk files open:                   0
    Internal Data Structures:
               155 StoreEntries
               150 StoreEntries with MemObjects
               149 Hot Object Cache Items
               103 on-disk objects
    
    


  • "Wired memory" is a BSD specific term for kernel memory that is "wired in place" and can't be swapped out by the memory management. What you're seeing might be normal depending on the circumstances, unless your system runs out of memory and starts swapping heavily there might be no problem to fix.



  • Squid only appears to be using ~256 MB.



  • Oh, based on your top(1) output you are using ZFS and the ARC cache of ZFS is using 3465 MBs of wired memory. Looks totally normal to me. ARC cache is auto tuned/scaled based on the available memory so you don't really have to worry about it. The rest of the wired memory is used by other kernel data structures that can't be swapped out.



  • Thanks kps, nothing to worry about is what I want to hear. I'll keep an eye on it.

    Thanks Kom, I think squid may not be an issue then. The changes I made to squid didn't affect the memory use, so that kind of hinted at that too. I'm hoping it's nothing to worry about. I'll worry when swap starts being used. Thanks for the info on squid anyway. I'm glad I learned something new today.



  • If it ever becomes a problem there's a system tunable that you can set to limit the amount of memory allocated for the ARC cache. Place this in /boot/loader.conf.local (assuming you want the limit to be at 2048 MBs).

    
    vfs.zfs.arc_max=2048M
    
    


  • Great! Thanks. I'll keep that in mind if it does become a problem.



  • Guess what... swap is now being used!!! I was fearing this and now I'll have nightmares. Users on the network as well as myself have been noticing a "slow connection". This could explain it. Why is my swap being used though? My memory according to the Dashboard is 90% used.

    0_1536266591943_cbd623cc-46de-4fbd-abcb-56472297afdc-image.png

    According to the last datapoint in the memory graph, it's nowhere near 90% ???
    0_1536266757389_afc6c1c5-5893-465c-94c8-86549e5f6e07-image.png

    According to the system activity, memory usage is not that high either... I think.
    0_1536266835043_47b8cbc6-6b32-4090-8e88-2d07c4b176c4-image.png

    And according to ps -aux, the memory usage is not that high either.

    /root: ps -aux
    USER      PID  %CPU %MEM     VSZ     RSS TT  STAT STARTED         TIME COMMAND
    root       11 397.5  0.0       0      64  -  RL    6Jun18 515112:38.18 [idle]
    root    44844   1.8  0.1   36800    8704  -  S    29Aug18    345:37.04 /usr/loc
    root    39540   0.2  2.1  305780  171600  -  Ss   Wed09       30:56.28 /usr/loc
    root    41892   0.2  0.4  289816   30204  -  S    29Aug18      0:22.93 php-fpm:
    root    43132   0.2  3.3  391404  275056  -  SNs  12:31        2:32.56 /usr/loc
    root    20840   0.1  0.4  287384   29540  -  S    29Aug18      0:20.24 php-fpm:
    root    47312   0.1  2.3  438604  190192  -  S    Wed09       13:06.22 /usr/loc
    root        0   0.0  0.1       0    4224  -  DLs   6Jun18    448:11.78 [kernel]
    root        1   0.0  0.0    5028     192  -  ILs   6Jun18      0:00.10 /sbin/in
    root        2   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [crypto]
    root        3   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [crypto
    root        4   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [crypto
    root        5   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [crypto
    root        6   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [crypto
    root        7   0.0  0.0       0      32  -  DL    6Jun18     16:34.13 [cam]
    root        8   0.0  0.0       0      16  -  DL    6Jun18      0:00.88 [soaiod1
    root        9   0.0  0.0       0      16  -  DL    6Jun18      0:00.88 [soaiod2
    root       10   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [audit]
    root       12   0.0  0.0       0     448  -  WL    6Jun18    431:29.90 [intr]
    root       13   0.0  0.0       0      64  -  DL    6Jun18      0:00.00 [ng_queu
    root       14   0.0  0.0       0      48  -  DL    6Jun18      0:00.01 [geom]
    root       15   0.0  0.0       0     160  -  DL    6Jun18     14:55.50 [usb]
    root       16   0.0  0.0       0      16  -  DL    6Jun18      0:00.89 [soaiod3
    root       17   0.0  0.0       0      16  -  DL    6Jun18      0:00.88 [soaiod4
    root       18   0.0  0.0       0     128  -  DL    6Jun18     36:58.08 [zfskern
    root       19   0.0  0.0       0      16  -  DL    6Jun18      0:00.00 [sctp_it
    root       20   0.0  0.0       0      16  -  DL    6Jun18     28:12.47 [pf purg
    root       21   0.0  0.0       0      16  -  DL    6Jun18      6:55.75 [rand_ha
    root       22   0.0  0.0       0      48  -  DL    6Jun18     14:46.96 [pagedae
    root       23   0.0  0.0       0      16  -  DL    6Jun18      0:00.17 [vmdaemo
    root       24   0.0  0.0       0      16  -  DL    6Jun18      0:00.09 [pagezer
    root       25   0.0  0.0       0      16  -  DL    6Jun18      0:29.09 [bufdaem
    root       26   0.0  0.0       0      16  -  DL    6Jun18      0:28.44 [bufspac
    root       27   0.0  0.0       0      16  -  DL    6Jun18      2:07.05 [syncer]
    root       28   0.0  0.0       0      16  -  DL    6Jun18      0:28.52 [vnlru]
    root       68   0.0  0.0       0      16  -  DL    6Jun18      0:07.04 [md0]
    squid     606   0.0  0.0   38424    4036  -  S    29Jul18      2:40.39 (pinger)
    root      640   0.0  0.1  280976   11656  -  Ss    6Jun18      2:58.07 php-fpm:
    root      655   0.0  0.0   19456    1180  -  INs   6Jun18      0:00.13 /usr/loc
    root      657   0.0  0.0   19456       0  -  IWN  -            0:00.00 check_re
    root      692   0.0  0.0    9176     892  -  Ss    6Jun18     58:28.76 /sbin/de
    squid    2191   0.0  0.0   38424    4052  -  S    21Aug18      1:08.03 (pinger)
    squid    2223   0.0  0.0   38424    2872  -  S    20Jun18      5:11.42 (pinger)
    squid    2291   0.0  0.0   38424    4036  -  S     8Jul18      4:02.11 (pinger)
    squid    2356   0.0  0.0   38424    4052  -  S    Wed00        0:08.79 (pinger)
    squid    3428   0.0  0.0   38424    4036  -  S    10Jul18      3:55.21 (pinger)
    squid    3979   0.0  0.0   38424    4052  -  S    27Aug18      0:46.81 (pinger)
    dhcpd    4706   0.0  0.1   16652    7020  -  Ss   10Jul18      4:54.32 /usr/loc
    root     4714   0.0  0.1   78872    7356  -  Ss   16:29        0:00.12 sshd: ad
    root     5445   0.0  0.0   13096    1000  -  SN   10Jul18     10:03.12 /bin/sh
    squid    5847   0.0  0.0   38424    4036  -  S    10Jul18      3:48.39 (pinger)
    root     6984   0.0  0.1   20352    6232  -  Ss   11Jul18      0:29.83 /usr/loc
    squid    7114   0.0  0.0   38424    4036  -  S    31Jul18      2:30.54 (pinger)
    root     8873   0.0  0.0   12736    1880  -  Ss    6Jun18     10:54.02 /usr/loc
    squid    9021   0.0  0.0   38424    2864  -  S     8Jun18      6:01.85 (pinger)
    clamav   9924   0.0  0.0   21292    2536  -  Ss    6Jun18      4:27.48 /usr/loc
    squid   10678   0.0  0.0   38424    2868  -  S    12Jun18      5:43.36 (pinger)
    root    10737   0.0  0.0   58800       0  -  IWs  -            0:00.00 /usr/loc
    squid   11038   0.0 22.2 2391456 1844992  -  S     6Jun18    173:11.61 (squid-1
    squid   11082   0.0  0.0   38424    4052  -  S    14Aug18      1:39.67 (pinger)
    squid   12406   0.0  0.0   38424    2864  -  S    11Jun18      5:41.73 (pinger)
    root    12585   0.0  0.0   53524    3816  -  Is    6Jun18      0:00.00 /usr/sbi
    squid   12946   0.0  0.0   38424    4036  -  S    10Jul18      3:49.26 (pinger)
    squid   13220   0.0  0.0   38424    4036  -  S    15Jul18      3:35.25 (pinger)
    squid   13484   0.0  0.0   38424    4052  -  S    15Aug18      1:33.61 (pinger)
    squid   14018   0.0  0.0   38424    4036  -  S    30Jul18      2:34.63 (pinger)
    squid   14504   0.0  0.0   38424    4052  -  S    19Aug18      1:19.54 (pinger)
    squid   15190   0.0  0.0   38300    2748  -  I     6Jun18      0:04.61 (unlinkd
    squid   15400   0.0  0.0   38424    2828  -  S     6Jun18      5:53.75 (pinger)
    squid   15496   0.0  0.0   38424    4036  -  S    20Jul18      3:17.71 (pinger)
    squid   15918   0.0  0.0   38424    4052  -  S    21Aug18      1:11.68 (pinger)
    squid   17576   0.0  0.0   38424    4052  -  S     7Aug18      2:08.22 (pinger)
    squid   18893   0.0  0.0   38424    3460  -  S     1Jul18      4:24.46 (pinger)
    squid   19381   0.0  0.0   38424    4036  -  S    10Jul18      3:50.83 (pinger)
    root    19842   0.0  0.0   10588    2012  -  Is    6Jun18      0:00.09 /usr/loc
    root    20864   0.0  0.1   41544    6616  -  I    Wed09        0:00.00 /usr/loc
    root    21018   0.0  0.1   50748    7832  -  Ss   Wed09        0:00.20 /usr/loc
    squid   21374   0.0  0.0   38424    4036  -  S    27Jul18      2:49.24 (pinger)
    squid   21394   0.0  0.0   38424    4036  -  S    24Jul18      2:58.48 (pinger)
    squid   21525   0.0  0.0   38424    3460  -  S    28Jun18      4:41.75 (pinger)
    squid   23125   0.0  0.0   38424    2864  -  S    17Jun18      5:24.03 (pinger)
    squid   23715   0.0  0.0   38424    2864  -  S     7Jun18      5:58.81 (pinger)
    squid   23729   0.0  0.0   38424    4036  -  S     1Aug18      2:28.38 (pinger)
    root    24601   0.0  0.0    6180    1936  -  I    16:29        0:00.00 sleep 55
    root    25108   0.0  0.4  287768   33056  -  I    29Aug18      0:22.28 php-fpm:
    squid   26087   0.0  0.0   38424    4036  -  S    17Jul18      3:25.45 (pinger)
    clamav  27135   0.0  0.1   43184    5520  -  S    23Aug18      1:29.01 /usr/loc
    squid   27295   0.0  0.0   38424    4052  -  S    Sun00        0:22.69 (pinger)
    squid   27607   0.0  0.0   38424    4036  -  S    22Jul18      3:04.78 (pinger)
    clamav  28126   0.0  0.1   43184    5504  -  S    23Aug18      1:31.43 /usr/loc
    squid   29328   0.0  0.0   38424    4036  -  S    19Jul18      3:18.48 (pinger)
    squid   29468   0.0  0.0   38424    2864  -  S    10Jun18      5:47.99 (pinger)
    squid   29578   0.0  0.0   38424    3460  -  S    26Jun18      4:45.75 (pinger)
    squid   30852   0.0  0.0   38424    4052  -  S     6Aug18      2:07.38 (pinger)
    squid   31370   0.0  0.0   38424    4052  -  S     8Aug18      2:03.54 (pinger)
    squid   32250   0.0  0.0   38424    4052  -  S     6Aug18      2:09.37 (pinger)
    squid   33971   0.0  0.0   38424    4036  -  S    21Jul18      3:10.41 (pinger)
    squid   36001   0.0  0.0   38424    4052  -  S    10Aug18      1:54.75 (pinger)
    squid   36202   0.0  0.0   38424    4052  -  S     6Aug18      2:08.37 (pinger)
    root    39254   0.0  0.1   22240    4332  -  S    Wed09        0:56.07 redis-se
    squid   40431   0.0  0.0   38424    4052  -  S    12Aug18      1:46.12 (pinger)
    squid   40627   0.0  0.0   38424    4052  -  S    29Aug18      0:37.84 (pinger)
    squid   40977   0.0  0.0   38424    4052  -  S    13Aug18      1:43.26 (pinger)
    clamav  41753   0.0  0.1   43184    5628  -  S    21Aug18      1:20.62 /usr/loc
    squid   42082   0.0  0.0   38424    4036  -  S    12Jul18      3:47.49 (pinger)
    clamav  43127   0.0  0.1   43184    5528  -  S    21Aug18      1:22.09 /usr/loc
    squid   44453   0.0  0.0   38424    4036  -  S    31Jul18      2:31.27 (pinger)
    squid   44793   0.0  0.0   38424    4052  -  S    28Aug18      0:43.06 (pinger)
    squid   45184   0.0  0.0   38424    4036  -  S    11Jul18      3:46.84 (pinger)
    squid   45219   0.0  0.0   38424    4036  -  S     2Aug18      2:25.88 (pinger)
    root    45682   0.0  0.0   25432       0  -  IWs  -            0:00.00 nginx: m
    root    45733   0.0  0.0   27480    2388  -  S     6Jun18      2:11.00 nginx: w
    root    45775   0.0  0.0   27480    2340  -  S     6Jun18      1:20.27 nginx: w
    squid   45917   0.0  0.0   38424    3460  -  S    24Jun18      4:57.20 (pinger)
    squid   46117   0.0  0.0   38424    4052  -  S    27Aug18      0:44.00 (pinger)
    root    46318   0.0  0.1   49904   12024  -  SNs  12:31        0:01.21 /usr/loc
    root    46381   0.0  0.0   12504    1220  -  Is    6Jun18      0:18.99 /usr/sbi
    squid   46627   0.0  0.0   38424    3460  -  S    30Jun18      4:34.74 (pinger)
    root    46847   0.0  0.2   25168   12992  -  Ss    6Jun18      6:09.93 /usr/loc
    squid   48371   0.0  0.0   38424    4052  -  S    Mon00        0:17.71 (pinger)
    squid   49034   0.0  0.0   38424    4052  -  S    23Aug18      1:04.26 (pinger)
    root    49564   0.0  0.4  289816   32416  -  I    29Aug18      0:25.08 php-fpm:
    squid   49671   0.0  0.0   38424    4052  -  S    17Aug18      1:28.36 (pinger)
    squid   51337   0.0  0.0   38424    3460  -  S    29Jun18      4:35.13 (pinger)
    squid   52136   0.0  0.0   38424    4052  -  S     5Aug18      2:13.85 (pinger)
    squid   52174   0.0  0.0   38424    4036  -  S    18Jul18      3:21.09 (pinger)
    squid   53310   0.0  0.0   38424    4052  -  S    26Aug18      0:51.16 (pinger)
    squid   53591   0.0  0.0   38424    4036  -  S    10Jul18      3:49.88 (pinger)
    squid   55044   0.0  0.0   38424    4036  -  S     9Jul18      3:56.14 (pinger)
    squid   55459   0.0  0.0   38424    3460  -  S    21Jun18      5:09.20 (pinger)
    squid   58694   0.0  0.0   38424    3460  -  S    22Jun18      5:01.95 (pinger)
    squid   59013   0.0  0.0   38424    4052  -  S    15Aug18      1:31.85 (pinger)
    squid   59817   0.0  0.0   38424    3460  -  S     2Jul18      4:19.35 (pinger)
    squid   60010   0.0  0.0   38424    4052  -  S    29Aug18      0:38.89 (pinger)
    squid   60727   0.0  0.0   38424    2864  -  S    15Jun18      5:25.47 (pinger)
    squid   61140   0.0  0.0   38424    4036  -  S    10Jul18      3:51.25 (pinger)
    squid   61995   0.0  0.0   38424    3816  -  S     4Jul18      4:15.84 (pinger)
    squid   62124   0.0  0.0   38424    4052  -  S    Tue00        0:13.18 (pinger)
    squid   62149   0.0  0.0   38424    4036  -  S    11Jul18      3:48.57 (pinger)
    clamav  62599   0.0 10.7  992496  884432  -  Is    6Jun18    512:18.87 /usr/loc
    squid   62916   0.0  0.0   38424    3460  -  S    27Jun18      4:46.58 (pinger)
    root    63363   0.0  0.0   10376    1788  -  Ss    6Jun18     19:31.44 /usr/sbi
    squid   63592   0.0  0.0   38424    3460  -  S    23Jun18      4:59.58 (pinger)
    squid   65505   0.0  0.0   38424    4052  -  S    18Aug18      1:23.73 (pinger)
    squid   65557   0.0  0.0   38424    4036  -  S    23Jul18      3:03.22 (pinger)
    squid   66880   0.0  0.0   38424    2864  -  S    14Jun18      5:34.61 (pinger)
    squid   67192   0.0  0.0   38424    4052  -  S    20Aug18      1:15.08 (pinger)
    squid   67994   0.0  0.0   38424    2864  -  S    19Jun18      5:14.22 (pinger)
    squid   68051   0.0  0.0   38424    4052  -  S     3Aug18      2:23.71 (pinger)
    root    68728   0.0  0.0   10988    2436  -  Is   10Jul18      9:04.05 /usr/loc
    root    69202   0.0  0.0   12584    2432  -  Is   Wed09        0:00.00 /usr/loc
    uucp    69283   0.0  0.0   12584    2440  -  S    Wed09        0:02.15 /usr/loc
    squid   69680   0.0  0.0   38424    4036  -  S    25Jul18      2:57.43 (pinger)
    root    70257   0.0  0.0    8232       0  -  IWs  -            0:00.00 /usr/loc
    root    70459   0.0  0.0    8232     336  -  I     6Jun18      0:02.52 minicron
    squid   70803   0.0  0.0   38424    4036  -  S    14Jul18      3:40.80 (pinger)
    root    70829   0.0  0.0    8232       0  -  IWs  -            0:00.00 /usr/loc
    root    71316   0.0  0.0    8232       0  -  IWs  -            0:00.00 /usr/loc
    root    71574   0.0  0.0    8232     336  -  I     6Jun18      0:00.16 minicron
    root    71973   0.0  0.0    8232     336  -  I     6Jun18      0:00.01 minicron
    clamav  72012   0.0  0.1   43184    5588  -  S    21Aug18      1:21.41 /usr/loc
    squid   74050   0.0  0.0   38424    4052  -  S    16Aug18      1:29.38 (pinger)
    squid   74312   0.0  0.0   38424    4036  -  S    13Jul18      3:44.20 (pinger)
    squid   75802   0.0  0.0   38424    4052  -  S     4Aug18      2:19.34 (pinger)
    squid   76249   0.0  0.0   38424    4052  -  S     9Aug18      1:59.01 (pinger)
    squid   76325   0.0  0.0   38424    4052  -  S    24Aug18      0:59.75 (pinger)
    root    76378   0.0  0.0   13096    2364  -  I    Wed09        0:01.41 /bin/sh
    squid   77431   0.0  0.0   38424    4036  -  S    11Jul18      3:49.73 (pinger)
    squid   77457   0.0  0.0   38424    4036  -  S    23Jul18      3:05.16 (pinger)
    squid   77742   0.0  0.0   38424    2864  -  S     9Jun18      5:55.06 (pinger)
    squid   78244   0.0  0.0   38424    4052  -  S    Wed09        0:06.88 (pinger)
    squid   78846   0.0  0.0   38424    3460  -  S     3Jul18      4:20.32 (pinger)
    squid   79799   0.0  0.0   38424    4036  -  S    26Jul18      2:52.31 (pinger)
    squid   81403   0.0  0.0   38424    2864  -  S    16Jun18      5:24.01 (pinger)
    squid   82969   0.0  0.0   38424    4036  -  S    28Jul18      2:41.91 (pinger)
    root    83208   0.0  0.0   10480    2020  -  Ss    6Jun18     28:30.85 /usr/sbi
    root    85070   0.0  0.0    6180    1936  -  SN   16:29        0:00.00 sleep 60
    squid   85222   0.0  0.0   38424    4036  -  S    16Jul18      3:29.70 (pinger)
    squid   86055   0.0  0.0   38424    4032  -  S     5Jul18      4:13.07 (pinger)
    squid   86562   0.0  0.0   38424    4052  -  S    Fri00        0:31.24 (pinger)
    root    86917   0.0  0.1   20352    6572  -  Ss   10Jul18     10:49.16 /usr/loc
    squid   88437   0.0  0.0   38424    3460  -  S    25Jun18      4:51.79 (pinger)
    squid   88499   0.0  0.0   38424    4036  -  S     1Aug18      2:25.88 (pinger)
    root    89183   0.0  0.4  287768   32212  -  S    29Aug18      0:20.98 php-fpm:
    squid   89679   0.0  0.0   38424    4052  -  S    11Aug18      1:50.45 (pinger)
    squid   90089   0.0  0.0   38424    4052  -  S    16Aug18      1:29.08 (pinger)
    squid   90192   0.0  0.0   38424    4052  -  S    22Aug18      1:07.29 (pinger)
    unbound 91907   0.0  1.2  142984   96324  -  Ss   10Jul18     30:40.27 /usr/loc
    squid   93517   0.0  0.0   38424    4036  -  S    10Jul18      3:52.45 (pinger)
    squid   93684   0.0  0.0   38424    2864  -  S    13Jun18      5:38.66 (pinger)
    clamav  93921   0.0  0.1   43184    4264  -  S    21Aug18      0:16.28 /usr/loc
    squid   94091   0.0  0.0   38424    2864  -  S    18Jun18      5:18.22 (pinger)
    squid   94285   0.0  0.0   38424    4032  -  S     6Jul18      4:08.19 (pinger)
    squid   95614   0.0  0.0   38424    4036  -  S    31Jul18      2:30.12 (pinger)
    squid   96134   0.0  0.0   38424    4052  -  S    25Aug18      0:55.23 (pinger)
    squid   97773   0.0  0.0   38424    4052  -  S    Sat00        0:27.35 (pinger)
    squid   97972   0.0  0.0   38424    4056  -  S    00:00        0:03.78 (pinger)
    squid   98189   0.0  0.0   38424    4052  -  S    30Aug18      0:35.59 (pinger)
    squid   99196   0.0  0.0   38424    4036  -  S    11Jul18      3:48.85 (pinger)
    squid   99264   0.0  0.0   38424    4032  -  S     7Jul18      4:03.40 (pinger)
    root    18208   0.0  0.0   39440       0 v0  IWs  -            0:00.00 login [p
    root    20140   0.0  0.0   13096       0 v0  IW   -            0:00.00 -sh (sh)
    root    20467   0.0  0.0   13096    1912 v0  I+    6Jun18      0:00.00 /bin/sh
    root    18327   0.0  0.0   10396    1720 v1  Is+   6Jun18      0:00.00 /usr/lib
    root    18601   0.0  0.0   10396    1720 v2  Is+   6Jun18      0:00.00 /usr/lib
    root    18924   0.0  0.0   10396    1720 v3  Is+   6Jun18      0:00.00 /usr/lib
    root    19168   0.0  0.0   10396    1720 v4  Is+   6Jun18      0:00.00 /usr/lib
    root    19480   0.0  0.0   10396    1720 v5  Is+   6Jun18      0:00.00 /usr/lib
    root    19816   0.0  0.0   10396    1720 v6  Is+   6Jun18      0:00.00 /usr/lib
    root    19833   0.0  0.0   10396    1720 v7  Is+   6Jun18      0:00.00 /usr/lib
    root    16887   0.0  0.0   21108    2768  0  R+   16:29        0:00.00 ps -aux
    root    25284   0.0  0.0   13096    2384  0  Ss   16:29        0:00.00 /bin/sh
    root    45372   0.0  0.0   13400    3396  0  S    16:29        0:00.01 /bin/tcs
    

    So, who is the gremlin in my pfSense box which is eating memory and causing the system to use swap??? Help!



  • @kpa Would this ARC cache be the reason my memory usage is high but it's not being reported by pfSense anywhere other than the Dashboard? I'm afraid to change anything without knowing what exactly it's doing or not doing.

    Thanks,
    Raffi



  • I made the change for the zfs arc cache in loader.conf and then rebooted. The memory is back down to normal and no swap usage as expected. Hopefully that will solve it for good. I'll keep an eye on it.