PFSense High Swap Usage

mimin28g

Hi Guys,

I recently Upgrade our C2758 1U pfSense Security Gateway Appliance memory to 32 GB (8x4)
and also added Four Port 1 GigaBit Intel Ethernet Adapter RJ45 (https://store.pfsense.org/AOC-SGP-I4/)
also upgrade the software from 2.2 to 2.3.2 …. and enable services like snort and squid
I can see that the system is using the swap memory even though there are inactive memory ....

last pid: 30231; load averages: 1.13, 1.21, 1.29 up 2+02:00:06 17:00:37
104 processes: 2 running, 102 sleeping
CPU: 3.3% user, 2.8% nice, 1.6% system, 3.1% interrupt, 89.2% idle
Mem: 14G Active, 14G Inact, 2519M Wired, 137M Cache, 1658M Buf, 219M Free
Swap: 64G Total, 1344K Used, 64G Free

Can you help .... why the system is using the Swap memory when there are 14GB of inactive memory...
And for the Squid i gave 20GB for memory caching
And for the system and snort 11GB ...

mimin28g

last pid: 42534; load averages: 1.24, 1.27, 1.25 up 2+03:11:57 18:12:28
104 processes: 1 running, 103 sleeping
CPU: 0.4% user, 2.2% nice, 1.0% system, 3.4% interrupt, 93.1% idle
Mem: 12G Active, 16G Inact, 2760M Wired, 537M Cache, 1664M Buf, 106M Free
Swap: 64G Total, 2822M Used, 61G Free, 4% Inuse

mimin28g

Hi Guys …. Please help...

mimin28g

last pid: 41931; load averages: 1.47, 1.19, 1.11 up 3+02:59:34 18:00:05
111 processes: 3 running, 108 sleeping
CPU: 4.7% user, 12.6% nice, 18.6% system, 4.8% interrupt, 59.4% idle
Mem: 5746M Active, 23G Inact, 2408M Wired, 563M Cache, 1655M Buf, 53M Free
Swap: 64G Total, 20G Used, 44G Free, 31% Inuse, 348K In

Please help …

jimp

More than likely your snort and squid settings are causing them to consume massive amounts of memory. There is not enough information in what you have shown to speculate about a cause with any accuracy. Post a full "ps uxawwd" output for starters.

mimin28g

Squid was suppose to write to disk once it utilize 20GB of the memory …. :-(

mimin28g

This is the details i get … Please let me if you need any more info ...

ps uxawwd
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 0 0.0 0.0 0 1184 - DLs Sat03PM 0:20.44 [kernel]
root 11 791.4 0.0 0 128 - RL Sat03PM 40317:33.65 - [idle]
root 12 0.4 0.0 0 1584 - WL Sat03PM 708:43.67 - [intr]
root 20 0.1 0.0 0 16 - DL Sat03PM 6:51.75 - [syncer]
root 1 0.0 0.0 9136 132 - ILs Sat03PM 0:00.08 - /sbin/init –
unbound 9935 18.4 0.2 92792 75292 - Ss 11:25AM 1:02.80 |-- /usr/local/sbin/unbound -c /var/unbound/unbound.conf
root 72283 0.8 0.4 516628 124872 - SNs 3:22AM 1:23.97 |-- /usr/local/bin/snort -R 50701 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan1650701 --pid-path /var/run --nolock-pidfile -G 50701 -c /usr/local/etc/snort/snort_50701_lagg0_vlan16/snort.conf -i lagg0_vlan16
root 45305 0.6 0.0 16676 2284 - Ss Sat03PM 40:28.87 |-- /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid
root 73638 0.6 0.4 516628 127644 - SNs 3:22AM 1:45.16 |-- /usr/local/bin/snort -R 37289 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan1937289 --pid-path /var/run --nolock-pidfile -G 37289 -c /usr/local/etc/snort/snort_37289_lagg0_vlan19/snort.conf -i lagg0_vlan19
root 82694 0.3 0.3 516628 98032 - SNs 3:22AM 0:27.04 |-- /usr/local/bin/snort -R 44081 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan4344081 --pid-path /var/run --nolock-pidfile -G 44081 -c /usr/local/etc/snort/snort_44081_lagg0_vlan43/snort.conf -i lagg0_vlan43
root 67357 0.1 0.4 516628 120168 - SNs 3:22AM 0:55.87 |-- /usr/local/bin/snort -R 37939 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan337939 --pid-path /var/run --nolock-pidfile -G 37939 -c /usr/local/etc/snort/snort_37939_lagg0_vlan3/snort.conf -i lagg0_vlan3
root 68118 0.1 0.3 516628 115368 - SNs 3:22AM 1:05.24 |-- /usr/local/bin/snort -R 49186 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan549186 --pid-path /var/run --nolock-pidfile -G 49186 -c /usr/local/etc/snort/snort_49186_lagg0_vlan5/snort.conf -i lagg0_vlan5
root 81363 0.1 0.3 516628 91680 - SNs 3:22AM 0:04.00 |-- /usr/local/bin/snort -R 6198 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan406198 --pid-path /var/run --nolock-pidfile -G 6198 -c /usr/local/etc/snort/snort_6198_lagg0_vlan40/snort.conf -i lagg0_vlan40
root 84875 0.1 0.4 516628 131728 - SNs 3:22AM 1:10.39 |-- /usr/local/bin/snort -R 53004 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan5053004 --pid-path /var/run --nolock-pidfile -G 53004 -c /usr/local/etc/snort/snort_53004_lagg0_vlan50/snort.conf -i lagg0_vlan50
root 263 0.0 0.0 268344 15616 - Ss Sat03PM 0:37.60 |-- php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm)
root 99856 7.1 0.1 297016 49160 - S 11:56AM 0:00.97 | |-- php-fpm: pool nginx (php-fpm)
root 2372 0.0 0.0 268344 15704 - S 11:56AM 0:00.00 | -- php-fpm: pool nginx (php-fpm) root 286 0.0 0.0 18888 1052 - INs Sat03PM 0:00.06 |-- /usr/local/sbin/check_reload_status root 288 0.0 0.0 18888 0 - IWN - 0:00.00 | -- check_reload_status: Monitoring daemon of check_reload_status
root 301 0.0 0.0 13624 1340 - Is Sat03PM 0:00.16 |-- /sbin/devd -q
root 3824 0.0 0.0 15012 2280 - Is 11:25AM 0:00.31 |-- /usr/local/bin/dpinger -S -r 0 -i GW -B x.x.x.x -p /var/run/dpinger_GW~x.x.x.x~y.y.y.y.pid -u /var/run/dpinger_GW~x.x.x.x~y.y.y.y.sock -C /etc/rc.gateway_alarm -d 0 -s 500 -l 2000 -t 60000 -A 1000 -D 500 -L 20 y.y.y.y
root 4211 0.0 0.0 15012 2284 - Is 11:25AM 0:00.32 |-- /usr/local/bin/dpinger -S -r 0 -i GW -B x.x.x.x4 -p /var/run/dpinger_GW~x.x.x.x~y.y.y.y.pid -u /var/run/dpinger_GW~x.x.x.x~y.y.y.y.sock -C /etc/rc.gateway_alarm -d 0 -s 500 -l 2000 -t 60000 -A 1000 -D 500 -L 20 y.y.y.y
root 10978 0.0 0.0 12272 2024 - Ss 11:25AM 0:02.41 |-- /usr/local/sbin/dhcpleases -l /var/dhcpd/var/db/dhcpd.leases -d ghy -p /var/run/unbound.pid -u /var/unbound/dhcpleases_entries.conf -h /var/etc/hosts
root 11128 0.0 0.0 17000 2308 - S 11:26AM 0:00.03 |-- /bin/sh /usr/local/pkg/sqpmon.sh
root 2150 0.0 0.0 8168 1828 - S 11:56AM 0:00.00 | -- sleep 55 root 25756 0.0 0.0 14408 1640 - Ss Sat03PM 0:04.63 |-- /usr/sbin/powerd -b max -a max -n max root 45566 0.0 0.0 12268 0 - IWs - 0:00.00 |-- /usr/local/bin/minicron 240 /var/run/ping_hosts.pid /usr/local/bin/ping_hosts.sh root 45855 0.0 0.0 12268 332 - I Sat03PM 0:00.13 | -- minicron: helper /usr/local/bin/ping_hosts.sh (minicron)
root 45893 0.0 0.1 70916 25544 - Ss 11:25AM 0:04.38 |-- /usr/sbin/bsnmpd -c /var/etc/snmpd.conf -p /var/run/snmpd.pid
root 46003 0.0 0.0 59956 13216 - Is 6:03PM 0:00.00 |-- /usr/local/sbin/squid -f /usr/local/etc/squid/squid.conf
squid 46331 1.9 34.7 12276128 11625780 - S 6:03PM 35:45.76 | -- (squid-1) -f /usr/local/etc/squid/squid.conf (squid) squid 12266 0.0 0.0 37752 3884 - S 11:26AM 0:01.53 | |-- (pinger) (pinger) squid 44290 0.0 0.0 37752 3848 - S 12:00AM 0:11.63 | |-- (pinger) (pinger) squid 46617 0.0 0.0 37616 3636 - S 6:03PM 0:04.71 | |-- (unlinkd) (unlinkd) squid 46741 0.0 0.0 37752 3844 - S 6:03PM 0:15.81 | |-- (pinger) (pinger) squid 97653 0.0 0.0 37752 3884 - S 11:25AM 0:01.48 | -- (pinger) (pinger)
root 46200 0.0 0.0 12268 0 - IWs - 0:00.00 |-- /usr/local/bin/minicron 3600 /var/run/expire_accounts.pid /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts
root 46671 0.0 0.0 12268 332 - I Sat03PM 0:00.01 | -- minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts (minicron) root 46508 0.0 0.0 18896 2120 - Is Sat03PM 0:00.02 |-- /usr/local/sbin/xinetd -syslog daemon -f /var/etc/xinetd.conf -pidfile /var/run/xinetd.pid root 46840 0.0 0.0 12268 0 - IWs - 0:00.00 |-- /usr/local/bin/minicron 86400 /var/run/update_alias_url_data.pid /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data root 46986 0.0 0.0 12268 0 - IW - 0:00.00 | -- minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data (minicron)
root 55175 0.0 0.0 39136 0 - IWs - 0:00.00 |-- nginx: master process /usr/local/sbin/nginx -c /var/etc/nginx-webConfigurator.conf (nginx)
root 55404 0.0 0.0 39136 3840 - S Sat03PM 0:25.58 | |-- nginx: worker process (nginx)
root 55663 0.0 0.0 39136 3520 - S Sat03PM 0:32.23 | -- nginx: worker process (nginx) root 56111 0.0 0.0 16532 1128 - Is Sat03PM 0:02.02 |-- /usr/sbin/cron -s dhcpd 58100 0.0 0.0 28908 16232 - Ss 11:25AM 0:01.29 |-- /usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid lagg0_vlan3 lagg0_vlan4 lagg0_vlan5 lagg0_vlan6 lagg0_vlan7 lagg0_vlan8 lagg0_vlan9 lagg0_vlan10 lagg0_vlan11 lagg0_vlan12 lagg0_vlan13 lagg0_vlan14 lagg0_vlan15 lagg0_vlan16 lagg0_vlan17 lagg0_vlan18 lagg0_vlan19 lagg0_vlan20 lagg0_vlan21 lagg0_vlan22 lagg0_vlan23 lagg0_vlan24 lagg0_vlan25 lagg0_vlan26 lagg0_vlan27 lagg0_vlan28 lagg0_vlan29 lagg0_vlan30 lagg0_vlan31 lagg0_vlan32 lagg0_vlan33 lagg0_vlan34 lagg0_vlan35 lagg0_vlan36 lagg0_vlan37 lagg0_vlan38 lagg0_vlan39 lagg0_vlan40 lagg0_vlan41 lagg0_vlan42 lagg0_vlan43 lagg0_vlan44 lagg0_vlan45 lagg0_vlan46 lagg0_vlan47 lagg0_vlan48 lagg0_vlan49 lagg0_vlan50 lagg0_vlan51 lagg0_vlan52 lagg0_vlan53 lagg0_vlan54 lagg0_vlan55 lagg0_vlan56 lagg0_vlan57 lagg0_vlan58 lagg0_vlan59 lagg0_vlan61 lagg0_vlan62 root 59124 0.0 0.0 17000 2740 - IN 11:25AM 0:02.61 |-- /bin/sh /var/db/rrd/updaterrd.sh root 96940 0.0 0.0 8168 1828 - IN 11:55AM 0:00.00 | -- sleep 60
root 60591 0.0 0.1 30140 17968 - Ss Sat03PM 1:11.86 |-- /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid
root 67882 0.0 0.3 516628 104044 - SNs 3:22AM 0:38.52 |-- /usr/local/bin/snort -R 19140 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan419140 --pid-path /var/run --nolock-pidfile -G 19140 -c /usr/local/etc/snort/snort_19140_lagg0_vlan4/snort.conf -i lagg0_vlan4
root 68723 0.0 0.3 516628 101496 - SNs 3:22AM 0:15.77 |-- /usr/local/bin/snort -R 13957 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan613957 --pid-path /var/run --nolock-pidfile -G 13957 -c /usr/local/etc/snort/snort_13957_lagg0_vlan6/snort.conf -i lagg0_vlan6
root 68976 0.0 0.3 516632 113396 - SNs 3:22AM 0:32.51 |-- /usr/local/bin/snort -R 12591 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan712591 --pid-path /var/run --nolock-pidfile -G 12591 -c /usr/local/etc/snort/snort_12591_lagg0_vlan7/snort.conf -i lagg0_vlan7
root 69219 0.0 0.3 516628 113324 - SNs 3:22AM 1:30.89 |-- /usr/local/bin/snort -R 595 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan8595 --pid-path /var/run --nolock-pidfile -G 595 -c /usr/local/etc/snort/snort_595_lagg0_vlan8/snort.conf -i lagg0_vlan8
root 69522 0.0 0.3 516628 111404 - SNs 3:22AM 0:26.83 |-- /usr/local/bin/snort -R 49963 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan949963 --pid-path /var/run --nolock-pidfile -G 49963 -c /usr/local/etc/snort/snort_49963_lagg0_vlan9/snort.conf -i lagg0_vlan9
root 69839 0.0 0.4 520724 131892 - SNs 3:22AM 1:36.96 |-- /usr/local/bin/snort -R 21940 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan1021940 --pid-path /var/run --nolock-pidfile -G 21940 -c /usr/local/etc/snort/snort_21940_lagg0_vlan10/snort.conf -i lagg0_vlan10
root 70438 0.0 0.3 516628 102980 - SNs 3:22AM 0:34.91 |-- /usr/local/bin/snort -R 761 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan11761 --pid-path /var/run --nolock-pidfile -G 761 -c /usr/local/etc/snort/snort_761_lagg0_vlan11/snort.conf -i lagg0_vlan11
root 70562 0.0 0.3 516628 101980 - SNs 3:22AM 0:30.12 |-- /usr/local/bin/snort -R 20985 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan1220985 --pid-path /var/run --nolock-pidfile -G 20985 -c /usr/local/etc/snort/snort_20985_lagg0_vlan12/snort.conf -i lagg0_vlan12
root 70931 0.0 0.3 516628 116560 - SNs 3:22AM 0:46.21 |-- /usr/local/bin/snort -R 52515 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan1352515 --pid-path /var/run --nolock-pidfile -G 52515 -c /usr/local/etc/snort/snort_52515_lagg0_vlan13/snort.conf -i lagg0_vlan13
root 71192 0.0 0.3 516628 95816 - SNs 3:22AM 0:07.62 |-- /usr/local/bin/snort -R 8530 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan148530 --pid-path /var/run --nolock-pidfile -G 8530 -c /usr/local/etc/snort/snort_8530_lagg0_vlan14/snort.conf -i lagg0_vlan14
root 71828 0.0 0.3 516628 117264 - SNs 3:22AM 0:43.70 |-- /usr/local/bin/snort -R 11082 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan1511082 --pid-path /var/run --nolock-pidfile -G 11082 -c /usr/local/etc/snort/snort_11082_lagg0_vlan15/snort.conf -i lagg0_vlan15
root 72635 0.0 0.4 516628 121260 - SNs 3:22AM 0:54.63 |-- /usr/local/bin/snort -R 9914 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan179914 --pid-path /var/run --nolock-pidfile -G 9914 -c /usr/local/etc/snort/snort_9914_lagg0_vlan17/snort.conf -i lagg0_vlan17
root 73182 0.0 0.3 516628 101068 - SNs 3:22AM 0:22.71 |-- /usr/local/bin/snort -R 3387 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan183387 --pid-path /var/run --nolock-pidfile -G 3387 -c /usr/local/etc/snort/snort_3387_lagg0_vlan18/snort.conf -i lagg0_vlan18
root 74113 0.0 0.4 516628 143048 - SNs 3:22AM 2:14.44 |-- /usr/local/bin/snort -R 54039 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2054039 --pid-path /var/run --nolock-pidfile -G 54039 -c /usr/local/etc/snort/snort_54039_lagg0_vlan20/snort.conf -i lagg0_vlan20
root 74324 0.0 0.3 516628 105104 - SNs 3:22AM 0:34.13 |-- /usr/local/bin/snort -R 8076 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan218076 --pid-path /var/run --nolock-pidfile -G 8076 -c /usr/local/etc/snort/snort_8076_lagg0_vlan21/snort.conf -i lagg0_vlan21
root 74743 0.0 0.3 516628 87792 - SNs 3:22AM 0:02.50 |-- /usr/local/bin/snort -R 7135 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan227135 --pid-path /var/run --nolock-pidfile -G 7135 -c /usr/local/etc/snort/snort_7135_lagg0_vlan22/snort.conf -i lagg0_vlan22
root 75105 0.0 0.3 516628 87792 - SNs 3:22AM 0:02.56 |-- /usr/local/bin/snort -R 36065 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2336065 --pid-path /var/run --nolock-pidfile -G 36065 -c /usr/local/etc/snort/snort_36065_lagg0_vlan23/snort.conf -i lagg0_vlan23
root 75596 0.0 0.3 516628 90804 - SNs 3:22AM 0:05.35 |-- /usr/local/bin/snort -R 31804 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2431804 --pid-path /var/run --nolock-pidfile -G 31804 -c /usr/local/etc/snort/snort_31804_lagg0_vlan24/snort.conf -i lagg0_vlan24
root 76138 0.0 0.3 516628 93692 - SNs 3:22AM 0:33.99 |-- /usr/local/bin/snort -R 23136 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2523136 --pid-path /var/run --nolock-pidfile -G 23136 -c /usr/local/etc/snort/snort_23136_lagg0_vlan25/snort.conf -i lagg0_vlan25
root 76326 0.0 0.3 516628 89768 - SNs 3:22AM 0:02.93 |-- /usr/local/bin/snort -R 38940 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2638940 --pid-path /var/run --nolock-pidfile -G 38940 -c /usr/local/etc/snort/snort_38940_lagg0_vlan26/snort.conf -i lagg0_vlan26
root 76626 0.0 0.3 516628 91028 - SNs 3:22AM 0:09.69 |-- /usr/local/bin/snort -R 26648 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2726648 --pid-path /var/run --nolock-pidfile -G 26648 -c /usr/local/etc/snort/snort_26648_lagg0_vlan27/snort.conf -i lagg0_vlan27
root 76950 0.0 0.3 516628 87740 - SNs 3:22AM 0:02.52 |-- /usr/local/bin/snort -R 55675 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan2855675 --pid-path /var/run --nolock-pidfile -G 55675 -c /usr/local/etc/snort/snort_55675_lagg0_vlan28/snort.conf -i lagg0_vlan28
root 77255 0.0 0.3 516628 89692 - SNs 3:22AM 0:02.81 |-- /usr/local/bin/snort -R 4271 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan294271 --pid-path /var/run --nolock-pidfile -G 4271 -c /usr/local/etc/snort/snort_4271_lagg0_vlan29/snort.conf -i lagg0_vlan29
root 77601 0.0 0.3 516628 89752 - SNs 3:22AM 0:02.76 |-- /usr/local/bin/snort -R 60552 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3060552 --pid-path /var/run --nolock-pidfile -G 60552 -c /usr/local/etc/snort/snort_60552_lagg0_vlan30/snort.conf -i lagg0_vlan30
root 77943 0.0 0.3 516628 91828 - SNs 3:22AM 0:11.10 |-- /usr/local/bin/snort -R 8856 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan318856 --pid-path /var/run --nolock-pidfile -G 8856 -c /usr/local/etc/snort/snort_8856_lagg0_vlan31/snort.conf -i lagg0_vlan31
root 78235 0.0 0.0 59068 4212 - Is Sat03PM 0:00.08 |-- /usr/sbin/sshd
root 10373 0.0 0.0 63736 6576 - Ss 11:49AM 0:00.14 | -- sshd: admin@pts/0 (sshd) root 27071 0.0 0.0 17000 2320 0 Is 11:50AM 0:00.01 | -- /bin/sh /etc/rc.initial
root 95956 0.0 0.0 17340 3840 0 S 11:50AM 0:00.03 | -- /bin/tcsh root 2377 0.0 0.0 18676 2324 0 R+ 11:56AM 0:00.00 | -- ps uxawwd
root 78271 0.0 0.3 516628 92716 - SNs 3:22AM 0:24.06 |-- /usr/local/bin/snort -R 37766 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3237766 --pid-path /var/run --nolock-pidfile -G 37766 -c /usr/local/etc/snort/snort_37766_lagg0_vlan32/snort.conf -i lagg0_vlan32
root 78320 0.0 0.3 516628 87764 - SNs 3:22AM 0:02.52 |-- /usr/local/bin/snort -R 12264 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3312264 --pid-path /var/run --nolock-pidfile -G 12264 -c /usr/local/etc/snort/snort_12264_lagg0_vlan33/snort.conf -i lagg0_vlan33
root 78689 0.0 0.3 516628 87764 - SNs 3:22AM 0:02.51 |-- /usr/local/bin/snort -R 46137 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3446137 --pid-path /var/run --nolock-pidfile -G 46137 -c /usr/local/etc/snort/snort_46137_lagg0_vlan34/snort.conf -i lagg0_vlan34
root 79243 0.0 0.3 516628 98324 - SNs 3:22AM 0:09.71 |-- /usr/local/bin/snort -R 62028 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3562028 --pid-path /var/run --nolock-pidfile -G 62028 -c /usr/local/etc/snort/snort_62028_lagg0_vlan35/snort.conf -i lagg0_vlan35
root 79659 0.0 0.3 516632 93736 - SNs 3:22AM 0:22.38 |-- /usr/local/bin/snort -R 64105 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3664105 --pid-path /var/run --nolock-pidfile -G 64105 -c /usr/local/etc/snort/snort_64105_lagg0_vlan36/snort.conf -i lagg0_vlan36
root 79715 0.0 0.3 516628 93276 - SNs 3:22AM 0:04.45 |-- /usr/local/bin/snort -R 26223 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3726223 --pid-path /var/run --nolock-pidfile -G 26223 -c /usr/local/etc/snort/snort_26223_lagg0_vlan37/snort.conf -i lagg0_vlan37
root 80399 0.0 0.3 516628 93896 - SNs 3:22AM 0:05.55 |-- /usr/local/bin/snort -R 4000 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan384000 --pid-path /var/run --nolock-pidfile -G 4000 -c /usr/local/etc/snort/snort_4000_lagg0_vlan38/snort.conf -i lagg0_vlan38
root 80790 0.0 0.3 516628 92296 - SNs 3:22AM 0:02.89 |-- /usr/local/bin/snort -R 40403 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan3940403 --pid-path /var/run --nolock-pidfile -G 40403 -c /usr/local/etc/snort/snort_40403_lagg0_vlan39/snort.conf -i lagg0_vlan39
root 81762 0.0 0.3 516628 93216 - SNs 3:22AM 0:04.19 |-- /usr/local/bin/snort -R 52487 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan4152487 --pid-path /var/run --nolock-pidfile -G 52487 -c /usr/local/etc/snort/snort_52487_lagg0_vlan41/snort.conf -i lagg0_vlan41
root 82176 0.0 0.3 516628 98360 - SNs 3:22AM 0:08.39 |-- /usr/local/bin/snort -R 9602 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan429602 --pid-path /var/run --nolock-pidfile -G 9602 -c /usr/local/etc/snort/snort_9602_lagg0_vlan42/snort.conf -i lagg0_vlan42
root 83145 0.0 0.3 516628 87764 - SNs 3:22AM 0:02.52 |-- /usr/local/bin/snort -R 41059 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan4441059 --pid-path /var/run --nolock-pidfile -G 41059 -c /usr/local/etc/snort/snort_41059_lagg0_vlan44/snort.conf -i lagg0_vlan44
root 83372 0.0 0.3 516628 87764 - SNs 3:22AM 0:02.58 |-- /usr/local/bin/snort -R 5010 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan455010 --pid-path /var/run --nolock-pidfile -G 5010 -c /usr/local/etc/snort/snort_5010_lagg0_vlan45/snort.conf -i lagg0_vlan45
root 83599 0.0 0.3 516628 99924 - SNs 3:22AM 0:23.61 |-- /usr/local/bin/snort -R 15555 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan4615555 --pid-path /var/run --nolock-pidfile -G 15555 -c /usr/local/etc/snort/snort_15555_lagg0_vlan46/snort.conf -i lagg0_vlan46
root 84089 0.0 0.3 516628 111416 - SNs 3:22AM 0:28.83 |-- /usr/local/bin/snort -R 52486 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan4752486 --pid-path /var/run --nolock-pidfile -G 52486 -c /usr/local/etc/snort/snort_52486_lagg0_vlan47/snort.conf -i lagg0_vlan47
root 84419 0.0 0.3 516628 106716 - SNs 3:22AM 0:27.37 |-- /usr/local/bin/snort -R 23887 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan4823887 --pid-path /var/run --nolock-pidfile -G 23887 -c /usr/local/etc/snort/snort_23887_lagg0_vlan48/snort.conf -i lagg0_vlan48
root 84506 0.0 0.3 516628 113608 - SNs 3:22AM 0:35.98 |-- /usr/local/bin/snort -R 1602 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan491602 --pid-path /var/run --nolock-pidfile -G 1602 -c /usr/local/etc/snort/snort_1602_lagg0_vlan49/snort.conf -i lagg0_vlan49
root 85175 0.0 0.3 516628 95288 - SNs 3:22AM 0:04.73 |-- /usr/local/bin/snort -R 7575 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan517575 --pid-path /var/run --nolock-pidfile -G 7575 -c /usr/local/etc/snort/snort_7575_lagg0_vlan51/snort.conf -i lagg0_vlan51
root 85497 0.0 0.3 516628 96352 - SNs 3:22AM 0:05.53 |-- /usr/local/bin/snort -R 9257 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan529257 --pid-path /var/run --nolock-pidfile -G 9257 -c /usr/local/etc/snort/snort_9257_lagg0_vlan52/snort.conf -i lagg0_vlan52
root 85712 0.0 0.3 516628 95448 - SNs 3:22AM 0:12.20 |-- /usr/local/bin/snort -R 4404 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan534404 --pid-path /var/run --nolock-pidfile -G 4404 -c /usr/local/etc/snort/snort_4404_lagg0_vlan53/snort.conf -i lagg0_vlan53
root 86283 0.0 0.3 516628 97544 - SNs 3:22AM 0:11.00 |-- /usr/local/bin/snort -R 4232 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan544232 --pid-path /var/run --nolock-pidfile -G 4232 -c /usr/local/etc/snort/snort_4232_lagg0_vlan54/snort.conf -i lagg0_vlan54
root 86665 0.0 0.3 516628 97912 - SNs 3:22AM 0:08.75 |-- /usr/local/bin/snort -R 9270 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan559270 --pid-path /var/run --nolock-pidfile -G 9270 -c /usr/local/etc/snort/snort_9270_lagg0_vlan55/snort.conf -i lagg0_vlan55
root 87221 0.0 0.3 516628 91924 - SNs 3:22AM 0:02.86 |-- /usr/local/bin/snort -R 50034 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan5650034 --pid-path /var/run --nolock-pidfile -G 50034 -c /usr/local/etc/snort/snort_50034_lagg0_vlan56/snort.conf -i lagg0_vlan56
root 87374 0.0 0.3 516632 98040 - SNs 3:22AM 3:08.31 |-- /usr/local/bin/snort -R 18564 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan5718564 --pid-path /var/run --nolock-pidfile -G 18564 -c /usr/local/etc/snort/snort_18564_lagg0_vlan57/snort.conf -i lagg0_vlan57
root 87684 0.0 0.3 516628 87792 - SNs 3:22AM 0:02.52 |-- /usr/local/bin/snort -R 15750 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan5815750 --pid-path /var/run --nolock-pidfile -G 15750 -c /usr/local/etc/snort/snort_15750_lagg0_vlan58/snort.conf -i lagg0_vlan58
root 87814 0.0 0.3 516628 101324 - SNs 3:22AM 0:39.51 |-- /usr/local/bin/snort -R 45401 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan5945401 --pid-path /var/run --nolock-pidfile -G 45401 -c /usr/local/etc/snort/snort_45401_lagg0_vlan59/snort.conf -i lagg0_vlan59
root 88059 0.0 0.7 545300 246880 - SNs 3:22AM 7:00.46 |-- /usr/local/bin/snort -R 46867 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan6146867 --pid-path /var/run --nolock-pidfile -G 46867 -c /usr/local/etc/snort/snort_46867_lagg0_vlan61/snort.conf -i lagg0_vlan61
root 88303 0.0 0.3 516628 111404 - SNs 3:22AM 8:38.26 |-- /usr/local/bin/snort -R 34342 -D -q --suppress-config-log -l /var/log/snort/snort_lagg0_vlan6234342 --pid-path /var/run --nolock-pidfile -G 34342 -c /usr/local/etc/snort/snort_34342_lagg0_vlan62/snort.conf -i lagg0_vlan62
root 90557 0.0 0.0 14436 1776 - Ss Sat03PM 14:29.63 |-- /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /var/etc/syslog.conf -b 1.1.2.1
root 65280 0.0 0.0 14428 1596 u0 Is+ Sat03PM 0:00.00 |-- /usr/libexec/getty std.38400 ttyu0
root 53400 0.0 0.0 14428 1596 v0 Is+ Mon11AM 0:00.00 `-- /usr/libexec/getty Pc ttyv0
root 2 0.0 0.0 0 16 - DL Sat03PM 0:00.00 - [crypto]
root 3 0.0 0.0 0 16 - DL Sat03PM 0:00.00 - [crypto returns]
root 4 0.0 0.0 0 48 - DL Sat03PM 0:00.00 - [cam]
root 5 0.0 0.0 0 16 - DL Sat03PM 3:26.63 - [pf purge]
root 6 0.0 0.0 0 16 - DL Sat03PM 0:00.00 - [sctp_iterator]
root 7 0.0 0.0 0 32 - DL Sat03PM 15:17.98 - [pagedaemon]
root 8 0.0 0.0 0 16 - DL Sat03PM 0:04.92 - [vmdaemon]
root 9 0.0 0.0 0 16 - DL Sat03PM 0:00.45 - [idlepoll]
root 10 0.0 0.0 0 16 - DL Sat03PM 0:00.00 - [audit]
root 13 0.0 0.0 0 128 - DL Sat03PM 0:00.00 - [ng_queue]
root 14 0.0 0.0 0 48 - DL Sat03PM 0:02.36 - [geom]
root 15 0.0 0.0 0 16 - DL Sat03PM 8:44.28 - [rand_harvestq]
root 16 0.0 0.0 0 160 - DL Sat03PM 0:20.78 - [usb]
root 17 0.0 0.0 0 16 - DL Sat03PM 0:00.01 - [pagezero]
root 18 0.0 0.0 0 32 - DL Sat03PM 0:18.44 - [bufdaemon]
root 19 0.0 0.0 0 16 - DL Sat03PM 0:03.64 - [vnlru]
root 51 0.0 0.0 0 16 - DL Sat03PM 0:00.77 - [md0]

pff.txt

jimp

Looking at VSZ, squid on its own is taking 12G, between all of your snort instances they're taking up 30G. That's not counting anything else running, either.
Looking at RSS, squid is using about the same, so 12G and snort adds up to just over 6G

All of that seems to agree with what you're seeing in terms of RAM+Swap usage in top.

Processes will claim access to more than they need, it doesn't mean they won't need it, it means it may not be active right that moment. Some things will be swapped to keep others in memory or for caching use by the OS.

Basically, you've still over-committed yourself memory-wise with all of those snort instances (59 of them!) and squid with your current settings. You're lucky it's not swapping more than it already is.

mimin28g

i actually restart Squid every two days ….
And previously on PF 2.2 the ran the same config with 16GB memory (excluding the https://store.pfsense.org/AOC-SGP-I4/ hardware) and not even once the swap momory is in use ...
When the allocated RAM 10 GB is full it write to disk and freeup the memory without me having to restart it ....
now we are on 32 GB memory with Squid 20 GB allocated ...

Let me reduce the allocated memory and may be give 16 GB for Snort and PF and 16 GB for Squid ...

Will post the result ....

And thank you so much .... for helping .... :-)

mimin28g

Reduce the allocated memory for Squid to 10GB … will wait and see ...
One thing i notice is that even when the swap memory usage is high...
On the pfsense dashboard ... i see the memory is only about 40% ....

mimin28g

after reducing the squid memory to 10GB … no more swap memory usage ....

Thanks you guys .....