Squid3 + squidGuard-devel travando após período
-
Olá pessoal.
Estou usando o pfSense numa máquina virtual do VirtualBox em cima de um Windows 2003 Server x64 para rotear a internet entre, aproximadamente, 60 usuários de rede e aplicar filtros ao conteúdo que pode ser mostrado para eles. Implementei autenticação NTLM com o Samba e aí uso o nome de usuário autenticado nele para aplicar nas regras do squidGuard. Tudo roda lindo, exceto por uma coisinha : alguns usuários têm reportado lentidão na conexão através do proxy. Minha máquina e algumas outras, que têm seu tráfego de internet roteado por NAT, não apresentam esse comportamento.
Eu tenho tentado isolar o problema e ontem descobri que quando eu reinicio os serviços do squid a lentidão desaparece instantaneamente. Busquei na internet algo sobre isso e encontrei algo relacionado com o parâmetro url_rewirte_children. O default dele é url_rewrite_children 16 startup=8 idle=4 concurrency=0. Imaginando que poderia ser um gargalo eu subi para url_rewrite_children 25 startup=10 idle=5 concurrency=0 dentro do arquivo squidguard_configurator.inc e gastei um belo tempo monitorando o status dos serviços por meio do comando ps axu | grep -i squidguard dentro de uma sessão SSH com o puTTY.
Os resultados, quando tudo está funcionando bem, mostram 10 processos do squidGuard, o que é o comportamento esperado mas, em algum momento entre 100 e 120 minutos, aproximadamente, por alguma razão, o squid inicia mais 5 processos do rewriter do squidGuard e a navegação através do proxy fica lenta. É como se os processos de rewriter do squidGuard travassem ou estivessem sob uma grande carga de trabalho, porém os indicadores de carga da memória e do processador não mostram nada diferente.
Para contornar, eu criei uma tarefa no Cron que roda um squid -k reconfigure de 15 em 15 minutos nos minutos 15, 30 e 45 e uma outra tarefa que reinicia os serviços do squid com o comando /usr/local/etc/rc.d/squid.sh restart de hora em hora, no minuto 0. Funciona, mas não me parece uma solução ideal.
Tem alguma coisa que eu posso checar ou modificar nos arquivos de configuração do squid ou do squidGuard para dar fim nesse problema ? Nos logs não aparece nada…
Minha configuração :
-
pfSense version is 2.2.4-RELEASE (amd64) built on Sat Jul 25 19:57:37 CDT 2015 (FreeBSD 10.1-RELEASE-p15)
-
squid3 3.4 pkg 0.4.7
-
squidGuard-devel pkg 1.5.10
As configurações da máquina virtual são:
-
Processador : Intel(R) Xeon(R) CPU E5-2420 v2 @ 2.20GHz - 4 CPUs: 1 package(s) x 4 core(s)
-
RAM : 4GB
-
Arquitetura : 64 bits
-
Disco : 240GB
-
Carga CPU média : 5%
-
Carga RAM média : 35%
-
Uso SWAP : 0 (de 8192MB)
-
Uso do disco em / (ufs) : 1% de 234GB
-
Uso do disco em /var/run (ufs na RAM) : 6% de 3,4MB
-
Uso do MBUF : 11% (3046/26584)
-
Tamanho da tabela de states : 0% (2004/406000)
-
WAN : 2
-
LAN : 1
Segue resumo do Sytstem Activity
last pid: 36391; load averages: 0.60, 0.75, 0.80 up 4+19:16:24 08:54:09 185 processes: 5 running, 161 sleeping, 19 waiting Mem: 771M Active, 1171M Inact, 497M Wired, 416M Buf, 1504M Free Swap: 8192M Total, 8192M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 155 ki31 0K 64K RUN 2 111.7H 100.00% [idle{idle: cpu2}] 11 root 155 ki31 0K 64K CPU1 1 110.9H 94.97% [idle{idle: cpu1}] 11 root 155 ki31 0K 64K CPU3 3 110.5H 93.99% [idle{idle: cpu3}] 11 root 155 ki31 0K 64K CPU0 0 104.4H 88.96% [idle{idle: cpu0}] 0 root -92 0 0K 208K - 1 58:48 2.98% [kernel{em0 taskq}] 0 root -92 0 0K 208K - 1 56:27 1.95% [kernel{em1 taskq}] 93901 proxy 21 0 252M 121M kqread 0 1:17 0.98% (squid-1) -f /usr/pbi/squid-amd64/local/et 0 root -16 0 0K 208K swapin 3 413.6H 0.00% [kernel{swapper}] 12 root -60 - 0K 304K WAIT 0 89:41 0.00% [intr{swi4: clock}] 30433 clamav 20 0 454M 364M select 1 20:22 0.00% /usr/local/sbin/clamd --config-file=/usr/p 15 root -16 - 0K 16K - 0 18:20 0.00% [rand_harvestq] 5 root -16 - 0K 16K pftm 0 15:38 0.00% [pf purge] 7 root -16 - 0K 16K psleep 0 9:13 0.00% [pagedaemon] 46290 root 20 0 21728K 6128K select 3 5:37 0.00% /usr/local/sbin/openvpn --config /var/etc/ 18918 root 20 0 12456K 2184K select 1 5:15 0.00% /usr/local/sbin/apinger -c /var/etc/apinge 4 root -16 - 0K 32K - 1 3:05 0.00% [cam{doneq0}] 23942 root 20 0 94848K 14832K select 1 2:59 0.00% /usr/local/sbin/winbindd -s /usr/local/etc 0 root -92 0 0K 208K - 3 2:56 0.00% [kernel{em2 taskq}]
Segue resultado do comando ps axu.
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 11 385.9 0.0 0 64 - RL Fri01PM 26342:49.47 [idle] root 0 5.9 0.0 0 208 - DLs Fri01PM 120:36.01 [kernel] proxy 52901 1.0 1.6 196520 65500 - S 9:00AM 0:20.89 (squid-1) -f /usr/pbi/squid-amd64/local/etc/squid/squid.conf (squid) root 1 0.0 0.0 9112 788 - ILs Fri01PM 0:00.08 /sbin/init -- root 2 0.0 0.0 0 16 - DL Fri01PM 0:00.00 [crypto] root 3 0.0 0.0 0 16 - DL Fri01PM 0:00.00 [crypto returns] root 4 0.0 0.0 0 32 - DL Fri01PM 3:06.65 [cam] root 5 0.0 0.0 0 16 - DL Fri01PM 15:40.36 [pf purge] root 6 0.0 0.0 0 16 - DL Fri01PM 0:00.00 [sctp_iterator] root 7 0.0 0.0 0 16 - DL Fri01PM 9:13.83 [pagedaemon] root 8 0.0 0.0 0 16 - DL Fri01PM 0:00.00 [vmdaemon] root 9 0.0 0.0 0 16 - DL Fri01PM 0:19.53 [idlepoll] root 10 0.0 0.0 0 16 - DL Fri01PM 0:00.00 [audit] root 12 0.0 0.0 0 304 - WL Fri01PM 94:36.81 [intr] root 13 0.0 0.0 0 64 - DL Fri01PM 0:00.00 [ng_queue] root 14 0.0 0.0 0 48 - DL Fri01PM 0:00.06 [geom] root 15 0.0 0.0 0 16 - DL Fri01PM 18:22.67 [rand_harvestq] root 16 0.0 0.0 0 64 - DL Fri01PM 0:16.02 [usb] root 17 0.0 0.0 0 16 - DL Fri01PM 0:00.21 [pagezero] root 18 0.0 0.0 0 32 - DL Fri01PM 3:21.33 [bufdaemon] root 19 0.0 0.0 0 16 - DL Fri01PM 0:59.87 [vnlru] root 20 0.0 0.0 0 16 - DL Fri01PM 2:40.55 [syncer] root 61 0.0 0.0 0 16 - DL Fri01PM 0:14.52 [md0] root 243 0.0 0.5 224228 21440 - Ss Fri01PM 2:16.12 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) root 259 0.0 0.1 19024 2584 - INs Fri01PM 0:00.27 /usr/local/sbin/check_reload_status root 261 0.0 0.1 19024 2412 - IN Fri01PM 0:00.00 check_reload_status: Monitoring daemon of check_reload_status root 275 0.0 0.1 13160 4456 - Ss Fri01PM 0:01.46 /sbin/devd -q proxy 2651 0.0 0.1 24080 3428 - S 9:00AM 0:00.22 (pinger) (pinger) root 5622 0.0 0.1 17136 2616 - I 9:00AM 0:00.03 /bin/sh /usr/local/pkg/sqpmon.sh root 6399 0.0 0.1 55628 6136 - Ss 9:17AM 0:00.03 sshd: root@notty (sshd) root 6711 0.0 0.1 25836 5252 - Is 9:17AM 0:00.01 /usr/libexec/sftp-server root 6953 0.0 0.1 55628 6136 - Ss 9:18AM 0:00.05 sshd: root@notty (sshd) root 7490 0.0 0.1 25836 5132 - Is 9:18AM 0:00.01 /usr/libexec/sftp-server root 7558 0.0 0.4 28204 18088 - Ss Mon10AM 0:15.74 /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid proxy 7602 0.0 0.2 80904 9420 - I 9:18AM 0:00.03 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) proxy 7723 0.0 0.2 80904 9420 - I 9:18AM 0:00.03 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) proxy 7747 0.0 0.2 80904 9420 - I 9:18AM 0:00.03 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) proxy 7921 0.0 0.2 80904 9420 - I 9:18AM 0:00.03 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) root 9940 0.0 0.0 8304 1956 - I 9:18AM 0:00.00 sleep 55 root 10049 0.0 0.1 55628 6136 - Ss 9:18AM 0:00.05 sshd: admin@pts/0 (sshd) root 11204 0.0 0.0 12404 1932 - Is Fri01PM 0:00.00 /usr/local/bin/minicron 240 /var/run/ping_hosts.pid /usr/local/bin/ping_hosts.sh root 11274 0.0 0.1 21728 5280 - Ss Fri01PM 1:38.67 /usr/local/sbin/openvpn --config /var/etc/openvpn/client3.conf root 11295 0.0 0.1 17136 2524 - SN Fri01PM 1:21.31 /bin/sh /var/db/rrd/updaterrd.sh root 11953 0.0 0.1 67768 5560 - I 8:29AM 0:02.12 /usr/pbi/squid-amd64/local/bin/c-icap -f /usr/pbi/squid-amd64/local/etc/c-icap/c-icap.conf root 12284 0.0 0.1 21728 5276 - Ss Fri01PM 1:40.07 /usr/local/sbin/openvpn --config /var/etc/openvpn/client4.conf root 12616 0.0 0.0 12404 1944 - I Fri01PM 0:00.63 minicron: helper /usr/local/bin/ping_hosts.sh (minicron) root 13337 0.0 0.0 12404 1932 - Is Fri01PM 0:00.00 /usr/local/bin/minicron 3600 /var/run/expire_accounts.pid /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts root 13358 0.0 0.1 21728 5280 - Ss Fri01PM 1:40.30 /usr/local/sbin/openvpn --config /var/etc/openvpn/client5.conf root 13514 0.0 0.0 12404 1944 - I Fri01PM 0:00.05 minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts (minicron) root 13553 0.0 0.0 12404 1932 - Is Fri01PM 0:00.00 /usr/local/bin/minicron 86400 /var/run/update_alias_url_data.pid /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data root 13640 0.0 0.1 16804 2440 - Ss Fri01PM 2:21.32 /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid root 14033 0.0 0.0 12404 1944 - I Fri01PM 0:00.00 minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data (minicron) root 14437 0.0 0.1 18780 2448 - Is Fri01PM 0:04.92 /usr/sbin/inetd -wW -R 0 -a 127.0.0.1 /var/etc/inetd.conf root 17184 0.0 0.1 32424 5300 - Ss Fri01PM 0:00.02 /usr/sbin/sshd root 17434 0.0 0.1 14748 2224 - Is Fri01PM 0:00.05 /usr/local/sbin/sshlockout_pf 15 root 18918 0.0 0.1 12456 2184 - Ss Fri01PM 5:16.00 /usr/local/sbin/apinger -c /var/etc/apinger.conf root 19064 0.0 0.1 28348 3008 - S Fri01PM 0:03.83 rrdtool - root 21240 0.0 0.2 84144 9172 - Ss Fri01PM 0:27.83 /usr/local/sbin/nmbd -D -s /usr/local/etc/smb.conf root 21938 0.0 0.3 93192 14408 - Is Fri01PM 0:00.18 /usr/local/sbin/smbd -D -s /usr/local/etc/smb.conf root 23697 0.0 0.3 90104 12648 - Ss Fri01PM 1:58.80 /usr/local/sbin/winbindd -s /usr/local/etc/smb.conf root 23942 0.0 0.4 94848 14832 - S Fri01PM 3:01.07 /usr/local/sbin/winbindd -s /usr/local/etc/smb.conf root 24013 0.0 0.4 93192 14628 - S Fri01PM 0:03.17 /usr/local/sbin/smbd -D -s /usr/local/etc/smb.conf root 26187 0.0 0.7 71280 27552 - S Fri01PM 2:45.46 /usr/local/sbin/lighttpd -f /var/etc/lighty-webConfigurator.conf proxy 27644 0.0 1.1 68328 43828 - S 9:15AM 0:00.91 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) root 27650 0.0 0.0 8304 1956 - SN 9:18AM 0:00.00 sleep 60 proxy 27900 0.0 1.0 64232 40804 - S 9:15AM 0:00.23 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 27996 0.0 0.9 60136 37744 - S 9:15AM 0:00.17 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28033 0.0 0.9 60136 37736 - S 9:15AM 0:00.16 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28213 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28229 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28392 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28594 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28646 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28914 0.0 0.8 56040 31800 - I 9:15AM 0:00.15 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 29084 0.0 0.1 24080 3428 - S 9:15AM 0:00.07 (pinger) (pinger) proxy 29294 0.0 0.2 80904 9448 - S 9:15AM 0:00.22 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) clamav 30433 0.0 9.0 465224 372716 - Is Fri01PM 28:06.55 /usr/local/sbin/clamd --config-file=/usr/pbi/squid-amd64/local/etc/clamd.conf root 33697 0.0 0.1 14532 2216 - S Fri01PM 0:37.99 /usr/local/sbin/radvd -p /var/run/radvd.pid -C /var/etc/radvd.conf -m syslog proxy 38365 0.0 0.2 80904 9444 - S 9:15AM 0:00.07 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) proxy 38450 0.0 0.2 80904 9444 - S 9:15AM 0:00.04 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) root 39307 0.0 0.1 67768 5600 - I 8:30AM 0:02.30 /usr/pbi/squid-amd64/local/bin/c-icap -f /usr/pbi/squid-amd64/local/etc/c-icap/c-icap.conf root 40129 0.0 0.1 21728 5688 - Ss Fri02PM 1:39.66 /usr/local/sbin/openvpn --config /var/etc/openvpn/client2.conf root 46290 0.0 0.1 21728 6128 - Ss Fri04PM 5:38.55 /usr/local/sbin/openvpn --config /var/etc/openvpn/server1.conf root 51888 0.0 0.1 16664 2364 - Is Fri01PM 0:02.56 /usr/sbin/cron -s root 52474 0.0 0.3 73640 13664 - Is 9:00AM 0:00.00 /usr/local/sbin/squid -f /usr/pbi/squid-amd64/local/etc/squid/squid.conf proxy 54957 0.0 0.1 26032 3236 - I 9:00AM 0:00.02 (unlinkd) (unlinkd) proxy 55042 0.0 0.1 24080 3428 - S 9:00AM 0:00.22 (pinger) (pinger) proxy 56884 0.0 0.2 80904 9420 - I 9:17AM 0:00.03 (ntlm_auth) --domain=SANTAINES --helper-protocol=squid-2.5-ntlmssp (ntlm_auth) root 63163 0.0 0.1 14664 2432 - Ss 12:24PM 0:13.49 /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /var/etc/syslog.conf root 64378 0.0 0.1 25380 2892 - Ss Fri01PM 1:22.13 /usr/pbi/squid-amd64/local/bin/c-icap -f /usr/pbi/squid-amd64/local/etc/c-icap/c-icap.conf root 66318 0.0 0.1 14748 2316 - Is Fri01PM 0:00.07 /usr/local/sbin/sshlockout_pf 15 root 66493 0.0 0.9 228324 38008 - S 9:17AM 0:00.13 php-fpm: pool lighty (php-fpm) root 66719 0.0 0.1 14748 2316 - Is 1:03PM 0:00.01 /usr/local/sbin/sshlockout_pf 15 root 69282 0.0 0.1 57008 3852 - Is Fri01PM 0:09.35 /usr/local/sbin/filterdns -p /var/run/filterdns.pid -i 300 -c /var/etc/filterdns.conf -d 1 root 81665 0.0 0.1 67768 5616 - I 8:03AM 0:02.51 /usr/pbi/squid-amd64/local/bin/c-icap -f /usr/pbi/squid-amd64/local/etc/c-icap/c-icap.conf nobody 94460 0.0 0.1 30264 4636 - S 1:19PM 0:46.14 /usr/local/sbin/dnsmasq --all-servers --rebind-localhost-ok --stop-dns-rebind --dns-forward-max=5000 --cache-size=10000 --local-ttl=1 root 17936 0.0 0.0 14560 2060 v0 Is+ Fri01PM 0:00.00 /usr/libexec/getty Pc ttyv0 root 27719 0.0 0.1 17136 2620 0 Ss 9:18AM 0:00.01 /bin/sh /etc/rc.initial root 29659 0.0 0.1 17476 3480 0 S 9:18AM 0:00.02 /bin/tcsh
E o resultado do ps axu | grep -i squidguard.
proxy 27644 0.0 1.1 68328 45676 - S 9:15AM 0:01.32 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 27900 0.0 1.0 64232 42376 - S 9:15AM 0:00.33 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 27996 0.0 1.0 64232 41056 - S 9:15AM 0:00.21 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28033 0.0 0.9 64232 38980 - S 9:15AM 0:00.17 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28213 0.0 0.9 60136 37700 - S 9:15AM 0:00.17 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28229 0.0 0.9 60136 37700 - S 9:15AM 0:00.17 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28392 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28594 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28646 0.0 0.8 56040 31800 - I 9:15AM 0:00.14 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) proxy 28914 0.0 0.8 56040 31800 - I 9:15AM 0:00.15 (squidGuard) -c /usr/pbi/squidguard-devel-amd64/etc/squidGuard/squidGuard.conf (squidGuard) ```![pfSense_monitor.png](/public/_imported_attachments_/1/pfSense_monitor.png) ![pfSense_monitor.png_thumb](/public/_imported_attachments_/1/pfSense_monitor.png_thumb)
-
-
Olá!
Nunca usei integração que você está trabalhando, logo, não sei dizer se por este motivo os processos estariam iniciando e acarretando a lentidão.
Tenho 3 senses rodando em proxy transparent e nunca notei esse comportamento, porém, todos com SQUID2+SQUIDGUARD, e todos são máquinas físicas.
Talvez o @marcelloc já tenha trabalhado dessa forma, e ele possa dar uma ajuda.
Abraços e boa sorte!
-
Também estou enfrentando problema parecido…..
-
Segue meu Cron (baixei o pacote no repositório) … isso resolve o problema, como disse, mas não creio ser correto ... Em tempo : o reconfigure roda em menos de 1 segundo e o restart roda em menos de 30 … não sei que tipo de impacto isso pode gerar no meu usuário (caso esteja fazendo um download grande, por exemplo, algo bem comum por aqui) ...