Consumo de cpu

[will23b]

saludos

Tengo una duda, actualmente tengo un equipo con el siguiente hardware :

Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz y memoria de 4 Gb

hoy me di cuenta que el cpu se me eleva entre 54 y 96 %, en la consola del pfsense le hagoun top y veo q el squid me consume un 100%, pero antes no pasaba eso, alguien me podria dar una mano de que podria ser…..Al qeuipo se conectan casi unos 600 usuarios.
los servicios que tengo corriendo son el dhcp,squid y squidGuard

[periko]

Que version de squid tienes?
  Que version de pfsense?
  Cual es tu configuracion de squid?
  Squid es el servicio mas pesado?

Muestra el resultado de:

mount

Cuando esos pasa, como se esta comportando tu disco, no esta tambien en cuello de botella?

Cuando estes viendo que squid se consume tu CPU, ejecuta:

iostat -c 100

Muestra el resultado.

Tienes reportes del cache?
  Saca las ultimas 300 lineas de access.log y cache.log para verlos aqui cuando se comporte feo pfsense.

Saludos.

[will23b]

saludos Periko

version de squid 2.7.9 pkg v.4.3.3
                squidGuard 1.4_4 pkg v.1.9.5
version de pfsense 2.0.2-RELEASE (i386)

configuracion del squid
Hard disk cache size 1048 Mb
Memory cache size 256
Maximum object size 4
modo transparente

solo esta corriendo dhcp, squid y el squidGuard

mount arroja esto

/dev/ad4s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/md0 on /var/run (ufs, local)
devfs on /var/dhcpd/dev (devfs, local)

lo de iostat -c 100
  tty            ad4              md0            cpu
tin  tout  KB/t tps  MB/s  KB/t tps  MB/s  us ni sy in id
  0  610 99.53  53  5.18  14.46  0  0.00  31  0 19  3 47
  0  183 128.00  42  5.25  0.00  0  0.00  35  0 15  3 47
  0    62 128.00  41  5.12  0.00  0  0.00  34  0 17  2 47
  0    62 125.33  42  5.14  0.00  0  0.00  33  0 17  3 47
  0    62 128.00  41  5.12  0.00  0  0.00  32  0 18  2 48
  0    62 122.67  42  5.03  0.00  0  0.00  31  0 19  3 47
  0    62 128.00  41  5.12  0.00  0  0.00  33  0 18  2 47
  0    62 122.91  44  5.28  0.00  0  0.00  32  0 18  1 49
  0    62 86.65  65  5.49  0.00  0  0.00  37  0 14  3 46
  0    61 74.92  72  5.26  0.00  0  0.00  26  0 18  1 56
  0    61 101.65  51  5.06  0.00  0  0.00  32  0 15  2 52
  0    62 107.00  48  5.01  0.00  0  0.00  29  0 17  0 53
  0    62 116.09  47  5.32  0.00  0  0.00  32  0 19  2 47
  0    62 125.27  41  5.01  0.00  0  0.00  34  0 16  3 48
  0    62 113.26  46  5.08  0.00  0  0.00  33  0 15  2 50
  0    62 128.00  43  5.37  0.00  0  0.00  35  0 16  2 47
  0    62 128.00  43  5.37  0.00  0  0.00  34  0 15  2 48
  0    62 122.67  42  5.03  0.00  0  0.00  34  0 14  3 49
  0    62 125.40  43  5.26  0.00  0  0.00  33  0 17  2 48
  0    62 128.00  42  5.24  0.00  0  0.00  31  0 19  2 48
  0    62 127.62  42  5.23  0.00  0  0.00  29  0 21  0 50
  0    62 128.00  43  5.37  0.00  0  0.00  33  0 18  2 48
  0    62 128.00  40  5.00  0.00  0  0.00  30  0 21  3 46
  0    62 120.19  43  5.04  0.00  0  0.00  34  0 16  2 48
  0    62 128.00  43  5.37  0.00  0  0.00  33  0 17  2 48

lo que me vota el cache.log
2014/03/07 13:49:46| Store rebuilding is 804.8% complete
2014/03/07 13:50:01| Store rebuilding is 95810.1% complete
2014/03/07 13:50:16| Store rebuilding is 30222.5% complete
2014/03/07 13:50:28| clientTryParseRequest: FD 95 (172.16.184.2:38767) Invalid Request
2014/03/07 13:50:31| Store rebuilding is 17934.5% complete
2014/03/07 13:50:35| httpAccept: FD 17: accept failure: (53) Software caused connection abort
2014/03/07 13:50:46| Store rebuilding is 12648.7% complete
2014/03/07 13:51:01| Store rebuilding is 9814.8% complete
2014/03/07 13:51:16| Store rebuilding is 8025.0% complete
2014/03/07 13:51:31| Store rebuilding is 6922.7% complete
2014/03/07 13:51:46| Store rebuilding is 6198.6% complete
2014/03/07 13:52:01| Store rebuilding is 5538.8% complete
2014/03/07 13:52:16| Store rebuilding is 4925.5% complete
2014/03/07 13:52:31| Store rebuilding is 4452.9% complete
2014/03/07 13:52:46| Store rebuilding is 4064.6% complete
2014/03/07 13:52:59| clientTryParseRequest: FD 37 (172.16.184.2:44348) Invalid Request
2014/03/07 13:53:01| Store rebuilding is 3743.2% complete
2014/03/07 13:53:15| clientTryParseRequest: FD 95 (172.16.77.28:51859) Invalid Request
2014/03/07 13:53:16| Store rebuilding is 3455.9% complete
2014/03/07 13:53:31| Store rebuilding is 3210.6% complete
2014/03/07 13:53:46| Store rebuilding is 3004.0% complete
2014/03/07 13:54:01| Store rebuilding is 2818.5% complete
2014/03/07 13:54:15| parseHttpRequest: Unsupported method ''
2014/03/07 13:54:15| clientTryParseRequest: FD 43 (172.16.184.2:39906) Invalid Request
2014/03/07 13:54:16| Store rebuilding is 2656.1% complete
2014/03/07 13:54:31| Store rebuilding is 2512.8% complete
2014/03/07 13:54:46| Store rebuilding is 2390.5% complete
2014/03/07 13:55:01| Store rebuilding is 2277.6% complete
2014/03/07 13:55:16| Store rebuilding is 2174.3% complete
2014/03/07 13:55:31| Store rebuilding is 2079.4% complete
2014/03/07 13:55:46| Store rebuilding is 1993.6% complete
2014/03/07 13:56:01| Store rebuilding is 1914.2% complete
2014/03/07 13:56:16| Store rebuilding is 1840.4% complete
2014/03/07 13:56:31| Store rebuilding is 1772.7% complete
2014/03/07 13:56:46| Store rebuilding is 1711.4% complete
2014/03/07 13:57:01| Store rebuilding is 1652.0% complete
2014/03/07 13:57:16| Store rebuilding is 1598.0% complete
2014/03/07 13:57:31| Store rebuilding is 1547.6% complete
2014/03/07 13:57:46| Store rebuilding is 1500.0% complete
2014/03/07 13:58:01| Store rebuilding is 1455.8% complete
2014/03/07 13:58:16| Store rebuilding is 1414.0% complete

lo de acces.log
[2.0.2-RELEASE][root@pfwifi.pfwifi.upse.edu.ec]/var/squid/logs(17): tail -f access.log
1394218736.788    144 172.16.63.245 TCP_MISS/200 564 POST http://www.google-analytics.com/collect - DIRECT/201.218.56.216 image/gif
1394218736.799  3745 172.16.84.154 TCP_MISS/200 213479 GET http://r16–-sn-nwj7kner.googlevideo.com/videoplayback? - DIRECT/74.125.170.245 application/octet-stream
1394218737.219    383 172.16.194.87 TCP_MISS/200 1462 POST http://z029.fma.fb.me/v1/979/p/228 - DIRECT/173.252.105.18 application/octet-stream
1394218737.245    476 172.16.188.123 TCP_MISS/200 263 POST http://z026.fma.fb.me/v1/454/p/901 - DIRECT/173.252.102.9 application/octet-stream
1394218737.468    391 172.16.194.138 TCP_MISS/200 1424 POST http://z024.fma.fb.me/v1/757/p/85 - DIRECT/69.171.248.56 application/octet-stream
1394218737.626    599 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218737.641      0 172.16.184.2 TCP_NEGATIVE_HIT/204 289 GET http://clients3.google.com/generate_204 - NONE/- text/html
1394218737.757    249 172.16.79.192 TCP_MISS/200 26072 GET http://adduplexv3.blob.core.windows.net/bannerads/60000/6089/4f415201-5db5-4fd9-99d5-dc6014c3db4b.png? - DIRECT/138.91.96.142 image/png
1394218737.789      0 172.16.184.2 TCP_NEGATIVE_HIT/204 289 GET http://clients3.google.com/generate_204 - NONE/- text/html
1394218737.910    338 172.16.125.107 TCP_MISS/200 264 GET http://soundfrost.org/update/commonLauncher.php? - DIRECT/217.23.15.146 text/html
1394218738.031    159 172.16.113.69 TCP_MISS/200 585 GET http://www.googleadservices.com/pagead/conversion/1001680686/? - DIRECT/74.125.229.186 text/html
1394218738.228  4906 172.16.191.193 TCP_MISS/206 716023 GET http://au.download.windowsupdate.com/msdownload/update/software/svpk/2011/06/clientshared32wwsp1-x-none_2c8173fc260afb350e6d76c1736e04cb00728e0a.cab - DIRECT/4.23.32.253 application/octet-stream
1394218738.262    167 172.16.74.120 TCP_MISS/200 585 GET http://www.googleadservices.com/pagead/conversion/1001680686/? - DIRECT/74.125.229.186 text/html
1394218738.350    441 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218738.357  1753 172.16.79.192 TCP_MISS/302 890 GET http://api.soundcloud.com/tracks/97628172/stream? - DIRECT/72.21.91.127 application/xml
1394218738.523    488 172.16.194.87 TCP_MISS/200 263 POST http://z029.fma.fb.me/v1/979/p/229 - DIRECT/173.252.105.18 application/octet-stream
1394218738.536    386 172.16.194.138 TCP_MISS/200 263 POST http://z024.fma.fb.me/v1/757/p/86 - DIRECT/69.171.248.56 application/octet-stream
1394218738.699      5 172.16.106.235 TCP_MISS/200 426 HEAD http://download.windowsupdate.com/v9/1/windowsupdate/redir/muv4wuredir.cab? - DIRECT/181.198.58.30 application/octet-stream
1394218738.894    295 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218739.469    106 172.16.113.69 TCP_MISS/200 2219 GET http://www.youtube.com/leanback_ajax? - DIRECT/201.218.56.227 application/json
1394218739.540  9238 172.16.77.248 TCP_MISS/200 749 GET http://fisica.uc.cl/templates/webucinterior/favicon.ico - DIRECT/146.155.99.247 image/x-icon
1394218739.577    296 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218739.642    458 172.16.194.138 TCP_MISS/200 263 POST http://z024.fma.fb.me/v1/757/p/87 - DIRECT/69.171.248.56 application/octet-stream
1394218740.091    159 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218740.098    392 172.16.194.136 TCP_MISS/200 263 POST http://z024.fma.fb.me/v1/302/p/32 - DIRECT/69.171.248.56 application/octet-stream
1394218740.132    130 172.16.113.69 TCP_MISS/204 558 GET http://www.youtube-nocookie.com/device_204? - DIRECT/201.218.56.208 text/html
1394218740.558    640 172.16.184.2 TCP_MISS/200 952 POST http://b.manboker.com/MessageInfoWebService.asmx/GetMessageByForm - DIRECT/223.6.251.124 text/xml
1394218740.778    382 172.16.188.123 TCP_MISS/200 263 POST http://z026.fma.fb.me/v1/454/p/902 - DIRECT/173.252.102.9 application/octet-stream
1394218740.821    158 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218740.874    87 172.16.79.192 TCP_MISS/200 1031 GET http://s.adduplex.com/Serve2/GetAd? - DIRECT/23.96.34.253 text/plain
1394218740.974      5 172.16.79.192 TCP_MISS/200 296 GET http://www.msftncsi.com/ncsi.txt - DIRECT/181.198.58.36 text/plain
1394218741.232    180 172.16.81.169 TCP_MISS/200 492 POST http://api.kiip.me/2.0/app/cache - DIRECT/107.23.207.64 application/json
1394218741.381    490 172.16.194.135 TCP_MISS/200 15763 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps? - DIRECT/168.62.11.165 application/atom+xml
1394218741.432    388 172.16.194.87 TCP_MISS/200 263 POST http://z029.fma.fb.me/v1/979/p/230 - DIRECT/173.252.105.18 application/octet-stream
1394218741.561    296 172.16.106.235 TCP_MISS/200 405 HEAD http://update.microsoft.com/v10/1/windowsupdate/selfupdate/wuident.cab? - DIRECT/65.55.163.222 application/octet-stream
1394218741.631    158 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218741.670    166 172.16.79.192 TCP_MISS/200 25976 GET http://adduplexv3.blob.core.windows.net/bannerads/70000/7650/f3d9dab6-1847-464b-9f46-aa2e32495ee3.jpg? - DIRECT/138.91.96.142 image/jpeg
1394218741.779      5 172.16.106.235 TCP_MISS/200 426 HEAD http://download.windowsupdate.com/v9/1/windowsupdate/b/selfupdate/WSUS3/x64/Vista/wsus3setup.cab? - DIRECT/181.198.58.30 application/octet-stream
1394218741.816      6 172.16.194.135 TCP_MISS/200 2672 GET http://cdn.marketplaceimages.windowsphone.com/v8/images/cc82bb81-277f-4966-8159-f45837da81a5? - DIRECT/181.198.58.30 image/jpeg
1394218741.941      6 172.16.194.135 TCP_MISS/200 2872 GET http://cdn.marketplaceimages.windowsphone.com/v8/images/6390b13e-a27b-46dd-854d-9ed65db4f265? - DIRECT/181.198.58.30 image/png
1394218742.011  3612 172.16.191.193 TCP_MISS/206 481431 GET http://au.download.windowsupdate.com/msdownload/update/software/svpk/2011/06/clientshared32wwsp1-x-none_2c8173fc260afb350e6d76c1736e04cb00728e0a.cab - DIRECT/4.23.32.253 application/octet-stream
1394218742.013    385 172.16.194.87 TCP_MISS/200 1437 POST http://z029.fma.fb.me/v1/979/p/231 - DIRECT/173.252.105.18 application/octet-stream
1394218742.498    189 172.16.191.193 TCP_MISS/200 2307 POST http://ocsp.verisign.com/ - DIRECT/23.13.171.27 application/ocsp-response
1394218742.539    158 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218742.969    309 172.16.81.169 TCP_MISS/200 271 POST http://data.flurry.com/aap.do - DIRECT/216.52.203.13 application/octet-stream
1394218743.008    384 172.16.194.87 TCP_MISS/200 502 POST http://z029.fma.fb.me/v1/979/p/232 - DIRECT/173.252.105.18 application/octet-stream
1394218743.238    158 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml
1394218743.314    342 172.16.125.107 TCP_MISS/200 264 GET http://soundfrost.org/update/commonLauncher.php? - DIRECT/217.23.15.146 text/html
1394218743.434    639 172.16.184.2 TCP_MISS/200 952 POST http://b.manboker.com/MessageInfoWebService.asmx/GetMessageByForm - DIRECT/223.6.251.124 text/xml
1394218743.492    704 172.16.194.138 TCP_MISS/200 263 POST http://z024.fma.fb.me/v1/757/p/88 - DIRECT/69.171.248.56 application/octet-stream
1394218743.687    390 172.16.194.136 TCP_MISS/200 263 POST http://z024.fma.fb.me/v1/302/p/33 - DIRECT/69.171.248.56 application/octet-stream
1394218743.901    159 172.16.194.135 TCP_MISS/200 11500 GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/82a23635-5bd9-df11-a844-00237de2db9e? - DIRECT/168.62.11.165 application/atom+xml

[periko]

Este log:

2014/03/07 13:56:46| Store rebuilding is 1711.4% complete
2014/03/07 13:57:01| Store rebuilding is 1652.0% complete
2014/03/07 13:57:16| Store rebuilding is 1598.0% complete
2014/03/07 13:57:31| Store rebuilding is 1547.6% complete
2014/03/07 13:57:46| Store rebuilding is 1500.0% complete
2014/03/07 13:58:01| Store rebuilding is 1455.8% complete
2014/03/07 13:58:16| Store rebuilding is 1414.0% complete

No es bueno, esta reconstruyendo su cache, y lo esta haciendo mientras surte a tus clientes.

Squiguard no lo vi en tus procesos, pero ahorita ese yo lo sacaria de la jugada.

Mira yo lo que haria es, borrar cache y crearlo de 0, el Store Rebuild lo hace durante el arranque, y si hay peticiones las va a responder, esta masacrando el disco y tratando de servir a tus clientes, o chiflas o platicas.
Hay un parametro que le indicamos que responda a tus clientes hasta que termine de reconstruir y revisar que esta listo para surtir pedidos, por default esta apagado y cuando esto sucede hace las 2 cosas a la vez,
Si le empiezan a llegar pedidos mientras lo hace y si es una red con altos usuarios nunca lo van a dejar terminar.

Parece que es lo que estoy viendo aqui.

600 Usuario, si te sobra memoria dale mas le  va a servir.

El borrado y generacion del los directorios del cache lo hace rapido por que es chico tu tamano, pero es hacerlo mientras nadie este conectado.

Saludos.

[will23b]

ok periko…gracias por la aclaracion....seguire tus consejos para ver como me va, le dare mas memoria, si me ayduas con la opcion que me dices que solo atienda peticiones y despues haga lo demas

[periko]

Deja nomas la busco, pero es un parametro simple, ahorita la cosa es que en cuanto tengas chance, detenerlo, borrar/crear cache y arrancar.

El parametro:

squid -F

Me equivoque, en la config no viene, pero es el parametro de arriba.

[will23b]

ok …gracias por la ayuda periko

[will23b]

bueno te comento que el problema se soluciono con los datos que me has dado Periko….muchas gracias por la ayuda