Squid3-dev ICAP Protocol Error on 32-bit



  • Pfsense 2.1.3 32-bit
    Ram 4 GB
    HDD 128 GB SSD
    CPU AMD Athlon 3400+ 64-bit
    1 Wan
    1 Lan

    Packages:
    pfBlocker 1.0.2
    Squid3-dev 3.3.10 pkg 2.2.2

    I am using Squid3-dev with antivirus and SSL enabled.  Works very well, but often it will give this error:

    ICAP Protocol Error, with a "no error" error code

    ERROR in the browser
    The following error was encountered while trying to retrieve the URL: http://google.com
        ICAP protocol error.

    The system returned: [No Error]

    This means that some aspect of the ICAP communication failed.

    Some possible problems are:

    *

    The ICAP server is not reachable.
        *

    An Illegal response was received from the ICAP server.

    The only way to fix it is to restart squid.

    I have read here:  http://squidclamav.darold.net/tuning.html  that changing some variables might help.  I have doubled the default amounts described on the darold.net page, but it does not seem to help.  Does anyone have any hints on what to look at next?  This page also mentions that this works "with bypass enabled" - any idea what that means?

    Thanks!



  • Potentially solved… the "bypass" feature can be added in the squid.inc file, as the squid.conf gets overwritten.  I'm not sure about adding it to the Custom fields in the GUI - maybe that would work(?), but I opted for the .inc edit instead.

    I also modified the other c-icap.conf file options that are relevant to these ICAP errors as stated on the developer's site. I ended up tripling the values before turning on the bypass=1 feature.  I might reduce the numbers now to double only, given that bypass seems to be working.  Clamav is presently running multiple processes and using (?) 600 MB of RAM, but I may not be reading the system activity correctly.

    So far, no ICAP error since turning bypass on.  Apparently, squid will ignore errors generated by ICAPs and not pass it on to the browser. I'm not sure if this breaks clamav for some period.  Squid3-dev is definately not for the ultra green  ;D



  • Hi MIT, can you please show how you fixed it step by step?

    Thanks.



  • Same problem…64-bit system here. ICAP problems start as soon as I enable antivirus in Squid interface. I tried the "bypass"trick and it works but I think it completely breaks the antivirus feature. I tested the same on EICAR test file and another test website; the antivirus didn't show any warning.

    So...that basically kills the purpose!



  • @golmaal:

    Same problem…64-bit system here. ICAP problems start as soon as I enable antivirus in Squid interface. I tried the "bypass"trick and it works but I think it completely breaks the antivirus feature. I tested the same on EICAR test file and another test website; the antivirus didn't show any warning.

    So...that basically kills the purpose!

    Strange.. I conitnue to have eicar blocking both http/https with bypass



  • @Bismarck:

    Hi MIT, can you please show how you fixed it step by step?

    Thanks.

    In pfsense web gui…

    For the bypass feature....

    Go to Diagnostics > edit file
    Browse to /usr/local/pkg
    Load squid.inc
    modify these two lines:

    icap_service service_req reqmod_precache bypass=0 icap://127.0.0.1:1344/squidclamav
    icap_service service_resp respmod_precache bypass=0 icap://127.0.0.1:1344/squidclamav

    TO THIS:

    icap_service service_req reqmod_precache bypass=1 icap://127.0.0.1:1344/squidclamav
    icap_service service_resp respmod_precache bypass=1 icap://127.0.0.1:1344/squidclamav

    Save file.

    Then I rebooted.  Done

    I ended up changing everyhing in the C-icap parameters back to the defaults (so you need not change those, found on the Antivurs tab of Squid3-dev) same goes for clam.conf, changed back to defaults.  Only the bypass=1 change was needed and no more ICAP error. I have tested with EICAR and it continues to stop those everytime in http and https.

    Good luck  ;)



  • Does it work with squidguard included in the equation too. I tried but didn't work.



  • @exograpix:

    Does it work with squidguard included in the equation too. I tried but didn't work.

    I don't use squidguard, so I am no help on that one.



  • I've got the same problem on PFSense 2.1.4-RELEASE (amd64)

    Enabling debug on squid, I've seen the following messages :

    2014/07/11 10:36:29.862 kid1| url.cc(386) urlParse: urlParse: Split URL 'icap://127.0.0.1:1344/squidclamav ICAP/1.0
    ' into proto='icap', host='127.0.0.1', port='1344', path='/squidclamav ICAP/1.0'
    2014/07/11 10:36:29.862 kid1| url.cc(422) urlParse: urlParse: URI has whitespace: {icap://127.0.0.1:1344/squidclamav ICAP/1.0
    RESPMOD icap://127.0.0.1:1344/squidclamav ICAP/1.0
    ICAP/1.0 204 Unmodified
    Server: C-ICAP/0.2.5
    2014/07/11 10:36:29.869 kid1| ModXact.cc(742) parseHeaders: parse ICAP headers
    2014/07/11 10:36:29.869 kid1| Xaction.cc(503) setOutcome: ICAP_ERR_OTHER
    2014/07/11 10:36:29.870 kid1| Server.cc(828) handleAdaptationAborted: creating ICAP error entry after ICAP failure
    2014/07/11 10:36:29.870 kid1| forward.cc(397) fail: ERR_ICAP_FAILURE "Internal Server Error"

    It seems the url used to contact ICAP server is malformed (there is a whitespace in it).



  • Sorry to necro this…

    Has anyone found a solution to this? I get this same error message and if the issue is a malformed url in a config file, which one is it?



  • Thanks MIT for the details, however in squid 3.4.10_2 pkg 0.2.6 there are changes in the squid.inc file.
    it's like this:
    icap_service service_avi_req reqmod_precache icap://[::1]:1344/squid_clamav bypass=off
    adaptation_access service_avi_req allow all
    icap_service service_avi_resp respmod_precache icap://[::1]:1344/squid_clamav bypass=on
    adaptation_access service_avi_resp allow all

    i changed the squid_clamav bypass=off to on and the eicar detection is working.

    thank you again :)



  • As I've posted on other many squid3 topics, clamav integration will work if you:

    • Enable antivirus on squid

    • fix config warnings alerts

    • wait first freshclam to finish

    • stop and start (not restart) squid and c-icap service

    Configure a clamav bypass has the same effect as disabling the antivirus integration.

    I've tested it on amd64 at least 3 times and had a working on all tests.



  • Can you fix the default config so that it works by default. While the GUI does tell you what to do if you save the page again, I'm sure that a lot of people on the forum and irc having issues with Squid do not go back there and save the page a second time.



  • @fragged:

    Can you fix the default config so that it works by default. While the GUI does tell you what to do if you save the page again, I'm sure that a lot of people on the forum and irc having issues with Squid do not go back there and save the page a second time.

    Decide what ip to use on sarg reports warn_php for example is not that simple. If I force Lan IP on package config then somebody will ask to listen on WLAN and/or internal http server.

    This is a first run configuration. Once configured, you do not need to check again antivurus options.



  • I have this problem on the 64 bit version RC 2.2 and I just go to the antivirus page and click save again and then the system comes back up .. but wish it stop messing up



  • Friends, help, please, how to solve a problem with this error ICAP?
    Configuring a clamav bypass=1 is disabling the antivirus integration!
    PFsense 2.1.5 x64, squid 3.3.10





  • @marcelloc:

    Did you read the topic first?
    https://forum.pfsense.org/index.php?topic=77264.msg485524#msg485524

    Friend, yes, I read it. But to my regret, I didn't understand part of instructions:
    fix config warnings alerts
    wait first freshclam to finish
    Please, explain more in detail which needs to be made here.
    Thanks!



  • Antonio, don't waste your time in pfSense 2.1.5 x64 i-cap ist still broken there, since it has never worked before.

    I guess you need to upgrade to pfSense 2.2 x64 to get it work, if I get marcelloc right?

    fix config warnings alerts = look in Status: System logs: General for errors and fix it

    wait first freshclam to finish = execute freshclam in the console/shell and watch via top till its finished

    Good luck.



  • Error in system log (PFsense 2.1.5 x64, squid 3.3.10):

    kernel: pid 85487 (c-icap), uid 9595: exited on signal 11
    

    It is possible to fix it, or it really nonremovable error in 2.1.5 x64 in ICAP?
    I don't like 2.2. With it I have many more problems with Squid+SquidGuard+Lightsquid. May be later, build of PFsense will be stable and I update it.



  • @Antonio_Grande:

    It is possible to fix it, or it really nonremovable error in 2.1.5 x64 in ICAP?

    Unfortunatelly no. the icap error are related to freebsd 8.x and icap, not pfsense itself. the same compile args and config options works fine on freebsd 8.x 32bit version.

    An workaround for pfsense 2.1.x 64bits if you are not using ssl interception is to use clamav on dansguardian ou havp.



  • I am receiving ICAP errors with squid3 on amd64 pfSense 2.2 but only on http sites. I think I must have something misconfigured because HTTPS is fine. How does one use HAVP with squid, I feel like I have too many redundant proxies with HAVP and Dansguardian.



  • I was having a similar problem until I saw this: https://forum.pfsense.org/index.php?topic=87424.msg480232#msg480232

    fresh 2.2 install
        Install squid3

        chech squid tabs, save, fix config options pointed by gui alerts
        On antivirus tab, save config twice as first time it will load sample files and second check config options.
        via console wait (repeating ps ax | grep -i fresclam  or tail -f /var/log/clamav/freshclam.log) clamav database first slow update
        enable transparent mode(do not select loopback on any squid option)
        stop and start squid via gui to force c-icap to restart too after first freshclam.

    Edited original post to describe my steps. The key part is the "save twice" on the AV tab. Fix the problems presented, each has its solution right in the message. I am now able to browse HTTP sites without the ICAP errors.



  • $ repeating ps ax | grep -i freshclam  or tail -f /var/log/clamav/freshclam.log
    grep: freshclam: No such file or directory
    grep: or: No such file or directory
    grep: tail: No such file or directory
    

    For some reason I can't freshclam



  • @jvamos:

    $ repeating ps ax | grep -i freshclam  or tail -f /var/log/clamav/freshclam.log
    

    This line means

    repeat this cmd on console every 30 seconds for example

    ps ax | grep -i freshclam 
    

    or this one once

    tail -f /var/log/clamav/freshclam.log
    


  • I think I just typed "freshclam" (without quotes) to update, as marcelloc says the other commands are to show the status of freshclam, not to execute it.



  • HI, Guys

    I got errors:

    ERROR
    The requested URL could not be retrieved

    The following error was encountered while trying to retrieve the URL: http://www.google.ca

    Connection to 127.0.0.1 failed.

    The system returned: (60) Operation timed out

    The remote host or network may be down. Please try the request again.

    Your cache administrator is admin@localhost.

    ERROR
    The requested URL could not be retrieved

    The following error was encountered while trying to retrieve the URL: http://www.dslreports.com/forum/rogers

    Unable to forward this request at this time.

    This request could not be forwarded to the origin server or to any parent caches.

    Some possible problems are:

    An Internet connection needed to access this domains origin servers may be down.
        All configured parent caches may be currently unreachable.
        The administrator may not allow this cache to make direct connections to origin servers.

    Your cache administrator is admin@localhost.

    I only installed snort, pfBlokerNG, and squid3,  for Squid3, all the settings were setup by default,  changed squid.inc, changed anti-virus configs, and execute freshclam, but I got above odd errors,  can't surf internet unless turn off the transparent HTTP proxy.

    What am I doing wrong?



  • Olá,

    Caso alguém ainda esteja com problemas. Segue abaixo como funcionou em minha rede:

    Pfsense 2.2.3 + Squid3 0.2.8 + SquidGuard 1.9.14 + i-cap/clamav

    Defina em squidclamav.conf:

    redirect http://IP_SEU_SERVIDOR/squid_clwarn.php

    Para o caso de possuir SquidGuard, descomente a linha:

    squidguard /usr/local/squidGuard/bin/squidGuard

    Adicione em i-cap.conf:

    Service squid_clamav squidclamav.so

    Apague essa linha de i-cap.conf(Mesmo que esteja comentada):

    ldap://cn=Directory Manager:Apassword@ldap.chtsanti.net?o=chtsanti?mermberUid?(&(objectClass=posixGroup)(cn=%s))

    Pra mim está funcionando ok.

    I hope it helps someone.  ;D



  • Just add domain in Whitelist with http and you`r issue will solve.