Any chance of getting a working transparent proxy again?



  • I - like many others - tried to get squid working in transparent mode on both i386- and 64-versions of recent pfense (tried squid 2 and 3). Both fresh installs.

    Followed the pfsense-Howto but it does not work, e.g. clients do not get through the proxy and are not able to get any data throughout port 80.

    If I disable transparent mode and set the clients manually to the proxy, squid works like a charm.

    As I have read here, there had been many complains about that, but the gurus always ended by: no problem here, has to be a user misconfiguration.

    But - as I stated - I also have tried it on two different machines with different hardware (one with i386, one 64bit), no difference, transparent mode does not run.

    Are the gurus working on it?



  • Can i ask you how i can see if i am using the cache or not?

    i have transparrent mode running with an growing cache? so it seems to work just fine here

    this is my syslog, as a proof ;)

    CPU Usage: 373.371 seconds = 196.398 user + 176.973 sys
    Maximum Resident Size: 1744000 KB
    Page faults with physical i/o: 0
    2015/04/02 14:25:22 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    2015/04/02 14:27:31 kid1| Starting Squid Cache version 3.4.10 for amd64-portbld-freebsd10.1...
    
    

  • Banned

    I have the same issue. Not working on transparent mode at all….only forwards HTTPS sites.

    So I had to uninstall and let it be...



  • I think it´s just because https uses port 443, not 80, as squid does.



  • @notaduck:

    i have transparrent mode running with an growing cache? so it seems to work just fine here

    Do u see any traffic in the WUI under "real time" in the squid section?



  • yes  i do ;)

    You have to enable loggig under logging settings.



  • I also had logging enabled.

    Strange to me how you have gotten transparent mode running without mods.



  • @peter808:

    I also had logging enabled.

    Strange to me how you have gotten transparent mode running without mods.

    This is theexact way i did my squid configuration http://techknight.eu/2015/03/13/pfsense-setup-and-configure-squid3-transparent-proxy/

    i just had to disable icap and enable loggin :)

    I hope it helps


  • Banned

    When using Squid3 then ClamAV is good to have.

    Can anyone post a working c-icap.conf file??



  • i don't think so :S
    i have seen so many topics without a solution :S



  • @notaduck:

    This is theexact way i did my squid configuration http://techknight.eu/2015/03/13/pfsense-setup-and-configure-squid3-transparent-proxy/

    Thanks, but I already knew about that article.

    Still no working transparent mode here  :'(



  • @peter808:

    @notaduck:

    This is theexact way i did my squid configuration http://techknight.eu/2015/03/13/pfsense-setup-and-configure-squid3-transparent-proxy/

    Thanks, but I already knew about that article.

    Still no working transparent mode here  :'(

    hmmmm…. I just don't understand why it shouldn't work for you.
    can you try to post your squid slog?

    btw. it annoys me that it isn't working because i am the one who wrote the article -_- and it works just fine for me



  • http_port 172.23.200.1:3128
    http_port 127.0.0.1:3128 intercept
    icp_port 0
    cache_effective_user proxy
    cache_effective_group proxy
    error_default_language de
    icon_directory /usr/pbi/squid-i386/local/etc/squid/icons
    visible_hostname firewall
    cache_log /var/squid/logs/cache.log
    cache_store_log none
    netdb_filename /var/squid/logs/netdb.state
    pinger_enable on
    pinger_program /usr/pbi/squid-i386/local/libexec/squid/pinger
    
    logfile_rotate 14
    debug_options rotate=14
    shutdown_lifetime 3 seconds
    # Allow local network(s) on interface(s)
    acl localnet src  172.23.200.0/24
    forwarded_for on
    uri_whitespace strip
    
    acl dynamic urlpath_regex cgi-bin \?
    cache deny dynamic
    
    cache_mem 1024 MB
    maximum_object_size_in_memory 32 KB
    memory_replacement_policy heap GDSF
    cache_replacement_policy heap LFUDA
    cache_dir aufs /var/squid/cache 500 16 256
    minimum_object_size 0 KB
    maximum_object_size 400 KB
    offline_mode off
    cache_swap_low 90
    cache_swap_high 95
    cache allow all
    
    # No redirector configured
    
    #Remote proxies
    
    # Setup some default acls
    # From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localh
    ost ACL definitions are now built-in.
    # acl localhost src 127.0.0.1/32
    acl allsrc src all
    acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901  3128 3127 1025-65535
    acl sslports port 443 563
    
    # From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localh
    ost ACL definitions are now built-in.
    #acl manager proto cache_object
    
    acl purge method PURGE
    acl connect method CONNECT
    
    # Define protocols used for redirects
    acl HTTP proto HTTP
    acl HTTPS proto HTTPS
    acl allowed_subnets src 172.23.200.0/24
    http_access allow manager localhost
    
    http_access deny manager
    http_access allow purge localhost
    http_access deny purge
    http_access deny !safeports
    http_access deny CONNECT !sslports
    
    # Always allow localhost connections
    # From 3.2 further configuration cleanups have been done to make things easier and safer.
    # The manager, localhost, and to_localhost ACL definitions are now built-in.
    # http_access allow localhost
    
    request_body_max_size 0 KB
    delay_pools 1
    delay_class 1 2
    delay_parameters 1 -1/-1 -1/-1
    delay_initial_bucket_level 100
    delay_access 1 allow allsrc
    
    # Reverse Proxy settings
    
    # Custom options before auth
    
    # Setup allowed acls
    # Allow local network(s) on interface(s)
    http_access allow allowed_subnets
    http_access allow localnet
    # Default block all to be sure
    http_access deny allsrc
    
    


  • cache.log.0

    2015/04/06 23:55:45| pinger: Unable to start ICMP pinger.
    2015/04/06 23:55:45|  icmp_sock: (1) Operation not permitted
    2015/04/06 23:55:45| pinger: Unable to start ICMPv6 pinger.
    2015/04/06 23:55:45| FATAL: pinger: Unable to open any ICMP sockets.
    

    No entries in access.log in transparent mode.



  • working for non https transparent mode

    do a fresh install of 2.2.1 @ x64

    install squid3 and squidguard dev

    create the necessary wdap files as per https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid

    use urlresolver instead of dhcp forwarder in 2.2.1! - same idea and setup

    using the file manager and editor

    create a /cache directiory and set it to 30gb or so in squid 3

    configure squidguard

    start squid 3 and put on transparent mode

    do not sue the antivirus atm as it gives icamp errors

    you may want to use the service watch to retreat squid3 if it stops,etc



  • I played around quite a bit and held off upgrading from 2.1.5 until just this week.  I've tried many times and always had to roll back to 2.1.5
    vmware workstation environment, 64bit pfsense.
    Uninstalled squid/squidguard.  Then ran the update.
    install squid 3
    install squidguard development (could never get the main version running).
    download blacklist.
    enable clamav in squid.  Then disable it.
    re-download blacklist.

    Seems to work now.  The main issue now is that after a reboot, everything crashes and fails to restart.
    Fix is to enable then disable clamav in squid.  Then re-download blacklist and it stays up and running.

    Definitely not good, but at least it's working.  Nowhere near as stable as when we were on 2.1.5 (or any previous version for that matter back to 1.2)

    Not pushing my luck on https proxy until this part is better.



  • @messerchmidt:

    working for non https transparent mode

    do a fresh install of 2.2.1 @ x64

    install squid3 and squidguard dev

    With squidguard dev installed and starting, squid now always crashes:

    Apr 7 07:53:07	squid[60452]: Squid Parent: (squid-1) process 69996 will not be restarted due to repeated, frequent failures
    Apr 7 07:53:07	squid[60452]: Squid Parent: (squid-1) process 69996 exited with status 1
    Apr 7 07:53:07	(squid-1): The redirector helpers are crashing too rapidly, need help!
    Apr 7 07:53:06	squid[60452]: Squid Parent: (squid-1) process 69996 started
    

    @messerchmidt:

    create the necessary wdap files as per https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid

    The howto you mentioned says: "To use squid authentication, squid cannot be used in transparent mode. "

    The rest you suggested does not seem to be related to the transparent proxy-problem, I guess, but thanks though.



  • @tester_02:

    I played around quite a bit and held off upgrading from 2.1.5 until just this week.  I've tried many times and always had to roll back to 2.1.5
    vmware workstation environment, 64bit pfsense.
    Uninstalled squid/squidguard.  Then ran the update.

    Did I get you right that squid transparent mode only seems to be running on pfsense 2.15?

    So what did you mean with "ran the upgrade"? You upgraded to the recent 2.2 after all, anyway?



  • Well i have it running on 2.2.1
    have you disabled icmp?



  • yep.



  • "Any chance of getting a working transparent proxy again?"

    Seems to depend on who you ask.




  • Having another go at this, taking things slowly and making sure I've not overlooked anything.

    I'm seeing the following in cache.log on startup.

    2015/04/07 22:04:13 kid1| Starting Squid Cache version 3.4.10 for i386-portbld-freebsd10.1...
    2015/04/07 22:04:14| pinger: Initialising ICMP pinger ...
    2015/04/07 22:04:14|  icmp_sock: (1) Operation not permitted
    2015/04/07 22:04:14| pinger: Unable to start ICMP pinger.
    2015/04/07 22:04:14|  icmp_sock: (1) Operation not permitted
    2015/04/07 22:04:14| pinger: Unable to start ICMPv6 pinger.
    2015/04/07 22:04:14| FATAL: pinger: Unable to open any ICMP sockets.
    2015/04/07 22:04:16 kid1| Starting Squid Cache version 3.4.10 for i386-portbld-freebsd10.1...
    2015/04/07 22:04:16| pinger: Initialising ICMP pinger ...
    2015/04/07 22:04:16|  icmp_sock: (1) Operation not permitted
    2015/04/07 22:04:16| pinger: Unable to start ICMP pinger.
    2015/04/07 22:04:16|  icmp_sock: (1) Operation not permitted
    2015/04/07 22:04:16| pinger: Unable to start ICMPv6 pinger.
    2015/04/07 22:04:16| FATAL: pinger: Unable to open any ICMP sockets.
    

    Is this normal. Do I need to disable IPv6?

    Steve


  • Banned

    I did this… not getting through to HTTP sites but only HTTPS as expected.

    Unchecked Suppress Squid Version -> clicked save.

    Rebooted.

    Now it worked with HTTP sites like it should.

    Squid version is 2.7.9 pkg v.4.3.6

    EDIT: pretty scary shit. It works for a short time, then it stops working again.

    And nothing in the logs at all!



  • Squid2 was crap from what I understand.  Have you tried Squid3?


  • Banned

    Yes. But I cant get the damned ClamAV working….

    V3 doesnt have the same issue.



  • When using Squid3, I just ignored or disabled the AV crap.  We already have a better solution running on the desktops, and it just slowed everything down when running on the router.


  • Banned

    It annoys me when disabled it still shows in services as not running together with the ICAP crap….

    Can someone tell me how to get rid of that so it doesnt trick my OCD all the time?? :D



  • If you know PHP you could take a look through /usr/local/www/status_services.php and do something with that.

    And yes, I fully agree that any services that are not enabled should not even show in the Service Status widget.  I don't know if they can programmatically distinguish between an enabled service that's stopped versus a disabled service.  Either way, it bugs me too that my eye keeps tripping on those little red icons.


  • Banned

    TRUE THAT!!



  • So coming back to my original post (and believing in the strenght of the devs :) ) :

    is there any chance of getting a working transparent proxy again?



  • This bugs me for some quite some time, too.
    Enabling transparent works for a couple of calls to websites - then it dies…
    Scarry is the right description...

    3.1.20 pkg 2.1.2 on pfsense 2.1.5
    I have to say that the previous package (whichever that was!?) was running just fine!


Log in to reply