Reverse squid proxy does not show always tomcat webpage



  • Hi,

    I use wonna use reverse squid proxy before our Novell Vibe server.
    Vibe is a Tomcat Application that uses internal 8080 and 8443.

    The config now is:

    vibe.domain.ext –> 80  --> squid --> 8080 --> Vibe server
    vibe.domain.ext --> 443 --> squid --> 8443 --> Vibe server

    With the ssl config i use an cert on the squid en one on Vibe.
    I made two webservers an two mappings in the config.

    The problem is that the Vibe site does not always show up. Its not stable. Its hanging or very very slow.
    I also use Groupwise and that goes right or just a simple site. I also set the firewall open, but no results

    Who can help me with this problem or what do i wrong. Is there a log that i can read except the realtime log?

    Pleas help!



  • check squid access.log and cache.log



  • Thanks for your reply.

    In the access log the http site does get logged correct one time.
    The second time nothing is logged. When i call the https one line get logged with a time out i gues.

    1340818529.227      3 82.168.159.164 TCP_MISS/302 291 GET http://vibe.domain.ext/ - FIRST_UP_PARENT/HOST1 -
    1340818529.289      7 82.168.159.164 TCP_MISS/302 536 GET http://vibe.domain.ext/ssf/a/ - FIRST_UP_PARENT/HOST1 -
    1340818529.365    25 82.168.159.164 TCP_MISS/200 30574 GET http://vibe.domain.ext/ssf/a/c/p_name/ss_forum/p_action/1/action/view_permalink/entityType/user/entryId/ss_user_id_place_holder/showWhatsNew/1/vibeonprem_url/1/vibeonprem_root/1 - FIRST_UP_PARENT/HOST1 text/html
    1340818529.940      3 82.168.159.164 TCP_MISS/200 6416 GET http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtteaming.nocache.js? - FIRST_UP_PARENT/HOST1 text/javascript
    1340818530.229      1 82.168.159.164 TCP_MISS/404 1621 GET http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/tiny_mce_3_2_7/tiny_mce/plugins/preelementfix/langs/nl.js? - FIRST_UP_PARENT/HOST1 text/html
    1340818531.125    60 82.168.159.164 TCP_MISS/200 727 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.233    10 82.168.159.164 TCP_MISS/200 564 GET http://vibe.domain.ext/ssf/a/do? - FIRST_UP_PARENT/HOST1 text/html
    1340818531.267    51 82.168.159.164 TCP_MISS/200 646 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.346    68 82.168.159.164 TCP_MISS/200 629 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.472    57 82.168.159.164 TCP_MISS/200 620 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.475    49 82.168.159.164 TCP_MISS/200 646 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.501    66 82.168.159.164 TCP_MISS/200 871 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.688    53 82.168.159.164 TCP_MISS/200 642 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.698    56 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.703    53 82.168.159.164 TCP_MISS/200 657 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.813    58 82.168.159.164 TCP_MISS/200 669 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.843    58 82.168.159.164 TCP_MISS/200 626 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818531.955    51 82.168.159.164 TCP_MISS/200 605 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818591.828    71 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818651.783    63 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
    1340818726.244  59270 82.168.159.164 TCP_MISS/000 0 GET https://vibe.domain.ext/ - FIRST_UP_PARENT/HOST2 -

    In the access log i get two errors that FD22 an FD24 already in use is.
    I think that the adresses are defined two time?

    2012/06/27 19:32:11| Starting Squid Cache version 3.1.20 for amd64-portbld-freebsd8.1…
    2012/06/27 19:32:11| Process ID 2482
    2012/06/27 19:32:11| With 11095 file descriptors available
    2012/06/27 19:32:11| Initializing IP Cache...
    2012/06/27 19:32:11| DNS Socket created at [::], FD 11
    2012/06/27 19:32:11| DNS Socket created at 0.0.0.0, FD 12
    2012/06/27 19:32:11| Adding domain annamaria.nl from /etc/resolv.conf
    2012/06/27 19:32:11| Adding nameserver 127.0.0.1 from /etc/resolv.conf
    2012/06/27 19:32:11| Adding nameserver 194.151.228.18 from /etc/resolv.conf
    2012/06/27 19:32:11| Adding nameserver 172.16.2.22 from /etc/resolv.conf
    2012/06/27 19:32:11| Adding nameserver 172.16.2.20 from /etc/resolv.conf
    2012/06/27 19:32:11| Adding nameserver 194.151.228.34 from /etc/resolv.conf
    2012/06/27 19:32:11| User-Agent logging is disabled.
    2012/06/27 19:32:11| Referer logging is disabled.
    2012/06/27 19:32:11| Unlinkd pipe opened on FD 17
    2012/06/27 19:32:11| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
    2012/06/27 19:32:11| Store logging disabled
    2012/06/27 19:32:11| Swap maxSize 102400 + 8192 KB, estimated 8507 objects
    2012/06/27 19:32:11| Target number of buckets: 425
    2012/06/27 19:32:11| Using 8192 Store buckets
    2012/06/27 19:32:11| Max Mem  size: 8192 KB
    2012/06/27 19:32:11| Max Swap size: 102400 KB
    2012/06/27 19:32:11| Version 1 of swap file with LFS support detected…
    2012/06/27 19:32:11| Rebuilding storage in /var/squid/cache (DIRTY)
    2012/06/27 19:32:11| Using Least Load store dir selection
    2012/06/27 19:32:11| Current Directory is /etc
    2012/06/27 19:32:11| Loaded Icons.
    2012/06/27 19:32:11| helperOpenServers: Starting 0/0 'ssl_crtd' processes
    2012/06/27 19:32:11| helperOpenServers: No 'ssl_crtd' processes needed.
    2012/06/27 19:32:11| Accepting  HTTP connections at 172.16.0.23:8090, FD 20.
    2012/06/27 19:32:11| Accepting  accelerated HTTP connections at xx.xx.xx.xx:80, FD 21.
    2012/06/27 19:32:11| commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
    2012/06/27 19:32:11| Accepting HTTPS connections at xx.xx.xx.xx:443, FD 23.
    2012/06/27 19:32:11| commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use
    2012/06/27 19:32:11| Accepting ICP messages at [::]:7, FD 25.
    2012/06/27 19:32:11| HTCP Disabled.
    2012/06/27 19:32:11| Configuring Parent 172.16.2.48/8080/0
    2012/06/27 19:32:11| Configuring Parent 172.16.2.48/8443/0
    2012/06/27 19:32:11| Ready to serve requests.
    2012/06/27 19:32:11| Done reading /var/squid/cache swaplog (263 entries)
    2012/06/27 19:32:11| Finished rebuilding storage from disk.
    2012/06/27 19:32:11|      263 Entries scanned
    2012/06/27 19:32:11|        0 Invalid entries.
    2012/06/27 19:32:11|        0 With invalid flags.
    2012/06/27 19:32:11|      263 Objects loaded.
    2012/06/27 19:32:11|        0 Objects expired.
    2012/06/27 19:32:11|        0 Objects cancelled.
    2012/06/27 19:32:11|        0 Duplicate URLs purged.
    2012/06/27 19:32:11|        0 Swapfile clashes avoided.
    2012/06/27 19:32:11|  Took 0.03 seconds (9953.83 objects/sec).
    2012/06/27 19:32:11| Beginning Validation Procedure
    2012/06/27 19:32:11|  Completed Validation Procedure
    2012/06/27 19:32:11|  Validated 551 Entries
    2012/06/27 19:32:11|  store_swap_size = 742
    2012/06/27 19:32:12| storeLateRelease: released 0 objects

    So i still cannot find what i'am doing wrong. Can you help me more?
    This is my squid config:

    This file is automatically generated by pfSense

    Do not edit manually !

    http_port 172.16.0.23:8090
    icp_port 7

    pid_filename /var/run/squid.pid
    cache_effective_user proxy
    cache_effective_group proxy
    error_default_language nl
    icon_directory /usr/local/etc/squid/icons
    visible_hostname localhost
    cache_mgr admin@localhost
    access_log /var/squid/logs/access.log
    cache_log /var/squid/logs/cache.log
    cache_store_log none
    sslcrtd_children 0
    logfile_rotate 14
    shutdown_lifetime 3 seconds

    Allow local network(s) on interface(s)

    acl localnet src  172.16.0.0/16
    uri_whitespace strip

    Break HTTP standard for flash videos. Keep them in cache even if asked not to.

    refresh_pattern -i .flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

    Let the clients favorite video site through with full caching

    acl youtube dstdomain .youtube.com
    cache allow youtube
    cache_mem 8 MB
    maximum_object_size_in_memory 32 KB
    memory_replacement_policy heap GDSF
    cache_replacement_policy heap LFUDA
    cache_dir ufs /var/squid/cache 100 16 256
    minimum_object_size 0 KB
    maximum_object_size 4 KB
    offline_mode offcache_swap_low 90
    cache_swap_high 95
    acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
    cache deny donotcache

    Add any of your own refresh_pattern entries above these.

    refresh_pattern ^ftp:    1440  20%  10080
    refresh_pattern ^gopher:  1440  0%  1440
    refresh_pattern -i (/cgi-bin/|?) 0  0%  0
    refresh_pattern .    0  20%  4320

    No redirector configured

    #Remote proxies

    Setup some default acls

    acl allsrc src all
    acl localhost src 127.0.0.1/32
    acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 85 3128 1025-65535
    acl sslports port 443 563 85
    acl manager proto cache_object
    acl purge method PURGE
    acl connect method CONNECT

    acl allowed_subnets src 172.16.0.0/16 172.17.20.0/24 172.17.30.0/24 172.17.40.0/24 172.17.50.0/24 172.17.60.0/24 172.17.70.0/24 172.17.80.0/24
    http_access allow manager localhost
     
    http_access deny manager
    http_access allow purge localhost
    http_access deny purge
    http_access deny !safeports
    http_access deny CONNECT !sslports

    Always allow localhost connections

    http_access allow localhost

    quick_abort_min -1 KB
    quick_abort_max 0 KB
    request_body_max_size 0 KB
    reply_body_max_size 4194304 KB allsrc
    delay_pools 1
    delay_class 1 2
    delay_parameters 1 -1/-1 -1/-1
    delay_initial_bucket_level 100

    Throttle extensions matched in the url

    acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
    delay_access 1 allow throttle_exts
    delay_access 1 deny allsrc

    Reverse Proxy settings

    http_port xx.xx.xx:80 accel defaultsite=vibe.domain.ext vhost
    https_port xx.xx.xx.xx:443 accel cert=/usr/local/etc/squid/4fe475cabe502.crt key=/usr/local/etc/squid/4fe475cabe502.key defaultsite=vibe.domain.ext
    http_port xx.xx.xx.xx:80 accel defaultsite=vibe.domain.ext vhost
    https_port xx.xx.xx.xx:443 accel cert=/usr/local/etc/squid/4fe475cabe502.crt key=/usr/local/etc/squid/4fe475cabe502.key defaultsite=vibe.domain.ext

    cache_peer 172.16.2.48 parent 8080 0 proxy-only no-query originserver login=PASS name=HOST1

    cache_peer 172.16.2.48 parent 8443 0 proxy-only no-query originserver login=PASS ssl sslflags=DONT_VERIFY_PEER front-end-https=auto name=HOST2

    acl URI1 url_regex -i http://vibe.domain.ext/.$
    acl URI2 url_regex -i https://vibe.domain.ext/.
    $
    cache_peer_access HOST1 allow URI1
    cache_peer_access HOST2 allow URI2
    cache_peer_access HOST1 deny allsrc
    cache_peer_access HOST2 deny allsrc
    never_direct allow URI1
    never_direct allow URI2
    http_access allow URI1
    http_access allow URI2

    deny_info TCP_RESET allsrc

    Custom options

    Setup allowed acls

    Allow local network(s) on interface(s)

    http_access allow allowed_subnets
    http_access allow localnet

    Default block all to be sure

    http_access deny allsrc



  • 2012/06/27 19:32:11| commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
    2012/06/27 19:32:11| commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use

    Did you changed pfsense gui to other port then 80 and 443 at system -> advanced?



  • That is correct. I have set it on port 85.

    The 4 lines are the same external ip address but FD differents.
    What does FD means?

    Accepting  accelerated HTTP connections at xx.xx.xx.xx:80, FD 21.
    commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
    Accepting HTTPS connections at xx.xx.xx.xx:443, FD 23.
    commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use



  • When i use only the http reverse mode the Novell Vibe site is stable reachable.

    As soon as i also use the https reverse mode the connection wil become unstable. This is also when i only use the https reverse mode. The site wil become unstable to connect.

    Do you have an other idea that i can check?



  • Has anyone else an idea? :'(



  • Solved! With the new package marcelloc made and the right regex its works good. Thanx! :D



  • @klokslag:

    Solved! With the new package marcelloc made and the right regex its works good. Thanx! :D

    Thanks for the feedback klokslag. Did it worked with both protocols(http and https)?

    att,
    Marcello Coutinho



  • It works with both protocols. I'm trying now an second url to work with https. But haven't succeeded yet. Because it still redirect to THE default/first site. Http works fine.

    Do you have an idea.



  • @klokslag:

    Do you have an idea.

    Do you see any erros/ missing config on squid.conf file?

    Just a note that may help:
    I'm finishing a new package gui version for apache+mod_security, I think this new package will do a much better job on http/https reverse proxy.
    I've tried to finish it last week but there is still missing features to include.

    att,
    Marcello Coutinho



  • I do not see any errors in the cache logn the or squid.conf.
    I think it has to do with the default site with the https protocol.
    When i put there the second url in it, that url works too but not the first.

    But if you think apche+mod_security will work better? I will wait when you finished it and will test it.
    Let me now when your finished it.

    Regards.


Locked