Reverse squid proxy does not show always tomcat webpage

klokslag

Hi,

I use wonna use reverse squid proxy before our Novell Vibe server.
Vibe is a Tomcat Application that uses internal 8080 and 8443.

The config now is:

vibe.domain.ext –> 80 --> squid --> 8080 --> Vibe server
vibe.domain.ext --> 443 --> squid --> 8443 --> Vibe server

With the ssl config i use an cert on the squid en one on Vibe.
I made two webservers an two mappings in the config.

The problem is that the Vibe site does not always show up. Its not stable. Its hanging or very very slow.
I also use Groupwise and that goes right or just a simple site. I also set the firewall open, but no results

Who can help me with this problem or what do i wrong. Is there a log that i can read except the realtime log?

Pleas help!

marcelloc

check squid access.log and cache.log

klokslag

Thanks for your reply.

In the access log the http site does get logged correct one time.
The second time nothing is logged. When i call the https one line get logged with a time out i gues.

1340818529.227 3 82.168.159.164 TCP_MISS/302 291 GET http://vibe.domain.ext/ - FIRST_UP_PARENT/HOST1 -
1340818529.289 7 82.168.159.164 TCP_MISS/302 536 GET http://vibe.domain.ext/ssf/a/ - FIRST_UP_PARENT/HOST1 -
1340818529.365 25 82.168.159.164 TCP_MISS/200 30574 GET http://vibe.domain.ext/ssf/a/c/p_name/ss_forum/p_action/1/action/view_permalink/entityType/user/entryId/ss_user_id_place_holder/showWhatsNew/1/vibeonprem_url/1/vibeonprem_root/1 - FIRST_UP_PARENT/HOST1 text/html
1340818529.940 3 82.168.159.164 TCP_MISS/200 6416 GET http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtteaming.nocache.js? - FIRST_UP_PARENT/HOST1 text/javascript
1340818530.229 1 82.168.159.164 TCP_MISS/404 1621 GET http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/tiny_mce_3_2_7/tiny_mce/plugins/preelementfix/langs/nl.js? - FIRST_UP_PARENT/HOST1 text/html
1340818531.125 60 82.168.159.164 TCP_MISS/200 727 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.233 10 82.168.159.164 TCP_MISS/200 564 GET http://vibe.domain.ext/ssf/a/do? - FIRST_UP_PARENT/HOST1 text/html
1340818531.267 51 82.168.159.164 TCP_MISS/200 646 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.346 68 82.168.159.164 TCP_MISS/200 629 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.472 57 82.168.159.164 TCP_MISS/200 620 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.475 49 82.168.159.164 TCP_MISS/200 646 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.501 66 82.168.159.164 TCP_MISS/200 871 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.688 53 82.168.159.164 TCP_MISS/200 642 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.698 56 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.703 53 82.168.159.164 TCP_MISS/200 657 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.813 58 82.168.159.164 TCP_MISS/200 669 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.843 58 82.168.159.164 TCP_MISS/200 626 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818531.955 51 82.168.159.164 TCP_MISS/200 605 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818591.828 71 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818651.783 63 82.168.159.164 TCP_MISS/200 612 POST http://vibe.domain.ext/ssf/static/02-11-2011-0137/js/gwt/gwtteaming/gwtTeaming.rpc - FIRST_UP_PARENT/HOST1 application/json
1340818726.244 59270 82.168.159.164 TCP_MISS/000 0 GET https://vibe.domain.ext/ - FIRST_UP_PARENT/HOST2 -

In the access log i get two errors that FD22 an FD24 already in use is.
I think that the adresses are defined two time?

2012/06/27 19:32:11| Starting Squid Cache version 3.1.20 for amd64-portbld-freebsd8.1…
2012/06/27 19:32:11| Process ID 2482
2012/06/27 19:32:11| With 11095 file descriptors available
2012/06/27 19:32:11| Initializing IP Cache...
2012/06/27 19:32:11| DNS Socket created at [::], FD 11
2012/06/27 19:32:11| DNS Socket created at 0.0.0.0, FD 12
2012/06/27 19:32:11| Adding domain annamaria.nl from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 127.0.0.1 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 194.151.228.18 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 172.16.2.22 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 172.16.2.20 from /etc/resolv.conf
2012/06/27 19:32:11| Adding nameserver 194.151.228.34 from /etc/resolv.conf
2012/06/27 19:32:11| User-Agent logging is disabled.
2012/06/27 19:32:11| Referer logging is disabled.
2012/06/27 19:32:11| Unlinkd pipe opened on FD 17
2012/06/27 19:32:11| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2012/06/27 19:32:11| Store logging disabled
2012/06/27 19:32:11| Swap maxSize 102400 + 8192 KB, estimated 8507 objects
2012/06/27 19:32:11| Target number of buckets: 425
2012/06/27 19:32:11| Using 8192 Store buckets
2012/06/27 19:32:11| Max Mem size: 8192 KB
2012/06/27 19:32:11| Max Swap size: 102400 KB
2012/06/27 19:32:11| Version 1 of swap file with LFS support detected…
2012/06/27 19:32:11| Rebuilding storage in /var/squid/cache (DIRTY)
2012/06/27 19:32:11| Using Least Load store dir selection
2012/06/27 19:32:11| Current Directory is /etc
2012/06/27 19:32:11| Loaded Icons.
2012/06/27 19:32:11| helperOpenServers: Starting 0/0 'ssl_crtd' processes
2012/06/27 19:32:11| helperOpenServers: No 'ssl_crtd' processes needed.
2012/06/27 19:32:11| Accepting HTTP connections at 172.16.0.23:8090, FD 20.
2012/06/27 19:32:11| Accepting accelerated HTTP connections at xx.xx.xx.xx:80, FD 21.
2012/06/27 19:32:11| commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
2012/06/27 19:32:11| Accepting HTTPS connections at xx.xx.xx.xx:443, FD 23.
2012/06/27 19:32:11| commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use
2012/06/27 19:32:11| Accepting ICP messages at [::]:7, FD 25.
2012/06/27 19:32:11| HTCP Disabled.
2012/06/27 19:32:11| Configuring Parent 172.16.2.48/8080/0
2012/06/27 19:32:11| Configuring Parent 172.16.2.48/8443/0
2012/06/27 19:32:11| Ready to serve requests.
2012/06/27 19:32:11| Done reading /var/squid/cache swaplog (263 entries)
2012/06/27 19:32:11| Finished rebuilding storage from disk.
2012/06/27 19:32:11| 263 Entries scanned
2012/06/27 19:32:11| 0 Invalid entries.
2012/06/27 19:32:11| 0 With invalid flags.
2012/06/27 19:32:11| 263 Objects loaded.
2012/06/27 19:32:11| 0 Objects expired.
2012/06/27 19:32:11| 0 Objects cancelled.
2012/06/27 19:32:11| 0 Duplicate URLs purged.
2012/06/27 19:32:11| 0 Swapfile clashes avoided.
2012/06/27 19:32:11| Took 0.03 seconds (9953.83 objects/sec).
2012/06/27 19:32:11| Beginning Validation Procedure
2012/06/27 19:32:11| Completed Validation Procedure
2012/06/27 19:32:11| Validated 551 Entries
2012/06/27 19:32:11| store_swap_size = 742
2012/06/27 19:32:12| storeLateRelease: released 0 objects

So i still cannot find what i'am doing wrong. Can you help me more?
This is my squid config:

This file is automatically generated by pfSense

Do not edit manually !

http_port 172.16.0.23:8090
icp_port 7

pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language nl
icon_directory /usr/local/etc/squid/icons
visible_hostname localhost
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
sslcrtd_children 0
logfile_rotate 14
shutdown_lifetime 3 seconds

Allow local network(s) on interface(s)

acl localnet src 172.16.0.0/16
uri_whitespace strip

Break HTTP standard for flash videos. Keep them in cache even if asked not to.

refresh_pattern -i .flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

Let the clients favorite video site through with full caching

acl youtube dstdomain .youtube.com
cache allow youtube
cache_mem 8 MB
maximum_object_size_in_memory 32 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir ufs /var/squid/cache 100 16 256
minimum_object_size 0 KB
maximum_object_size 4 KB
offline_mode offcache_swap_low 90
cache_swap_high 95
acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
cache deny donotcache

Add any of your own refresh_pattern entries above these.

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|?) 0 0% 0
refresh_pattern . 0 20% 4320

No redirector configured

#Remote proxies

Setup some default acls

acl allsrc src all
acl localhost src 127.0.0.1/32
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 85 3128 1025-65535
acl sslports port 443 563 85
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT

acl allowed_subnets src 172.16.0.0/16 172.17.20.0/24 172.17.30.0/24 172.17.40.0/24 172.17.50.0/24 172.17.60.0/24 172.17.70.0/24 172.17.80.0/24
http_access allow manager localhost

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

Always allow localhost connections

http_access allow localhost

quick_abort_min -1 KB
quick_abort_max 0 KB
request_body_max_size 0 KB
reply_body_max_size 4194304 KB allsrc
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100

Throttle extensions matched in the url

acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
delay_access 1 allow throttle_exts
delay_access 1 deny allsrc

Reverse Proxy settings

http_port xx.xx.xx:80 accel defaultsite=vibe.domain.ext vhost
https_port xx.xx.xx.xx:443 accel cert=/usr/local/etc/squid/4fe475cabe502.crt key=/usr/local/etc/squid/4fe475cabe502.key defaultsite=vibe.domain.ext
http_port xx.xx.xx.xx:80 accel defaultsite=vibe.domain.ext vhost
https_port xx.xx.xx.xx:443 accel cert=/usr/local/etc/squid/4fe475cabe502.crt key=/usr/local/etc/squid/4fe475cabe502.key defaultsite=vibe.domain.ext

cache_peer 172.16.2.48 parent 8080 0 proxy-only no-query originserver login=PASS name=HOST1

cache_peer 172.16.2.48 parent 8443 0 proxy-only no-query originserver login=PASS ssl sslflags=DONT_VERIFY_PEER front-end-https=auto name=HOST2

acl URI1 url_regex -i http://vibe.domain.ext/.$
acl URI2 url_regex -i https://vibe.domain.ext/.$
cache_peer_access HOST1 allow URI1
cache_peer_access HOST2 allow URI2
cache_peer_access HOST1 deny allsrc
cache_peer_access HOST2 deny allsrc
never_direct allow URI1
never_direct allow URI2
http_access allow URI1
http_access allow URI2

deny_info TCP_RESET allsrc

Custom options

Setup allowed acls

Allow local network(s) on interface(s)

http_access allow allowed_subnets
http_access allow localnet

Default block all to be sure

http_access deny allsrc

marcelloc

2012/06/27 19:32:11| commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
2012/06/27 19:32:11| commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use

Did you changed pfsense gui to other port then 80 and 443 at system -> advanced?

klokslag

That is correct. I have set it on port 85.

The 4 lines are the same external ip address but FD differents.
What does FD means?

Accepting accelerated HTTP connections at xx.xx.xx.xx:80, FD 21.
commBind: Cannot bind socket FD 22 to xx.xx.xx.xx:80: (48) Address already in use
Accepting HTTPS connections at xx.xx.xx.xx:443, FD 23.
commBind: Cannot bind socket FD 24 to xx.xx.xx.xx:443: (48) Address already in use

klokslag

When i use only the http reverse mode the Novell Vibe site is stable reachable.

As soon as i also use the https reverse mode the connection wil become unstable. This is also when i only use the https reverse mode. The site wil become unstable to connect.

Do you have an other idea that i can check?

klokslag

Has anyone else an idea? :'(

klokslag

Solved! With the new package marcelloc made and the right regex its works good. Thanx! :D

marcelloc

@klokslag:

Solved! With the new package marcelloc made and the right regex its works good. Thanx! :D

Thanks for the feedback klokslag. Did it worked with both protocols(http and https)?

att,
Marcello Coutinho

klokslag

It works with both protocols. I'm trying now an second url to work with https. But haven't succeeded yet. Because it still redirect to THE default/first site. Http works fine.

Do you have an idea.

marcelloc

@klokslag:

Do you have an idea.

Do you see any erros/ missing config on squid.conf file?

Just a note that may help:
I'm finishing a new package gui version for apache+mod_security, I think this new package will do a much better job on http/https reverse proxy.
I've tried to finish it last week but there is still missing features to include.

att,
Marcello Coutinho

klokslag

I do not see any errors in the cache logn the or squid.conf.
I think it has to do with the default site with the https protocol.
When i put there the second url in it, that url works too but not the first.

But if you think apche+mod_security will work better? I will wait when you finished it and will test it.
Let me now when your finished it.

Regards.