Squid3 - New GUI with sync, normal and reverse proxy

ccesario · Apr 17, 2012, 11:54 AM

So I am using squid2.7 and squidguard here on work and posting many times on the forum and there is not that "bug". Perhaps some parameters on squid3 which causes this problems. Perhaps POST HEADER size or something like that.

Do you have an URL where we can "spam" posts to test this ? Probably it is not the best to do with pfsense forum ;o)

Hehehehh no, I don't have URL to can "spam" posts. But using pfsense forum its possible.

Edit your posts and save-it :) … I my tests I usage this to reproduce many times the error :)

Thanks

Nachtfalke · Apr 17, 2012, 8:17 PM

@ccesario:

@Nachtfalke:

So I am using squid2.7 and squidguard here on work and posting many times on the forum and there is not that "bug". Perhaps some parameters on squid3 which causes this problems. Perhaps POST HEADER size or something like that.

Do you have an URL where we can "spam" posts to test this ? Probably it is not the best to do with pfsense forum ;o)

Hehehehh no, I don't have URL to can "spam" posts. But using pfsense forum its possible.

Edit your posts and save-it :) … I my tests I usage this to reproduce many times the error :)

Thanks

Will do this perhaps this afternoon/night when I am at home. Perhaps we can tweak something if it's not a bug. :-)

marcelloc · Apr 17, 2012, 1:11 PM

@Nachtfalke:

Will do this perhaps this afternoon/night when I am at home. Perhaps we can tweak something if it's not a bug. :-)

Maybe a compile option like 'Be strictly HTTP compliant'

ccesario · Apr 17, 2012, 7:23 PM

@Nachtfalke:

@ccesario:

@Nachtfalke:

So I am using squid2.7 and squidguard here on work and posting many times on the forum and there is not that "bug". Perhaps some parameters on squid3 which causes this problems. Perhaps POST HEADER size or something like that.

Do you have an URL where we can "spam" posts to test this ? Probably it is not the best to do with pfsense forum ;o)

Hehehehh no, I don't have URL to can "spam" posts. But using pfsense forum its possible.

Edit your posts and save-it :) … I my tests I usage this to reproduce many times the error :)

Thanks

Will do this perhaps this afternoon/night when I am at home. Perhaps we can tweak something if it's not a bug. :-)

Thanks…. if possible report your tests!

Nachtfalke · Apr 17, 2012, 8:23 PM

Hmm, no luck till now.

tried with different browsers (IE8 and FF11)
tried with (re)moving some refresh_patterns
tried with different DNS servers for squid (8.8.8.8 and 127.0.0.1)
added this to custom options to get more information from access.log

strip_query_terms off

This is a difference I found on squid access.log


540 192.168.0.112 TCP_MISS/302 601 POST http://forum.pfsense.org/index.php?action=post2;start=45;msg=255851;sesc=b98e34206a1c8d9eb69521c441186ad3;board=15 - DIRECT/69.64.6.7 text/html
71 192.168.0.112 TCP_MISS/503 5000 POST http://forum.pfsense.org/index.php?action=post2;start=45;msg=255851;sesc=b98e34206a1c8d9eb69521c441186ad3;board=15 - DIRECT/forum.pfsense.org text/html

The 503 line uses DNS and the 302 uses an IP address…

If I click on the URL posted in the error page I get returned to the post edit page and got an error message from pfsense forum:


Your session timed out while posting. Please try to re-submit your message. No subject was filled in. The message body was left empty.

hmmm…

canefield · Apr 17, 2012, 8:33 PM

Marcello and others,

I've still got problems configuring Squid 3 as a reverse proxy. Somehow I can't manage it to work properly.
As you illustrated in the forst postings I did exactly the same and added NAT and Firewall rules. I'm using port 8080 and 8443.

How come…?!?!

Thanks a lot,
Canefield

ccesario · Apr 17, 2012, 8:35 PM

@Nachtfalke:

This is a difference I found on squid access.log


540 192.168.0.112 TCP_MISS/302 601 POST http://forum.pfsense.org/index.php?action=post2;start=45;msg=255851;sesc=b98e34206a1c8d9eb69521c441186ad3;board=15 - DIRECT/69.64.6.7 text/html
71 192.168.0.112 TCP_MISS/503 5000 POST http://forum.pfsense.org/index.php?action=post2;start=45;msg=255851;sesc=b98e34206a1c8d9eb69521c441186ad3;board=15 - DIRECT/forum.pfsense.org text/html

The 503 line uses DNS and the 302 uses an IP address…

Hmmmm this can be a hint.. O_o

marcelloc · Apr 18, 2012, 2:55 AM

@canefield:

Marcello and others,

I've still got problems configuring Squid 3 as a reverse proxy. Somehow I can't manage it to work properly.
As you illustrated in the forst postings I did exactly the same and added NAT and Firewall rules. I'm using port 8080 and 8443.

How come…?!?!

Thanks a lot,
Canefield

Still the same issue from TMG post? Did you removed the Nat?

Donny · Apr 18, 2012, 8:01 AM

Another bugs found on system log when I use revers proxy.

Apr 18 08:41:51
php: : The command '/usr/local/sbin/squid -k reconfigure' returned exit code '1', the output was '2012/04/18 08:41:51| redreshAddToList: Unknown option 'my.windowsupdate.website.com/..(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip)': reload-into-imsrange_offset_limit 2012/04/18 08:41:51| redreshAddToList: Unknown option 'my.windowsupdate.website.com/..(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip)': -1 2012/04/18 08:41:51| redreshAddToList: Unknown option 'symantecliveupdate.com/..(cab|exe|dll|msi)': reload-into-imsrange_offset_limit 2012/04/18 08:41:51| redreshAddToList: Unknown option 'symantecliveupdate.com/..(cab|exe|dll|msi)': -1 2012/04/18 08:41:51| redreshAddToList: Unknown option 'avast.com/..(vpu|cab|stamp|exe)': reload-into-imscache_mem 2012/04/18 08:41:51| redreshAddToList: Unknown option 'avast.com/..(vpu|cab|stamp|exe)': 1024 2012/04/18 08:41:51| redreshAddToList: Unknown option 'avast.com/.*.(vpu|cab|stamp|exe)': MB 2012/04/18 08:41:51| Warning: empty ACL: acl throttle_exts url

Solved! I found this problem because at dansquardian has banned "extension files". After I disable banned at extension tab, the error has gone.

marcelloc · Apr 18, 2012, 1:00 PM

Donny,
Check if does not happen if you uncheck dynamic content options on squid cache tab.

Donny · Apr 18, 2012, 6:43 PM

@marcelloc:

Donny,
Check if does not happen if you uncheck dynamic content options on squid cache tab.

Now, If I check or uncheck dynamic content options. The error is disappear.

al_reidy · Apr 18, 2012, 1:56 PM

i can't seem to get this package to cache files at all… no errors and whatismyip.com detects the proxy is working just it always downloads files without looking at the cache store... anyone else got this problem?

marcelloc · Apr 18, 2012, 3:38 PM

@al_reidy:

i can't seem to get this package to cache files at all… no errors and whatismyip.com detects the proxy is working just it always downloads files without looking at the cache store... anyone else got this problem?

If you know how to handle squid.conf files, can you check if your squid.conf file is ok?

al_reidy · Apr 18, 2012, 3:50 PM

@marcelloc:

@al_reidy:

i can't seem to get this package to cache files at all… no errors and whatismyip.com detects the proxy is working just it always downloads files without looking at the cache store... anyone else got this problem?

If you know how to handle squid.conf files, can you check if your squid.conf file is ok?


# This file is automatically generated by pfSense
# Do not edit manually !
http_port 192.168.168.150:3128
http_port 127.0.0.1:3128 intercept
icp_port 0

pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_directory /usr/local/etc/squid/errors/en
icon_directory /usr/local/etc/squid/icons
visible_hostname hostname.org
cache_mgr user@domain.org
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
sslcrtd_children 0
logfile_rotate 1
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  192.168.168.0/24
forwarded_for off
uri_whitespace strip

# Break HTTP standard for flash videos. Keep them in cache even if asked not to.
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

# Let the clients favorite video site through with full caching
acl youtube dstdomain .youtube.com
cache allow youtube
cache_mem 1024 MB
maximum_object_size_in_memory 5000 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir diskd /var/squid/cache 429000 16 256
minimum_object_size 0 KB
maximum_object_size 5242880 KB
offline_mode offcache_swap_low 90
cache_swap_high 95
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:		1440	20%	10080
refresh_pattern ^gopher:	1440	0%	1440
refresh_pattern -i (/cgi-bin/|\?) 0	0%	0
refresh_pattern .		0	20%	4320
# No redirector configured

# Setup some default acls
acl allsrc src all
acl localhost src 127.0.0.1/32
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901  3128 1025-65535 
acl sslports port 443 563  
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT

http_access allow manager localhost

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

quick_abort_min 0 KB
quick_abort_max 0 KB
request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
# Throttle extensions matched in the url
acl throttle_exts urlpath_regex -i "/var/squid/acl/throttle_exts.acl"
delay_access 1 allow throttle_exts
delay_access 1 deny allsrc

# Reverse Proxy settings

deny_info TCP_RESET allsrc

# Package Integration

# Custom options

# Setup allowed acls
# Allow local network(s) on interface(s)
http_access allow localnet
# Default block all to be sure
http_access deny allsrc

marcelloc · Apr 18, 2012, 3:57 PM

The cache info is there…

Can you grep for TCP_CACHE your squid access_log file?

al_reidy · Apr 18, 2012, 4:07 PM

@marcelloc:

The cache info is there…

Can you grep for TCP_CACHE your squid access_log file?

returns nothing back :'(

it does have TCP_MISS…. / DIRECT every line...

Pahtzo · Apr 19, 2012, 9:06 PM

Squid 3.1.19 pkg 2.0.3
2.0.1-RELEASE (amd64)
built on Mon Dec 12 18:43:51 EST 2011
FreeBSD 8.1-RELEASE-p6

Entering PEM intermediate CA certificate in the Reverse Proxy General screen: "intermediate CA certificate (if needed)" field.

Receiving the following error:

Fatal error: Call to undefined function sq_text_area_decodedecode() in /usr/local/pkg/squid_reverse.inc on line 61

Thank you

marcelloc · Apr 19, 2012, 9:20 PM

@Pahtzo:

Fatal error: Call to undefined function sq_text_area_decodedecode() in /usr/local/pkg/squid_reverse.inc on line 61

typo, I'll fix it. :)

marcelloc · Apr 20, 2012, 5:29 AM

Version 2.0.4 is out with

bug and typo fixes
Upstream tab is now remote cache to enable multiple peer/sibling cache config
New compilation to avoid refresh_pattern and transparent mode errors

Thanks for all feedback specially to ccesario.

I recommend a package uninstall/install instead of reinstall to be sure old squid3 binaries are removed.

att,
Marcello Coutinho

al_reidy · Apr 20, 2012, 7:54 AM

@marcelloc:

Version 2.0.4 is out with

bug and typo fixes

Upstream tab is now remote cache to enable multiple peer/sibling cache config

New compilation to avoid refresh_pattern and transparent mode errors

Thanks for all feedback specially to ccesario.

I recommend a package uninstall/install instead of reinstall to be sure old squid3 binaries are removed.

att,
Marcello Coutinho

i appreciate all your time on this, it is however broken still. i have done a fresh install also…

2012/04/20 07:51:25| WARNING: dnsserver #1 (FD 11) exited
2012/04/20 07:51:25| ipcacheParse: Got <null>reply
2012/04/20 07:51:25| WARNING: dnsserver #2 (FD 13) exited
2012/04/20 07:51:25| ipcacheParse: Got <null>reply
2012/04/20 07:51:25| WARNING: dnsserver #3 (FD 15) exited
2012/04/20 07:51:25| ipcacheParse: Got <null>reply
2012/04/20 07:51:25| WARNING: dnsserver #4 (FD 17) exited
2012/04/20 07:51:25| Too few dnsserver processes are running
2012/04/20 07:51:25| storeDirWriteCleanLogs: Starting...
2012/04/20 07:51:25|   Finished.  Wrote 222 entries.
2012/04/20 07:51:25|   Took 0.01 seconds (29264.43 entries/sec).
FATAL: The dnsserver helpers are crashing too rapidly, need help!

Squid Cache (Version 3.1.19): Terminated abnormally.
CPU Usage: 0.242 seconds = 0.195 user + 0.047 sys
Maximum Resident Size: 22836 KB
Page faults with physical i/o: 0</null></null></null>

transparent mode : which leaves nothing able to get DNS :'(