Cant get squid to work properly.

[ddemuro]

Sure,
On the logs, appears squid started properly, loaded 72 objects in the folders, yet when I check the folders there only is 72 cached items.

The actual config is:
Proxy interface: LAN
Proxy port: 3128
ICP port: none
Allow users on interface: checked
Transparent proxy: checked
Enable logging: check
Log rotate: 10 days
Visible hostname: localhost
Supress Squid Version: checked.
–-------------------------------------
Local cache:
Cache replacement policy: Heap LFUDA
Low watermark: 90
High watermark: 95
External cache managers: 127.0.0.1;192.168.1.80;
Hard disk cache size: 55000
Hard disk cache system: diskd
Level1 subdirectories: 128
Hard disk location: /var/squid/cache (tried to put it in home)
Minimum object size: 0
Maximum object size: 400000
Memory cache size: 450
Maximum object size in ram: 128
Memory replacement policy: Heap GDSF
Cache dynamic content: checked
Refresh patterns: Youtube, Windows Update, Symantec Antivirus.

Assume that I did not post default values.

Thank you in advance.

[ddemuro]

Im wondering we're not the only ones.

I did a simple cache script when rebuilding a static page from php. Using a variable serial printed on pages. And it constantly changes plus I can see the WAN card blowing up when doing connection tests.

The weird thing is those 72 objects in the cache and not being able to cache much.

Im wondering if other people are having the same problems. I picked pfsense because of the great distro itself and the caching + queues.

Bummer I cant get squid to actually work as it should. Now im trying a clean setup on a virtualbox machine on a brand new Ubuntu Server install.

I'll keep this updated, it didnt work with squid also.

Squid or Squid3 no difference in the bottom line.

[ddemuro]

Yes, I second that.

Hehe yep, something similar, to that website, including images and such. Actually I'm not being able to cache anything. When I put a video from youtube for example, that I recently watched, I still see the WAN going nuts.

Those 72 cache items are  :-\ who knows.

I tried changing sizes, and stuff, more ram less ram, more disk, less disk, changing cache location and such no effect. Running pfsense natively no change, going without transparent proxy… No effect...

I dont know whats going on actually, if I get to solve it I'll post it here. But still no effect. hehe this is driving me crazy. At least u get the same serial. I don't.

[ddemuro]

Following by, with the config I posted the logical stuff to happen is cache going nuts as I try to reload and cache everything. Yet disk space hasnt changed in HOURS.

[2.0.1-RELEASE][admin@takelan.router.takelan.com]/root(1): df -h
Filesystem    Size    Used  Avail Capacity  Mounted on
/dev/ad0s1a    61G    330M    56G    1%    /
devfs          1.0K    1.0K      0B  100%    /dev
/dev/md0      3.6M    52K    3.3M    2%    /var/run
devfs          1.0K    1.0K      0B  100%    /var/dhcpd/dev

[marcelloc]

Check created squid conf with a made by hand config and see If you find a misconfigured option.

YouTube and many sites are dynamic, the same watch video URL cand send you to a different server but I agree with you that no cache hit is not normal

Similar problem solved on this post
http://forum.pfsense.org/index.php/topic,54182.msg289826.html#msg289826

[ddemuro]

@marcelloc:

Check created squid conf with a made by hand config and see If you find a misconfigured option.

YouTube and many sites are dynamic, the same watch video URL cand send you to a different server but I agree with you that no cache hit is not normal

Similar problem solved on this post
http://forum.pfsense.org/index.php/topic,54182.msg289826.html#msg289826

Marcelloc just got it working, I had ticked "cache dynamic content" I unchecked it and now its working.  ::) Seems like the directive cache dynamic content has something to do. I'll check that Im running the stable release 2.0.1

[marcelloc]

Thanks for the feedback,  I 'll check it soon.

[ddemuro]

I'll post as soon as I have news.

[ddemuro]

The lines added seem correct according to Squid Doc:

This file is automatically generated by pfSense

Do not edit manually !

http_port 192.168.1.80:3128
http_port 127.0.0.1:3128 intercept
icp_port 7
dns_v4_first on
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language es
icon_directory /usr/local/etc/squid/icons
visible_hostname localhost
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
sslcrtd_children 0
logfile_rotate 10
shutdown_lifetime 3 seconds

Allow local network(s) on interface(s)

acl localnet src  192.168.1.0/24
httpd_suppress_version_string on
uri_whitespace strip

acl dynamic urlpath_regex cgi-bin ?
cache deny dynamic
cache_mem 250 MB
maximum_object_size_in_memory 960 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir aufs /var/squid/cache 42680 64 256
minimum_object_size 0 KB
maximum_object_size 393216 KB
offline_mode oncache_swap_low 90
cache_swap_high 95

No redirector configured

#Remote proxies

Setup some default acls

acl allsrc src all
acl localhost src 127.0.0.1/32
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 808 3128 1025-65535
acl sslports port 443 563 808
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT

http_access allow manager localhost
 
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

Always allow localhost connections

http_access allow localhost

quick_abort_min 0 KB
quick_abort_max 0 KB
request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow allsrc

Reverse Proxy settings

Custom options

Setup allowed acls

Allow local network(s) on interface(s)

http_access allow localnet

Default block all to be sure

http_access deny allsrc

This file is automatically generated by pfSense

Do not edit manually !

http_port 192.168.1.80:3128
http_port 127.0.0.1:3128 intercept
icp_port 7
dns_v4_first on
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language es
icon_directory /usr/local/etc/squid/icons
visible_hostname localhost
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
sslcrtd_children 0
logfile_rotate 10
shutdown_lifetime 3 seconds

Allow local network(s) on interface(s)

acl localnet src  192.168.1.0/24
httpd_suppress_version_string on
uri_whitespace strip

Break HTTP standard for flash videos. Keep them in cache even if asked not to.

refresh_pattern -i .flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

Let the clients favorite video site through with full caching

acl youtube dstdomain .youtube.com
cache allow youtube

Windows Update refresh_pattern

range_offset_limit -1
refresh_pattern -i microsoft.com/..(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i my.windowsupdate.website.com/.*.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

Symantec refresh_pattern

range_offset_limit -1
refresh_pattern liveupdate.symantecliveupdate.com/..(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims
refresh_pattern symantecliveupdate.com/..(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims

Avast refresh_pattern

range_offset_limit -1
refresh_pattern avast.com/.*.(vpu|cab|stamp|exe) 10080 100% 43200 reload-into-ims

Avira refresh_pattern

range_offset_limit -1
refresh_pattern personal.avira-update.com/.*.(cab|exe|dll|msi|gz) 10080 100% 43200 reload-into-ims
cache_mem 250 MB
maximum_object_size_in_memory 960 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir aufs /var/squid/cache 42680 64 256
minimum_object_size 0 KB
maximum_object_size 393216 KB
offline_mode oncache_swap_low 90
cache_swap_high 95

Add any of your own refresh_pattern entries above these.

refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|?) 0  0%  0
refresh_pattern .    0  20%  4320

No redirector configured

#Remote proxies

Setup some default acls

acl allsrc src all
acl localhost src 127.0.0.1/32
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 808 3128 1025-65535
acl sslports port 443 563 808
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT

http_access allow manager localhost
 
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

Always allow localhost connections

http_access allow localhost

quick_abort_min 0 KB
quick_abort_max 0 KB
request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow allsrc

Reverse Proxy settings

Custom options

Setup allowed acls

Allow local network(s) on interface(s)

http_access allow localnet

Default block all to be sure

http_access deny allsrc

Difference between the two configs:

Add any of your own refresh_pattern entries above these.

refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|?) 0  0%  0
refresh_pattern .    0  20%  4320

Break HTTP standard for flash videos. Keep them in cache even if asked not to.

refresh_pattern -i .flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

Let the clients favorite video site through with full caching

acl youtube dstdomain .youtube.com
cache allow youtube

Windows Update refresh_pattern

range_offset_limit -1
refresh_pattern -i microsoft.com/..(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/..(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i my.windowsupdate.website.com/.*.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

Symantec refresh_pattern

range_offset_limit -1
refresh_pattern liveupdate.symantecliveupdate.com/..(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims
refresh_pattern symantecliveupdate.com/..(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims

Avast refresh_pattern

range_offset_limit -1
refresh_pattern avast.com/.*.(vpu|cab|stamp|exe) 10080 100% 43200 reload-into-ims

Avira refresh_pattern

range_offset_limit -1
refresh_pattern personal.avira-update.com/.*.(cab|exe|dll|msi|gz) 10080 100% 43200 reload-into-ims

Ideas?

[ddemuro]

Nope I unchecked it and now its simply caching static stuff. There is no config I can make it cache dynamic as youtube, windows update or other.

Simply when I tick cache dynamic content, all I get are TCP MISS 200, if I uncheck it, I get hits for static content.