Squid3 - New GUI with sync, normal and reverse proxy

[edanpedragosa]

It seems that squid does not cache anything at all.

I only get TCP_MISS/200 in the access log.

Here's my current squid config settings:

# This file is automatically generated by pfSense
# Do not edit manually !

http_port 177.7.0.1:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=64MB cert=/usr/pbi/squid-amd64/etc/squid/serverkey.pem capath=/usr/pbi/squid-amd64/share/certs/

http_port 172.16.0.1:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=64MB cert=/usr/pbi/squid-amd64/etc/squid/serverkey.pem capath=/usr/pbi/squid-amd64/share/certs/

http_port 127.0.0.1:3128 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=64MB cert=/usr/pbi/squid-amd64/etc/squid/serverkey.pem capath=/usr/pbi/squid-amd64/share/certs/

https_port 127.0.0.1:3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=64MB cert=/usr/pbi/squid-amd64/etc/squid/serverkey.pem capath=/usr/pbi/squid-amd64/share/certs/

icp_port 0
dns_v4_first on
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language en
icon_directory /usr/pbi/squid-amd64/etc/squid/icons
visible_hostname AIMSNet_Gateway
cache_mgr edan@aims.ac.th
access_log /var/squid/log/access.log
cache_log /var/squid/log/cache.log
cache_store_log none
netdb_filename /var/squid/log/netdb.state
pinger_enable on
pinger_program /usr/pbi/squid-amd64/libexec/squid/pinger
sslcrtd_program /usr/pbi/squid-amd64/libexec/squid/ssl_crtd -s /var/squid/lib/ssl_db -M 4MB -b 2048
sslcrtd_children 32
sslproxy_capath /usr/pbi/squid-amd64/share/certs/

logfile_rotate 180
debug_options rotate=180
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  177.7.0.0/16 172.16.0.0/16
forwarded_for off
via off
httpd_suppress_version_string on
uri_whitespace encode

# Break HTTP standard for flash videos. Keep them in cache even if asked not to.
refresh_pattern -i .flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

# Let the clients favorite video site through with full caching
acl youtube dstdomain .youtube.com
cache allow youtube

# Windows Update refresh_pattern
range_offset_limit -1
refresh_pattern -i microsoft.com/.*.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/.*.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windows.com/.*.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims

# Symantec refresh_pattern
range_offset_limit -1
refresh_pattern liveupdate.symantecliveupdate.com/.*.(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims
refresh_pattern symantecliveupdate.com/.*.(cab|exe|dll|msi) 10080 100% 43200 reload-into-ims

# Avast refresh_pattern
range_offset_limit -1
refresh_pattern avast.com/.*.(vpu|cab|stamp|exe) 10080 100% 43200 reload-into-ims

# Avira refresh_pattern
range_offset_limit -1
refresh_pattern personal.avira-update.com/.*.(cab|exe|dll|msi|gz) 10080 100% 43200 reload-into-ims

cache_dir aufs /var/squid/cache 77777 256 256
cache_mem 15777 MB
maximum_object_size_in_memory 1024 KB
memory_replacement_policy heap LFUDA
cache_replacement_policy heap LFUDA
minimum_object_size 0 KB
maximum_object_size 1000000 KB
offline_mode on
cache_swap_low 90
cache_swap_high 95
cache allow all

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|?) 0  0%  0
refresh_pattern .    0  20%  4320

# No redirector configured

#Remote proxies

# Setup some default acls
# From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localhost ACL definitions are now built-in.
# acl localhost src 127.0.0.1/32
acl allsrc src all
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901  3128 3127 1025-65535 
acl sslports port 443 563  

# From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localhost ACL definitions are now built-in.
#acl manager proto cache_object

acl purge method PURGE
acl connect method CONNECT

# Define protocols used for redirects
acl HTTP proto HTTP
acl HTTPS proto HTTPS
http_access allow manager localhost

# Allow external cache managers
acl ext_manager src 172.16.0.1
acl ext_manager src 177.7.0.1
http_access allow manager ext_manager

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
# From 3.2 further configuration cleanups have been done to make things easier and safer. 
# The manager, localhost, and to_localhost ACL definitions are now built-in.
# http_access allow localhost

quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 70
request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow allsrc

# Reverse Proxy settings

# Package Integration
url_rewrite_program /usr/pbi/squidguard-squid3-amd64/bin/squidGuard -c /usr/pbi/squidguard-squid3-amd64/etc/squidGuard/squidGuard.conf
url_rewrite_bypass off
url_rewrite_children 5

# Custom options before auth
always_direct allow all
ssl_bump server-first all

# Setup allowed acls
# Allow local network(s) on interface(s)
http_access allow localnet
# Default block all to be sure
http_access deny allsrc

What could be wrong with it? Note that it was working fine with squid3-dev 3.3.10 pkg 2.2.6

Packages installed:
Cron
pfBlocker
Sarg
squid3-dev
squidGuard-squid3
System Patches

My hardware/software config is:

Dell PowerEdge T110 II
Intel Xeon processor E3-1200 V2 product family
32GB Server RAM
256GB SSD Drive

Version 2.1.5-RELEASE (amd64)
built on Mon Aug 25 07:44:45 EDT 2014
FreeBSD 8.3-RELEASE-p16

Platform pfSense
CPU Type
Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz
4 CPUs: 1 package(s) x 4 core(s)

DNS server(s)
208.67.222.222
208.67.220.220

State table size
0% (1367/1777777)

MBUF Usage
4% (6292/177778)

Temperature 29.8°C

Load average
0.20, 0.14, 0.15

CPU usage
0%

Memory usage
7% of 32716 MB

SWAP usage
0% of 65536 MB

Disk usage
1% of 169G

[finalcut]

i made update it work flawless ,, thanks for update

[sunghost]

Hello,
first you make a very good job. Pfsense as well all package developer. I look since weeks for a solution to use an webproxy with antivirus and contentfilter. First i want to use dansguardian, but the last version is from 2012 - very old. than i wanted to use havp, but this version is from 2010! and it looks like the latest security fix is not included in pfsense havp package. I use always stable version for my production environment so squid whould a good choice, but the version of the package is from 2008-2010! and the squid3 beta is also old 2012!

After reading lots of post here in this forum and the most of this over 23 pages in this thread i am not alight with a positiv feeling of using it. The package Info link of squid3 in pfsense leads also to this thread which is a bad overview for information to this package. In my eyes this part must created new with infos over the package and not to a thread with errors and other parts of an information to a package. So in short why are the packages old and what whould be the choice of using a webproxy with antivirus and content filter/security?
thx and this is no offense to anybody just my impression.

[finalcut]

then push $$$$upport to programmer for those updates
i try to send him donation for his great job but my country is blocked

[Tikimotel]

@finalcut:

then push $$$$upport to programmer for those updates
i try to send him donation for his great job but my country is blocked

My country wasn't blocked, so I donated today!

[finalcut]

i am not kidding
also it happen with pfsense i try to buy two stickers (as simple first  donation) and they send the money back to me

any way i try to encourage my company to get pfsense support , they can buy any thing from anywhere

[sunghost]

You are right, donation for the developer and the project is always good. Thats not the question. i think its not good to use package which are older than 1 or 2 or more years. or packages which seems not longer develop. you know? All things getting faster, the development the bad guys and also the features of some packages which we cant use, while our older than 1 year. i also think its not in any case good to use the latest version, but one which is older than 6 month?! not good if the features or fixes alot. i think a security solution must be up to date. But the product pfsense and the idea behind it is great. dont misunderstand me.

[edanpedragosa]

@finalcut:

i made update it work flawless ,, thanks for update

Are you on a 32-bit or 64-bit version?

Caching does not seem to work right now… I only get TCP_MISS....

Can you share us your configuration and settings for squid?

Thank you in advance!

[finalcut]

64-bit
do you have tcp_swapfail ?

[edanpedragosa]

No, I don't have TCP_SWAPFAIL_MISS, only TCP_MISS/200. What I'm waiting to see in the logs is the HIT but I don't see TCP_HIT.

[edanpedragosa]

Now I'm getting a HIT by setting the Minimum object size to 0.

It's just the large downloads that don't get cached.

Hope this will get sorted out again in the next update….

[Escorpiom]

Perhaps someone can help a confused n00b on PfSense 2.2 Beta…

I have at the moment Squid "3.3.11_1 pkg 2.2.8" installed, that was a dev package from some weeks ago.
But looking at the available packages list, there now is a "beta 3.4.9 pkg 0.1".

To me this seems to be an update, is this correct?
And is it possible to update from 3.3.11 dev to 3.4.9 beta? Should I uninstall the 3.3.11 dev package first?

Cheers.

[dig1234]

That would be very exciting news but I'm still only seeing  3.3.10 pkg 2.2.8.
I've actually been experimenting with compiling from source on a freebsd 8.3 machine so this would really save me some hassle.

@Escorpiom:

Perhaps someone can help a confused n00b on PfSense 2.2 Beta…

I have at the moment Squid "3.3.11_1 pkg 2.2.8" installed, that was a dev package from some weeks ago.
But looking at the available packages list, there now is a "beta 3.4.9 pkg 0.1".

To me this seems to be an update, is this correct?
And is it possible to update from 3.3.11 dev to 3.4.9 beta? Should I uninstall the 3.3.11 dev package first?

Cheers.

[Escorpiom]

Probably your not on 2.2 beta. The package wouldn't be compatible.

Cheers.

[dig1234]

ah indeed you are correct. That makes sense as 2.2 is using FreeBSD 10.1…

[Escorpiom]

Please can anybody outline the correct procedure and confirm Squid was updated for PfSense 2.2 beta?

Cheers.

[longhorn]

@edanpedragosa:

Now I'm getting a HIT by setting the Minimum object size to 0.

It's just the large downloads that don't get cached.

Hope this will get sorted out again in the next update….

It could also be how you utilize the proxy. It took me some time to figure out for many applications, Squid's caching function is useless. Unless you have a network where multiple computers hit the exact same content, within a period of time (while the data is still in the cache).

I use pfSense for my home network. Almost nothing is cached. However, I find Squid very useful for logging and filtering Web traffic.

YMMV, but just pointing out that seeing a lot of MISSes reported in Squid's log is most often WAD behavior (Working As Designed)

[Topper727]

I have it installed in 2.2RC as of 12/30's release.  Does not run.  Starts then stops then starts then stops and if I try to run through transparent I get no data through unless it is application that uses and IP instead of a domain name to get the connection.  Is there a problem with DNS ?I use DNS Resolver the new one they say.. The DNS Fordwarder didn't work for me so I use that now.. But all is well without Squid.

Why it don't stay running and keep restarting ?

2.2-RC (amd64)
built on Tue Dec 30 15:16:19 CST 2014
FreeBSD pfSense.localdomain 10.1-RELEASE-p3 FreeBSD 10.1-RELEASE-p3 #0 8bdb2f8(releng/10.1)-dirty: Tue Dec 30 15:58:58 CST 2014 root@pfsense-22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10 amd64

[marcelloc]

@Topper727:

Why it don't stay running and keep restarting ?

It looks like current pbi was build with debug options. Once we get the pull request mergerd and pbi build again with all used options, I think we get a 'ready to go' version

[Cino]

@marcelloc did you update the symbolic links for the install also? I recall I had to create a bunch to get 3.4 to start on a fresh 2.2RC install. I have to check my notes, but I believe I was able to get clamd to run but c-icap wouldn't start up.