Squid 3.3.4 package for pfsense with ssl filtering

Pankar

Hi,

I have read this thread and several topics/tutorials on the web but I still cant figure out whether Squid3 and especially 3.3.8-dev is able to cach dynamic content.

I understand that 3.3.8-dev has solved an issue on squid3 stable where enabling dynamic content wouldn't let anything to cache: I have enabled dynamic content on 3.3.8 and static content (pictures, etc) is caching correctly.

However, enabling dynamic content doesnt seem to serve its purpose: cache any URL that includes a ? (it returns a TCP_MISS/200)

This is my squid.conf. My main goal is to cache iOS updates and ibooks since I am a school IT Manager and there are 50+ identical ibook downloads per day through the iTunesU platform. The iBook URLS are like:

http://.phobos.apple.com///*/example.ibooks?downloadKey=12345678

# This file is automatically generated by pfSense
# Do not edit manually !

http_port 10.0.10.2:3128
http_port 127.0.0.1:3128 intercept
icp_port 7
dns_v4_first off
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language el
icon_directory /usr/pbi/squid-i386/etc/squid/icons
visible_hostname localhost
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none

logfile_rotate 30
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  10.0.0.0/16
httpd_suppress_version_string on
uri_whitespace strip

# Windows Update refresh_pattern
range_offset_limit -1
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i my.windowsupdate.website.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i ([^.]+.|)(appldnld|update).(apple.|)com/.* 43200 100% 43200 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate
refresh_pattern -i ([^.]+.|)(phobos).(apple.|)com/.*\.(zip|ibooks|ipa) 10080 100% 10080 ignore-reload ignore-no-store override-expire override-lastmod ignore-must-revalidate

cache_mem 1600 MB
maximum_object_size_in_memory 4096 KB
memory_replacement_policy lru
cache_replacement_policy heap LFUDA
cache_dir ufs /var/squid/cache 30000 16 256
minimum_object_size 0 KB
maximum_object_size 2048000 KB
offline_mode off
cache_swap_low 90
cache_swap_high 95
cache allow all

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
refresh_pattern .    0  20%  4320

# No redirector configured

#Remote proxies

# Setup some default acls
# From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localhost ACL definitions are now built-in.
# acl localhost src 127.0.0.1/32
acl allsrc src all
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901  3128 3127 1025-65535 80 443
acl sslports port 443 563  443

# From 3.2 further configuration cleanups have been done to make things easier and safer. The manager, localhost, and to_localhost ACL definitions are now built-in.
#acl manager proto cache_object

acl purge method PURGE
acl connect method CONNECT

# Define protocols used for redirects
acl HTTP proto HTTP
acl HTTPS proto HTTPS
acl allowed_subnets src 10.0.5.1/24 10.0.10.2/24
http_access allow manager localhost

# Allow external cache managers
acl ext_manager src 10.0.10.2
http_access allow manager ext_manager

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
# From 3.2 further configuration cleanups have been done to make things easier and safer. 
# The manager, localhost, and to_localhost ACL definitions are now built-in.
# http_access allow localhost

quick_abort_min -1 KB
quick_abort_max 0 KB
quick_abort_pct 20
request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow allsrc

# Reverse Proxy settings

# Custom options

# Setup allowed acls
# Allow local network(s) on interface(s)
http_access allow allowed_subnets
http_access allow localnet
# Default block all to be sure
http_access deny allsrc

Do you have any suggestion? Would it be wise to uninstall 3.3.8-dev and then install Squid 2.7 and manually add the refresh_patterns found in older tutorials?

Thanks in advance,

Panos

exograpix

Is there is any way to run havp with squid-dev package, squid is working ok, but without antivirus, it is bit incomplete, please suggest.

marcelloc

IMHO, havp is old and obsolete but if you are not using ssl filtering, you may configure the same way you do on squid2

I'm working to get antivirus tab working for this -dev version.

exograpix

Any alternate antivirus working with ssl filtering.

marcelloc

@exograpix:

Any alternate antivirus working with ssl filtering.

Not yet.

Pankar

Hello,

I have installed 3.3.8-dev and reinstalled lightsquid.

Problem is that the Proxysquid realtime stat, is either not showing client sessions at all, or, when it does, the list lines remain null and the following warning occurs several times (as many as the sessions):

gethostbyaddr(): Address is not a valid IPv4 or IPv6 address in /usr/local/www/sqstat/sqstat.class.php on line 426

Has anyone seen that before?

Thank you in advance.

ccarreraalza

I have a question.
I'm using squid 3.3.8-devel, well, when I go to facebook page shows me the warning I give exception and shows me confirm the facebook page but incompletely, as well as also the paypal page, and this because it is within the site of faccebook or https paypal there are other pages that are to be displayed, but they all come with an X.

as could validate the certificate on them to the web page or paypal facebook full display?

thanks

Google translator, sorry not mastered the English perfectly

totalimpact

Any tips to getting this working with dansguardian?

downloaded all the libs, installed Dans and then squid-dev, checked the boxes under squid to enable transparent mode and https MITM, installed Dans, and selected the same CA/Cert on pf.

Can see pages getting proxy'd, but no filtering is happening.

Mozart

Is it possible to update the Squid3-dev package so that I can add a couple of options to the reverse proxy part? I want to disable certain unsafe protocols like sslv2. Here is an example of the line in the config file that I updated:
https_port 172.28.252.101:443 accel cert=/usr/pbi/squid-amd64/etc/squid/52038c08be9a9.crt key=/usr/pbi/squid-amd64/etc/squid/52038c08be9a9.key defaultsite=some.site.com vhost options=NO_SSLv2:NO_TLSv1 cipher=ALL:!aNULL:!eNULL:!LOW:!EXP:!ADH:!RC4+RSA:+HIGH:+MEDIUM:!SSLv2

I would like it if there would be an option in the GUI to do this. My workaround at the moment is editing the /usr/local/pkg/squid_reverse.inc file on line 85. I added the options there so they will stay in place in case I make an update to the reverse proxy.

exograpix

Any news or timeline or alternative of antivirus package compatible with squid -dev version.

Thanks

leojoe

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/ldd/

i386
http://e-sac.siteseguro.ws/pfsense/8/All/ldd/

Hello All, Have tried to open the link above but displays forbidden? Anyone can check whats going on. Cannot start the Squid3 - dev for missing libraries.

Nachtfalke

@leojoe:

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/ldd/

i386
http://e-sac.siteseguro.ws/pfsense/8/All/ldd/

Hello All, Have tried to open the link above but displays forbidden? Anyone can check whats going on. Cannot start the Squid3 - dev for missing libraries.

I don't have any problems with these links. Try another browser or clear your browser's cache.

leojoe

@Nachtfalke:

@leojoe:

amd64
http://e-sac.siteseguro.ws/pfsense/8/amd64/All/ldd/

i386
http://e-sac.siteseguro.ws/pfsense/8/All/ldd/

Hello All, Have tried to open the link above but displays forbidden? Anyone can check whats going on. Cannot start the Squid3 - dev for missing libraries.

I don't have any problems with these links. Try another browser or clear your browser's cache.

yeah it's working now.

ZGruk

I'm having a really weird issue with squid. I've got squid and dansguardian, and I added local authentication to squid.
I defined 3 users in the squid GUI, one named admin, and two others.
The "admin" user works as intended. It blocks access unless I log in, and once I do, it works fine. None of the other users work.
When I try to log in with them, it acts like they don't exist or I put the wrong password in. I thought maybe it was linked somehow to the pfSense user "admin" but

A) the squid admin password is different from the pfSense password and
B) adding a pfSense user with one of the other names doesn't help.

Any idea what could be causing this?

Separately, do we know if SSL man-in-the-middle works with dansguardian? Older posts suggest that it doesn't, but I wondered if it had been fixed more recently.

wheelz

I'm seeing some SSL disconnects which are very similar to this bug: http://bugs.squid-cache.org/show_bug.cgi?id=3659. However this was supposed to have been resolved some time ago and changing the read_timeout does not fix it. Any ideas?

ZGruk

Update to add: I tried adding an "administrator" and an "admin2" account. "administrator" works, "admin2" doesn't. Very strange. Perhaps I should try "administrador" (spanish) or "administrateur" (french) next :P

jomcy

hi,
i tried to install squid 3.3.5 followed your document and all went smooth. but not showing in package list. please help me install it.
i just want transparent proxy for dynamic caching nothing more. can you please suggest me a stable proxy version for cache YouTube and other video sites
jomcy

duanes

Has anyone tried to use the "Time" setting to change squidguard filtering on this version of squid?

I don't know if this is a squid or a squidguard error, but the filters NEVER switch from On-Time to Off-Time and vice-versa…. ie, the "recalc trigger time" event never happens.

This has been a bug for YEARS, but I cannot seem to get any of the code developers interested in looking at the problem. They all point to a client caching problem, but that is definitely not the case. Squidguard log shows the redirect every time.

:(
Thanks for listening.

See:
https://forum.pfsense.org/index.php?topic=72251
https://forum.pfsense.org/index.php?topic=59671
https://forum.pfsense.org/index.php/topic,43352.0.html (nearly 7000 views!)

erwintwr

Hi guys

I am having a problem using transparent SSL man in the middle, and skype. Skype client connects, but it doesnt seem to complete the process, as no contants come online.

I currently managed to get transparent Squid3 Dev and Squidguard 3 working with SSL Man in the middle configured with a CA cert generated by the pfsense box - it works beaufitully, especially since this allows squidguard to apply access rules to HTTPS sites, which previously required cumbersome DNS work arounds.)

If i bypass the source IP on the proxy, skype works, and also if i switch off SSL man in the middle it works as well, which both has obvious drawbacks.

Can somebody maybe point me in the right direction as to what could cause this issue? (or what to do to resolve this)
I have not changed any filter rules or added any NAT on the firewall side ( thus it is stock)

If there is a way to bypass skype traffic from the squid proxy, we can look at that as well

some entries from the squid log is generated below :

1393247653.671 680 192.168.2.11 TCP_MISS/200 394 GET http://conn.skype.com/ - HIER_DIRECT/91.190.216.81 text/html
1393247654.337 640 192.168.2.11 TCP_MISS/200 515 GET http://ui.skype.com/ui/0/6.3.60.105./en/getlatestversion? - HIER_DIRECT/157.56.114.104 text/html
1393247654.403 0 192.168.2.11 NONE/400 3535 CNT 1CON185 - HIER_NONE/- text/html
1393247654.640 700 192.168.2.11 TCP_MISS/200 1290 GET http://api.skype.com/users/adriaan.klopper/profile/avatar - HIER_DIRECT/91.190.216.7 image/jpeg
1393247654.914 259 192.168.2.11 TCP_MISS/200 1290 GET http://api.skype.com/users/alfred.modiba/profile/avatar - HIER_DIRECT/91.190.216.7 image/jpeg
1393247655.256 317 192.168.2.11 TCP_MISS/200 3338 GET http://api.skype.com/users/annelize.smit3/profile/avatar - HIER_DIRECT/91.190.216.7 image/jpeg
1393247655.391 0 192.168.2.11 NONE/400 3535 CNT 2CON185 - HIER_NONE/- text/html
1393247655.522 256 192.168.2.11 TCP_MISS/200 1290 GET http://api.skype.com/users/live:bernardi222/profile/avatar - HIER_DIRECT/91.190.216.7 image/jpeg
1393247663.465 0 192.168.2.11 NONE/400 3535 CNT 3CON185 - HIER_NONE/- text/html
1393247686.127 0 192.168.2.11 NONE_ABORTED/000 0 GET https://forum.pfsense.org/Themes/flagrantly202/images/bbc/bbc_hoverbg.gif - HIER_NONE/- -
1393247710.685 0 192.168.2.11 NONE/400 3535 CNT 4CON185 - HIER_NONE/- text/html

any ideas welcome

thank you guys!

serialdie

@duanes:

Has anyone tried to use the "Time" setting to change squidguard filtering on this version of squid?

I don't know if this is a squid or a squidguard error, but the filters NEVER switch from On-Time to Off-Time and vice-versa…. ie, the "recalc trigger time" event never happens.

This has been a bug for YEARS, but I cannot seem to get any of the code developers interested in looking at the problem. They all point to a client caching problem, but that is definitely not the case. Squidguard log shows the redirect every time.

:(
Thanks for listening.

See:
https://forum.pfsense.org/index.php?topic=72251
https://forum.pfsense.org/index.php?topic=59671
https://forum.pfsense.org/index.php/topic,43352.0.html (nearly 7000 views!)

I feel your pain.
My work around this was to create a cronjob to HUP squid each off time.