Squid3 - New GUI with sync, normal and reverse proxy

TheNetStriker · Jun 10, 2013, 9:22 PM

I've commited my changes on GitHub: https://github.com/TheNetStriker/pfsense-packages/commit/7d926f3d44cee817475c20bde44fe6471bab4ba7 (I hope it worked, it was my first commit on GitHub)

I've added a new page to configure additional ports and added a configuration for client certificates.

marcelloc · Jun 10, 2013, 10:29 PM

@TheNetStriker:

I've added a new page to configure additional ports and added a configuration for client certificates.

I'm doing it on web servers page but I'll take a look on your code and see how can I merge it with my current uncommitted code.

Thanks for your code contribution. :)

technical · Jun 12, 2013, 1:14 PM

squid access.log rotate problems

http://forum.pfsense.org/index.php/topic,59557.msg341017.html#msg341017

cron jobs not work
/bin/rm /var/squid/cache/swap.state; /usr/local/sbin/squid -k rotate -f /usr/local/etc/squid/squid.conf

returns with error.

check it please marchelloc

/var/squid/logs/access.log
and my cache not works to

my squid .conf


# This file is automatically generated by pfSense
# Do not edit manually !
http_port 10.50.2.1:3949
http_port 10.50.3.1:3949
http_port 127.0.0.1:3949 intercept
icp_port 7
dns_v4_first on
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_default_language tr
icon_directory /usr/local/etc/squid/icons
visible_hostname localhost
cache_mgr admin@localhost
access_log /var/squid/logs/access.log
cache_log /var/squid/logs/cache.log
cache_store_log none
sslcrtd_children 0
logfile_rotate 1
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src  10.50.2.0/24 10.50.3.0/24
forwarded_for off
via off
httpd_suppress_version_string on
uri_whitespace strip

# Break HTTP standard for flash videos. Keep them in cache even if asked not to.
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private

# Let the clients favorite video site through with full caching
acl youtube dstdomain .youtube.com
cache allow youtube

# Windows Update refresh_pattern
range_offset_limit -1
refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
refresh_pattern -i my.windowsupdate.website.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims
cache_mem 512 MB
maximum_object_size_in_memory 128 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
cache_dir ufs /var/squid/cache 140000 16 256
minimum_object_size 0 KB
maximum_object_size 4500000 KB
offline_mode offcache_swap_low 90
cache_swap_high 95

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:    1440  20%  10080
refresh_pattern ^gopher:  1440  0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0  0%  0
refresh_pattern .    0  20%  4320
# No redirector configured

#Remote proxies

# Setup some default acls
acl allsrc src all
acl localhost src 127.0.0.1/32
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 443 3128 1025-65535 
acl sslports port 443 563 443 
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT

# Define protocols used for redirects
acl HTTP proto HTTP
acl HTTPS proto HTTPS

http_access allow manager localhost

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

# Always allow localhost connections
http_access allow localhost

request_body_max_size 0 KB
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow allsrc

# Reverse Proxy settings

deny_info TCP_RESET allsrc

# Package Integration
redirect_program /usr/local/bin/squidGuard -c /usr/local/etc/squidGuard/squidGuard.conf
redirector_bypass off
url_rewrite_children 5

# Custom options

# Setup allowed acls
# Allow local network(s) on interface(s)
http_access allow localnet
# Default block all to be sure
http_access deny allsrc

file is now 539 mb and become larger more than …

marcelloc · Jun 12, 2013, 1:52 PM

@TechnicaL:

/bin/rm /var/squid/cache/swap.state; /usr/local/sbin/squid -k rotate -f /usr/local/etc/squid/squid.conf

the squid rotate cmd is the same from squid2

what errors do you get running it on console?

If you use sarg, you can disable squid rotate logs and enable it on sarg after report creation.

technical · Jun 12, 2013, 3:14 PM

@marcelloc:

the squid rotate cmd is the same from squid2

what errors do you get running it on console?

If you use sarg, you can disable squid rotate logs and enable it on sarg after report creation.

i cant using sarg.

command is working but access.log file not rotating i delete today manuel that bigger than 600 mb.

access.log file rotate at 00:00 dially is important for us.

i ll check tomorrow morning.

technical · Jun 13, 2013, 6:04 AM

Update :13-06-2013 09:00 PM access.log rotate is not working, still including yesterday log events.

1371044860.589 1479 10.50.2.115 TCP_MISS/200 39264 GET http://www.youtube.com/ - DIRECT/173.194.39.199 text/html

time stamp = GMT: Wed, 12 Jun 2013 13:47:40 GMT

marcelloc · Jun 13, 2013, 2:23 PM

If the script is on cron and if you can successful run it by hand, It's not related to squid.

Are you on nanobsd?

technical · Jun 13, 2013, 3:03 PM

@marcelloc:

If the script is on cron and if you can successful run it by hand, It's not related to squid.

Are you on nanobsd?

no not nanobsd, system running at P4 PC

how to create script without delete the access.log file
if im create rm -rf command log file will be deleted. how to clean the only inside access.log file.

marcelloc · Jun 13, 2013, 6:58 PM

@TechnicaL:

how to create script without delete the access.log file
if im create rm -rf command log file will be deleted. how to clean the only inside access.log file.

squid -k rotate -f /usr/local/etc/squid/squid.conf does it.

technical · Jun 14, 2013, 5:08 AM

@marcelloc:

@TechnicaL:

how to create script without delete the access.log file
if im create rm -rf command log file will be deleted. how to clean the only inside access.log file.

squid -k rotate -f /usr/local/etc/squid/squid.conf does it.

oke i ll try it and inform you.

arch113 · Jun 14, 2013, 9:00 PM

Does any version of Squidguard work with this -dev version of squid? I cant seem to get the Service to START.

marcelloc · Jun 14, 2013, 9:08 PM

@arch113:

Does any version of Squidguard work with this -dev version of squid? I cant seem to get the Service to START.

Since squid 3, it starts on demand. If you have access then squidguard will run.

technical · Jun 15, 2013, 5:50 AM

@marcelloc:

squid -k rotate -f /usr/local/etc/squid/squid.conf does it.

marchelloc it does not work. still access.log file include 12.06.2013 events.
installed packages

squid3 3.1.20 pkg 2.0.6
squidguard 1.4_4 pkg v.1.9.5
Lightsquid 1.8.2 pkg v.2.32

Tikimotel · Jun 15, 2013, 8:22 AM

@arch113:

Does any version of Squidguard work with this -dev version of squid? I cant seem to get the Service to START.

If you reinstalled the packages, sometimes squid is already started without "kids" or "children" active.
Somehow when squid is running it won't take the squidguard process as a child-process.
When that happens, just stop squid via the dashboard or services page and restart using the save button in the proxy server page.

I use squid3-dev with squidGuard-squid3.
With squid3 and onwards I always manually edit the "squidguard_configurator.inc" to use squid3 options for child-processes.


# ------------------------------------------------------------------------------
# squid config options
# ------------------------------------------------------------------------------
define('REDIRECTOR_OPTIONS_REM',   '# squidGuard options');
define('REDIRECTOR_PROGRAM_OPT',   'url_rewrite_program');
define('REDIRECT_BYPASS_OPT',      'url_rewrite_bypass');
define('REDIRECT_CHILDREN_OPT',    'url_rewrite_children');
define('REDIRECTOR_PROCESS_COUNT', '12 startup=8 idle=6 concurrency=0'); # redirector processes count will start with 8 do a max of 12 and idles at 6 (with v3.3+)

technical · Jun 17, 2013, 2:21 PM

marcelloc any news ?

technical · Jun 21, 2013, 5:19 AM

Access.log file still rise marchelloc can u check log rotation on squid3 ?

we are still looking your help. thank you.

marcelloc · Jun 21, 2013, 11:14 AM

Did you tried sarg instead of lightsquid?

You can disable log rotate on squid gui and enable it on sarg.

technical · Jun 21, 2013, 12:26 PM

@marcelloc:

Did you tried sarg instead of lightsquid?

You can disable log rotate on squid gui and enable it on sarg.

many turkish ppl using lightsquid because we translated lightsquid from turkish language http://forum.pfsense.org/index.php/topic,62008.0.html

many ppl must use lightsquid can u check the problem for lightsquid package.

and i ll try it with sarg and inform you.

technical · Jun 22, 2013, 6:32 AM

I ll tryed sarg with log rotate.

there is no change
still not rotated access.log file

check the image's.

marcelloc · Jun 24, 2013, 4:39 AM

Do you have sarg reports created? On my machine it rotates successfully.

Remember to disable log rotating on squid and check sarg script on cron.

Did you tried to reinstall squid package to any other version?

What permissions do you have on squid logs and dir?