Squid3 - New GUI with sync, normal and reverse proxy

dig1234

ah indeed you are correct. That makes sense as 2.2 is using FreeBSD 10.1…

Escorpiom

Please can anybody outline the correct procedure and confirm Squid was updated for PfSense 2.2 beta?

Cheers.

longhorn

Now I'm getting a HIT by setting the Minimum object size to 0.

It's just the large downloads that don't get cached.

Hope this will get sorted out again in the next update….

It could also be how you utilize the proxy. It took me some time to figure out for many applications, Squid's caching function is useless. Unless you have a network where multiple computers hit the exact same content, within a period of time (while the data is still in the cache).

I use pfSense for my home network. Almost nothing is cached. However, I find Squid very useful for logging and filtering Web traffic.

YMMV, but just pointing out that seeing a lot of MISSes reported in Squid's log is most often WAD behavior (Working As Designed)

Topper727

I have it installed in 2.2RC as of 12/30's release. Does not run. Starts then stops then starts then stops and if I try to run through transparent I get no data through unless it is application that uses and IP instead of a domain name to get the connection. Is there a problem with DNS ?I use DNS Resolver the new one they say.. The DNS Fordwarder didn't work for me so I use that now.. But all is well without Squid.

Why it don't stay running and keep restarting ?

2.2-RC (amd64)
built on Tue Dec 30 15:16:19 CST 2014
FreeBSD pfSense.localdomain 10.1-RELEASE-p3 FreeBSD 10.1-RELEASE-p3 #0 8bdb2f8(releng/10.1)-dirty: Tue Dec 30 15:58:58 CST 2014 root@pfsense-22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10 amd64

marcelloc

@Topper727:

Why it don't stay running and keep restarting ?

It looks like current pbi was build with debug options. Once we get the pull request mergerd and pbi build again with all used options, I think we get a 'ready to go' version

Cino

@marcelloc did you update the symbolic links for the install also? I recall I had to create a bunch to get 3.4 to start on a fresh 2.2RC install. I have to check my notes, but I believe I was able to get clamd to run but c-icap wouldn't start up.

marcelloc

Not yet. I've fixed compile args but we need to wait a new pbi run for squid.

Cino

thanks marcelloc!! Looking forward to it! waiting for a few more other packages to be updated and i'll be ready to switch over to 2.2

Escorpiom

@Topper727:

…and if I try to run through transparent I get no data through unless it is application that uses and IP instead of a domain name to get the connection. Is there a problem with DNS ?I use DNS Resolver the new one they say.. The DNS Fordwarder didn't work for me so I use that now.. But all is well without Squid.

Sadly, the issue persists after updating to 3.4.10_2 pkg 0.2.1.

Cheers.

Topper727

Same here on 3.4.10_2 pkg 0.2.5.. Transparent still gives no net unless secure https .. then of course it don't go through proxy (not have it setup to do https) so all works there.. but all other pages show can't find page

marcelloc

@Topper727:

Same here on 3.4.10_2 pkg 0.2.5.. Transparent still gives no net unless secure https .. then of course it don't go through proxy (not have it setup to do https) so all works there.. but all other pages show can't find page

Squid Transparent proxy is working on 2.2 with or without ssl interception.

What you get on squid/cache logs?

dalex

Hello.

I have 2.2-RELEASE (i386) with Squid 2.7.9 pkg v.4.3.6 (and Lightsquid who stopped working after last upgrade).

I need IPv6 support, so i think Squid3 is my only option.

Shall i uninstall Squid 2.7.9 first?

Is Lightsquid compatible with Squid3 (when they fix new problems)?

Thanks

Cino

@dalex:

Hello.

I have 2.2-RELEASE (i386) with Squid 2.7.9 pkg v.4.3.6 (and Lightsquid who stopped working after last upgrade).

I need IPv6 support, so i think Squid3 is my only option.

Shall i uninstall Squid 2.7.9 first?

Is Lightsquid compatible with Squid3 (when they fix new problems)?

Thanks

uninstall 2.7.9 then install squid3… squid 3 does support IPv6, but it doesn't listen on an IPv6 address tho. So clients would need to connect to ipv4:3128. You can however in advance options add an IPv6 interface e.g http_port [xxxx.xxxx.xxxx.xxxx.xxx.xxx]:3182

lightsquid still has issues with its pbi but search for my workaround to get it working. It is compatible with squid3

dalex

@Cino:

uninstall 2.7.9 then install squid3… squid 3 does support IPv6, but it doesn't listen on an IPv6 address tho. So clients would need to connect to ipv4:3128. You can however in advance options add an IPv6 interface e.g http_port [xxxx.xxxx.xxxx.xxxx.xxx.xxx]:3182

lightsquid still has issues with its pbi but search for my workaround to get it working. It is compatible with squid3

Thanks a lot.

I can live with v4 clients, and i will try the custom addition.

I want IPv6 Internet behavior for Squid because my 6to4 Tunnel ends in a (very) faster local Gate (International circuits much larger/faster than my ISP's one).

TheBanHammer

Hello pfSense Gurus!

I have been using pfSense for about 2 years now and I love it. One of my favorite features has come to be Squid2 and Squid3. Right now I am using Squid3 to run both proxy and reverse proxy.

Using this forum thread and other threads on this forum I have my reverse proxy running. There are two questions if have to make it work perfectly:

1. How can I force Squid3 (the reverse proxy part in pfSense) to suppress the addition headers identifying the host as Squid? (picture of my headers attached.[Contained in the red box])

2. How can I get Squid (reverse proxy) to forcefully cache static (text/HTML) content?

Squid3 will query my web server on the local network every time I connect. I have two remote machines that I use for testing. When I look in my Squid logs both machines show their remote IP address separately and each time it says "TCP_MISS/200". (Picture attached)

I have a openVPN connection between two networks. One network is the public IP(site B) I use to run my website and the other network is my private network(site A). It works right now and the DNS has no problems resolving.

When i check the headers (in my example[contained in blue box]) it says that is has not changed for two days and has an eTag. That header is from Apache server itself(I have checked headers on localhost using [curl -I]).

If I understand how the reverse proxy is supposed to be working. Since it is being presented with a not changed since date it should be caching it at (site B), but instead it queries (site A) for each call.

tl;dr - Need help suppressing Squid's headers in reverse proxy mode and forcing the reverse proxy to cache static content.

![TCP Headers.png](/public/imported_attachments/1/TCP Headers.png)
![Reverse Proxy Cache.png](/public/imported_attachments/1/Reverse Proxy Cache.png)
![Reverse Proxy Cache.png_thumb](/public/imported_attachments/1/Reverse Proxy Cache.png_thumb)
![TCP Headers.png_thumb](/public/imported_attachments/1/TCP Headers.png_thumb)

workingman

Hi marcelloc/pfsense squid folks.

I've been using squid3/squidguard setup in a redundant CARP pair setup for quite some time and was just cooking up a new pair for a new site and ran into a problem that I guess I've pretty much always had.

Sync to configured system backup server - never sync's to the backup. The configured system backup server is working as expected sharing my DHCP/DNS/Firewall rules but I have always had to turn on..

Sync to host(s) defined below - set the same server as I use in the system config and then everything does get copied over to the backup.

Since it was easy to fix/workaround it never really bothered me much but I'm just setting up a brand fresh install of 2.2.1 and it seems to still happen. Am I the only one?

Thanks!

varazir

Hi,

How do I add this list http://winhelp2002.mvps.org/hosts.txt to squid3? or do I need squidGuard ?

KR
Daniel

messerchmidt

@varazir:

Hi,

How do I add this list http://winhelp2002.mvps.org/hosts.txt to squid3? or do I need squidGuard ?

KR
Daniel

i think you need squid guard

varazir

@messerchmidt:

@varazir:

Hi,

How do I add this list http://winhelp2002.mvps.org/hosts.txt to squid3? or do I need squidGuard ?

KR
Daniel

i think you need squid guard

Okay, have idea how? or if there are any good guide…

spittlbm

@varazir:

@messerchmidt:

@varazir:

Hi,

How do I add this list http://winhelp2002.mvps.org/hosts.txt to squid3? or do I need squidGuard ?

KR
Daniel

i think you need squid guard

Okay, have idea how? or if there are any good guide…

I'd be VERY interested in the same feature.

qinohe

@spittlbm:

@varazir:

@messerchmidt:

@varazir:

Hi,

How do I add this list http://winhelp2002.mvps.org/hosts.txt to squid3? or do I need squidGuard ?

KR
Daniel

i think you need squid guard

Okay, have idea how? or if there are any good guide…

I'd be VERY interested in the same feature.

I create this list myself. It's just one (domains) big list, which I pack together with f.i. shalla or mesd.
My main OS is Arch Linux, I use a program called hosts-update see https://aur.archlinux.org/packages/hosts-update/
From the hosts file, which is basically the MVPS list, I create the 'domains' file, which can be used by squidguard.

If there is interest, I could place this list (mvps only) on github.

robatwork

Does the version on the pfsense packages ie:

beta
0.2.8
platform: 2.2

contain support for HTTP/1.1 ?

We are having a problem with keep-alives that need this support.
Thanks

BTW the link https://github.com/pfsense/pfsense-packages/commits/master/config/34 that is on the version number leads to a 404.

SisterOfMercy

@robatwork:

BTW the link https://github.com/pfsense/pfsense-packages/commits/master/config/34 that is on the version number leads to a 404.

That should probably be this one then:
https://github.com/pfsense/pfsense-packages/tree/master/config/squid3/34

ismaelnoble

@hackersoft:

I am still experiencing the same problem.

I've got the time last week to reinstall pfsense amd-64 and the same problem exists.

So my solution right now is to reboot pfsense everyday using CRON @ 1 am. That way pfsense will be alive the next working day.

I'm suspecting one of the packages installed is messing it up at 12 midnight so I do a cron to reboot the machine @ 1am daily.

To heimdal, you can install the CRON package then add a cron task:

0 1 * * * root /sbin/reboot

I hope the fix will be released soon.

for some wierd reason im getting this too, a look at the logs doesnt show much so its been one of those evil bugs that dont leave any traces. i did mitigate it by simply seting up a cron job that runs squid at 12.15 am …

currently i have squid, squidguard, ntopng and freeradius setup and running its just this but that is "bugging" me

dkrizic

@Cino:

@dalex:

Hello.
I need IPv6 support, so i think Squid3 is my only option.

squid 3 does support IPv6, but it doesn't listen on an IPv6 address tho. So clients would need to connect to ipv4:3128. You can however in advance options add an IPv6 interface e.g http_port [xxxx.xxxx.xxxx.xxxx.xxx.xxx]:3182

How can I use IPv6 with the Reverse Proxy? I currently listen on IPv4/80 and IPv4/443 and would like to additionally listen on IPv6.

hell bomb

Interesting error, while AV is enabled while navigating to most websites (not all websites) I get TAG_NONE/500 Status error. Any suggestions?

SisterOfMercy

@varazir:

How do I add this list http://winhelp2002.mvps.org/hosts.txt to squid3? or do I need squidGuard ?

Wouldn't pfBlockerNG work better in this case?

aGeekhere

whats new in 0.2.9?

S. Kirschner

@aGeekHere:

whats new in 0.2.9?

Hm , its still the same version.

Here are the build options of 0.2.9

Squid Cache: Version 3.4.10
configure options:  '--with-default-user=squid' '--bindir=/usr/local/sbin' '--sbindir=/usr/local/sbin' '--datadir=/usr/local/etc/squid' '--libexecdir=/usr/local/libexec/squid'
 '--localstatedir=/var' '--sysconfdir=/usr/local/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid/squid.pid' '--with-swapdir=/var/squid/cache' '--enable-auth'
 '--enable-build-info' '--enable-loadable-modules' '--enable-removal-policies=lru heap' '--disable-epoll' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-translation'
 '--disable-arch-native' '--enable-eui' '--enable-cache-digests' '--enable-delay-pools' '--enable-ecap' '--disable-esi' '--enable-follow-x-forwarded-for' '--enable-htcp'
 '--enable-icap-client' '--enable-icmp' '--enable-ident-lookups' '--enable-ipv6' '--enable-kqueue' '--with-large-files' '--enable-http-violations' '--without-nettle' '--enable-snmp' '--enable-ssl'
 '--enable-ssl-crtd' '--disable-stacktraces' '--disable-ipf-transparent' '--disable-ipfw-transparent' '--enable-pf-transparent' '--with-nat-devpf' '--disable-forw-via-db' '--enable-wccp'
 '--enable-wccpv2' '--enable-auth-basic=DB MSNT MSNT-multi-domain NCSA PAM POP3 RADIUS fake getpwnam LDAP NIS' '--enable-auth-digest=file'
 '--enable-external-acl-helpers=file_userip time_quota unix_group LDAP_group' '--enable-auth-negotiate=kerberos wrapper' '--enable-auth-ntlm=fake smb_lm'
 '--enable-storeio=ufs aufs diskd' '--enable-disk-io=AIO Blocking IpcIo Mmapped DiskThreads DiskDaemon' '--enable-log-daemon-helpers=file' '--enable-url-rewrite-helpers=fake'
 '--enable-storeid-rewrite-helpers=file' '--with-openssl=/usr' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=amd64-portbld-freebsd10.1' 
'build_alias=amd64-portbld-freebsd10.1' 'CC=cc' 'CFLAGS=-O2 -pipe  -I/usr/local/include -I/usr/local/include -I/usr/include -fstack-protector -DLDAP_DEPRECATED -fno-strict-aliasing'
 'LDFLAGS= -L/usr/local/lib -L/usr/local/lib -pthread -Wl,-rpath,/usr/lib:/usr/local/lib -L/usr/lib -fstack-protector' 'LIBS=' 'CPPFLAGS=' 'CXX=c++'
 'CXXFLAGS=-O2 -pipe -I/usr/local/include -I/usr/local/include -I/usr/include -fstack-protector -DLDAP_DEPRECATED -fno-strict-aliasing  -Wno-unused-private-field' 'CPP=cpp' 'PKG_CONFIG=pkgconf' --enable-ltdl-convenience

aGeekhere

well I updated,all went well, just had to remove the cache after the install or else i get page loading issues.

magicduck

Hi there.

I found a "bug" / "feature" when we have IPv6.
Seems that I have to add manually the IPv6 to listen on the squid (on the integration box)

Is this normal ?

Cino

@magicduck:

Hi there.

I found a "bug" / "feature" when we have IPv6.
Seems that I have to add manually the IPv6 to listen on the squid (on the integration box)

Is this normal ?

Yes its normal. Its been that way since squid3 was added as a package. I would say its a feature that needs to be added to the GUI. To enable it and grab the interface IP from the interfaces you select

rataosinho

I installed version 0.3.0 today and ended my life. Computers with setted proxy manually, can not load pages. I had to take out the proxy settings of the LAN and did not want to do that.
Is there any way for me to go back to version 0.2.9? :'(
hugs!

whitexp

@rataosinho:

I installed version 0.3.0 today and ended my life. Computers with setted proxy manually, can not load pages. I had to take out the proxy settings of the LAN and did not want to do that.
Is there any way for me to go back to version 0.2.9? :'(
hugs!

same here

aGeekhere

I installed version 0.3.0 today and ended my life. Computers with setted proxy manually, can not load pages. I had to take out the proxy settings of the LAN and did not want to do that.
Is there any way for me to go back to version 0.2.9? :'(
hugs!

Did you delete the squid cache?

default666

squid 0.3.0 beta

cache cleared

acl > blacklist - not working
acl > banned host adresses - not working
authentication - not working

reinstalled
transparent mode - not working
port change - not working

updated to 0.3.1 - same sh.

doktornotor

@rataosinho:

Is there any way for me to go back to version 0.2.9? :'(

Sure. Revert these patches via System Patches:
https://github.com/doktornotor/pfsense-packages/commit/a928e6df410383d00c19318b5c20459c50249a5a
https://github.com/doktornotor/pfsense-packages/commit/3f881d1e7c55ea875017764e3c6c8522fb344b84

I can assure you that reverting that will fix absolutely none of the issues you are having. Nothing changed in the Squid binaries, nothing changed in blacklists, banned hosts or anything like that, nothing changed in authentication.

P.S. For complete job, do not forget to revert https://github.com/pfsense/pfsense-packages/pull/1053 as well and chmod the pinger binary back to 0755 to get it nicely broken again.

default666

Sure. Revert these patches via System Patches:

Used this instruction: https://doc.pfsense.org/index.php/System_Patches

Patch can NOT be reverted cleanly.

Output of full patch revert test:
/usr/bin/patch --directory=/usr/local/www/ -f -p1 -i /var/patches/55f0f3997bde1.patch --check --reverse --ignore-whitespace

Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|From a928e6df410383d00c19318b5c20459c50249a5a Mon Sep 17 00:00:00 2001
|From: doktornotor 
|Date: Fri, 4 Sep 2015 22:17:03 +0200
|Subject: [PATCH] squid3 - use install_cron_job() instead of reimplementing it
|
|Also, use platform_booting() for boot check.
|---
| config/squid3/34/squid.inc | 90 +++++++++-------------------------------------
| 1 file changed, 17 insertions(+), 73 deletions(-)
|
|diff --git a/config/squid3/34/squid.inc b/config/squid3/34/squid.inc
|index 2705ceb..481b010 100755
|--- a/config/squid3/34/squid.inc
|+++ b/config/squid3/34/squid.inc
--------------------------
No file to patch.  Skipping...
Hunk #1 ignored at 772.
1 out of 1 hunks ignored while patching config/squid3/34/squid.inc
done

doktornotor

You have the paths wrong, very obviously.

marcelloc

@aGeekHere:

whats new in 0.2.9?

You can click o package version to see it's changelog.