Squid3 problems
-
Where to start…
I have set up a pfsense box at work to support the tech and the sales.... We were using a single TP-Link router everyone had full access to everything.
We have comcast business 35 down and 5 up, running a fresh install of pfsense and only essentials configured to make internet work and keep sales from entering the tech subnet I can get the full speed from our provider.
My problem comes in when I add Squid3, Everything runs great for 6 hours or more. Then sometime throughout the day my speeds drop 5 down 1 up. If I check system activity it shows swap is untouched but almost 4gb of memory is used.
current set up is:
2.1-BETA0 (amd64)
built on Fri Oct 12 09:44:16 EDT 2012
FreeBSD 8.3-RELEASE-p4Squid3 (latest from packages manager)
Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz
4gb memory
500gb hdd seagate something 7200rpmi thinktwo intel cards (WAN/LAN tech) and onboard realtek (sales)
nmbclusters set to 32768
vfs.read=128
squid:
cache:8000mb
largest object:600mb
memory_cache: 512mb
largest_object: 10mbI can't remember if there is anything else that is important that I might have changed.
Why do my speeds drop so bad? Any help is appreciated!
Thanks, Josh
P.s. if I forgot something or you just want to know more of something, run a test or what not let me know I will be glad to do it! -
did you enabled soft updates on /usr during install to improve disk performance?
-
I appreciate the reply! Sorry it took me so long to get back to you I've been sick all weekend.
Back on topic.. I don't remember seeing that option but at the time of install i think i choose the easy/quick option. :-\
Is there a way to enable it after its been installed?
Thanks again for the help!
-
Is there a way to enable it after its been installed?
I've tried once but no success. :(
Another option could be a second hard disk for caching and logging.
-
I've tried once but no success. Sad
Sad day…
so if soft updates were enabled I wouldn't be having this issue where my internet slowly stops responding?
If that is the case I may just reinstall pfsense!!! (probably not but the thought did cross my mind) -
Try to set vfs.read_max to 128 on system->advanced->system tunables.
It will increase disk performance on most systems.
-
Try to set vfs.read_max to 128 on system->advanced->system tunables.
It will increase disk performance on most systems.
…Things just got a little more sad...
I've already increased that
@jwhostet:nmbclusters set to 32768
vfs.read=128No big deal. Thanks for all the help! |If I manage to start over fresh again… I'll let you know my results! (things are slow right now, so the possibility is good!) :)
-
did you enabled soft updates on /usr during install to improve disk performance?
One last question… if soft updates were turned on during the install would that have solved my problem with the speed issue I'm having?
-
I do not install any squid/dansguardian/postfix install without it.
I've never tried read_max 128.
-
UPDATE
From a live cd 20121018-0934 64bit I reinstalled pfsense I then made sure everything was working like it was. Everything is good to go no problems.
To make things better I now have soft updates on for /var and /usr:[2.1-BETA0][admin@pfsense.pcd.domain]/root(3): mount
/dev/ad6s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad6s1d on /var (ufs, local, soft-updates)
/dev/ad6s1e on /usr (ufs, local, soft-updates)
devfs on /var/dhcpd/dev (devfs, local)I have enabled squid and things are not looking so good. :(
After enabling my speeds have dropped dramatically within the first hour of enabling squid.
Same settings as before!What now?! Why is this happening? :(
Any other ideas?
Thanks!
-
Well I spoke to soon seems soft updates might have solved the speed dropping issue. It seems to use almost all 4gb of memory but not complaining too much.
Is it normal to use next to all of the available memory? What if I had other packages that needed to use memory too? ( As of right now I do not but soon I wish to implement others.)
-
Is it normal to use next to all of the available memory? What if I had other packages that needed to use memory too? ( As of right now I do not but soon I wish to implement others.)
yes, It's normal.
You can try to reduce squid memory cache or add more memory to keep a good performance :)
-
:(
well… apparently I spoke too soon again!
Today I get a message saying the internet is got a 2 second ping and 200kb download which is terrible... after removing squid package everything goes back to normal 20ms ping 32mb download.
I'm assuming squid has been running normal and fine because the load on the router was minimal 5 systems hitting the system hard. Whereas today with a couple more technicians I'm assuming more system continually pulling data.
My question now is why is this happening? Is it hardware related? I estimate about 10gb of bandwidth a day is used. Are we accessing so much data that the system can't keep up?
Any help is once again appreciated, hopefully we can get this solved. If you would like to know any hardware specifics or a specific configuration file let me know and I will get you what you need.
Thanks, Josh
-
Is my thread dead? :(
I have a system on the network that I thought might have been causing squid to go crazy and hang draining internet to a stop.
So I added it to the list that are supposed to bypass proxy but it did not help solve the issue.Any body have any ideas?
-
Any body have any ideas?
Can you try to include this on squid.sh startup script after line 4.
Remember to do not save squid config on gui while you test.sysctl kern.ipc.nmbclusters=65536 sysctl kern.ipc.somaxconn=16384 sysctl kern.maxfiles=131072 sysctl kern.maxfilesperproc=104856 sysctl kern.threads.max_threads_per_proc=4096 -
Hey thanks for getting back to me!! :)
sorry I didn't get back sooner, school got kinda hectic on me for a couple days.
I can definitely try to do that!
One question though, how do i not save squid config on gui?Thanks again for the help…. I feel repetitive but I do truly mean it!!!!!
-
One question though, how do i not save squid config on gui?
do not press save on squid gui after changing rc.d file ;)
-
Haha…. Oh that makes sense. Thanks for the help I forgot this went on to a second page Till right now. I'll probably start testing that tomorrow and hope things go well to report back with good news!!
-
Well we are having major ISP issues here at work so it might be a couple days before I can truly test your possible solutions will post back asap with results!
-
Ok :)
-
I love squid but i find too many limitations, at least with when i use it in pfsense. (i don't like how it works in other platforms)
What is going on right now is that https pages cannot be access with the squid3 package installed from pfsense.
I can use lusca and it works well with both http and https pages but it's very outdated and is not being updated an maintained anymore.So i want to update my pfsense platform but i really feel that squid3 is useless if it doesn't allow me to work with https just like lusca and squid 2.7 without using crazy configurations.
Also squid crashes when i set the maximum object size in RAM above about 450 KB.
In fact i assign 32 GB of RAM to PFSense of which 20 GB are for dedicated for Squid RAM caching and the only way to use all that RAM and make it useful is having the maximum limit of at least 512 KB or even 1024KB for objects stored in RAM.
-
What is going on right now is that https pages cannot be access with the squid3 package installed from pfsense.
check dns_v4_first option on squid config gui and try again.
-
Marcellok so you say that i have to set dns_v4_first off or on?
This is a brand new command that i just discovered thanks to you Marcellok
The problem is that in the squid guide it doesn't say all the values, just the default
and talks about this command: –enable-internal-dnsI wonder if i have to stick --enable-internal-dns with all the other squid settings
http://www.squid-cache.org/Doc/config/dns_v4_first/
With the IPv6 Internet being as fast or faster than IPv4 Internet
for most networks Squid prefers to contact websites over IPv6.This option reverses the order of preference to make Squid contact
dual-stack websites over IPv4 first. Squid will still perform both
IPv6 and IPv4 DNS lookups before connecting.WARNING:
This option will restrict the situations under which IPv6
connectivity is used (and tested). Hiding network problems
which would otherwise be detected and warned about.Also do you know what are maximum size for objects in RAM?
Can i safely set 4096 KB as maximum Object size in RAM?
I used to cache in disk objects big as 9 Gigabytes (a full dvd) with lusca and it works, but if i go beyond 512Kilobytes of object size in RAM sometimes quid crashes.
Marcellok do you work for the squid guys or you just work in the squid3 package for pfsense.
If you work for the squid-cache people, can you please import the ultra aggressive caching features of lusca inside squid? Because lusca is actually a tuned up squid.
Thanks for your support!
-
Well major issues here… first ISP issue then after they finally came back online I find out there is some sort of failover that my GM added me into. So now im limited to 10Mbps till a new card is installed in his failover box. So I might have to scrap my whole setup/squid may just end up being a basic router... Or nothing.
Thanks for all the help, if something changes in the next few days and I get everything I had before back I will try your fix and report back.
Or I will create another post and start over new.
Again thanks!
Josh
