Squid3 problems

marcelloc

did you enabled soft updates on /usr during install to improve disk performance?

jwhostet

I appreciate the reply! Sorry it took me so long to get back to you I've been sick all weekend.

Back on topic.. I don't remember seeing that option but at the time of install i think i choose the easy/quick option. :-\

Is there a way to enable it after its been installed?

Thanks again for the help!

marcelloc

@jwhostet:

Is there a way to enable it after its been installed?

I've tried once but no success. :(

Another option could be a second hard disk for caching and logging.

jwhostet

I've tried once but no success. Sad

Sad day…

so if soft updates were enabled I wouldn't be having this issue where my internet slowly stops responding?
If that is the case I may just reinstall pfsense!!! (probably not but the thought did cross my mind)

marcelloc

Try to set vfs.read_max to 128 on system->advanced->system tunables.

It will increase disk performance on most systems.

jwhostet

@marcelloc:

Try to set vfs.read_max to 128 on system->advanced->system tunables.

It will increase disk performance on most systems.

…Things just got a little more sad...

I've already increased that
@jwhostet:

nmbclusters set to 32768
vfs.read=128

No big deal. Thanks for all the help! |If I manage to start over fresh again… I'll let you know my results! (things are slow right now, so the possibility is good!) :)

jwhostet

@marcelloc:

did you enabled soft updates on /usr during install to improve disk performance?

One last question… if soft updates were turned on during the install would that have solved my problem with the speed issue I'm having?

marcelloc

I do not install any squid/dansguardian/postfix install without it.

I've never tried read_max 128.

jwhostet

UPDATE

From a live cd 20121018-0934 64bit I reinstalled pfsense I then made sure everything was working like it was. Everything is good to go no problems.
To make things better I now have soft updates on for /var and /usr:

[2.1-BETA0][admin@pfsense.pcd.domain]/root(3): mount
/dev/ad6s1a on / (ufs, local)
devfs on /dev (devfs, local)
/dev/ad6s1d on /var (ufs, local, soft-updates)
/dev/ad6s1e on /usr (ufs, local, soft-updates)
devfs on /var/dhcpd/dev (devfs, local)

I have enabled squid and things are not looking so good. :(
After enabling my speeds have dropped dramatically within the first hour of enabling squid.
Same settings as before!

What now?! Why is this happening? :(

Any other ideas?

Thanks!

jwhostet

Well I spoke to soon seems soft updates might have solved the speed dropping issue. It seems to use almost all 4gb of memory but not complaining too much.

Is it normal to use next to all of the available memory? What if I had other packages that needed to use memory too? ( As of right now I do not but soon I wish to implement others.)

marcelloc

@jwhostet:

Is it normal to use next to all of the available memory? What if I had other packages that needed to use memory too? ( As of right now I do not but soon I wish to implement others.)

yes, It's normal.

You can try to reduce squid memory cache or add more memory to keep a good performance :)

jwhostet

:(

well… apparently I spoke too soon again!

Today I get a message saying the internet is got a 2 second ping and 200kb download which is terrible... after removing squid package everything goes back to normal 20ms ping 32mb download.

I'm assuming squid has been running normal and fine because the load on the router was minimal 5 systems hitting the system hard. Whereas today with a couple more technicians I'm assuming more system continually pulling data.

My question now is why is this happening? Is it hardware related? I estimate about 10gb of bandwidth a day is used. Are we accessing so much data that the system can't keep up?

Any help is once again appreciated, hopefully we can get this solved. If you would like to know any hardware specifics or a specific configuration file let me know and I will get you what you need.

Thanks, Josh

jwhostet

Is my thread dead? :(

I have a system on the network that I thought might have been causing squid to go crazy and hang draining internet to a stop.
So I added it to the list that are supposed to bypass proxy but it did not help solve the issue.

Any body have any ideas?

marcelloc

@jwhostet:

Any body have any ideas?

Can you try to include this on squid.sh startup script after line 4.
Remember to do not save squid config on gui while you test.

sysctl kern.ipc.nmbclusters=65536
sysctl kern.ipc.somaxconn=16384
sysctl kern.maxfiles=131072
sysctl kern.maxfilesperproc=104856
sysctl kern.threads.max_threads_per_proc=4096

jwhostet

Hey thanks for getting back to me!! :)

sorry I didn't get back sooner, school got kinda hectic on me for a couple days.

I can definitely try to do that!
One question though, how do i not save squid config on gui?

Thanks again for the help…. I feel repetitive but I do truly mean it!!!!!

marcelloc

@jwhostet:

One question though, how do i not save squid config on gui?

do not press save on squid gui after changing rc.d file ;)

jwhostet

Haha…. Oh that makes sense. Thanks for the help I forgot this went on to a second page Till right now. I'll probably start testing that tomorrow and hope things go well to report back with good news!!

jwhostet

Well we are having major ISP issues here at work so it might be a couple days before I can truly test your possible solutions will post back asap with results!

marcelloc

Ok :)

quetzalcoatl

I love squid but i find too many limitations, at least with when i use it in pfsense. (i don't like how it works in other platforms)

What is going on right now is that https pages cannot be access with the squid3 package installed from pfsense.
I can use lusca and it works well with both http and https pages but it's very outdated and is not being updated an maintained anymore.

So i want to update my pfsense platform but i really feel that squid3 is useless if it doesn't allow me to work with https just like lusca and squid 2.7 without using crazy configurations.

Also squid crashes when i set the maximum object size in RAM above about 450 KB.

In fact i assign 32 GB of RAM to PFSense of which 20 GB are for dedicated for Squid RAM caching and the only way to use all that RAM and make it useful is having the maximum limit of at least 512 KB or even 1024KB for objects stored in RAM.