Squid 3 eating my bandwidth

hansaya

I noticed two time today, my bandwidth getting eaten up completely. When i checked it was getting used by the router it self. When I switched off the transparent proxy, it stopped. What causes it? how to fix it? I would really do love to use this feature and seems to work marvelously when its working.

Harvy66

Others with more experience will eventually post something, but until then, I've seen others have similar issues and it was because stuff like Windows Updates and other things were being proxied. Huge files.

You can check to see what IP addresses it's communicating with. Assuming not HTTPS traffic, you can inspect the data to see what it's requesting.

KOM

It's not that the Windows Updates files are big, it's that the default config doesn't handle WU well at all and even selecting the Dynamic Content box with Windows Update doesn't help (last time I checked anyway). The dynamic range requests by Windows get turned into a full file download. For example, when Windows wants a file, it will request a chunk of that file. Squid will fetch the entire file every time when just a chunk is requested. That means that a 100 MB update might take 1 GB of bandwidth as it downloads the entire 100MB for every 10 MB requested. If you really need to cache WU, install WSUS in your AD domain.

Guest

I noticed two time today, my bandwidth getting eaten up completely.

Squid can be really fine tuned and configured that it will be matching all the criterias
at your network and the usage.

Home, SOHO, SMB or enterprise network and their needs are often not even the same.
That means, pfSense & VPN there fore the hardware will be sufficient enough, but
with Squid it would be going lmaing the entire system.

When i checked it was getting used by the router it self.

I really and personally think it is something around with the cache capabilities,
the free RAM and the power of the CPU. That is not a must be, but more then often.
What is your Squid configuration?

I would really do love to use this feature and seems to work marvelously when its working.

For sure who will not do so? I think you should be sure about what you will or must be inspecting by Squid!
Do you need inspecting Linux or BSD ISO images? YouTube or netflix? Large files?
All this should be configured by you.

How many RAM do you own in this pfSense?
The default is 32 MB, if you have a huge amount of RAM you are able to high up this number
How many employees or peoples must be served?
If the entire given hardware is not powerful enough and then Squid on top…
What is the entire Cache dimension you was setting up?
A mSATA or SSD can really speed up those action
Is the CPU not strong enough to handle all this load?
Pentium G3260, C2758 or a Intel Xeon E3 is a big difference
Too many services perhaps like, Snort, Squid & SquidGuard, HAVP, DPI, VLAN, QoS, Traffic Shaping,…....?
I really mean the used hardware should also be matching the entire services
To small or to slow cache perhaps.
Would be in my eyes an urgent point

hansaya

Thanks for the replies. Im hoping to at least get some amount bandwidth saving and if i could cash things like windows updates and youtube, that will be great. You guys pointed out that i might not have correct config, then can someone point out to the best config i can have?. I could not find much on this, while googling. BTW I'm using this in a house environment with 5 people, heavily using internet. I had previous experience 2 years ago when I saved a school dorm network with pfsense, even then with better hardware and 600 clients. I could not get the cashing to work properly, ultimately keeping it turned off.

im using dual core intel Celeron with 2gb of ram(in the process of getting 4gb) with 32gb SSD

This is one of the last things log spit out before it started to download something full bandwidth

14.07.2015 23:19:19 172.168.1.18 TCP_MISS_ABORTED/206 http://108.175.38.241/? - 108.175.38.241
14.07.2015 23:19:19 172.168.1.18 TCP_MISS_ABORTED/206 http://108.175.38.241/? - 108.175.38.241

Guest

At general to talk about your amount of RAM, a safe number is 10 MB RAM for every 1 GB of cache space on disk.
Please have also a look to all other services that should be running smooth, like Snort, like pfSense it selfs,…....

Squid Package Tuning would be a good
point to start.

Squid Proxy Server 3.1: Beginner's Guide & pfSense: The Definitive Guide Version 2.1
I think this two books would be also really interesting for you if you want to dive deeper in the
material, for longer winter's evenings.