Everything related to squid is bugged as hell
-
Hi everyone,
I'm an unhappy customer who foolishly bought a SG 8860 appliance thinking that it would replace my SonicWall appliance without any issues. What a fool i was….
We use Squid, SquiGuard, ClamAV and Squidanalyser. Squid is configured as explicit proxy wiithout auth BUT with LDAP active.
We heavily depend on content filtering for internet and i have the following issues:
- Squid is almost at 100% all the time. why? dunno... spent countless hours to figure out why. And when it's at 100% it just slow down everything. Tried almost everything on the net without any imporvement.
- Squid in transparent mode is even worst when active, it's real 100% CPU all the time and considerably slowing things down.
- Squid for some reason open a LOT of random ports tcpwrapped on the WAN. Again don't know why. It's not Snort (uninstalled) neither WAN rules. Clearly related to above issue
- SquidAnalyser is just the worst. Cron is not working with it. Randomly selecting dates give errors accessing report page.
- SquidGuard is maybe the only working part of squid, i didn't have that much issues appart sometime where service stops after a restart without any reasons.
- ClamAV, depending of wind direction, can start or not. Sometime complaining about not being able to download signatures, sometimes about socket in use, sometimes about ClamAV to be upgraded...
Lastly FYI, i did a PoC on a standard Dell workstation with 2 Nics and didn't have that much issues tahn with dedicated appliance. Maybe i did wrong to backup config from PoC and applying to appliance. But backup/Restore is for that isn't it?
I do'nt remember performances issues with PoC on standard Dell so i don't know if CPU was running high for Squid. but for information, i could even use transparent proxy without issues.
To be honnest we are ready to pay for professional support but if we pay and they say that they don't support Squid, it's a total non sense for us.
Did you experienced professionnal support of PfSense regarding Squid?
-
Hi,
Those seem to be pretty strange issues, it almost certainly looks like bad configuration. Can you check if PowerD is enabled? Simply go to System > Advanced > Miscellaneous and enable PowerD. That might have been off on your proof of concept box and with config restore it was carried to new box. It can result in slow speeds if it's turned off.
Try that first. Regarding our support, they can help with Squid but not ClamAV or reverse proxy which shouldn't be a deal breaker for you. ClamAV is pretty much just set it and forget it.
-
Indeed, PowerD was unactive.
I just have activated it and will tell you if it's any better.
Thanks for your answer, it's quite reassuring and will talk to my boss to buy a commercial support.
TBH, i really suspect that restoring config from other box where interfaces where absolutly not the same, screwed everything interfaces wise (lot of VLANs on PoC box and just two on dedeicated appliance plus normal interface). Should i give a try to reconfigure everything from scratch? i would like to avoid that as it would take me a lot of time to do so and without any guarantee it would be better, it's not a simple choice)
-
After activating PowerD, my squid is running at 25% which is quite good. Have to wait some more to be sure.
-
Glad to see turning on PowerD helps! If you continue to experience issues I would reconfigure everything from scratch as the old config seems to cause issues.
-
It's really better! I just activated transparent proxy for only one interface, not the main one which stay as explicit and i'm happy to see squid not getting too high with his CPU use (it's around 70-80% now instead of the struggling 99-101% that i was getting anyway with transparent proxy).
Now maybe in the end the false open ports are not related to squid in the end? when i did test yesterday and i stopped squid, nmap was reporting ports that were really open. But squid is restarrting is service by itself so no way to really make sure of it…
Is there any security functionnality implemented in PfSense that would act as an IDS by opening false ports to confuse the port scanner? to me it's quite useless as port scanner is testing ports afterward anyway. But i already removed Snort :( so maybe someting native to PfSense?
Sorry for the stupid questions if it's sounding stupid :)
-
For information, PowerD did very well on our appliance. BUT, it wasn't solved completly. Was still having CPU peak from time to time from Squid.
I think i found what was causing that:
Tracker website that were blocked by SquidGuard like watson.telemetry.microsoft.com but there were too many requests on Squid (like 100 per seconds maybe more) for Squid to follow. Once i put most redundant domains in Squid ACL blacklist, Squid was happy again and not going higher than 50% CPU usage. Even on low firm activity aka no users in office, Squid is running at 5% and i never saw a lower level than this.
Now CPU is mostly took by Traffic Shaper but that's ok, it's efficient.
-
I'd utilize pfBlockerNG for blocking domains via its DNSBL (or IP for that matter). It's much faster and better. What's your network size, how many clients are there?
