Squid 5.8 ---> 5.9
-
Again after that announcement Squid has been updated to 5.9 in Plus on ARM so something was fixed in the package, if they were going to abandon it why take the time to release a new version... right?
-
@Gertjan 6.6 ??? So cool
-
@JonathanLee said in Squid 5.8 ---> 5.9:
@DBMandrake That's amazing so the free version of pfsense non plus version has 6.3?
Also do you use the cache?
if you do I highly recommend you enable
pipeline_prefetch 100 in custom area this lets Squid request more than one item at a time really speeds it upCan't say I've really noticed any performance issues with Squid - and that's with around 800 clients, but I will check out this option anyway, thanks.
The only real issue I have with squid is this issue....
-
@JonathanLee said in Squid 5.8 ---> 5.9:
Again after that announcement Squid has been updated to 5.9 in Plus on ARM so something was fixed in the package, if they were going to abandon it why take the time to release a new version... right?
Continuing to update the package (at least in Plus, the last time it was updated in CE was with 2.7.0 I think) despite saying they're deprecating it is a bit odd I agree, but I don't think that means they've changed their mind. I guess we will see.
Unless they explicitly come out and say they've changed their mind, anybody who relies on the feature will already be actively investigating alternatives - I know I am.
-
Thanks for your very detailed words.
You're some one who knows about the why part, as it is clear that you've been looking for the real reasons on the right places
My post wasn't about finger pointing to some one or something like that : everybody is free to decide what they do with their software.
I do get some times a bit upset as I read that a lot is taken for granted, so I started to enumerate possible reasons. And by no means I've listed all the reasons, or the correct reasons. Just that these projects are driven by people and they can have 'reasons'.As you've stated very clearly, and I guess a lot of us don't know : "pfSense" (the pfSense squid package maintainer) grabs the squid source, adapts it so it compile in a pfSense environment, then he write, a whole lot of GUI glue script code and whatever else that is needed so that it shows up and can be used as a package under pfSense. Then rince and repeat, as this is an never ending story.
I'm not sure if it is a Netgate employee, or some other, 'freelance' maintainer.The reason of my post above was not at all because I know why Netgate posted this : Deprecation of Squid Add-On Package For pfSense Software I've just read the blog post and interpreted my thoughts about it. I saw it as a message like a "be aware that we could pull the plug on this package because ..." as people (pfSense users) depend on it, so that they were warned that Netgate can pull the plug.
edit : your case 1) is somewhat what I thought. Not that I would use the word 'blame', as it might be the simple result of the equation : what does it deliver, and what are the costs.
Ripping it out is the easy, least effort path to follow.
and probably the only choice they have.I was surprised of the reaction of some pfSEnse users that were asking ; can Netgate not just make Squid better, fork it, or do something to solve the issue, as they don't know that squid is ... HUGE ....
I agree with everything you said, although I really think that :
Personally I think that if they really do believe Web proxy/content filter is not an important feature they're making a big mistake and it's an incredibly short sighted move
this can't be the case. Or, differently, I hope that's not the case.
Because of what you said :All the removal of Squid will do is lose them userbase and also paying customers.
Important info : I'm not a squid user myself.
And yes, It might be possible that my humor was a bit borderline. -
@DBMandrake said in Squid 5.8 ---> 5.9:
I think you're being more than a little disingenuous here.
@Gertjan said in Squid 5.8 ---> 5.9:
The issue was (my words) : squid wasn't supported anymore by the maintainers, they couldn't follow the passe, lost interest - or it became just to complex.
What maintainers ? Upstream squid maintainers ? Squid package maintainers at Netgate ?
At the time the deprecation announcement was made the Squid version in PFSense was version 5.8 and version 6.5 of Squid was available. Since then three more point releases of Squid have come out. Whose fault was it that PFSense was 7 point releases and one major release version behind and complaining about bugs not being fixed ?
How is squid "not supported" anymore by the developers ? Have you even looked at the releases page on the squid website ?
https://www.squid-cache.org/Versions/
The most recent version is 6.8 released on March 4th this year. There have been 10 point releases of Squid in the last year alone from 6.0 through to 6.8, in that time there has been one update to the Squid package in PFSense CE. (Not sure about Plus)
Squid 5 saw 18 point releases from 2020 to 2023, so development has actually accelerated if anything since then.
@Gertjan said in Squid 5.8 ---> 5.9:
This can have big repercussions on the security of the package, so Netgate "warned' that they have thresholds that can make them decide to not include Squid as a package. So see the Netgate blog post as a public message to all parties : move your a** or wi'll stop including it. This means that most active squid users now contact the squid support , the maintainers, directly, to motivate them (in the modern world, this means : sending a check would do it - or harassing them on social media, very popular).
This is incredibly arse about face and shows what is wrong with how open source projects are treated these days. The Squid developers don't owe anyone ANYTHING, least of all Netgate.
Netgate is taking an open source project - Squid, for free, building it into their firewall product, and charging people for it, at least for PFSense Plus, and making money selling hardware whose software feature list includes (or used to include) a web proxy/filter which included transparent proxying and filtering.
Many people will have bought Netgate hardware or started using PFSense plus partially on the basis that a content filtering web proxy (an essential part of any enterprise firewall) is in the feature set. I know I would not have started using PFSense if it did not have Squid/Squidguard in it as it is a feature I require, and the lack of this feature will force me to abandon PFSense when the time comes as I doubt they are going to change course on removing Squid.
Ok, so lets say Netgate have identified some specific security bugs in Squid that concern them, first of all, see if these bugs are fixed in more recent versions and actually update your package to the latest squid. If you're 7 versions and nearly a year behind then yeah, there's going to be bugs, self inflicted bugs because you're not keeping the package up to date.
But lets say that the specific bugs that concern them are not fixed in the latest Squid version either - guess what, it's open source, they can fix the bugs directly, compile a patched version of the package including the fixes for their customers and contribute the fixes upstream! Just like they do with the FreeBSD kernel... I'm sure the Squid guys would be very appreciative to get some help and get some fixes pushed upstream in the spirit of Open Source cooperation rather than just get demands and threats that bugs must be fixed.
Netgate's blog post is basically saying "We're annoyed that this package that we get for free and build into our commercial product has some security bugs but we don't think Squid is a high priority feature so we don't want to spend the effort in house to fix these bugs ourselves and contribute the fixes upstream so we're going to threaten to remove the package in the hopes that something will happen upstream or that our users will complain upstream"
That's just a terrible take on how open source works. The upstream Squid developers don't care if PFSense drops Squid support - why would they ? If PFSense users go and complain to the Squid developers as you're suggesting they're complaining to the wrong people.
If Netgate wants to sell a firewall product that includes a web proxy with filtering (I would argue a core piece of functionality of any enterprise edge firewall/router) and the only open source project that meets that need, albeit in an imperfect way is Squid and Squidguard, then it's beholden on Netgate to make the effort to help address any perceived security bugs that open source project might have, and in doing so, contribute back to the open source community that their PFSense product is built on. (Nearly everything in PFSense below the web GUI is open source projects including the FreeBSD OS base)
I can only see two possible reasons why Netgate is really taking the action they currently are with Squid:
-
They don't have the interest or resources to have a Netgate staffer work on the Squid source code to attempt to fix security bugs, (and upstream them) but instead of admitting this they want to pass all the blame upstream and just call the project that they've included in PFSense for free for over a decade a steaming pile of security bugs and "not their problem". Ripping it out is the easy, least effort path to follow.
-
They genuinely don't think a web proxy/filter is an important feature of a firewall, and have no qualms about removing a feature that many people bought their hardware for.
Granted, I am using PFSense CE (in an education environment) so I haven't bought their hardware, but I have seriously considered over the last year I've been using CE upgrading to PFSense Plus on the white box hardware that I'm using to get official TAC support, but now that Squid removal has been announced that definitely won't be happening, and in fact I will be forced to leave PFSense when Squid is actually removed. I would have been pretty pissed if I'd upgraded to Plus and a few months later one of the key feature I need was removed with no remorse and no workable alternatives. (Other than leaving PFSense)
Personally I think that if they really do believe Web proxy/content filter is not an important feature they're making a big mistake and it's an incredibly short sighted move. I can't think of any other comparable firewall/router from other vendors that does not have some kind of web content filtering in it. All the removal of Squid will do is lose them userbase and also paying customers.
Everything I’ve been saying in the forums since the announcement of the removal of Squid in pfsense. Glad I’m not alone
edit: To be fair to netgate, their documentation does point out that using proxies is no longer effective. https://docs.netgate.com/pfsense/en/latest/recipes/block-websites.html#using-a-proxy
That being the case there really is no excuse to be behind on numerous releases and possible package fixes and then at the same time state the package is insecure. I always found the Netgate response to be nonsensical at best and at worst extremely disingenuous because as I and other have pointed out - they are taking freeware , bundling it into a paid product and then complaining that the package is insecure without contributing anything upstream to the project. The same criticism they give to the OPNsense devs may i add (They pull from upstream but contribute nothing back). A classic example of don't throw rocks in a glass house.
edit2: One last thing. There is no current mainteainer of the Squid package for pfSense. I have reached out to the maintainer of record for SquidGuard and he informed me over email that he hasn't been involved in the package for quite some time. So the package may be getting updated but improvements to it will require someone to take charge of the plugin. I believe Netgate's position should be rather - because there is no maintainer we cannot continue to offer the package as we cant ensure its continued security. Thats much better than saying "Upstream doesn't fix anything" which turns out to be silly.
-
-
@DBMandrake I have never seen that error with my configuration, I will send it to you once it reboots
-
@JonathanLee said in Squid 5.8 ---> 5.9:
@DBMandrake I have never seen that error with my configuration, I will send it to you once it reboots
You won't see that error in the logs unless you add the debug options that I describe in the ticket. If you add the debug options in the (normally hidden) advanced settings, restart squid, then use the transparent proxy with some clients for a while and check the cache.log (at /var/squid/logs/cache.log) you'll see lines like those, particularly with any large CDN.
This is the single biggest limitation of Squid by far at the moment, and it's actually a bug, as the configuration option relating to this dns test does not behave as described in the squid documentation. Anyway, it's all in the ticket.
-
@michmoor said in Squid 5.8 ---> 5.9:
Everything I’ve been saying in the forums since the announcement of the removal of Squid in pfsense. Glad I’m not alone
I'm sad because I think PFSense is one of the best and most feature rich firewalls out there - although I've heard about it for years I only tried it the first time about a year and a half ago and started using it seriously a year ago.
I was forced at very short notice to abandon the firewall system we had previously been running, (vendor went bankrupt) and spent a lot of time setting up PFSense in a fairly complex environment that makes full use of a large percentage of the features (Things like OpenVPN, DHCP relay, Avahi, udpbroadcast forwarder, etc) and after a couple of teething problems it has been solid as a rock and has never failed to have a feature that I have needed.
Removal of Squid/Squidguard will force me to yet again look for another firewall and to be honest there is little out there that can compete with the feature rich functionality of PFSense, because I have come to rely on so many of the features in the configuration I'm now running.
edit: To be fair to netgate, their documentation does point out that using proxies is no longer effective. https://docs.netgate.com/pfsense/en/latest/recipes/block-websites.html#using-a-proxy
"No longer effective" depends what you're using it for... Is a Squid proxy still effective as a bandwidth saving cache ? No, and it hasn't been for about 8 years due to ubiquitous use of SSL/TLS. I don't have caching enabled.
Is it still useful as an explicit proxy server (proxy configured on the client) optionally with user authentication ? Yes.
Is it still useful as a content filter ? Yes, partially. Without MITM inspection and client side certificates (which is complex and problematic to set up) it's impossible for it to do any kind of deep inspection like keyword matching, but SNI inspection still works.
SNI inspection doesn't allow the full URL to be found but it allows the domain name for the request to be found whether you're using explicit proxy or transparent proxying.
This is way more useful that it sounds. Coupled with Squidguard this makes a powerful domain name based block list - and importantly for us, you can apply different domain blocklists for different IP ranges / VLAN's, something that is very difficult to do with DNS based blocking.
As a means of filtering out specific websites (based on categories and/or individual domain names) the Squid/Squidguard combination is more powerful and more thorough that any form of DNS blocking like PFBlocker.
SNI based domain name filtering is what we use Squid/Squidguard for - as an explicit proxy for devices where this can be configured, and also as a transparent proxy as a fall-back. Unfortunately the bug I linked to earlier does cause some issues in transparent mode, but this bug should be fixable.
That being the case there really is no excuse to be behind on numerous releases and possible package fixes and then at the same time state the package is insecure. I always found the Netgate response to be nonsensical at best and at worst extremely disingenuous because as I and other have pointed out - they are taking freeware , bundling it into a paid product and then complaining that the package is insecure without contributing anything upstream to the project. The same criticism they give to the OPNsense devs may i add (They pull from upstream but contribute nothing back). A classic example of don't throw rocks in a glass house.
I don't know for certain that Netgate don't or haven't contributed to Squid in the past (maybe they have) but if the issue they're not happy with is security bugs, the best thing to do is dig into the code and help fix those bugs and upstream the fixes.
Bug fixes are generally accepted into upstream projects much more readily than new features or major code refactors.
edit2: One last thing. There is no current mainteainer of the Squid package for pfSense. I have reached out to the maintainer of record for SquidGuard and he informed me over email that he hasn't been involved in the package for quite some time. So the package may be getting updated but improvements to it will require someone to take charge of the plugin. I believe Netgate's position should be rather - because there is no maintainer we cannot continue to offer the package as we cant ensure its continued security. Thats much better than saying "Upstream doesn't fix anything" which turns out to be silly.
Some of the packages in PFSense are indeed maintained by volunteers who don't work for Netgate, although I'm not sure if that was the case with Squid and/or Squidguard.
If volunteers for key packages are burnt out / busy / not interested anymore then that's an issue Netgate needs to find a solution to. The proxy features were listed on the website as features of the product right up until the announcement that it was going to be removed.
As I said, I'd have been pretty annoyed if I'd forked out for hardware (or even the TAC subscription) based on the feature set at the time I bought it, only to have the feature deprecated a few months later.
While I'm not hopeful, I do hope they reconsider, because if nothing else PFSense will no longer be the "swiss army knife" of firewalls that it is now if a key feature like this is removed. It's hard to recommend a firewall with a key feature like that missing.
The ideal solution from the point of view of end users would be if Netgate put the resources into maintaining packages such as Squid and actively working on security fixes for them where necessary. Here's hoping.
-
FYI------>
Hello fellow Netgate community
Squid 6.6 is available in bata version 24 I am working on some bugs on it but it's more secure and fixes all the concerns as it is the latest version of Squid. Squid is very effective at blocking URLS and I am working on finding a way to access the menu with Squid support also. I have some GitHub pulls open for the issues. More to come -
@slu that stateement was released before 6.6 was available
-
@JonathanLee said in Squid 5.8 ---> 5.9:
FYI------>
Hello fellow Netgate community
Squid 6.6 is available in bata version 24 I am working on some bugs on it but it's more secure and fixes all the concerns as it is the latest version of Squid. Squid is very effective at blocking URLS and I am working on finding a way to access the menu with Squid support also. I have some GitHub pulls open for the issues. More to comeAre you working on the official squid package in PFSense or on an alternative ?
If the former this is interesting news...
-
@DBMandrake Official Version only issue with it is
Per Squid Support Amos Jeffries
"You do have direct proxy (and thus manager) access via the 192.168.1.1:3128 so this URL should work:
http://192.168.1.1:3128/squid-internal-mgr/menu"Per Alex Rousskov Squid Support
"Currently, you may need to figure out what hostname Squid considers to self-identify as and use that hostname in cache manager requests. The following bug report may help, but there are several overlapping problems here, and that makes it difficult to triage without more information: https://bugs.squid-cache.org/show_bug.cgi?id=5283"https://bugs.squid-cache.org/show_bug.cgi?id=5283
It works great Blocks URLs some software convergence issues but nothing really major might need a new SSL/TLS cert made but that's about it
-
@JonathanLee said in Squid 5.8 ---> 5.9:
@DBMandrake Official Version only issue with it is
Per Squid Support Amos Jeffries
"You do have direct proxy (and thus manager) access via the 192.168.1.1:3128 so this URL should work:
http://192.168.1.1:3128/squid-internal-mgr/menu"Per Alex Rousskov Squid Support
"Currently, you may need to figure out what hostname Squid considers to self-identify as and use that hostname in cache manager requests. The following bug report may help, but there are several overlapping problems here, and that makes it difficult to triage without more information: https://bugs.squid-cache.org/show_bug.cgi?id=5283"https://bugs.squid-cache.org/show_bug.cgi?id=5283
It works great Blocks URLs some software convergence issues but nothing really major might need a new SSL/TLS cert made but that's about it
What about this bug ?
https://redmine.pfsense.org/issues/14390
This is a biggy - it has existed in Squid for over 10 years and causes major problems with CDN networks with rapidly rotated multiple IP address hostnames. (only with transparent proxying)
The bug has been there for years, what has changed is CDN's have started to very aggressively rotate DNS entries with TTL's as short as 30 seconds or less, this has made the symptoms trigger far more often than in the past, and this single issue is responsible for nearly all intermittent behaviour and connection failures (HTTP/409) in transparent proxy mode.
A fix for this would be massive.
-
@DBMandrake Was the Bug also listed in Squid Bugs? I thought that was closed out in Squid 6.6 they had an open report on the Squid side also. Again they now have the version 6.6 installed if you have the ability to do BE (Boot Environments) Check it out I opened some pulls for it just small issues with Squid -k parse. Again the menu issue with making squid know its identity. Main thing is it works, I can block URLS and cache traffic in 24. I might need a new certificate. All just seem like Squid to GUI software convergence for user convenience is all. Key issue is resolved the security concerns it is running Squid 6.6 and it has all the updates in that version. The Squid DEV version is 7.X right now the stable version is 6.X
I wish I knew more I love this package I am glad Netgate did not give up on it. Squid really shines when you configure it correctly.
-
@DBMandrake They have had to do something they use acceleration systems globally. Squid is used all over it's huge. It is technically green technology as items are not downloaded a million times over and over they are downloaded once and stored closer to the client. Again software hardware convergence is a bit complicated.
This really interests me (see below), Facebook is actively working on improving cacheing, they even have an open source project. It does save energy.
2016
https://research.facebook.com/blog/2016/4/the-evolution-of-advanced-caching-in-the-facebook-cdn/
2021
https://engineering.fb.com/2021/09/02/core-infra/cachelib/
https://cachelib.orgOverall cacheing fascinates me as a computer science student, the complications, the protocols, and the challenge. It is amazing when it works correctly. This tool is amazing when it's configured correctly.
-
@JonathanLee said in Squid 5.8 ---> 5.9:
@DBMandrake Was the Bug also listed in Squid Bugs? I thought that was closed out in Squid 6.6 they had an open report on the Squid side also.
I originally reported this bug 11 months ago and only to PFSense (discussed on the forum here in another thread then I opened the ticket) as at the time it was running a very out of date version of squid.
I haven't seen any upstream reports for this issue, do you have a link to that if you think it is fixed in 6.6 ?
There has been no update in the original redmine ticket so I assumed that the problem has not been fixed.
Again they now have the version 6.6 installed if you have the ability to do BE (Boot Environments) Check it out I opened some pulls for it just small issues with Squid -k parse. Again the menu issue with making squid know its identity. Main thing is it works, I can block URLS and cache traffic in 24. I might need a new certificate. All just seem like Squid to GUI software convergence for user convenience is all. Key issue is resolved the security concerns it is running Squid 6.6 and it has all the updates in that version. The Squid DEV version is 7.X right now the stable version is 6.X
Unfortunately I'm running 2.7.2 CE which currently only has Squid 6.3, and when I checked recently version 6.3 still has this issue, and boot environments are not supported in CE either. Based on previous release cycles of CE it could be a long time (6 months or more) before 6.6 found its way into CE.
Are you able to try the Python test script I attached to the PFSense ticket ? This can be run on any PC with Python installed.
For the test to be valid transparent proxying needs to be enabled on PFSense with the client going through the transparent proxy with no explicit proxy settings.
The way it works is it resolves a test hostname known to cause issues (I have at least 4 in the script that can be tried) and saves the IP address, it then crafts an HTTPS query to the same IP address every 30 seconds instead of freshly looking up the hostname. This is to simulate use of a "stale" DNS record.
When this IP address times out of the (rapidly rotated) DNS records, Squid will start to refuse the requests with HTTP/409.
If the same test is run with no transparent proxy there will be no errors, so this script is a reliable way to reproduce the issue.
I wish I knew more I love this package I am glad Netgate did not give up on it. Squid really shines when you configure it correctly.
So far I've seen no official word from Netgate that they've reversed their decision, so I hope you're right.
If they have changed their mind and are bringing the package right up to date and as a side effect this long standing Squid bug is fixed that would be absolutely fantastic.
-
@DBMandrake I test it when I get some time, I can’t do tests while family is home so I have what I call the everything bagel boot environment running sometime in the week I will check it the script.
-
@JonathanLee I've done a little testing myself today.
Using the following I was able to install a Squid 6.8 build on 2.7.2 CE on a spare test firewall:
https://forum.netgate.com/topic/186911/squid-6-8-available
Unfortunately this does not fix the issue as it still fails my python testing script (note that some of the domain names I've given as examples in the script are no longer valid so you'll need to find some that work) and also the errors are still logged in cache.log.
So as of Squid version 6.8 this problem still exists unfortunately. I have also not been able to find mention of any fixes for this issue in recent squid release notes.
-
@JonathanLee Here is the discussion about this issue:
https://bugs.squid-cache.org/show_bug.cgi?id=4940
A patch was first provided nearly 4 years ago but it has still not been accepted. (!)
The problem has been known about for over 10 years, as the discussion shows.
There is a patch attached to the ticket which is basically a one liner change that fixes the issue. This is something that netgate could choose to include in their version of the package if they wanted to. (And if they were satisfied it was safe)
