Squid 6.5 !! Nov 6th

jrey

pfBlockerNG is not a very good solution as anyone can dodge a DNS blocklist quite easily regardless of the guardrails put in place by the network admin to prevent this.

Enlighten us, I'm curious about this statement.

I don't run squid, no need in this environment, but if I did need a proxy I don't think running it on the firewall would be my first choice, but that's just me. I can certainly understand how many would appreciate having it all-in-one. Has Netgate given a specific reason for removal?

jdeloach

@JonathanLee

Sounds to me like if you think Squid is that important, you need to step up to the plate and volunteer to be the maintainer, to make whatever changes that it needs to work with future versions of pfSense. Netgate has already stated that they do not have the resources to maintain it inhouse. Most all of the addons are maintained by unpaid, volunteers. When the volunteers gets tired of maintaining the package, it usually goes BYE, BYE unless some other user takes ownership of it.

michmoor

@jdeloach said in Squid 6.5 !! Nov 6th:

Sounds to me like if you think Squid is that important, you need to step up to the plate and volunteer to be the maintainer, to make whatever changes that it needs to work with future versions of pfSense.

Multiple ways to be a volunteer and it doesnt include coding (why do people always think it involves that i dont know...).
Secondly as already pointed above there are updates for Squid that addresses some of the security concerns.

jc1976

i'll probably get flagged and banned for saying this...

But opnsense has a full-on proxy server with caching and icap antivirus, and documentation on how to configure transparent autoconfig proxy with wpad-pac..

i'm guessing since squid is open source, the developers at opnsense integrated squid into opnsense like they did with suricata..

do i want/need squid for blocking? no.. i've tried it but in my for me pfblockerng-dev seems to be a more elegant solution in terms of blocklists and updates (since anyone can become part of it).. (yeah, i LOVE pfblockerng (God forbid THAT ever goes away..) and there are A LOT of pfsense users who come back to pfsense for that one package..).
but i DO want the ability to proxy.. i'm no firewall guru by any means but it makes sense to me that if suricata and icap had access to unencrypted streams, they'd be able to perform their function better..

yeah, not having the ability to set up a proxy on the firewall is a pretty big deal..

JonathanLee

Lot's of enterprise class firewalls have certificate issued proxy use for deep packet inspections/submissions.

Also Squid GUI Needs a Do not Cache and a Splice Always button inside the package. That is one thing it really needs.

Screenshot 2023-11-16 at 3.13.46 PM.png

I get hits all day on it once the cache is full, again I do not want it trying to cache everything or splice everything. It should have quick create rule.

https://redmine.pfsense.org/issues/14998

greenlight

pfsense was once a very popular firewall in Turkey and was used by most small businesses. In addition, with a law passed in 2007 in Turkey, content filtering was obligatory in places where public internet was used. There were people who solved most of this problem with squid. (Connections were usually http so it was relatively easy). Of course, later on, there were people who improved this and filtered https, and even commercial software was produced by creating a fork of pfsense.

We owe it all to squid. But nowadays squid is almost dead for pfsense and we can say that it is ignored because very few people need a proxy. With the updates made in the law in Turkey, the need for pfsense directly decreased, and with the increase in commercial software and the new generation moving away from IT, its community almost disappeared.

So what I'm saying is that Squid is orphaned and I can't say his future is very bright after Shalla's closure.

JonathanLee

@greenlight Shalla's closure?

There is other blacklist sites

greenlight

@JonathanLee i know and now i'm using other sites. My point is that squid with HTTPS was not useful enough. MITM is not successful and squid can still cause problems in some installations.

JonathanLee

@greenlight Yes for some however, for others it is still a GREAT cyber security tool. I will most likely follow the vendor where Squid is still supported thus jump to OpenSense. Within Pfblocking you can also see the writing on the wall. The Squid tool set really needs to be part of a system for me. End of story. That was the number one reason I selected Netgate appliances in the first place. I don't need to be told repeatedly Squid doesn't work I flat know it does. It is an advanced configuration and most users don't want to spend the time setting it up. Squid worked perfectly for me, always did.

Screenshot 2023-03-14 at 8.50.09 AM (1).png

michmoor

@JonathanLee said in Squid 6.5 !! Nov 6th:

I don't need to be told repeatedly Squid doesn't work I flat know it does. It is an advanced configuration and most users don't want to spend the time setting it up. Squid worked perfectly for me, always did.

What you are saying is perfectly reasonable. Squid is an advanced type of setup and configuration. But the reality is that enterprise vendors such as Fortinet or Palo use a customized version of Squid that is built to be easy for the admin to use but also because a lot of the advanced tools of those firewalls require the ssl sessions to be broken in order to do the deep packet analysis.

I say all this to say that anyone who says Squid is out of style or not relevant simply doesn't work in the industries i am in. Fintech specifically, MITM is used quite often. Squid on pfSense never really got the love that was needed to make it a better tool (read all the open redmines for examples). I also dont expect Netgate devs to spend time customizing this tool either so now we are in a situation where there is no proxy support and no way of having customized filtering options.
Certain..ahem....Youtube personalities that are very fond of pfSense and see no fault in it, are screaming "protect the endpoint" and although its true, the more accurate way of handling security is to do things in depth. I absolutely want my firewall breaking TLS sessions to inspect packets. I absolutely want my endpoint protection to catch what my firewall didnt.

mcury

I understand what you guys are saying, I thought about this for a long time..

Fortinet and etc, they have teams and robots to categorize websites, they have their own URL categorization, this is expensive to maintain and update. If you report a website that is wrongly categorized, the team will check and work to fix that in a few hours, it never exceeds one day.
Do you guys want a service like that ? Well, you need to pay for it.

Now, when speaking about Squid/Squidguard, if you plan to deploy it for 10 customers, prepare for a lot of trouble because, splice all method is going away, yes, encrypted SNI headers are becoming a reality and the only option you will have is to bump everything (MITM) and you guys know how that can be...
You have to worry about websites as financial, governamental, social media, windows updates, messaging apps and etc....

Now think, you have deployed Squid for 10 customers, can you handle the storm of problems that this will bring ?
If you think that you can tune everything for everyone, and you can handle every problem that appears, even the small ones such as an image in a site don't opening, go ahead and install Squid, but I would say, do it in another device due to the security risks already mentioned.

Now, when speaking about pfblockerNG DNSBL, it is much easier to maintain than Squid, you can tune it much more easily and block everything you want, but it also has problems.
DoH, DOT and QUIC, these can be a problem but as I see it, you wcan work around that but you will be leaving performance on the table because the world is moving forward, protocols are evolving.

So, as I see it, if you want to perform URL filtering based on categories, with everything working and be able to deploy it for a customers without being overwhelmed by problems ? Do it at the end point, this will save you so much trouble and performance and will be so much better.. Or, pay for it, have you guys heard about Zorus project ?

JonathanLee

@michmoor

I am once again lost in the awesome mountain of power that is big tech. To have such tools ripped away under the context of vulnerabilities, is counterintuitive to end user based cybersecurity.

lg1980

Hello,

For anyone interested, I compiled the Squid update for version 6.5 (pfSense 2.7.1) as per the release of the Squid project and made it available here:

https://pkg.pf2ad.com/pfsense/2.7.1/amd64/All/squid-6.5.pkg

How am I going to maintain the pf2ad project (https://pf2ad.com) and for customers who use it want to continue. I will maintain the update and repository for Squid and Squidguard (updating any version/changes).

In the installation script I have already prepared the check and update for the latest version of Squid.

Regards,

Luiz Costa

JonathanLee

@lg1980 I am so happy bro!!! This is amazing. Epic, epicness. Take that spyware!!!! Booyeahhhhh

Thank you

jc1976

@lg1980

Luiz,

I just got home last night from the holiday to see this..

I can't thank you enough!

i am in no way shape or form a firewall/security guru, however in my limited experience i absolute think pfsense is beyond incredible, and it's the independent devs such as bbcan (pfblockerng) and yourself that make this firewall so much better than the be companies ever could..

i'd rather donate to devs such as yourself and bbcan and anyone else who takes time out to contribute because they're devoted to the cause and love their craft than the folks at the big companies who sub-out the work to some sweat-shop halfway full of crappy programmers halfway around the globe all in the name of saving money..

Thanks again!

jc1976

@lg1980

One question?
again, i'm a novice so..

how do we install the version of squid you compiled? i don't have any AD integration for my setup, so i'm only looking to update my installation to the latest.

Also, what is the order of updating/installation between squid and pfsense? i'm currently on pfsense 2.7CE. do i update squid to 6.5 first and then pfsense to 2.7.1?

Thanks!!

lg1980

@jc1976 said in Squid 6.5 !! Nov 6th:

@lg1980

One question?
again, i'm a novice so..

how do we install the version of squid you compiled? i don't have any AD integration for my setup, so i'm only looking to update my installation to the latest.

Also, what is the order of updating/installation between squid and pfsense? i'm currently on pfsense 2.7CE. do i update squid to 6.5 first and then pfsense to 2.7.1?

Thanks!!

I made a little snippet to automate this:

https://gitlab.labexposed.com/-/snippets/14

1 - Update all pfsense and all packages, including squid
2 - Just run the command: "fetch -q -o - https://gitlab.labexposed.com/-/snippets/14/raw/main/update-squid.sh | sh"

Basically it will add the pf2ad repository and update squid, using this repository

I hope it helps !

Welcome !

Luiz Costa

jc1976

@lg1980

i copied and pasted the command you gave me after ssh'ing

"fetch -q -o - https://gitlab.labexposed.com/-/snippets/14/raw/main/update-squid.sh | sh"

(without quotes) into my pfsense box as root and it gave me the following error:

ld-elf.so.1: Shared object "libssl.so.30" not found, required by "pkg"
ld-elf.so.1: Shared object "libssl.so.30" not found, required by "pkg"

the packages i have installed are as follows:

pfsense packages.JPG

i'm still on 2.7.0 (stable).

Thanks!!

JonathanLee

@jc1976
Try this I had the same issue

https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html#upgrade-not-offered-library-errors

jc1976

@JonathanLee

cool, thanks! i'll try it when i get home.

should i bother uninstalling the old squid first?

what will it take to get this package put into the regular package manager? Luiz makes this look pretty effortless, why are the devs fighting the update?