23.01 Squid issue

hugoeyng

@rcoleman-netgate sorry for my lack of knowledge. I installed pfSense on a personal computer that uses its own firmaware. So, what other firmware should I update?

rcoleman-netgate

@hugoeyng I believe they're suggesting reinstallation of pfSense but you'll have to ask @JonathanLee

hugoeyng

@rcoleman-netgate Hmmm ... I got It. I think that is insane to reinstall pfSense as a solution to a package of pfSense to work.

Also, I think that even I reinstall pfSense will no solve the root of this question: squidGuard package will not install correctly while they do not correct the bug that causes this issue.

JonathanLee

@hugoeyng reinstall software is considered 'firmware' in PfSense Netgate models

JonathanLee

@hugoeyng Hypervisor, VirtualBox, VMware? Anything that you can spin up a machine you can load PfSense on to test without using your real equipment. They create a virtualized machine that you can install software on to test with. Windows 10-11 pro the software under features enable Hypervisor-V, Virtual box is free it's made by Oracle, VMware costs money. I recommend just using Hyper-V in Windows, it's really simple to use create a blank Virtual machine and load the software like you would on a normal machine it runs in a virtualized environment and you can test save snapshots test again and if you find something that works and doesn't have issues, repeat it with the real hardware. If you have access to a VM you can test it with a clean install, the reason is maybe your configuration is bonked up. You can test it outside the machine this way. I like virtual box too I use to use it with Palm Pre

https://www.ceos3c.com/pfsense/install-pfsense-on-virtualbox/

rcoleman-netgate

@jonathanlee A better guide is here: https://docs.netgate.com/pfsense/en/latest/virtualization/index.html

stephenw10

@hugoeyng said in 23.01 Squid issue:

I think that is insane to reinstall pfSense as a solution to a package of pfSense to work.

Yeah, that really should never be required. It looks like some part of the install/deinstall script is failing there. Since that doesn't happen for most people it's probably something in the config that already exists.

hugoeyng

@stephenw10 What config would you bet on, since I've already uninstalled and reinstalled both Squid and SquidGuard.

In my point of view, the problem is that the squidguard.conf comes with the installation of SquiGuard and it is this "bugged" squidguard.conf that prevents the correct installation of the package.

Once the installation failed, changing the squidguard.conf doesn't solve it anymore, even applying the proposed patch.

The first question to fix is making SquidGuard appear in the menu and in the list of services, even stopped (red 'x' icon). If that happens, I guess the patch could solve the initial bug, about what this post discusses.

JonathanLee

@stephenw10 I agree, a blank install and no user config should fix it. After apply the patches and once that is done put the config back on.

stephenw10

The package install script is hitting some php error and failing before it has added the require menu and service items into the main pfSense config.
Usually when I've seen that it's because the system has been upgraded and the php versions mismatch. But as I understand it that shouldn't be the case here?

JonathanLee

@stephenw10 Remember this was the same issue I had and it turned out I had to resave the reverse proxy to get it to stop. With that being said I thought maybe there is a different configuration item bonked up with his settings that is missing.

stephenw10

That issue prevented you making a config change in Squid but it did not prevent installing/uninstalling AFAIK.

hugoeyng

@stephenw10 said in 23.01 Squid issue:

The package install script is hitting some php error and failing before it has added the require menu and service items into the main pfSense config.
Usually when I've seen that it's because the system has been upgraded and the php versions mismatch. But as I understand it that shouldn't be the case here?

I tried to manually insert the SquidGuard in the Menu Options.

First I made a backup of the current installed pfSense .

Then, I took a old backup XML file where the SquidGuard was present in the menu, copied the lines that referred toSG to the recent XML and then I restored it.

Did not work. Raised an error and I did not try again.

stephenw10

If you remove your squidguard config from the config file and then try to reinstall squidguard does it succeed?

hugoeyng

@stephenw10 No. It doesn´t.

stephenw10

Hmm, so it throws those same errors even with no squidguard config present?

And that's after uninstalling and reinstalling the pkg?

hugoeyng

@stephenw10 Yes. A clean installation of squidGuard will fail showing the same errors.

stephenw10

Is that with those patches applied still?

I can't replicate this. It installs fine in 23.01 on a test box here:

>>> Installing pfSense-pkg-squidGuard... 
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 16 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	arc: 5.21p [pfSense]
	arj: 3.10.22_9 [pfSense]
	brotli: 1.0.9,1 [pfSense]
	c-icap: 0.5.10,2 [pfSense]
	c-icap-modules: 0.5.5 [pfSense]
	clamav: 0.105.1_1,1 [pfSense]
	db5: 5.3.28_8 [pfSense]
	krb5: 1.20 [pfSense]
	libmspack: 0.10.1 [pfSense]
	pfSense-pkg-squid: 0.4.45_10 [pfSense]
	pfSense-pkg-squidGuard: 1.16.18_20 [pfSense]
	squid: 5.7 [pfSense]
	squidGuard: 1.4_15 [pfSense]
	squid_radius_auth: 1.10 [pfSense]
	squidclamav: 7.2 [pfSense]
	unzoo: 4.4_2 [pfSense]

Number of packages to be installed: 16

The process will require 47 MiB more space.
10 MiB to be downloaded.
[1/16] Fetching libmspack-0.10.1.pkg: .......... done
[2/16] Fetching pfSense-pkg-squidGuard-1.16.18_20.pkg: ...... done
[3/16] Fetching krb5-1.20.pkg: .......... done
[4/16] Fetching db5-5.3.28_8.pkg: .......... done
[5/16] Fetching squidGuard-1.4_15.pkg: ...... done
[6/16] Fetching squidclamav-7.2.pkg: .......... done
[7/16] Fetching clamav-0.105.1_1,1.pkg: .......... done
[8/16] Fetching arj-3.10.22_9.pkg: .......... done
[9/16] Fetching arc-5.21p.pkg: ..... done
[10/16] Fetching c-icap-0.5.10,2.pkg: .......... done
[11/16] Fetching brotli-1.0.9,1.pkg: .......... done
[12/16] Fetching squid_radius_auth-1.10.pkg: .. done
[13/16] Fetching c-icap-modules-0.5.5.pkg: .......... done
[14/16] Fetching pfSense-pkg-squid-0.4.45_10.pkg: ......... done
[15/16] Fetching squid-5.7.pkg: .......... done
[16/16] Fetching unzoo-4.4_2.pkg: ... done
Checking integrity... done (0 conflicting)
[1/16] Installing libmspack-0.10.1...
[1/16] Extracting libmspack-0.10.1: ......... done
[2/16] Installing arj-3.10.22_9...
[2/16] Extracting arj-3.10.22_9: .......... done
[3/16] Installing arc-5.21p...
[3/16] Extracting arc-5.21p: ...... done
[4/16] Installing brotli-1.0.9,1...
[4/16] Extracting brotli-1.0.9,1: .......... done
[5/16] Installing unzoo-4.4_2...
[5/16] Extracting unzoo-4.4_2: ..... done
[6/16] Installing krb5-1.20...
[6/16] Extracting krb5-1.20: .......... done
[7/16] Installing clamav-0.105.1_1,1...
===> Creating groups.
Creating group 'clamav' with gid '106'.
Using existing group 'mail'.
===> Creating users
Creating user 'clamav' with uid '106'.
Adding user 'clamav' to group 'mail'.
[7/16] Extracting clamav-0.105.1_1,1: .......... done
[8/16] Installing c-icap-0.5.10,2...
===> Creating groups.
Creating group 'c_icap' with gid '959'.
===> Creating users
Creating user 'c_icap' with uid '959'.
[8/16] Extracting c-icap-0.5.10,2: .......... done
[9/16] Installing db5-5.3.28_8...
[9/16] Extracting db5-5.3.28_8: .......... done
[10/16] Installing squidclamav-7.2...
[10/16] Extracting squidclamav-7.2: .......... done
[11/16] Installing squid_radius_auth-1.10...
[11/16] Extracting squid_radius_auth-1.10: .... done
[12/16] Installing c-icap-modules-0.5.5...
[12/16] Extracting c-icap-modules-0.5.5: .......... done
[13/16] Installing squid-5.7...
===> Creating groups.
Creating group 'squid' with gid '100'.
===> Creating users
Creating user 'squid' with uid '100'.
===> Creating homedir(s)
===> Pre-installation configuration for squid-5.7
[13/16] Extracting squid-5.7: .......... done
[14/16] Installing squidGuard-1.4_15...
[14/16] Extracting squidGuard-1.4_15: ...... done
[15/16] Installing pfSense-pkg-squid-0.4.45_10...
[15/16] Extracting pfSense-pkg-squid-0.4.45_10: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...done.
Executing custom_php_resync_config_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.
[16/16] Installing pfSense-pkg-squidGuard-1.16.18_20...
[16/16] Extracting pfSense-pkg-squidGuard-1.16.18_20: .......... done
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Executing custom_php_install_command()...done.
Executing custom_php_resync_config_command()...done.
Menu items... done.
Services... done.
Writing configuration... done.
Please visit Services - SquidGuard Proxy Filter - Target Categories and set up at least one category there before enabling SquidGuard. See https://docs.netgate.com/pfsense/en/latest/packages/cache-proxy/squidguard.html for details.=====
Message from db5-5.3.28_8:

--
===>   NOTICE:

The db5 port currently does not have a maintainer. As a result, it is
more likely to have unresolved issues, not be up-to-date, or even be removed in
the future. To volunteer to maintain this port, please create an issue at:

https://bugs.freebsd.org/bugzilla

More information about port maintainership is available at:

https://docs.freebsd.org/en/articles/contributing/#ports-contributing
--
===>   NOTICE:

This port is deprecated; you may wish to reconsider installing it:

EOLd, potential security issues, maybe use db18 instead.

It is scheduled to be removed on or after 2022-06-30.
=====
Message from squid_radius_auth-1.10:

--
===>   NOTICE:

The squid_radius_auth port currently does not have a maintainer. As a result, it is
more likely to have unresolved issues, not be up-to-date, or even be removed in
the future. To volunteer to maintain this port, please create an issue at:

https://bugs.freebsd.org/bugzilla

More information about port maintainership is available at:

https://docs.freebsd.org/en/articles/contributing/#ports-contributing
=====
Message from squid-5.7:

--
o You can find the configuration files for this package in the
       directory /usr/local/etc/squid.

     o The default cache directory is /var/squid/cache/.
       The default log directory is /var/log/squid/.

       Note:
       You must initialize new cache directories before you can start
       squid.  Do this by running "squid -z" as 'root' or 'squid'.
       If your cache directories are already initialized (e.g. after an
       upgrade of squid) you do not need to initialize them again.

     o When using DiskD storage scheme remember to read documentation:
         http://wiki.squid-cache.org/Features/DiskDaemon
       and alter your kern.ipc defaults in /boot/loader.conf. DiskD will not
       work reliably without this. Last recomendations were:

         kern.ipc.msgmnb=8192
         kern.ipc.msgssz=64
         kern.ipc.msgtql=2048

     o The default configuration will deny everyone but the local host and
       local networks as defined in RFC 1918 for IPv4 and RFCs 4193 and
       4291 for IPv6 access to the proxy service.  Edit the "http_access
       allow/deny" directives in /usr/local/etc/squid/squid.conf
       to suit your needs.

     o If AUTH_SQL option is set, please, don't forget to install one of
       following perl modules depending on database you like:
         databases/p5-DBD-mysql
         databases/p5-DBD-Pg
         databases/p5-DBD-SQLite

     To enable Squid, set squid_enable=yes in either
     /etc/rc.conf, /etc/rc.conf.local or /etc/rc.conf.d/squid
     Please see /usr/local/etc/rc.d/squid for further details.

     Note:
     If you just updated your Squid installation from an earlier version,
     make sure to check your Squid configuration against the 3.4 default
     configuration file /usr/local/etc/squid/squid.conf.sample.

     /usr/local/etc/squid/squid.conf.documented is a fully annotated
     configuration file you can consult for further reference.

     Additionally, you should check your configuration by calling
     'squid -f /path/to/squid.conf -k parse' before starting Squid.
=====
Message from squidGuard-1.4_15:

--
In order to activate squidGuard you have to edit squid.conf
 To the contain "url_rewrite_program /usr/local/bin/squidGuard"
 and create a configuration file for squidGuard.

 Sample blacklists have been installed in /usr/local/share/examples/squidGuard.

 A sample configuration file has beeen installed in
 /usr/local/etc/squid/squidGuard.conf.sample.

 You need to edit the configuration and compile the blacklist
 you choose to use with:
 squidGuard -d -C all

 Please bear in mind that this is just a sample configuration file
 and for any real world usage you need to download or create your
 own updated blacklists and create your own configuration file.

 Check documentation here:

 http://www.squidguard.org/Doc/

 To activate the changes do a /usr/local/sbin/squid -k reconfigure
=====
Message from pfSense-pkg-squid-0.4.45_10:

--
Please visit Services - Squid Proxy Server menu to configure the package and enable the proxy.
=====
Message from pfSense-pkg-squidGuard-1.16.18_20:

--
Please visit Services - SquidGuard Proxy Filter - Target Categories and set up at least one category there before enabling SquidGuard. See https://docs.netgate.com/pfsense/en/latest/packages/cache-proxy/squidguard.html for details.
>>> Cleaning up cache... done.
Success

hugoeyng

@stephenw10 @stephenw10 I upgraded from 22.05 to 23.01.

22.05 was working fine.

After upgrading Squid was running e squidGuard was not.

SquidGuard was appearing in "installed packages" but not in the menu options and neither in the services runnig, and a notification about the mentioned error started to be shown.

I applied the patch and the error has gone. But squidGuard did no appear in the menu options and neither in services running.

I removed squidGuard package and reinstalled. The error was back and squidGuard appears in the installed packages but not in the menu options and neither in services running.

Then I applied the patch and the error has gone, but squidGuard did not appears in the menu options and neither in the services running.

And so on, as many times as I can uninstall and reinstall.

The way I can see to solve this issue is the suidguard.conf to be corrected in the installion package.

The error in the squidguard.conf occurs because a ")" is not present in a line of the code.

stephenw10

@hugoeyng said in 23.01 Squid issue:

The error in the squidguard.conf occurs because a ")" is not present in a line of the code.

I'm not sure where you're seeing that. It looks like the errors you're seeing are in the squidguard pkg install script. It fails before it's able to add the menu and service items.
It doesn't fail on a clean install to 23.01 that never had squidguard so it pretty much has to be something in your existing config that's tripping it up. To be clear this is a bug. It should handle existing squidguard config.

If you remove the existing squidguard entris fromn your config file it will probably install fine.

What squidguard values do you have in the config? My test box basic config does not hit this:

		<squidguardgeneral>
			<config>
				<squidguard_enable>on</squidguard_enable>
				<ldap_enable></ldap_enable>
				<ldapbinddn></ldapbinddn>
				<ldapbindpass></ldapbindpass>
				<ldapcachetime>0</ldapcachetime>
				<stripntdomain></stripntdomain>
				<striprealm></striprealm>
				<ldapversion>3</ldapversion>
				<rewrite_children>16</rewrite_children>
				<rewrite_children_startup>8</rewrite_children_startup>
				<rewrite_children_idle>4</rewrite_children_idle>
				<enable_guilog>on</enable_guilog>
				<enable_log>on</enable_log>
				<log_rotation>on</log_rotation>
				<adv_blankimg>on</adv_blankimg>
				<blacklist></blacklist>
				<blacklist_proxy></blacklist_proxy>
				<blacklist_url></blacklist_url>
			</config>
		</squidguardgeneral>