Snort and Suricata problems with the new PHP 8.1 and FreeBSD Main Snapshots

bmeeks

Currently both Snort and Suricata have some issues in the PHP GUI code with the new PHP 8.1 update and the move to FreeBSD Main that is now part of the latest pfSense 2.7.0 and pfSense Plus 22.11 DEVEL snapshots.

Please install and help test pfSense with the new snapshots, but don't try to install or run either of the two IDS/IPS packages just yet with the new 2.7.0 DEVEL snapshots. I have to make some fixes in the PHP code to adjust for the move from PHP 7.4 to PHP 8.1 in the new snapshots. I will post an update to this thread (and mark it RESOLVED) once the fixes for Snort and Suricata are posted to the DEVEL branch of pfSense.

So until you see an update in this thread from me, be aware that Snort and Suricata will both have a broken GUI on the latest pfSense 2.7.0 DEVEL (and also pfSense Plus 22.11 DEVEL) snapshots. I am actively working on the fixes and expect to have them ready to go within about a week.

Update: it obviously took much more than a week, but Snort and Suricata both will now run on the latest pfSense CE and pfSense Plus snapshots with PHP 8.1.

bmeeks

The initial Pull Request updating the Snort package for PHP 8.1 has been posted for the Netgate team to review and merge soon. The Pull Request is here: https://github.com/pfsense/FreeBSD-ports/pull/1191.

Note that the changes are simply posted for Netgate's review for now. After they are approved and merged, a new Snort package will appear in the Package Manager with version 4.1.6_1.

I will post a further update when the changes have been merged into the DEVEL Package Repo.

Later Update: this is a bit more involved than I initially estimated. Working through it, but it is taking longer than anticipated. Lots of changes required in the PHP code.

bmeeks

The pull request containing the initial PHP8 fixes for the Suricata package have been posted for review and merge. Details are available on GitHub here: https://github.com/pfsense/FreeBSD-ports/pull/1192.

I will post further updates in this thread once the fixes have been approved, merged, and new packages built for the pfSense 2.7.0-DEVEL and pfSense Plus DEVEL branches.

bmeeks

Sorry this is dragging along, but it turned out to be a bit more involved than I initially thought. I can see the light at the end of the tunnel, though. Hopefully it is not another oncoming train .

Currently fighting an issue with installs of the package when there is an existing previous configuration. Something in the new pfSense code is getting tripped up in this circumstance and it is not recognizing when the package installation is actually complete, so the installation status progress bar within pfSense never turns "green" to signal success.

bmeeks

We finally have the new Snort package for the 2.7.0-DEVEL CE and 23.01 pfSense Plus branches ready for preliminary testing. It should show up in the Package Manager as an update (package version 4.1.6_1). This update contains only fixes for PHP 8.1.

Note:
There is still a problem with the installation seeming to "hang" when you are installing with a prior existing configuration or when you upgrade (since the previous installation's configuration will be detected).

The problem is a generic issue in pfSense's pkg installer system and is not specific to Snort or Suricata. The Netgate team is actively investigating the issue. It currently impacts Snort, Suricata, pfBlockerNG, and Squid.

To work around the problem when installing, follow these steps:

Install or update as usual via SYSTEM > PACKAGE MANAGER.
If the install stops scrolling text in the status update window, but the progress bar never turns green, and you don't see a SUCCESS message, you are probably experiencing the hang. This should only happen with an upgrade of the package when there is an existing configuration present. Give the install process several minutes to be sure it is hung as downloading and updating the Snort rules can also take a while.
To clear the "hung" process and have the install complete, open a CLI session on the firewall either via the console or using SSH for a remote shell session. Issue this command:

ps -ax | grep rc.packages

You should see a single running instance of that process. Note its Process ID (PID).
Kill the rc.packages process with this command:

kill <pid>

where <pid> is the Process ID discovered in step 4 above.
6. Close the shell session and return to the pfSense GUI and you should see the installation status screen has a green progress bar. Snort should be installed and started if you had a valid previous configuration.

bmeeks

A new Suricata package version for pfSense 2.7.0-DEVEL and 23.01 pfSense Plus DEVEL is now available. This package contains fixes for PHP 8.1. There are no new features and the binary, for the moment, remains at 6.0.6.

The new package version is 6.0.6_1 and should show up in the Package Manager for users on the 2.7.0 and 23.01 development snapshots.

Note:
There is still a problem with the installation seeming to "hang" when you are installing with a prior existing configuration or when you upgrade (since the previous installation's configuration will be detected).

The problem is a generic issue in pfSense's pkg installer system and is not specific to Snort or Suricata. The Netgate team is actively investigating the issue. It currently impacts Snort, Suricata, pfBlockerNG, and Squid.

To work around the problem when installing, follow these steps:

Install or update as usual via SYSTEM > PACKAGE MANAGER.
If the install stops scrolling text in the status update window, but the progress bar never turns green, and you don't see a SUCCESS message, you are probably experiencing the hang. This should only happen with an upgrade of the package when there is an existing configuration present. Give the install process several minutes to be sure it is hung as downloading and updating the rules can also take a while.
To clear the "hung" process and have the install complete, open a CLI session on the firewall either via the console or using SSH for a remote shell session. Issue this command:

ps -ax | grep rc.packages

You should see a single running instance of that process. Note its Process ID (PID).
Kill the rc.packages process with this command:

kill <pid>

where <pid> is the Process ID discovered in step 4 above.
6. Close the shell session and return to the pfSense GUI and you should see the installation status screen has a green progress bar. Suricata should be installed and started if you had a valid previous configuration.

The next item on the list for Suricata is to update the binary to the latest 6.0.8 version from upstream.

nhscan

@bmeeks I posted this somewhere else on the forms however I'm in the latest snapshot of November 8th 2022 still unable to install snort.

I still get the same error forgive me if I'm posting this in the wrong place. But maybe you have some insight on how I can fix this obviously running the command that it suggests breaks the web interface completely and I have to reinstall pfsense.

lua-resty-core: 0.1.23
lua-resty-lrucache: 0.13
luajit-openresty: 2.1.20220915
nginx: 1.22.0_9,3
pfSense: 2.7.0.a.20221108.0600

New packages to be INSTALLED:
daq: 2.2.2_3 [pfSense]
libdnet: 1.13_3 [pfSense]
libpcap: 1.10.1_2 [pfSense]
luajit-devel: 2.1.0.20221004_1 [pfSense]
pfSense-pkg-snort: 4.1.6_1 [pfSense]
snort: 2.9.20_1 [pfSense]

Number of packages to be removed: 5
Number of packages to be installed: 6

The process will require 3 MiB more space.
pkg-static: Cannot delete vital package: pfSense!
pkg-static: If you are sure you want to remove pfSense,
pkg-static: unset the 'vital' flag with: pkg set -v 0 pfSense
Failed

bmeeks

@nhscan, I am aware of the issue. It was caused by an upstream change in the FreeBSD ports tree. There is an open pfSense Redmine Issue here: https://redmine.pfsense.org/issues/13623. I am waiting on the Netgate developer team to tell me which of the available options for correcting this issue is best for the long term.

nhscan

@bmeeks Thank you so much!

nhscan

@bmeeks It also happens with suricata but I'm sure you're aware of that. Again thank you for the information and your help just glad somebody's looking into it. Thanks again.

bmeeks

@nhscan said in Snort and Suricata problems with the new PHP 8.1 and FreeBSD Main Snapshots:

@bmeeks It also happens with suricata but I'm sure you're aware of that. Again thank you for the information and your help just glad somebody's looking into it. Thanks again.

Yes. Anything using lua is going to be impacted by that upstream FreeBSD change.

nhscan

@bmeeks Is there any update after reading some of the bug reports it looked like snort ver 4.1.6_1 snort-2.9.20_1 was the php fix however i am still unable to install it still getting this for an error. I am running Current Base System2.7.0.a.20221118.0600

Installing pfSense-pkg-snort...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity... done (2 conflicting)

luajit-devel-2.1.0.20221004_1 [pfSense] conflicts with luajit-openresty-2.1.20220915 [installed] on /usr/local/bin/luajit
luajit-devel-2.1.0.20221004_1 [pfSense] conflicts with luajit-openresty-2.1.20220915 [pfSense] on /usr/local/bin/luajit
Checking integrity... done (0 conflicting)
The following 12 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
lua-resty-core: 0.1.23
lua-resty-lrucache: 0.13
luajit-openresty: 2.1.20220915
nginx: 1.22.0_9,3
pfSense: 2.7.0.a.20221118.0600

New packages to be INSTALLED:
daq: 2.2.2_3 [pfSense]
libdnet: 1.13_3 [pfSense]
libpcap: 1.10.1_2 [pfSense]
luajit-devel: 2.1.0.20221004_1 [pfSense]
pfSense-pkg-snort: 4.1.6_1 [pfSense]
snort: 2.9.20_1 [pfSense]

Installed packages to be REINSTALLED:
pkg-1.18.4_1 [pfSense]

Number of packages to be removed: 5
Number of packages to be installed: 6
Number of packages to be reinstalled: 1

The process will require 3 MiB more space.
pkg-static: Cannot delete vital package: pfSense!
pkg-static: If you are sure you want to remove pfSense,
pkg-static: unset the 'vital' flag with: pkg set -v 0 pfSense
Failed

bmeeks

@nhscan said in Snort and Suricata problems with the new PHP 8.1 and FreeBSD Main Snapshots:

@bmeeks Is there any update after reading some of the bug reports it looked like snort ver 4.1.6_1 snort-2.9.20_1 was the php fix however i am still unable to install it still getting this for an error. I am running Current Base System2.7.0.a.20221118.0600

Installing pfSense-pkg-snort...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity... done (2 conflicting)

luajit-devel-2.1.0.20221004_1 [pfSense] conflicts with luajit-openresty-2.1.20220915 [installed] on /usr/local/bin/luajit

luajit-devel-2.1.0.20221004_1 [pfSense] conflicts with luajit-openresty-2.1.20220915 [pfSense] on /usr/local/bin/luajit
Checking integrity... done (0 conflicting)
The following 12 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
lua-resty-core: 0.1.23
lua-resty-lrucache: 0.13
luajit-openresty: 2.1.20220915
nginx: 1.22.0_9,3
pfSense: 2.7.0.a.20221118.0600

New packages to be INSTALLED:
daq: 2.2.2_3 [pfSense]
libdnet: 1.13_3 [pfSense]
libpcap: 1.10.1_2 [pfSense]
luajit-devel: 2.1.0.20221004_1 [pfSense]
pfSense-pkg-snort: 4.1.6_1 [pfSense]
snort: 2.9.20_1 [pfSense]

Installed packages to be REINSTALLED:
pkg-1.18.4_1 [pfSense]

Number of packages to be removed: 5
Number of packages to be installed: 6
Number of packages to be reinstalled: 1

The process will require 3 MiB more space.
pkg-static: Cannot delete vital package: pfSense!
pkg-static: If you are sure you want to remove pfSense,
pkg-static: unset the 'vital' flag with: pkg set -v 0 pfSense
Failed

Unfortunately, no news yet from the Netgate team. I suspect they are sort of covered up with bug fixes in the base system, and issues with add-on packages are taking a back seat at the moment. I will bump them up again to get a status.

bmeeks

@nhscan, the problem with library dependency errors when trying to install the latest Snort package on pfSense DEVEL snapshots should be fixed in the next snapshot build.

The fix was merged about 11:00 AM Eastern Time (USA) on 11/23/2022. So the fix should be in any snapshot update with a date and time after that time.

RabidSasquatch

@bmeeks Does a similar change need to be made to the Suricata makefile as well? The pull request appears to apply only to Snort.

bmeeks

@rabidsasquatch said in Snort and Suricata problems with the new PHP 8.1 and FreeBSD Main Snapshots:

@bmeeks Does a similar change need to be made to the Suricata makefile as well? The pull request appears to apply only to Snort.

Yeah, probably so. Forgot about that one. I'll get one created in the next few days and submitted.

bmeeks

@rabidsasquatch said in Snort and Suricata problems with the new PHP 8.1 and FreeBSD Main Snapshots:

@bmeeks Does a similar change need to be made to the Suricata makefile as well? The pull request appears to apply only to Snort.

I submitted a fix for Suricata as well for the luajit-openresty library conflict. It was merged around 10:00 AM US Eastern on 11/28/2022, and so will appear in the next snapshot build after that time. The new Suricata binary version will be 6.0.8_2 (updated from 6.0.8_1).

NollipfSense

@bmeeks Suricata 6.0.8_2 not working (2.7) and log doesn't say why, just refresh and that does nothing...

Screenshot 2022-12-15 at 12.09.04 PM.png

Screenshot 2022-12-15 at 12.08.20 PM.png

bmeeks

Have you checked the pfSense System Log for anything that might be logged there?

I just fired up my 2.7.0 Snapshot Virtual Machine with Suricata installed and everything came up fine. Here is a screenshot of Suricata running on the WAN --

If Suricata does not get far enough along in its startup to create the suricata.log file and write to it, then something pretty drastic is messed up on the box.

You can check the System Log to see what may be logged there. You could also try a remove and reinstall operation with the Suricata package.

NollipfSense

@bmeeks said in Snort and Suricata problems with the new PHP 8.1 and FreeBSD Main Snapshots:

You can check the System Log to see what may be logged there. You could also try a remove and reinstall operation with the Suricata package.

I only reinstall once but did not completely remove before the reinstall...all pfSense system log says is that Suricata was stopped then upgraded. Will remove, reinstall, and report back.

Dec 15 11:37:34 pkg-static 67395 suricata upgraded: 6.0.8_1 -> 6.0.8_2
Dec 15 11:37:34 SuricataStartup 76447 Suricata STOP for WAN(25152_em0)...