Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module

bmeeks

@NogBadTheBad said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

@bmeeks Is this the new version that's just come out today, if it is its still dumping core:-

Nov 17 15:58:20 kernel pid 93766 (suricata), jid 0, uid 0: exited on signal 11 (core dumped)
Nov 17 15:58:19 suricata 92214 [634254] <Notice> -- This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
Nov 17 15:58:19 php 82334 [Suricata] Suricata START for LAN(igb0)...

No, there is no new version yet. You are likely seeing the old code if you are on 23.09 Plus. There was a problem with the package builder for 23.09, so what was the "new" Suricata package for 2.7.0 and 2.7.1 CE users did not get into 23.09 at the same time. 23.09 continued with the old 7.0.0 package until probably this morning. They were still working on the 23.09 package builder server yesterday, but expected it to be working last night.

So, long story is you do not have the fixed package. In fact, it may not get posted until either later tonight, over the weekend, or potentially it might be Monday. Not sure right now.

bmeeks

@SteveITS said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

@bmeeks said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

fix requires the publishing of a new libpfctl library package

I might be low on coffee but does this mean it would be distributed as part of the Suricata/Snort packages, and not a pfSense version update?

Thanks,

Yes, indirectly. Because libpfctl is now a build dependency for the Snort and Suricata binaries, when Snort or Suricata is rebuilt the updated libpfctl will be used.

At some point over the next four days (since this is a Friday), new versions of the Snort and Suricata packages will show up for 2.7.1 CE and 23.09 Plus users. Because we are spanning a weekend, things may pause over Saturday and Sunday and pick up on Monday. That will be Netgate's call. I am working now to get my changes in.

computerhousecalls

2.7.0-RELEASE (amd64)
built on Wed Jun 28 03:53:34 UTC 2023
FreeBSD 14.0-CURRENT

After update last week, snort status on interfaces keeps crashing so far every four to five mins. Snort version 4.1.6-13 is using legacy blocking with kill states enabled. At this point I decided to switch to Suricata 7.0.2 and still the same thing. Suricata would also crash about every five mins. So then I switched back to snort and disabled killstates. Then the service was not error 11 status. So as to the Original Note it is affecting both Snort and Suricata packages.

bmeeks

@computerhousecalls said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

2.7.0-RELEASE (amd64)
built on Wed Jun 28 03:53:34 UTC 2023
FreeBSD 14.0-CURRENT

After update last week, snort status on interfaces keeps crashing so far every four to five mins. Snort version 4.1.6-13 is using legacy blocking with kill states enabled. At this point I decided to switch to Suricata 7.0.2 and still the same thing. Suricata would also crash about every five mins. So then I switched back to snort and disabled killstates. Then the service was not error 11 status. So as to the Original Note it is affecting both Snort and Suricata packages.

Yes, both packages are impacted when using Legacy Blocking Mode. A fix has been indentified and is on the way. Just taking a little time for final extra testing and packaging things up.

computerhousecalls

@bmeeks Thank you & Thank you again.

JonathanLee

I have 23.05.01 and with the old version of snort it works fine. When I stay at 23.05.01 and update snort the core dump bug starts.
I run a sg-2100max. I sure you already have that info. I just didn't know if anyone has used it on 23.05.01 yet

Gerard64

@bmeeks

I upgrade to 2.7.1 this morning and Snort didn't stop anymore not once.
So seams all is good again.

Thank you man you are the best

bmeeks

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

I have 23.05.01 and with the old version of snort it works fine. When I stay at 23.05.01 and update snort the core dump bug starts.
I run a sg-2100max. I sure you already have that info. I just didn't know if anyone has used it on 23.05.01 yet

Anything earlier than 23.09 will have the defective libpfctl library version on it. If you update to a Snort package version after 4.1.6_9 you will hit the bug, because beginning with 4.1.6_11 the updated custom blocking module code that calls functions in the defective libpfctl library was included. Same thing applies to pfSense CE.

JonathanLee

@bmeeks thanks for the reply. I can assure you I still see it, however much less than the new version of Snort. I am running the .11 and it does seem very stable without the core crashes. As soon as I update the snort package it crashes every couple mins when adjusting supress lists. After I downgraded I have no more logs for core dumps. I am stable with the version before it.

I went back to the old version.

The errors in logs are from when I had the updated snort.

JonathanLee

Will the update work on 23.05.01??

bmeeks

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

Will the update work on 23.05.01??

No. The kernel there does not contain some of the necessary functions. But there should not be a problem with the Snort version in 23.05.1. Everything there on the binary side is using the old code.

The problem surfaced when Snort was updated to cope with changes that appeared in 23.09 Plus and 2.7.1 CE.

SteveITS

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

Will the update work on 23.05.01??

If you're on 23.05 don't install a package from 23.09...change your update branch to Previous Stable per my sig.

slu

@bmeeks

I installed today 4.1.6_14 and there was also a libpfctl upgrade, do this version fix the issue?

[1/3] Fetching snort-2.9.20_7.pkg: .......... done
[2/3] Fetching pfSense-pkg-snort-4.1.6_14.pkg: .......... done
[3/3] Fetching libpfctl-0.8.pkg: . done
Checking integrity... done (0 conflicting)
[1/3] Installing libpfctl-0.8...
[1/3] Extracting libpfctl-0.8: ...... done
[2/3] Upgrading snort from 2.9.20_5 to 2.9.20_7...
[2/3] Extracting snort-2.9.20_7: .......... done
[3/3] Upgrading pfSense-pkg-snort from 4.1.6_13 to 4.1.6_14...

fireodo

@slu said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:
@bmeeks

I installed today 4.1.6_14 and there was also a libpfctl upgrade

Here too 4.1.6_14 (under pfsense 2.7.1) - no issues so far ... keep observing.

bmeeks

Yes, the 4.1.6_14 version of the Snort package should correct the Signal 11 crashing issue when Kill States is enabled with Legacy Mode Blocking. There is also an update for Suricata to 7.0.2_1 that fixes the same problem in Suricata's Legacy Blocking Mode.

The developer working with me on this fix resides in Europe, and he merged the fixes during his working hours- some of which are still during my bedtime here in the Eastern US. So, just now seeing the news.

slu

@bmeeks
so cool, thank you @bmeeks and Netgate.

Like the way to communicate here directly and open.

bmeeks

The 2.7.1 CE updates are in place and available to users. There is a problem with package builds in the 23.09 branch that is unrelated to the Snort and Suricata fixes. So, the updated packages are not yet showing up for Plus 23.09 users. The changes are in place in that repo, but for unrelated reasons package building is failing there. The Netgate guys are working on it.

bmeeks

@slu said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

[1/3] Installing libpfctl-0.8...
[1/3] Extracting libpfctl-0.8: ...... done

These two lines are the actual fix for the problem. The libpfctl library that comes bundled with pfSense natively is version 0.4 and it has the bug. That buggy library is still there for now, but packages that need to do libpfctl things have been recompiled and instructed to use the package version of that library now stored in /usr/local/lib/ instead of the system-bundled version in /usr/lib/.

In the next release of pfSense (whenever that happens), the bundled library will be removed and ports recompiled to use only the package version of libpfctl. This will make any future updates to the library easier. Packages that need libfpctl functionality will automatically install that library package if it is not already present-- or update the installed version if necessary.

JonathanLee

@bmeeks when will 23.09 plus users get the update? Anytime soon or should users go back the their old boot environments?

bmeeks

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

@bmeeks when will 23.09 plus users get the update? Anytime soon or should users go back the their old boot environments?

Whenever the 23.09 package builder server successfully builds all of the packages and copies them over to the 23.09 repo web server. I have no inside information on what's wrong. All I was told is that no packages are succesfully building on that infrastructure. right now.

The 2.7.1 CE builders are working fine, so the Snort and Suricata updates are available there.

In today's world, with all the encrypted traffic on networks, I would not consider the IDS/IPS important enough to warrant rolling back to 23.05.1. I would simply disable the IDS/IPS until the package update becomes available for 23.09.