Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module

bmeeks

@SteveITS said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

2.7.1 released yesterday actually…didn’t notice any posts here but it’s on Netgate’s blog.

I've been buried head down chasing the IDS/IPS package bugs and have not checked the release status.

I do know the kernel developer and author of all the recent libpfctl library changes is working to post the fixes and get them into at least 2.7.1 and 23.09. That is all happening this morning.

bmeeks

I've updated the original post at the top of this thread with new information.

I'll briefly repeat that here --

A fix has been identified and implemented for the bug exposed by the custom blocking module changes in the most recent Snort and Suricata package updates. The fix requires the publishing of a new libpfctl library package and then rebuilding impacted packages where libpfctl is a build dependency. That will happen soon in the pfSense CE 2.7.1 and pfSense Plus 23.09 branches.

Not sure at this point if the fix can be fully migrated back to CE 2.7.0 because of the kernel age difference between 2.7.0 CE and the just releasted 2.7.1 CE. Some kernel functions used by the updated libpfctl library code are not present in the 2.7.0 CE kernel. It may turn out that the only fix for 2.7.0 CE users is to update to 2.7.1 CE after the fixed packages are present there. But this question is still being looked at by the Netgate developer team.

fireodo

@bmeeks OFF Topic:

Thanks for your effort and work chasing that annoying bug! (has to be said)

NogBadTheBad

@bmeeks Is this the new version that's just come out today, if it is its still dumping core:-

Nov 17 15:58:20 kernel pid 93766 (suricata), jid 0, uid 0: exited on signal 11 (core dumped)
Nov 17 15:58:19 suricata 92214 [634254] <Notice> -- This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
Nov 17 15:58:19 php 82334 [Suricata] Suricata START for LAN(igb0)...

SteveITS

@bmeeks said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

fix requires the publishing of a new libpfctl library package

I might be low on coffee but does this mean it would be distributed as part of the Suricata/Snort packages, and not a pfSense version update?

Thanks,

bmeeks

@NogBadTheBad said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

@bmeeks Is this the new version that's just come out today, if it is its still dumping core:-

Nov 17 15:58:20 kernel pid 93766 (suricata), jid 0, uid 0: exited on signal 11 (core dumped)
Nov 17 15:58:19 suricata 92214 [634254] <Notice> -- This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
Nov 17 15:58:19 php 82334 [Suricata] Suricata START for LAN(igb0)...

No, there is no new version yet. You are likely seeing the old code if you are on 23.09 Plus. There was a problem with the package builder for 23.09, so what was the "new" Suricata package for 2.7.0 and 2.7.1 CE users did not get into 23.09 at the same time. 23.09 continued with the old 7.0.0 package until probably this morning. They were still working on the 23.09 package builder server yesterday, but expected it to be working last night.

So, long story is you do not have the fixed package. In fact, it may not get posted until either later tonight, over the weekend, or potentially it might be Monday. Not sure right now.

bmeeks

@SteveITS said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

@bmeeks said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

fix requires the publishing of a new libpfctl library package

I might be low on coffee but does this mean it would be distributed as part of the Suricata/Snort packages, and not a pfSense version update?

Thanks,

Yes, indirectly. Because libpfctl is now a build dependency for the Snort and Suricata binaries, when Snort or Suricata is rebuilt the updated libpfctl will be used.

At some point over the next four days (since this is a Friday), new versions of the Snort and Suricata packages will show up for 2.7.1 CE and 23.09 Plus users. Because we are spanning a weekend, things may pause over Saturday and Sunday and pick up on Monday. That will be Netgate's call. I am working now to get my changes in.

computerhousecalls

2.7.0-RELEASE (amd64)
built on Wed Jun 28 03:53:34 UTC 2023
FreeBSD 14.0-CURRENT

After update last week, snort status on interfaces keeps crashing so far every four to five mins. Snort version 4.1.6-13 is using legacy blocking with kill states enabled. At this point I decided to switch to Suricata 7.0.2 and still the same thing. Suricata would also crash about every five mins. So then I switched back to snort and disabled killstates. Then the service was not error 11 status. So as to the Original Note it is affecting both Snort and Suricata packages.

bmeeks

@computerhousecalls said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

2.7.0-RELEASE (amd64)
built on Wed Jun 28 03:53:34 UTC 2023
FreeBSD 14.0-CURRENT

After update last week, snort status on interfaces keeps crashing so far every four to five mins. Snort version 4.1.6-13 is using legacy blocking with kill states enabled. At this point I decided to switch to Suricata 7.0.2 and still the same thing. Suricata would also crash about every five mins. So then I switched back to snort and disabled killstates. Then the service was not error 11 status. So as to the Original Note it is affecting both Snort and Suricata packages.

Yes, both packages are impacted when using Legacy Blocking Mode. A fix has been indentified and is on the way. Just taking a little time for final extra testing and packaging things up.

computerhousecalls

@bmeeks Thank you & Thank you again.

JonathanLee

I have 23.05.01 and with the old version of snort it works fine. When I stay at 23.05.01 and update snort the core dump bug starts.
I run a sg-2100max. I sure you already have that info. I just didn't know if anyone has used it on 23.05.01 yet

Gerard64

@bmeeks

I upgrade to 2.7.1 this morning and Snort didn't stop anymore not once.
So seams all is good again.

Thank you man you are the best

bmeeks

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

I have 23.05.01 and with the old version of snort it works fine. When I stay at 23.05.01 and update snort the core dump bug starts.
I run a sg-2100max. I sure you already have that info. I just didn't know if anyone has used it on 23.05.01 yet

Anything earlier than 23.09 will have the defective libpfctl library version on it. If you update to a Snort package version after 4.1.6_9 you will hit the bug, because beginning with 4.1.6_11 the updated custom blocking module code that calls functions in the defective libpfctl library was included. Same thing applies to pfSense CE.

JonathanLee

@bmeeks thanks for the reply. I can assure you I still see it, however much less than the new version of Snort. I am running the .11 and it does seem very stable without the core crashes. As soon as I update the snort package it crashes every couple mins when adjusting supress lists. After I downgraded I have no more logs for core dumps. I am stable with the version before it.

I went back to the old version.

The errors in logs are from when I had the updated snort.

JonathanLee

Will the update work on 23.05.01??

bmeeks

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

Will the update work on 23.05.01??

No. The kernel there does not contain some of the necessary functions. But there should not be a problem with the Snort version in 23.05.1. Everything there on the binary side is using the old code.

The problem surfaced when Snort was updated to cope with changes that appeared in 23.09 Plus and 2.7.1 CE.

SteveITS

@JonathanLee said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:

Will the update work on 23.05.01??

If you're on 23.05 don't install a package from 23.09...change your update branch to Previous Stable per my sig.

slu

@bmeeks

I installed today 4.1.6_14 and there was also a libpfctl upgrade, do this version fix the issue?

[1/3] Fetching snort-2.9.20_7.pkg: .......... done
[2/3] Fetching pfSense-pkg-snort-4.1.6_14.pkg: .......... done
[3/3] Fetching libpfctl-0.8.pkg: . done
Checking integrity... done (0 conflicting)
[1/3] Installing libpfctl-0.8...
[1/3] Extracting libpfctl-0.8: ...... done
[2/3] Upgrading snort from 2.9.20_5 to 2.9.20_7...
[2/3] Extracting snort-2.9.20_7: .......... done
[3/3] Upgrading pfSense-pkg-snort from 4.1.6_13 to 4.1.6_14...

fireodo

@slu said in Important Snort and Suricata Package Announcement -- probable bug in Legacy Blocking Module:
@bmeeks

I installed today 4.1.6_14 and there was also a libpfctl upgrade

Here too 4.1.6_14 (under pfsense 2.7.1) - no issues so far ... keep observing.

bmeeks

Yes, the 4.1.6_14 version of the Snort package should correct the Signal 11 crashing issue when Kill States is enabled with Legacy Mode Blocking. There is also an update for Suricata to 7.0.2_1 that fixes the same problem in Suricata's Legacy Blocking Mode.

The developer working with me on this fix resides in Europe, and he merged the fixes during his working hours- some of which are still during my bedtime here in the Eastern US. So, just now seeing the news.