Suricata core dumping after 2.4.5 upgrade
-
Yes sir I did read the posts above. I just added some additional details regarding the specific errors thrown in the off chance that it might be helpful. If it is not, ignore away.
Appreciate the feedback particularly the expectation management in terms of a fix. While I'm bummed that what once worked does not now, I will take the opportunity to learn a bit about Snort and give it a go.
-
@h3llo said in Suricata core dumping after 2.4.5 upgrade:
Yes sir I did read the posts above. I just added some additional details regarding the specific errors thrown in the off chance that it might be helpful. If it is not, ignore away.
Appreciate the feedback particularly the expectation management in terms of a fix. While I'm bummed that what once worked does not now, I will take the opportunity to learn a bit about Snort and give it a go.
Blame this problem on two things. One is the Suricata upstream team deciding to use Rust in the binary and to make it mandatory starting with version 5.x. The other is the folks who created the llvm compiler used to create armv6/v7 and aarch64 binary from C source code. That compiler makes some binary instruction choices that are "how shall I say?" -- not optimal ...
. -
I just bought a SG-1100 and have this same issue. Sad to find out that it may not be fixed soon. I'll look into Snort in the meantime.
-
I just posted the same issue after the 2.4.5 Release. I've been working on this for 2 days now. I see I'm not the only one having this issue. See my post below.
Suricata will not start using pfsense. I currently have the SG-1100 appliance.
Added the Suricata package with no issues.
Configured Global Setting using the ETOpen Emerging Threats rules only. No Issues
Performed the Emerging Threats Open Rules update with no issues.
Enabled emerging rules. No Issues
The SG-1100 appliance only has 2 CPU's with 1GB of Ram. It should still run the Suricata application.
When i go to start the service on the WAN interface, it runs for 3 seconds then stops.
Log View message is below
5/4/2020 -- 18:34:18 - <Notice> -- This is Suricata version 5.0.2 RELEASE running in SYSTEM mode
5/4/2020 -- 18:34:18 - <Info> -- CPUs/cores online: 2
5/4/2020 -- 18:34:18 - <Info> -- HTTP memcap: 67108864
5/4/2020 -- 18:34:18 - <Notice> -- using flow hash instead of active packets -
@blaytrail said in Suricata core dumping after 2.4.5 upgrade:
I just posted the same issue after the 2.4.5 Release. I've been working on this for 2 days now. I see I'm not the only one having this issue. See my post below.
Suricata will not start using pfsense. I currently have the SG-1100 appliance.
Added the Suricata package with no issues.
Configured Global Setting using the ETOpen Emerging Threats rules only. No Issues
Performed the Emerging Threats Open Rules update with no issues.
Enabled emerging rules. No Issues
The SG-1100 appliance only has 2 CPU's with 1GB of Ram. It should still run the Suricata application.
When i go to start the service on the WAN interface, it runs for 3 seconds then stops.
Log View message is below
5/4/2020 -- 18:34:18 - <Notice> -- This is Suricata version 5.0.2 RELEASE running in SYSTEM mode
5/4/2020 -- 18:34:18 - <Info> -- CPUs/cores online: 2
5/4/2020 -- 18:34:18 - <Info> -- HTTP memcap: 67108864
5/4/2020 -- 18:34:18 - <Notice> -- using flow hash instead of active packetsLook in the pfSense system log and I bet you will see the Signal 4 core dump message disscussed in the posts above. Suricata 5.x on SG-1100 hardware is a no-go for now. You will need to either switch over to Snort or just wait for the fix.
There is absolutely no setting or configuration change a user can make that will change this problem. It is an issue with the compiled binary itself.
-
Thanks for the update. You are correct. I just looked in the system logs and found the entry below.
Apr 5 20:09:32 kernel pid 38129 (suricata), jid 0, uid 0: exited on signal 4 (core dumped)
I may use AlienVault with the built in Suricata software and create a span port off the netgate device.
Can you create a span port on the Netgate device using the OPT port?
-
@blaytrail said in Suricata core dumping after 2.4.5 upgrade:
Thanks for the update. You are correct. I just looked in the system logs and found the entry below.
Apr 5 20:09:32 kernel pid 38129 (suricata), jid 0, uid 0: exited on signal 4 (core dumped)
I may use AlienVault with the built in Suricata software and create a span port off the netgate device.
Can you create a span port on the Netgate device using the OPT port?
Not to my knowledge, but I'm no expert on the Marvel switch chip inside the SG-1100. You might want to ask that question in the Netgate Hardware forum on here.
Signal 4 means ILLEGAL INSTRUCTION.
-
Thanks. I'm looking forward to learning more about IDS\IPS. Surricata seems pretty simple to setup until i ran into the issue. If you don't mind me asking are you doing to switch to another IDS\IPS solution?
-
@blaytrail said in Suricata core dumping after 2.4.5 upgrade:
Thanks. I'm looking forward to learning more about IDS\IPS. Surricata seems pretty simple to setup until i ran into the issue. If you don't mind me asking are you doing to switch to another IDS\IPS solution?
I am the developer/maintainer for Suricata and Snort on pfSense (volunteer, not paid or affiliated with Netgate). I started with Snort years ago and have never changed. Not because it is better or Suricata is worse, just lazy and didn't want to change out my production firewall.
So for now I'm running Snort on a Netgate SG-5100 appliance. Prior to that I ran it on custom-built hardware (mostly Supermicro 1U servers).
-
Cool! It must be fun to be the developer/maintainer for the applications. I really enjoy working with pfSense. I just setup the VPN and it works perfectly. I guess i can move up to the SG-5100. I want to set getting more familiar with IDS/IPS. I take it I will need to use something else beside the SG-1100 for IDS/IPS.
-
@blaytrail said in Suricata core dumping after 2.4.5 upgrade:
Cool! It must be fun to be the developer/maintainer for the applications. I really enjoy working with pfSense. I just setup the VPN and it works perfectly. I guess i can move up to the SG-5100. I want to set getting more familiar with IDS/IPS. I take it I will need to use something else beside the SG-1100 for IDS/IPS.
The SG-1100 is a good starter box for IDS/IPS, but to be honest the amount of RAM it has can limit it when it comes to a full-blown IDS/IPS setup. You would need to be a bit choosy about which rules, and how many in total, you enabled to control RAM usage.
The SG-5100 is more capable both in terms of CPU and RAM. Of course it costs quite a bit more. But I looked around and discovered that getting a chassis, motherboard, CPU, RAM and the other required bits totaled up to be at least as much as the SG-5100 (or so close it was really a wash).
-
I will order the SG-5100 appliance when i get paid next week. I don't mind spending the money for educational purposes. Thanks so much for your time this morning.
I have the instructions for setting up Suricata. Is there a good place to get basic instructions for setting up Snort?
-
@blaytrail said in Suricata core dumping after 2.4.5 upgrade:
I will order the SG-5100 appliance when i get paid next week. I don't mind spending the money for educational purposes. Thanks so much for your time this morning.
I have the instructions for setting up Suricata. Is there a good place to get basic instructions for setting up Snort?
There is a link in the offical Netgate documentation. Both packages work essentially the exact same way in terms of set up on pfSense. In fact, a large percentage of the PHP source is copy/paste with "Snort" changed to "Suricata". The Snort package existed first.
You should be able to run Snort currently on the SG-1100. I don't think it has the same compiler issue (but I'm not 100% positive).
Here is the link: https://docs.netgate.com/pfsense/en/latest/ids-ips/setup-snort-package.html.
-
Excellent!
I just realize there is a separate snort package install on the SG-1100. I will get this setup this morning.
-
I am glad to know this is an existing bug. My SG-1000 upgrade to 2.4.5 failed and I had to restore the factory image from USB key. I just assumed my backup configuration was somehow corrupt and causing an issue.
Even after installing the factory image and Suricata package with minimal configuration Suricata would die after about 10 seconds. Only when I ran Suricata from the command line did I see it print "Illegal Instruction" to stderr when it crashed - it was not captured to the log file.
I installed Snort and it works just fine, but I miss Suricata and hope the upstream issue is corrected soon.
-
@sholekamp said in Suricata core dumping after 2.4.5 upgrade:
I am glad to know this is an existing bug. My SG-1000 upgrade to 2.4.5 failed and I had to restore the factory image from USB key. I just assumed my backup configuration was somehow corrupt and causing an issue.
Even after installing the factory image and Suricata package with minimal configuration Suricata would die after about 10 seconds. Only when I ran Suricata from the command line did I see it print "Illegal Instruction" to stderr when it crashed - it was not captured to the log file.
I installed Snort and it works just fine, but I miss Suricata and hope the upstream issue is corrected soon.
The Signal 4 code (which is the ILLEGAL INSTRUCTION error) is printed to the system log of pfSense by the operating system. The reason it is not printed to the Suricata log is that Suricata is the process executing the illegal instruction. As a result, the operating system immediately terminates the process and there is no "suricata" process to write anything to its log.
-
@bmeeks
That all makes sense. I missed the entries in the system log:
Apr 9 15:31:09 pfSense kernel: pid 51904 (suricata), jid 0, uid 0: exited on signal 4 (core dumped)Seeing "Illegal Instruction" printed to the terminal is what led me to this thread. In the end my SG-100 is back up and running and the problem has been identified. I will keep an eye out for a Suricata update. Thank you for the support.
*** UPDATE ***
For anyone that stumbles upon this thread, the issue has been corrected with pfSense 2.4.5-p1 released June 9, 2020. See the release notes: https://docs.netgate.com/pfsense/en/latest/releases/2-4-5-p1-new-features-and-changes.htmlGreat work!
