everyday at 6am suricata crashes

bmeeks

Your PHP errors stem from trying to load a file that is too big to fit into the finite amount of RAM the PHP process reserves. That is a "normal" error message when attempting to load and view very large files. You can't load up and view a core dump file in the GUI. You will need to export the file off the firewall and load it into another editor (or post its contents back here).

Your Suricata problem is not related to that PHP error at all. A Signal 10 error is a BUS FAULT. That was a common error on 32-bit ARM hardware, but should almost never happen on Intel/AMD64 hardware. Your system detail says you have an SG-6100.

The GeoLite2 database update appears to be completing as the last log message shows the temp subdirectory used during that process being cleaned up.

Euman

I'm going on the record here. I had Suricata use Wan interface because I have this stupid AT&T internet and the gateway had to be setup with Use non-local gateway

Use non-local gateway through interface specific route. This will allow use of a gateway outside of this interface's subnet.
This is usually indicative of a configuration error, but is required for some scenarios.

I have a block of IP's and the Wan I setup is one of those in the block of ip's I purchased from AT&T
Wan is supposed to block by default inbound activity so Herein lies my mistake.

I have just now shifted from using Suricata on Wan to the 4 LAN's my Netgate 6100 has. I do hope this doesn't prove to be wrong and fixes this particular issue.

Euman

Thank you all for your help and apologies for my ignorance and making an issue where misconfiguration was the issue.

bmeeks

@Euman said in everyday at 6am suricata crashes:

Thank you all for your help and apologies for my ignorance and making an issue where misconfiguration was the issue.

The Signal 10 Bus Error is very unusual. I would essentially expect to never see that error on Intel/AMD hardware, no matter what interface you run Suricata on.

That error usually means the running code attempted to access memory on a non-word aligned boundary. But Intel hardware always automatically fixes up such access by converting unaligned memory accesses into a series of sequential reads followed by bit shifting to properly align the resulting read data.

Euman

@bmeeks How can I help debug this further? I'm a novice programmer (in advance) and your msg read like the titanic is sinking.

Euman

suricata.core is definitely 633276.00 kb.. I will download and gather any detail

bmeeks

@Euman said in everyday at 6am suricata crashes:

@bmeeks How can I help debug this further? I'm a novice programmer (in advance) and your msg read like the titanic is sinking.

Not easily debugged without compiling your own binary will full debugging enabled. That really can't be done unless you create a pfSense package builder.

There can be a few other things that generate a Signal 10 Bus Error, but non-aligned memory access is the most common. It's also possible some obscure hardware fault has occurred that some piece of Suricata code is tickling just right to trigger the problem.

While I don't know precisely how many SG-6100 users are out there running Suricata with the GeoIPLite option enabled, if there was a widespread problem I would expect to have seen at least a few other similar posts. Not seeing any yet from other users would be a trend favoring a potential hardware issue of some type in your setup.

bmeeks

@Euman said in everyday at 6am suricata crashes:

suricata.core is definitely 633276.00 kb.. I will download and gather any detail

That file will be a binary memory image you would load into the GDB debugger package (which you would need to install separately). But since the production pfSense packages are compiled WITHOUT debugging info or symbols, the utility of the core dump can be limited.

Euman

apparently the "save button" for the dialog window that I had opened for the file (0) zero'd the file contents, I lost it.. sighs heavily!

Old people mice clicking should be denied..

Euman

@bmeeks I believe I know why suricata would crash when geolite2 was updated and I believe suricata was using lots of data and holding ip address's well over 5000 them in snort2c tables, that, coupled with using too large a RAM Disk for /var & /tmp, I was simply out of ram. I have changed the ram disk size and adjusted suricata to NOT keep ip's longer than 7 days and this helped as I've had no more 6am suricata crash nor core dumps have occurred.

I really appreciate all of you guys help here on the forum :) Thank you again!