Suricata 2.0.3 Package Preview
-
@jflsakfja:
Hey Bill,
As far as I can tell, the bug with the logs not getting properly removed happens with the packet captures as well. Dunno if it happens with other logs, those are the first that went over the limits.Clarification and Update on this issue:
There are configuration parameters on the INTERFACE SETTINGS tab where packet capture files are enabled that allow you specify the size limit in megabytes for each pcap file and the total number of pcap files to retain in the log directory. The defaults are 32 MB for file size and 1000 pcap files maintained. Once the number of pcap files to retain value is exceeded, the oldest file overwritten.
With that said, there is also a type of "fail safe" feature associated with both Snort and Suricata that lets you set an overall limit in MB on the amount of disk space in /var/log that will be consumed by ALL of the Suricata (or Snort, in that package) log files. It's this latter fail-safe code that was neglecting to remove the pcap files when the log directory size limit was exceeded. That will be fixed in the next update of Suricata.
Bill
-
Clarification and Update on this issue:
There are configuration parameters on the INTERFACE SETTINGS tab where packet capture files are enabled that allow you specify the size limit in megabytes for each pcap file and the total number of pcap files to retain in the log directory. The defaults are 32 MB for file size and 1000 pcap files maintained. Once the number of pcap files to retain value is exceeded, the oldest file overwritten.
With that said, there is also a type of "fail safe" feature associated with both Snort and Suricata that lets you set an overall limit in MB on the amount of disk space in /var/log that will be consumed by ALL of the Suricata (or Snort, in that package) log files. It's this latter fail-safe code that was neglecting to remove the pcap files when the log directory size limit was exceeded. That will be fixed in the next update of Suricata.
Bill
No no, I meant that the log rotation ignores the limits set for the packet logs. If setting a limit of 100MB, and a log number of 100, logs get cut off at 100MB as expected, and a new one is created until the size is reached, but the number of logs increases to 160, for example.
I identified the bug when the packet logs started filling up all the space that was assigned to suricata's log files. For some strange reason the blocked tab would cut off at a random number of hosts, and suricata would constantly complain that the log directory is full. Did an ls on the directory, which showed that packet logs weren't properly getting rotated, which led to the directory becoming full, and suricata freaking out and cutting off the alerts/blocks/other logs, but keeping the packet logs intact.
-
@jflsakfja:
No no, I meant that the log rotation ignores the limits set for the packet logs. If setting a limit of 100MB, and a log number of 100, logs get cut off at 100MB as expected, and a new one is created until the size is reached, but the number of logs increases to 160, for example.
I identified the bug when the packet logs started filling up all the space that was assigned to suricata's log files. For some strange reason the blocked tab would cut off at a random number of hosts, and suricata would constantly complain that the log directory is full. Did an ls on the directory, which showed that packet logs weren't properly getting rotated, which led to the directory becoming full, and suricata freaking out and cutting off the alerts/blocks/other logs, but keeping the packet logs intact.
Oh…that sounds like perhaps a bug in the Suricata binary. The limits set on the INTERFACE SETTINGS tab (where the pcap feature is turned on or off) are all handled by built-in parts of the binary. Only settings on the LOG MGMT tab are controlled by PHP code I wrote. I will test the binary rotation in a VM by setting the number very low (like 1 file) to see what happens.
Bill
-
@jflsakfja:
No no, I meant that the log rotation ignores the limits set for the packet logs. If setting a limit of 100MB, and a log number of 100, logs get cut off at 100MB as expected, and a new one is created until the size is reached, but the number of logs increases to 160, for example.
I identified the bug when the packet logs started filling up all the space that was assigned to suricata's log files. For some strange reason the blocked tab would cut off at a random number of hosts, and suricata would constantly complain that the log directory is full. Did an ls on the directory, which showed that packet logs weren't properly getting rotated, which led to the directory becoming full, and suricata freaking out and cutting off the alerts/blocks/other logs, but keeping the packet logs intact.
Looked into the binary source code and this is not really a bug. It's more a case of a bit misleading feature. Suricata keeps track of the number of pcap files it creates during a given session and correctly rotates them. However, the key here is "a given session". It does not maintain state through startup/shutdown cycles. That means as far as Suricata is concerned, when it starts up, it has opened "zero files". So any that existed prior to the last Suricata shutdown are ignored. So the misleading part is that the "max-files" parameter really only works so long as Suricata is never restarted… :-\
So to address this I will need to put something in the GUI code's cron job. I will do that in the update I am currently working on.
Bill
-
@jflsakfja:
No no, I meant that the log rotation ignores the limits set for the packet logs. If setting a limit of 100MB, and a log number of 100, logs get cut off at 100MB as expected, and a new one is created until the size is reached, but the number of logs increases to 160, for example.
I identified the bug when the packet logs started filling up all the space that was assigned to suricata's log files. For some strange reason the blocked tab would cut off at a random number of hosts, and suricata would constantly complain that the log directory is full. Did an ls on the directory, which showed that packet logs weren't properly getting rotated, which led to the directory becoming full, and suricata freaking out and cutting off the alerts/blocks/other logs, but keeping the packet logs intact.
Looked into the binary source code and this is not really a bug. It's more a case of a bit misleading feature. Suricata keeps track of the number of pcap files it creates during a given session and correctly rotates them. However, the key here is "a given session". It does not maintain state through startup/shutdown cycles. That means as far as Suricata is concerned, when it starts up, it has opened "zero files". So any that existed prior to the last Suricata shutdown are ignored. So the misleading part is that the "max-files" parameter really only works so long as Suricata is never restarted… :-\
So to address this I will need to put something in the GUI code's cron job. I will do that in the update I am currently working on.
Bill
You've got it worse than I have it in my day job. Thumbs up to you, sir.
-
Has this issue been resolved?
I am also running a PPPoE Connection and trying to get Suricata configured but my logs are flooded with "SC_ERR_DATALINK_UNIMPLEMENTED" errors.
Suricata will not start automatically after a pfsense restart, is this by design or caused by these errors?
Regards,
-
Not sure (since I've never needed pppoe), and mind is a bit fuzzy right now (been working straight for the past 50 hours, with a pass-out-on-the-office-chair in between) but have you tried making suricata listen on the underlying interface, instead of the pppoe one? Oh well, someone more knowledgeable on this will come along. Please correct me if I'm wrong in any way.
-
I have not tried that yet as it was noted earlier in the thread it was a bad idea to do that as Suricata will receive header information from the PPPoE interface which it wont know how to handle.
I will have to research how to do this as at the moment suricata is not generating any alerts at all.
-
No alerts usually means that it's not recognizing the subnets (technically home_net). Are they declared correctly?
-
Has this issue been resolved?
I am also running a PPPoE Connection and trying to get Suricata configured but my logs are flooded with "SC_ERR_DATALINK_UNIMPLEMENTED" errors.
Suricata will not start automatically after a pfsense restart, is this by design or caused by these errors?
Regards,
No, not resolved and can't be really resolved until there is an upstream patch to Suricata so it recognizes the DLT_NULL data link type that FreeBSD assigns to PPPoE interfaces. This is a problem in the Suricata binary that comes from upstream.
You can make it partially work by hand-editing the suricata.yaml file and telling it the interface is an Ethernet physical link (by using the real physical NIC driver name), but you will still get occasional errors due to the PPPoE frame headers since Suricata does not expect to see them on a physical Ethernet interface. Also, I can't guarantee you that all the detection signatures will work properly. In short, this kind of hack is unsupported and you would be on your own.
If you have a PPPoE interface and want IDS/IPS on it, then you should use Snort instead of Suricata. The Snort binary fully supports the DLT_NULL data link type that FreeBSD uses.
Bill