Suricata 2.0.3 pkg v2.0.2 - Release Notes
-
I have a new install of pfSense and Suricata. This is the only version of Suricata I've used.
Services: Suricata 2.0.3 pkg v2.0.2 - Intrusion Detection System
2.1.5-RELEASE (amd64)
built on Mon Aug 25 07:44:45 EDT 2014
FreeBSD pfsense.bedrock 8.3-RELEASE-p16 FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014I hit the CSV comma bug in block.log that I thought was fixed in the previous release, so I'm confused.
This is the entry in block.log
10/09/2014-21:10:46.431197,Block Src,1,2001972,17,ET SCAN Behavioral Unusually fast Terminal Server Traffic, Potential Scan or Infection (Inbound),Misc activity,3,TCP,124.232.152.37,18785This is the error in the system log
php: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:10:46.431197 [1] => Block Src [2] => 1 [3] => 2001972 [4] => 17 [5] => ET SCAN Behavioral Unusually fast Terminal Server Traffic [6] => Potential Scan or Infection (Inbound) [7] => Misc activity [8] => 3 [9] => TCP [10] => 124.232.152.37 [11] => 18785 )I do see alerts and blocks in the tabs, but the error keeps getting logged. I deleted block.log and that fixed it for now, but I thought this wasn't supposed to happen again with this version?
-Stuart
-
I have a new install of pfSense and Suricata. This is the only version of Suricata I've used.
Services: Suricata 2.0.3 pkg v2.0.2 - Intrusion Detection System
2.1.5-RELEASE (amd64)
built on Mon Aug 25 07:44:45 EDT 2014
FreeBSD pfsense.bedrock 8.3-RELEASE-p16 FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014I hit the CSV comma bug in block.log that I thought was fixed in the previous release, so I'm confused.
This is the entry in block.log
10/09/2014-21:10:46.431197,Block Src,1,2001972,17,ET SCAN Behavioral Unusually fast Terminal Server Traffic, Potential Scan or Infection (Inbound),Misc activity,3,TCP,124.232.152.37,18785This is the error in the system log
php: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:10:46.431197 [1] => Block Src [2] => 1 [3] => 2001972 [4] => 17 [5] => ET SCAN Behavioral Unusually fast Terminal Server Traffic [6] => Potential Scan or Infection (Inbound) [7] => Misc activity [8] => 3 [9] => TCP [10] => 124.232.152.37 [11] => 18785 )I do see alerts and blocks in the tabs, but the error keeps getting logged. I deleted block.log and that fixed it for now, but I thought this wasn't supposed to happen again with this version?
-Stuart
Well, the error specifically fixed was in the ALERTS tab. The BLOCKED tab gets its data from a log file that is still in CSV format. I did put a hack in so it should not corrupt the HTML display, and it logs the offending entry to the system log as you see. I will put this fix on my TODO list.
Bill
-
Thanks Bill for the correction. I missed the that distinction. I now see the difference in the format of alerts.log and block.log.
Being new to pfSense and Suricata, I found the learning curve quite short and the product very good, a pleasant surprise for a firewall / IDS!
I do have a couple of feature suggestions.
It would be nice to be able to set the highest priority level to block. I ended up suppressing some priority 3 rules so that I could enable blocking. I'd rather still see/log most of those priority 3 alerts but have only priority 1 and 2 alerts cause blocks to be generated.
When installing the Snort VRT rules you need to enter the rules filename. It took a while to find the filename to use and be confident I had the correct one (which turned out was the same as the GUI example). In the release description of the change to add the filename field, there was a good explanation of why it was needed, where to get it and how frequently it would change. It would be nice to capture that info into the GUI.
Thanks,
-Stuart -
Thanks Bill for the correction. I missed the that distinction. I now see the difference in the format of alerts.log and block.log.
Being new to pfSense and Suricata, I found the learning curve quite short and the product very good, a pleasant surprise for a firewall / IDS!
I do have a couple of feature suggestions.
It would be nice to be able to set the highest priority level to block. I ended up suppressing some priority 3 rules so that I could enable blocking. I'd rather still see/log most of those priority 3 alerts but have only priority 1 and 2 alerts cause blocks to be generated.
When installing the Snort VRT rules you need to enter the rules filename. It took a while to find the filename to use and be confident I had the correct one (which turned out was the same as the GUI example). In the release description of the change to add the filename field, there was a good explanation of why it was needed, where to get it and how frequently it would change. It would be nice to capture that info into the GUI.
Thanks,
-StuartI intend to fix the Block log problem. Might can sneak it into the next update. If not, it will go in the following one. That log is written by the custom blocking plugin I created for Suricata, so it's familiar territory for me.
I had actually thought about adding an option to block by priority. If more folks are interested, I could add it. I will see what I can do about the VRT filename. The Sourcefire guys messed up a good thing when they discontinued their "edge" file that always contained the latest rule update. The problem I face on the GUI is somewhat limited space, but maybe there is enough room without making it too cluttered to put a URL link to the rules file or something.
Bill
-
Related to the block.log… I get this only with long IPv6 address : ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...
Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:09:57.398584 [1] => Block Dst [2] => 1 [3] => 770058 [4] => 1 [5] => PB-PROTOCOL ICMP for IPv6 RFC 4443 [6] => RFC 4884 [7] => Detection of a Non-Standard Protocol or Event [8] => 2 [9] => IPV6-ICMP [10] => ff02:0000:0000:0000:0000:0000:0000:0001 [11] => 0 )
Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry…F.
-
Related to the block.log… I get this only with long IPv6 address : ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...
Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:09:57.398584 [1] => Block Dst [2] => 1 [3] => 770058 [4] => 1 [5] => PB-PROTOCOL ICMP for IPv6 RFC 4443 [6] => RFC 4884 [7] => Detection of a Non-Standard Protocol or Event [8] => 2 [9] => IPV6-ICMP [10] => ff02:0000:0000:0000:0000:0000:0000:0001 [11] => 0 )
Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry…F.
Can you also post text from the ALERTS and BLOCK log for this exact event? You can correlate the timestamp to be sure you get the exact event. I'm working now on coding a fix for this problem on the BLOCK tab. You can find these two files in the /var/log/suricata subdirectory for the interface, or you can go to the LOGS VIEW tab and select the interface and log file to view.
Bill
-
bmeeks: do we have plane to update to suricata 2.0.4 or 2.1 ? :)
-
Bill, can you change the blocks tab to show most recent first like the alerts tab?
-Stuart
-
Bill, can you change the blocks tab to show most recent first like the alerts tab?
-Stuart
The column headers are sortable. Just click on any one of the headers.
-
Bill, can you change the blocks tab to show most recent first like the alerts tab?
-Stuart
The column headers are sortable. Just click on any one of the headers.
There's no "Date" column on Blocks like there is on "Alerts", so nothing to click to get most recent first.
I have auto refresh turned on, so having the default would be nice since the column sort doesn't stick through an auto refresh.
What is the default sort order of the Blocks? It seems mostly "recent last", but the first part of the list doesn't quite match that. Is it ordered by "earliest seen last"? (I have a 1hr expiration on blocks)
-
Logs rotated from that event, but heres a new one, same error and this time not from a custom rule.
Cheers.
sys.log
Oct 12 14:10:11 php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/11/2014-21:27:04.782000 [1] => Block Src [2] => 1 [3] => 22 [4] => 1 [5] => FILE pdf claimed [6] => but not pdf [7] => (null) [8] => 3 [9] => TCP [10] => 128.119.103.XXX [11] => 80 ) Oct 12 14:10:11 php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...alert.log
10/11/2014-21:27:04.782000 [**] [1:22:1] FILE pdf claimed, but not pdf [**] [Classification: (null)] [Priority: 3] {TCP} 128.119.103.XXX:80 -> 192.168.1.XXX:47798block.log
10/11/2014-21:27:04.782000,Block Src,1,22,1,FILE pdf claimed, but not pdf,(null),3,TCP,128.119.103.XXX,80rule
alert http any any -> any any (msg:"FILE pdf claimed, but not pdf"; flow:established,to_client; fileext:"pdf"; filemagic:!"PDF document"; filestore; sid:22; rev:1;) -
Logs rotated from that event, but heres a new one, same error and this time not from a custom rule.
Cheers.
sys.log
Oct 12 14:10:11 php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/11/2014-21:27:04.782000 [1] => Block Src [2] => 1 [3] => 22 [4] => 1 [5] => FILE pdf claimed [6] => but not pdf [7] => (null) [8] => 3 [9] => TCP [10] => 128.119.103.XXX [11] => 80 ) Oct 12 14:10:11 php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...alert.log
10/11/2014-21:27:04.782000 [**] [1:22:1] FILE pdf claimed, but not pdf [**] [Classification: (null)] [Priority: 3] {TCP} 128.119.103.XXX:80 -> 192.168.1.XXX:47798block.log
10/11/2014-21:27:04.782000,Block Src,1,22,1,FILE pdf claimed, but not pdf,(null),3,TCP,128.119.103.XXX,80rule
alert http any any -> any any (msg:"FILE pdf claimed, but not pdf"; flow:established,to_client; fileext:"pdf"; filemagic:!"PDF document"; filestore; sid:22; rev:1;)I'm working on the fix, but this is really just a cosmetic error. The custom module that inserts a block also writes some of the fields provided by the Suricata alert module to a log file (the block.log file). To make it easy to parse in PHP, I configured the blocking module to separate those Suricata fields with commas. PHP has a built-in function for reading a CSV text file and placing each field into an array. This technique has been used for ages in the Snort package. However, in Suricata an unforeseen problem popped up because a tiny handful of the text rules have embedded commas in the rule message. That messes up the CSV parser because it expects the 11 fields of data to be delimited by commas. When you have extra commas as part of the data of a field, the CSV parser winds up presenting PHP with 12, 13 or possibly more fields. That messes up the order in which the BLOCK tab code expects the fields to be in.
I have a fix working in my test lab and hope to push it out soon for the pfSense developers to review and hopefully approve.
Bill
-
Hey Bill,
Thanks again. Another question; I dont understand how the SID MGMT works.
Lets say I want to remove the fast_pattern:only from those SID 31944,29895,28807,28406,14081,21474,26722
I then edit the modifysid-sample.conf and add:
31944,29895,28807,28406,14081,21474,26722 "fast_pattern:only; " ""Then select the proper interface (LAN) in the Interface SID Management File Assignements and select modify-sample.conf under Modify SID File, click save and nothing happens ;(
Please let me know what I doing wrong.
Thanks!
F.
-
Hi fsansfil,
I would suggest you make a new modify-sid (and enable/disable sid also) because if you use the "Sample" Ones, they will be overwritten when you do any re-install or Package Updates.
Did you click the "Rebuild" Checkboxes beside the Interface Names?
-
Hey there,
Thanks for the quick reply. Just tested it again, this time with a newly created modify.config, did checked rebuild in front of the interface (LAN), still nothing. Seems the check rebuild button dont save, every refresh it goes back to uncheck…
Thanks.
F.
-
The Rebuild Checkbox does not stay checked after you hit save…
Once you make you Sid Mgmt Changes, Click "Rebuild" then save.
Goto the Interface (ie: LAN) click "Edit"
LAN Rules: and Look for the Rule that you modified to see if the changes were made to the Rule.
-
Here's a good example of the need to have control over blocking by the alert priority level.
These are from windows update trying to download new updates today.
10/15/2014-08:57:59.102113,Block Src,1,2015744,3,ET INFO EXE IsDebuggerPresent (Used in Malware Anti-Debugging),Misc activity,3,TCP,96.17.15.72,80 10/15/2014-08:59:32.716282,Block Src,1,2014819,3,ET INFO Packed Executable Download,Misc activity,3,TCP,96.17.15.96,80If the IPS cries wolf too much it'll get turned off. Other than during the initial setup, I've only had one alert with pri < 3 that's been a false positive. I've had a couple dozen pri 3 that have been false positives since the initial setup about a week ago. The choice seems to be between constant monitoring for false positives or turning off large blocks or categories of rules, neither of which I'd like to do.
Is there a simple solution to this situation I've missed?
-Stuart
-
Here's a good example of the need to have control over blocking by the alert priority level.
These are from windows update trying to download new updates today.
10/15/2014-08:57:59.102113,Block Src,1,2015744,3,ET INFO EXE IsDebuggerPresent (Used in Malware Anti-Debugging),Misc activity,3,TCP,96.17.15.72,80 10/15/2014-08:59:32.716282,Block Src,1,2014819,3,ET INFO Packed Executable Download,Misc activity,3,TCP,96.17.15.96,80If the IPS cries wolf too much it'll get turned off. Other than during the initial setup, I've only had one alert with pri < 3 that's been a false positive. I've had a couple dozen pri 3 that have been false positives since the initial setup about a week ago. The choice seems to be between constant monitoring for false positives or turning off large blocks or categories of rules, neither of which I'd like to do.
Is there a simple solution to this situation I've missed?
-Stuart
There is currently no capability for filtering by PRIORITY for block actions in the Suricata or Snort packages. That will require rewrites of the custom patch that is compiled into both Snort and Suricata in order to implement blocking by interfacing with the pf firewall in pfSense. Just thinking about it off the cuff, I don't see too much of a problem implementing it in the near future. But that is just off the cuff without a detailed code review. Could be there is something that might prove to be an obstacle to implementing the feature.
Bill
-
Hey there,
Thanks for the quick reply. Just tested it again, this time with a newly created modify.config, did checked rebuild in front of the interface (LAN), still nothing. Seems the check rebuild button dont save, every refresh it goes back to uncheck…
Thanks.
F.
BBcan177 is correct in describing the steps. The new SID MGMT tab takes the configuration files and uses the instructions within to create a list of rules Suricata will use for inspecting traffic. Literally, the primary purpose of the SID MGMT tab itself is to let you upload, edit and assign various text configuration files to your interfaces. Saving simply does this part. The actual creation of the new list of rules for Suricata to use for network traffic inspection happens in these three cases:
1. You click the REBUILD checkbox next to the interface on the SID MGMT tab. That will not only save the file assignments as described above, but will trigger the creation of a new list of rules for the interface. Once the list is created, the running Suricata process is sent a "live rule reload" signal. You will see this in the suricata.log file viewed in the LOGS VIEW tab for the interface. As BBcan177 said, you can then go to the CATEGORIES or RULES tabs for the interface and see the new color-coded icons beside the impacted text rules.
2. Once auto-SID management is enabled, every time you click SAVE on any other interface tabs, the list of rules is rebuilt using the logic in the assigned SID MGMT configuration files.
3. Finally, when an automatic download of updated rules occurs via the scheduled update job, a new rules list is generated and Suricata is signaled to load it.
Bill
