Suricata 2.0.3 pkg v2.0.2 - Release Notes



  • Suricata 2.0.3 pkg v2.0.2

    This update for the Suricata package corrects five reported bugs and adds two enhancements.

    Important:  if applying this upgrade to CARP sync hosts, first temporarily disable Suricata package replication on the master, and then upgrade the master and all slaves to the new package version.  When all members are upgraded, you may re-enable CARP sync of the Suricata package.

    Bug Fixes

    1. Barnyard tab gives the following error: "Fatal error: Can't use function return value in write context in /usr/local/www/suricata/suricata_barnyard.php on line 99" when trying to save an edited value.

    2. Changes to configuration on some tabs are not synced to CARP slaves when hitting SAVE or APPLY buttons.

    3. The word CANCEL is misspelled in the tooltip message for the CLEAR button on the BLOCKED tab.

    4. Entries left over in the block.log from older Suricata versions can cause display errors when viewing blocked hosts.

    5. New interfaces created with the DUP (clone interface) icon introduced in the 1.4.6 version of the package share the UUID of their parent instead of getting their own unique ID.

    New Features/Enhancements

    1. Added a count of User Enabled and User Disabled rules to the summary information displayed at the bottom of the RULES tab.

    2. Improve security by using $_POST for Suppress List deletion.



  • Thanks a lot Bill. The automated fix for duplicate UUID works perfectly (tried it on an old backup)!



  • @digdug3:

    Thanks a lot Bill. The automated fix for duplicate UUID works perfectly (tried it on an old backup)!

    Great!  Thanks for the feedback.



  • Well, it was running OK until sometime this AM when it seemingly randomly crashed and fails to restart. The only recent change to the system was enabling the DHCPv6 server, I disabled that again but it did not fix the issue.. Rebooting and reinstalling so far haven't helped. The only relevant error I found is at the end of the suricata.log below

    22/9/2014 -- 10:54:02 - <notice>-- This is Suricata version 2.0.3 RELEASE
    22/9/2014 -- 10:54:02 - <info>-- CPUs/cores online: 4
    22/9/2014 -- 10:54:02 - <info>-- Live rule reloads enabled
    22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
    22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
    22/9/2014 -- 10:54:02 - <info>-- HTTP memcap: 67108864
    22/9/2014 -- 10:54:02 - <info>-- DNS request flood protection level: 500
    22/9/2014 -- 10:54:02 - <info>-- DNS per flow memcap (state-memcap): 524288
    22/9/2014 -- 10:54:02 - <info>-- DNS global memcap: 16777216
    22/9/2014 -- 10:54:02 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24
    22/9/2014 -- 10:54:02 - <info>-- preallocated 65535 defrag trackers of size 144
    22/9/2014 -- 10:54:02 - <info>-- defrag memory usage: 11009904 bytes, maximum: 33554432
    22/9/2014 -- 10:54:02 - <info>-- AutoFP mode using "Active Packets" flow load balancer
    22/9/2014 -- 10:54:02 - <info>-- preallocated 1024 packets. Total memory 3508224
    22/9/2014 -- 10:54:03 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
    22/9/2014 -- 10:54:03 - <info>-- preallocated 1000 hosts of size 88
    22/9/2014 -- 10:54:03 - <info>-- host memory usage: 366144 bytes, maximum: 16777216
    22/9/2014 -- 10:54:03 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
    22/9/2014 -- 10:54:03 - <info>-- preallocated 10000 flows of size 240
    22/9/2014 -- 10:54:03 - <info>-- flow memory usage: 6674304 bytes, maximum: 33554432
    22/9/2014 -- 10:54:03 - <info>-- IP reputation disabled
    22/9/2014 -- 10:54:03 - <info>-- Added "35" classification types from the classification file
    22/9/2014 -- 10:54:03 - <info>-- Added "19" reference types from the reference.config file
    22/9/2014 -- 10:54:03 - <info>-- using magic-file /usr/share/misc/magic
    22/9/2014 -- 10:54:03 - <info>-- Delayed detect disabled
    22/9/2014 -- 10:54:06 - <info>-- 2 rule files processed. 3736 rules successfully loaded, 0 rules failed
    22/9/2014 -- 10:54:07 - <info>-- 3736 signatures processed. 190 are IP-only rules, 1004 are inspecting packet payload, 2886 inspect application layer, 0 are decoder event only
    22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
    22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
    22/9/2014 -- 10:54:12 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
    22/9/2014 -- 10:54:13 - <info>-- Threshold config parsed: 3 rule(s) found
    22/9/2014 -- 10:54:13 - <info>-- Core dump size is unlimited.
    22/9/2014 -- 10:54:13 - <info>-- alert-pf output device (regular) initialized: block.log
    22/9/2014 -- 10:54:13 - <error>-- [ERRCODE: SC_ERR_INVALID_ARGUMENTS(52)] - prefix or user NULL</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice> 
    

    running v2.0.2 on 2.1.5-RELEASE (amd64).



  • I checked this morning and my WAN interface monitoring was down, log looked like this

    
    20/9/2014 -- 23:37:34 - <notice>-- This is Suricata version 2.0.3 RELEASE
    20/9/2014 -- 23:37:34 - <info>-- CPUs/cores online: 1
    20/9/2014 -- 23:37:34 - <info>-- Live rule reloads enabled
    20/9/2014 -- 23:37:34 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
    20/9/2014 -- 23:37:34 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
    20/9/2014 -- 23:37:34 - <info>-- HTTP memcap: 67108864
    20/9/2014 -- 23:37:34 - <info>-- DNS request flood protection level: 500
    20/9/2014 -- 23:37:34 - <info>-- DNS per flow memcap (state-memcap): 524288
    20/9/2014 -- 23:37:34 - <info>-- DNS global memcap: 16777216
    20/9/2014 -- 23:37:34 - <info>-- allocated 786432 bytes of memory for the defrag hash... 65536 buckets of size 12
    20/9/2014 -- 23:37:34 - <info>-- preallocated 65535 defrag trackers of size 100
    20/9/2014 -- 23:37:34 - <info>-- defrag memory usage: 7339932 bytes, maximum: 33554432
    20/9/2014 -- 23:37:34 - <info>-- AutoFP mode using "Active Packets" flow load balancer
    20/9/2014 -- 23:37:34 - <info>-- preallocated 1024 packets. Total memory 2754560
    20/9/2014 -- 23:37:34 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
    20/9/2014 -- 23:37:34 - <info>-- preallocated 1000 hosts of size 56
    20/9/2014 -- 23:37:34 - <info>-- host memory usage: 326144 bytes, maximum: 16777216
    20/9/2014 -- 23:37:34 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
    20/9/2014 -- 23:37:34 - <info>-- preallocated 10000 flows of size 156
    20/9/2014 -- 23:37:34 - <info>-- flow memory usage: 5794304 bytes, maximum: 33554432
    20/9/2014 -- 23:37:34 - <info>-- IP reputation disabled
    20/9/2014 -- 23:37:34 - <info>-- Added "35" classification types from the classification file
    20/9/2014 -- 23:37:34 - <info>-- Added "19" reference types from the reference.config file
    20/9/2014 -- 23:37:34 - <info>-- using magic-file /usr/share/misc/magic
    20/9/2014 -- 23:37:34 - <info>-- Delayed detect disabled
    20/9/2014 -- 23:38:04 - <info>-- 2 rule files processed. 14895 rules successfully loaded, 0 rules failed
    20/9/2014 -- 23:38:04 - <info>-- 14896 signatures processed. 912 are IP-only rules, 4166 are inspecting packet payload, 11516 inspect application layer, 0 are decoder event only
    20/9/2014 -- 23:38:04 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
    20/9/2014 -- 23:38:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
    20/9/2014 -- 23:38:35 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
    20/9/2014 -- 23:38:46 - <info>-- Threshold config parsed: 1 rule(s) found
    20/9/2014 -- 23:38:46 - <info>-- Core dump size is unlimited.
    20/9/2014 -- 23:38:46 - <info>-- fast output device (regular) initialized: alerts.log
    20/9/2014 -- 23:38:46 - <info>-- Using 1 live device(s).
    20/9/2014 -- 23:38:46 - <info>-- using interface nfe0
    20/9/2014 -- 23:38:46 - <info>-- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
    20/9/2014 -- 23:38:46 - <info>-- Found an MTU of 1500 for 'nfe0'
    20/9/2014 -- 23:38:46 - <info>-- Set snaplen to 1516 for 'nfe0'
    20/9/2014 -- 23:38:46 - <info>-- RunModeIdsPcapAutoFp initialised
    20/9/2014 -- 23:38:46 - <info>-- stream "prealloc-sessions": 32768 (per thread)
    20/9/2014 -- 23:38:46 - <info>-- stream "memcap": 33554432
    20/9/2014 -- 23:38:46 - <info>-- stream "midstream" session pickups: disabled
    20/9/2014 -- 23:38:46 - <info>-- stream "async-oneside": disabled
    20/9/2014 -- 23:38:46 - <info>-- stream "checksum-validation": disabled
    20/9/2014 -- 23:38:46 - <info>-- stream."inline": disabled
    20/9/2014 -- 23:38:46 - <info>-- stream "max-synack-queued": 5
    20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "memcap": 67108864
    20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "depth": 0
    20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "toserver-chunk-size": 2610
    20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "toclient-chunk-size": 2663
    20/9/2014 -- 23:38:46 - <info>-- stream.reassembly.raw: enabled
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 4, prealloc 256
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 16, prealloc 512
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 112, prealloc 512
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 248, prealloc 512
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 512, prealloc 512
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 768, prealloc 1024
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 1448, prealloc 1024
    20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 65535, prealloc 128
    20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "chunk-prealloc": 250
    20/9/2014 -- 23:38:46 - <notice>-- all 2 packet processing threads, 1 management threads initialized, engine started.
    20/9/2014 -- 23:38:46 - <notice>-- Signal Received.  Stopping engine.
    20/9/2014 -- 23:38:46 - <info>-- 0 new flows, 0 established flows were timed out, 0 flows in closed state
    20/9/2014 -- 23:38:46 - <info>-- time elapsed 0.125s
    20/9/2014 -- 23:38:46 - <info>-- (RxPcapnfe01) Packets 0, bytes 0
    20/9/2014 -- 23:38:46 - <info>-- (RxPcapnfe01) Pcap Total:0 Recv:0 Drop:0 (nan%).
    20/9/2014 -- 23:38:46 - <info>-- AutoFP - Total flow handler queues - 1
    20/9/2014 -- 23:38:46 - <info>-- AutoFP - Queue 0  - pkts: 0            flows: 0           
    20/9/2014 -- 23:38:46 - <info>-- Stream TCP processed 0 TCP packets
    20/9/2014 -- 23:38:46 - <info>-- Fast log output wrote 0 alerts
    20/9/2014 -- 23:38:46 - <info>-- host memory usage: 326144 bytes, maximum: 16777216
    20/9/2014 -- 23:38:47 - <info>-- cleaning up signature grouping structure... complete
    20/9/2014 -- 23:38:47 - <notice>-- Stats for 'nfe0':  pkts: 0, drop: 0 (nan%), invalid chksum: 0
    20/9/2014 -- 23:38:48 - <error>-- [ERRCODE: SC_ERR_DAEMON(87)] - Child died unexpectedly</error></notice></info></info></info></info></info></info></info></info></info></info></notice></notice></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice> 
    


  • @DigitalDeviant:

    Well, it was running OK until sometime this AM when it seemingly randomly crashed and fails to restart. The only recent change to the system was enabling the DHCPv6 server, I disabled that again but it did not fix the issue.. Rebooting and reinstalling so far haven't helped. The only relevant error I found is at the end of the suricata.log below

    22/9/2014 -- 10:54:02 - <notice>-- This is Suricata version 2.0.3 RELEASE
    22/9/2014 -- 10:54:02 - <info>-- CPUs/cores online: 4
    22/9/2014 -- 10:54:02 - <info>-- Live rule reloads enabled
    22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
    22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
    22/9/2014 -- 10:54:02 - <info>-- HTTP memcap: 67108864
    22/9/2014 -- 10:54:02 - <info>-- DNS request flood protection level: 500
    22/9/2014 -- 10:54:02 - <info>-- DNS per flow memcap (state-memcap): 524288
    22/9/2014 -- 10:54:02 - <info>-- DNS global memcap: 16777216
    22/9/2014 -- 10:54:02 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24
    22/9/2014 -- 10:54:02 - <info>-- preallocated 65535 defrag trackers of size 144
    22/9/2014 -- 10:54:02 - <info>-- defrag memory usage: 11009904 bytes, maximum: 33554432
    22/9/2014 -- 10:54:02 - <info>-- AutoFP mode using "Active Packets" flow load balancer
    22/9/2014 -- 10:54:02 - <info>-- preallocated 1024 packets. Total memory 3508224
    22/9/2014 -- 10:54:03 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
    22/9/2014 -- 10:54:03 - <info>-- preallocated 1000 hosts of size 88
    22/9/2014 -- 10:54:03 - <info>-- host memory usage: 366144 bytes, maximum: 16777216
    22/9/2014 -- 10:54:03 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
    22/9/2014 -- 10:54:03 - <info>-- preallocated 10000 flows of size 240
    22/9/2014 -- 10:54:03 - <info>-- flow memory usage: 6674304 bytes, maximum: 33554432
    22/9/2014 -- 10:54:03 - <info>-- IP reputation disabled
    22/9/2014 -- 10:54:03 - <info>-- Added "35" classification types from the classification file
    22/9/2014 -- 10:54:03 - <info>-- Added "19" reference types from the reference.config file
    22/9/2014 -- 10:54:03 - <info>-- using magic-file /usr/share/misc/magic
    22/9/2014 -- 10:54:03 - <info>-- Delayed detect disabled
    22/9/2014 -- 10:54:06 - <info>-- 2 rule files processed. 3736 rules successfully loaded, 0 rules failed
    22/9/2014 -- 10:54:07 - <info>-- 3736 signatures processed. 190 are IP-only rules, 1004 are inspecting packet payload, 2886 inspect application layer, 0 are decoder event only
    22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
    22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
    22/9/2014 -- 10:54:12 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
    22/9/2014 -- 10:54:13 - <info>-- Threshold config parsed: 3 rule(s) found
    22/9/2014 -- 10:54:13 - <info>-- Core dump size is unlimited.
    22/9/2014 -- 10:54:13 - <info>-- alert-pf output device (regular) initialized: block.log
    22/9/2014 -- 10:54:13 - <error>-- [ERRCODE: SC_ERR_INVALID_ARGUMENTS(52)] - prefix or user NULL</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice> 
    

    running v2.0.2 on 2.1.5-RELEASE (amd64).

    Actually, my issue ended up being the IPV6 addresses of my DNS servers added to the pass list from "Add WAN DNS servers to the list." I guess it took a day or two to update the pass lists automatically since I re-enabled IPv6 for the first time in a few months a couple days ago and added the DNS servers shortly after.



  • @DigitalDeviant:

    @DigitalDeviant:

    Well, it was running OK until sometime this AM when it seemingly randomly crashed and fails to restart. The only recent change to the system was enabling the DHCPv6 server, I disabled that again but it did not fix the issue.. Rebooting and reinstalling so far haven't helped. The only relevant error I found is at the end of the suricata.log below

    22/9/2014 -- 10:54:02 - <notice>-- This is Suricata version 2.0.3 RELEASE
    22/9/2014 -- 10:54:02 - <info>-- CPUs/cores online: 4
    22/9/2014 -- 10:54:02 - <info>-- Live rule reloads enabled
    22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
    22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
    22/9/2014 -- 10:54:02 - <info>-- HTTP memcap: 67108864
    22/9/2014 -- 10:54:02 - <info>-- DNS request flood protection level: 500
    22/9/2014 -- 10:54:02 - <info>-- DNS per flow memcap (state-memcap): 524288
    22/9/2014 -- 10:54:02 - <info>-- DNS global memcap: 16777216
    22/9/2014 -- 10:54:02 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24
    22/9/2014 -- 10:54:02 - <info>-- preallocated 65535 defrag trackers of size 144
    22/9/2014 -- 10:54:02 - <info>-- defrag memory usage: 11009904 bytes, maximum: 33554432
    22/9/2014 -- 10:54:02 - <info>-- AutoFP mode using "Active Packets" flow load balancer
    22/9/2014 -- 10:54:02 - <info>-- preallocated 1024 packets. Total memory 3508224
    22/9/2014 -- 10:54:03 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
    22/9/2014 -- 10:54:03 - <info>-- preallocated 1000 hosts of size 88
    22/9/2014 -- 10:54:03 - <info>-- host memory usage: 366144 bytes, maximum: 16777216
    22/9/2014 -- 10:54:03 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
    22/9/2014 -- 10:54:03 - <info>-- preallocated 10000 flows of size 240
    22/9/2014 -- 10:54:03 - <info>-- flow memory usage: 6674304 bytes, maximum: 33554432
    22/9/2014 -- 10:54:03 - <info>-- IP reputation disabled
    22/9/2014 -- 10:54:03 - <info>-- Added "35" classification types from the classification file
    22/9/2014 -- 10:54:03 - <info>-- Added "19" reference types from the reference.config file
    22/9/2014 -- 10:54:03 - <info>-- using magic-file /usr/share/misc/magic
    22/9/2014 -- 10:54:03 - <info>-- Delayed detect disabled
    22/9/2014 -- 10:54:06 - <info>-- 2 rule files processed. 3736 rules successfully loaded, 0 rules failed
    22/9/2014 -- 10:54:07 - <info>-- 3736 signatures processed. 190 are IP-only rules, 1004 are inspecting packet payload, 2886 inspect application layer, 0 are decoder event only
    22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
    22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
    22/9/2014 -- 10:54:12 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
    22/9/2014 -- 10:54:13 - <info>-- Threshold config parsed: 3 rule(s) found
    22/9/2014 -- 10:54:13 - <info>-- Core dump size is unlimited.
    22/9/2014 -- 10:54:13 - <info>-- alert-pf output device (regular) initialized: block.log
    22/9/2014 -- 10:54:13 - <error>-- [ERRCODE: SC_ERR_INVALID_ARGUMENTS(52)] - prefix or user NULL</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice> 
    

    running v2.0.2 on 2.1.5-RELEASE (amd64).

    Actually, my issue ended up being the IPV6 addresses of my DNS servers added to the pass list from "Add WAN DNS servers to the list." I guess it took a day or two to update the pass lists automatically since I re-enabled IPv6 for the first time in a few months a couple days ago and added the DNS servers shortly after.

    Values in a PASS LIST are only updated with a Suricata restart.  They cannot be dynamically updated due to limitations within the Suricata binary.  So when you make a change to a PASS LIST, you must stop and restart Suricata for those changes to become active.  Could be that an overnight automatic rule update (assuming you have those enabled) finally restarted Suricata and it picked up something in the updated PASS LIST.  The rules update process only restarts Suricata if an actual rules package download and update is necessary (that is, the new rules have been posted by the rules vendor).

    Bill



  • I have a new install of pfSense and Suricata.  This is the only version of Suricata I've used.

    Services: Suricata 2.0.3 pkg v2.0.2 - Intrusion Detection System

    2.1.5-RELEASE (amd64)
    built on Mon Aug 25 07:44:45 EDT 2014
    FreeBSD pfsense.bedrock 8.3-RELEASE-p16 FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014

    I hit the CSV comma bug in block.log that I thought was fixed in the previous release, so I'm confused.

    This is the entry in block.log

    10/09/2014-21:10:46.431197,Block Src,1,2001972,17,ET SCAN Behavioral Unusually fast Terminal Server Traffic, Potential Scan or Infection (Inbound),Misc activity,3,TCP,124.232.152.37,18785
    
    

    This is the error in the system log

    php: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:10:46.431197 [1] => Block Src [2] => 1 [3] => 2001972 [4] => 17 [5] => ET SCAN Behavioral Unusually fast Terminal Server Traffic [6] => Potential Scan or Infection (Inbound) [7] => Misc activity [8] => 3 [9] => TCP [10] => 124.232.152.37 [11] => 18785 )
    
    

    I do see alerts and blocks in the tabs, but the error keeps getting logged.  I deleted block.log and that fixed it for now, but I thought this wasn't supposed to happen again with this version?

    -Stuart



  • @gogglespisano:

    I have a new install of pfSense and Suricata.  This is the only version of Suricata I've used.

    Services: Suricata 2.0.3 pkg v2.0.2 - Intrusion Detection System

    2.1.5-RELEASE (amd64)
    built on Mon Aug 25 07:44:45 EDT 2014
    FreeBSD pfsense.bedrock 8.3-RELEASE-p16 FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014

    I hit the CSV comma bug in block.log that I thought was fixed in the previous release, so I'm confused.

    This is the entry in block.log

    10/09/2014-21:10:46.431197,Block Src,1,2001972,17,ET SCAN Behavioral Unusually fast Terminal Server Traffic, Potential Scan or Infection (Inbound),Misc activity,3,TCP,124.232.152.37,18785
    
    

    This is the error in the system log

    php: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:10:46.431197 [1] => Block Src [2] => 1 [3] => 2001972 [4] => 17 [5] => ET SCAN Behavioral Unusually fast Terminal Server Traffic [6] => Potential Scan or Infection (Inbound) [7] => Misc activity [8] => 3 [9] => TCP [10] => 124.232.152.37 [11] => 18785 )
    
    

    I do see alerts and blocks in the tabs, but the error keeps getting logged.  I deleted block.log and that fixed it for now, but I thought this wasn't supposed to happen again with this version?

    -Stuart

    Well, the error specifically fixed was in the ALERTS tab.  The BLOCKED tab gets its data from a log file that is still in CSV format.  I did put a hack in so it should not corrupt the HTML display, and it logs the offending entry to the system log as you see.  I will put this fix on my TODO list.

    Bill



  • Thanks Bill for the correction.  I missed the that distinction.  I now see the difference in the format of alerts.log and block.log.

    Being new to pfSense and Suricata, I found the learning curve quite short and the product very good, a pleasant surprise for a firewall / IDS!

    I do have a couple of feature suggestions.

    It would be nice to be able to set the highest priority level to block.  I ended up suppressing some priority 3 rules so that I could enable blocking.  I'd rather still see/log most of those priority 3 alerts but have only priority 1 and 2 alerts cause blocks to be generated.

    When installing the Snort VRT rules you need to enter the rules filename.  It took a while to find the filename to use and be confident I had the correct one (which turned out was the same as the GUI example).  In the release description of the change to add the filename field, there was a good explanation of why it was needed, where to get it and how frequently it would change.  It would be nice to capture that info into the GUI.

    Thanks,
    -Stuart



  • @gogglespisano:

    Thanks Bill for the correction.  I missed the that distinction.  I now see the difference in the format of alerts.log and block.log.

    Being new to pfSense and Suricata, I found the learning curve quite short and the product very good, a pleasant surprise for a firewall / IDS!

    I do have a couple of feature suggestions.

    It would be nice to be able to set the highest priority level to block.  I ended up suppressing some priority 3 rules so that I could enable blocking.  I'd rather still see/log most of those priority 3 alerts but have only priority 1 and 2 alerts cause blocks to be generated.

    When installing the Snort VRT rules you need to enter the rules filename.  It took a while to find the filename to use and be confident I had the correct one (which turned out was the same as the GUI example).  In the release description of the change to add the filename field, there was a good explanation of why it was needed, where to get it and how frequently it would change.  It would be nice to capture that info into the GUI.

    Thanks,
    -Stuart

    I intend to fix the Block log problem.  Might can sneak it into the next update.  If not, it will go in the following one.  That log is written by the custom blocking plugin I created for Suricata, so it's familiar territory for me.

    I had actually thought about adding an option to block by priority.  If more folks are interested, I could add it.  I will see what I can do about the VRT filename.  The Sourcefire guys messed up a good thing when they discontinued their "edge" file that always contained the latest rule update.  The problem I face on the GUI is somewhat limited space, but maybe there is enough room without making it too cluttered to put a URL link to the rules file or something.

    Bill



  • Related to the block.log… I get this only with long IPv6 address : ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...

    Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:09:57.398584 [1] => Block Dst [2] => 1 [3] => 770058 [4] => 1 [5] => PB-PROTOCOL ICMP for IPv6 RFC 4443 [6] => RFC 4884 [7] => Detection of a Non-Standard Protocol or Event [8] => 2 [9] => IPV6-ICMP [10] => ff02:0000:0000:0000:0000:0000:0000:0001 [11] => 0 )
    Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry…

    F.



  • @fsansfil:

    Related to the block.log… I get this only with long IPv6 address : ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...

    Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:09:57.398584 [1] => Block Dst [2] => 1 [3] => 770058 [4] => 1 [5] => PB-PROTOCOL ICMP for IPv6 RFC 4443 [6] => RFC 4884 [7] => Detection of a Non-Standard Protocol or Event [8] => 2 [9] => IPV6-ICMP [10] => ff02:0000:0000:0000:0000:0000:0000:0001 [11] => 0 )
    Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry…

    F.

    Can you also post text from the ALERTS and BLOCK log for this exact event?  You can correlate the timestamp to be sure you get the exact event.  I'm working now on coding a fix for this problem on the BLOCK tab.  You can find these two files in the /var/log/suricata subdirectory for the interface, or you can go to the LOGS VIEW tab and select the interface and log file to view.

    Bill



  • bmeeks: do we have plane to update to suricata 2.0.4 or 2.1 ? :)



  • Bill, can you change the blocks tab to show most recent first like the alerts tab?

    -Stuart


  • Moderator

    @gogglespisano:

    Bill, can you change the blocks tab to show most recent first like the alerts tab?

    -Stuart

    The column headers are sortable. Just click on any one of the headers.



  • @BBcan177:

    @gogglespisano:

    Bill, can you change the blocks tab to show most recent first like the alerts tab?

    -Stuart

    The column headers are sortable. Just click on any one of the headers.

    There's no "Date" column on Blocks like there is on "Alerts", so nothing to click to get most recent first.

    I have auto refresh turned on, so having the default would be nice since the column sort doesn't stick through an auto refresh.

    What is the default sort order of the Blocks?  It seems mostly "recent last", but the first part of the list doesn't quite match that.  Is it ordered by "earliest seen last"?  (I have a 1hr expiration on blocks)



  • Logs rotated from that event, but heres a new one, same error and this time not from a custom rule.

    Cheers.

    sys.log

    Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/11/2014-21:27:04.782000 [1] => Block Src [2] => 1 [3] => 22 [4] => 1 [5] => FILE pdf claimed [6] => but not pdf [7] => (null) [8] => 3 [9] => TCP [10] => 128.119.103.XXX [11] => 80 )
    Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...
    

    alert.log

    10/11/2014-21:27:04.782000  [**] [1:22:1] FILE pdf claimed, but not pdf [**] [Classification: (null)] [Priority: 3] {TCP} 128.119.103.XXX:80 -> 192.168.1.XXX:47798
    

    block.log

    10/11/2014-21:27:04.782000,Block Src,1,22,1,FILE pdf claimed, but not pdf,(null),3,TCP,128.119.103.XXX,80
    

    rule

    alert http any any -> any any (msg:"FILE pdf claimed, but not pdf"; flow:established,to_client; fileext:"pdf"; filemagic:!"PDF document"; filestore; sid:22; rev:1;)
    


  • @fsansfil:

    Logs rotated from that event, but heres a new one, same error and this time not from a custom rule.

    Cheers.

    sys.log

    Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/11/2014-21:27:04.782000 [1] => Block Src [2] => 1 [3] => 22 [4] => 1 [5] => FILE pdf claimed [6] => but not pdf [7] => (null) [8] => 3 [9] => TCP [10] => 128.119.103.XXX [11] => 80 )
    Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...
    

    alert.log

    10/11/2014-21:27:04.782000  [**] [1:22:1] FILE pdf claimed, but not pdf [**] [Classification: (null)] [Priority: 3] {TCP} 128.119.103.XXX:80 -> 192.168.1.XXX:47798
    

    block.log

    10/11/2014-21:27:04.782000,Block Src,1,22,1,FILE pdf claimed, but not pdf,(null),3,TCP,128.119.103.XXX,80
    

    rule

    alert http any any -> any any (msg:"FILE pdf claimed, but not pdf"; flow:established,to_client; fileext:"pdf"; filemagic:!"PDF document"; filestore; sid:22; rev:1;)
    

    I'm working on the fix, but this is really just a cosmetic error.  The custom module that inserts a block also writes some of the fields provided by the Suricata alert module to a log file (the block.log file).  To make it easy to parse in PHP, I configured the blocking module to separate those Suricata fields with commas.  PHP has a built-in function for reading a CSV text file and placing each field into an array.  This technique has been used for ages in the Snort package.  However, in Suricata an unforeseen problem popped up because a tiny handful of the text rules have embedded commas in the rule message.  That messes up the CSV parser because it expects the 11 fields of data to be delimited by commas.  When you have extra commas as part of the data of a field, the CSV parser winds up presenting PHP with 12, 13 or possibly more fields.  That messes up the order in which the BLOCK tab code expects the fields to be in.

    I have a fix working in my test lab and hope to push it out soon for the pfSense developers to review and hopefully approve.

    Bill



  • Hey Bill,

    Thanks again. Another question; I dont understand how the SID MGMT works.

    Lets say I want to remove the fast_pattern:only from those SID 31944,29895,28807,28406,14081,21474,26722

    I then edit the modifysid-sample.conf and add:
    31944,29895,28807,28406,14081,21474,26722 "fast_pattern:only; " ""

    Then select the proper interface (LAN) in the Interface SID Management File Assignements and select modify-sample.conf under Modify SID File, click save and nothing happens ;(

    Please let me know what I doing wrong.

    Thanks!

    F.


  • Moderator

    Hi fsansfil,

    I would suggest you make a new modify-sid (and enable/disable sid also) because if you use the "Sample" Ones, they will be overwritten when you do any re-install or Package Updates.

    Did you click the "Rebuild" Checkboxes beside the Interface Names?



  • Hey there,

    Thanks for the quick reply. Just tested it again, this time with a newly created modify.config, did checked rebuild in front of the interface (LAN), still nothing. Seems the check rebuild button dont save, every refresh it goes back to uncheck…

    Thanks.

    F.


  • Moderator

    The Rebuild Checkbox does not stay checked after you hit save…

    Once you make you Sid Mgmt Changes, Click "Rebuild" then save.

    Goto the Interface (ie: LAN) click "Edit"

    LAN Rules: and Look for the Rule that you modified to see if the changes were made to the Rule.



  • Here's a good example of the need to have control over blocking by the alert priority level.

    These are from windows update trying to download new updates today.

    10/15/2014-08:57:59.102113,Block Src,1,2015744,3,ET INFO EXE IsDebuggerPresent (Used in Malware Anti-Debugging),Misc activity,3,TCP,96.17.15.72,80
    10/15/2014-08:59:32.716282,Block Src,1,2014819,3,ET INFO Packed Executable Download,Misc activity,3,TCP,96.17.15.96,80
    
    

    If the IPS cries wolf too much it'll get turned off.  Other than during the initial setup, I've only had one alert with pri < 3 that's been a false positive.  I've had a couple dozen pri 3 that have been false positives since the initial setup about a week ago.  The choice seems to be between constant monitoring for false positives or turning off large blocks or categories of rules, neither of which I'd like to do.

    Is there a simple solution to this situation I've missed?

    -Stuart



  • @gogglespisano:

    Here's a good example of the need to have control over blocking by the alert priority level.

    These are from windows update trying to download new updates today.

    10/15/2014-08:57:59.102113,Block Src,1,2015744,3,ET INFO EXE IsDebuggerPresent (Used in Malware Anti-Debugging),Misc activity,3,TCP,96.17.15.72,80
    10/15/2014-08:59:32.716282,Block Src,1,2014819,3,ET INFO Packed Executable Download,Misc activity,3,TCP,96.17.15.96,80
    
    

    If the IPS cries wolf too much it'll get turned off.  Other than during the initial setup, I've only had one alert with pri < 3 that's been a false positive.  I've had a couple dozen pri 3 that have been false positives since the initial setup about a week ago.  The choice seems to be between constant monitoring for false positives or turning off large blocks or categories of rules, neither of which I'd like to do.

    Is there a simple solution to this situation I've missed?

    -Stuart

    There is currently no capability for filtering by PRIORITY for block actions in the Suricata or Snort packages.  That will require rewrites of the custom patch that is compiled into both Snort and Suricata in order to implement blocking by interfacing with the pf firewall in pfSense.  Just thinking about it off the cuff, I don't see too much of a problem implementing it in the near future.  But that is just off the cuff without a detailed code review.  Could be there is something that might prove to be an obstacle to implementing the feature.

    Bill



  • @fsansfil:

    Hey there,

    Thanks for the quick reply. Just tested it again, this time with a newly created modify.config, did checked rebuild in front of the interface (LAN), still nothing. Seems the check rebuild button dont save, every refresh it goes back to uncheck…

    Thanks.

    F.

    BBcan177 is correct in describing the steps.  The new SID MGMT tab takes the configuration files and uses the instructions within to create a list of rules Suricata will use for inspecting traffic.  Literally, the primary purpose of the SID MGMT tab itself is to let you upload, edit and assign various text configuration files to your interfaces.  Saving simply does this part.  The actual creation of the new list of rules for Suricata to use for network traffic inspection happens in these three cases:

    1.  You click the REBUILD checkbox next to the interface on the SID MGMT tab.  That will not only save the file assignments as described above, but will trigger the creation of a new list of rules for the interface.  Once the list is created, the running Suricata process is sent a "live rule reload" signal.  You will see this in the suricata.log file viewed in the LOGS VIEW tab for the interface.  As BBcan177 said, you can then go to the CATEGORIES or RULES tabs for the interface and see the new color-coded icons beside the impacted text rules.

    2.  Once auto-SID management is enabled, every time you click SAVE on any other interface tabs, the list of rules is rebuilt using the logic in the assigned SID MGMT configuration files.

    3.  Finally, when an automatic download of updated rules occurs via the scheduled update job, a new rules list is generated and Suricata is signaled to load it.

    Bill


Log in to reply