Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Suricata 2.0.3 pkg v2.0.2 - Release Notes

    Scheduled Pinned Locked Moved pfSense Packages
    26 Posts 8 Posters 5.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      DigitalDeviant
      last edited by

      Well, it was running OK until sometime this AM when it seemingly randomly crashed and fails to restart. The only recent change to the system was enabling the DHCPv6 server, I disabled that again but it did not fix the issue.. Rebooting and reinstalling so far haven't helped. The only relevant error I found is at the end of the suricata.log below

      22/9/2014 -- 10:54:02 - <notice>-- This is Suricata version 2.0.3 RELEASE
22/9/2014 -- 10:54:02 - <info>-- CPUs/cores online: 4
22/9/2014 -- 10:54:02 - <info>-- Live rule reloads enabled
22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
22/9/2014 -- 10:54:02 - <info>-- HTTP memcap: 67108864
22/9/2014 -- 10:54:02 - <info>-- DNS request flood protection level: 500
22/9/2014 -- 10:54:02 - <info>-- DNS per flow memcap (state-memcap): 524288
22/9/2014 -- 10:54:02 - <info>-- DNS global memcap: 16777216
22/9/2014 -- 10:54:02 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24
22/9/2014 -- 10:54:02 - <info>-- preallocated 65535 defrag trackers of size 144
22/9/2014 -- 10:54:02 - <info>-- defrag memory usage: 11009904 bytes, maximum: 33554432
22/9/2014 -- 10:54:02 - <info>-- AutoFP mode using "Active Packets" flow load balancer
22/9/2014 -- 10:54:02 - <info>-- preallocated 1024 packets. Total memory 3508224
22/9/2014 -- 10:54:03 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
22/9/2014 -- 10:54:03 - <info>-- preallocated 1000 hosts of size 88
22/9/2014 -- 10:54:03 - <info>-- host memory usage: 366144 bytes, maximum: 16777216
22/9/2014 -- 10:54:03 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
22/9/2014 -- 10:54:03 - <info>-- preallocated 10000 flows of size 240
22/9/2014 -- 10:54:03 - <info>-- flow memory usage: 6674304 bytes, maximum: 33554432
22/9/2014 -- 10:54:03 - <info>-- IP reputation disabled
22/9/2014 -- 10:54:03 - <info>-- Added "35" classification types from the classification file
22/9/2014 -- 10:54:03 - <info>-- Added "19" reference types from the reference.config file
22/9/2014 -- 10:54:03 - <info>-- using magic-file /usr/share/misc/magic
22/9/2014 -- 10:54:03 - <info>-- Delayed detect disabled
22/9/2014 -- 10:54:06 - <info>-- 2 rule files processed. 3736 rules successfully loaded, 0 rules failed
22/9/2014 -- 10:54:07 - <info>-- 3736 signatures processed. 190 are IP-only rules, 1004 are inspecting packet payload, 2886 inspect application layer, 0 are decoder event only
22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
22/9/2014 -- 10:54:12 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
22/9/2014 -- 10:54:13 - <info>-- Threshold config parsed: 3 rule(s) found
22/9/2014 -- 10:54:13 - <info>-- Core dump size is unlimited.
22/9/2014 -- 10:54:13 - <info>-- alert-pf output device (regular) initialized: block.log
22/9/2014 -- 10:54:13 - <error>-- [ERRCODE: SC_ERR_INVALID_ARGUMENTS(52)] - prefix or user NULL</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice>

      running v2.0.2 on 2.1.5-RELEASE (amd64).

      1 Reply Last reply Reply Quote 0
      • T
        Trel
        last edited by

        I checked this morning and my WAN interface monitoring was down, log looked like this

        
20/9/2014 -- 23:37:34 - <notice>-- This is Suricata version 2.0.3 RELEASE
20/9/2014 -- 23:37:34 - <info>-- CPUs/cores online: 1
20/9/2014 -- 23:37:34 - <info>-- Live rule reloads enabled
20/9/2014 -- 23:37:34 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
20/9/2014 -- 23:37:34 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
20/9/2014 -- 23:37:34 - <info>-- HTTP memcap: 67108864
20/9/2014 -- 23:37:34 - <info>-- DNS request flood protection level: 500
20/9/2014 -- 23:37:34 - <info>-- DNS per flow memcap (state-memcap): 524288
20/9/2014 -- 23:37:34 - <info>-- DNS global memcap: 16777216
20/9/2014 -- 23:37:34 - <info>-- allocated 786432 bytes of memory for the defrag hash... 65536 buckets of size 12
20/9/2014 -- 23:37:34 - <info>-- preallocated 65535 defrag trackers of size 100
20/9/2014 -- 23:37:34 - <info>-- defrag memory usage: 7339932 bytes, maximum: 33554432
20/9/2014 -- 23:37:34 - <info>-- AutoFP mode using "Active Packets" flow load balancer
20/9/2014 -- 23:37:34 - <info>-- preallocated 1024 packets. Total memory 2754560
20/9/2014 -- 23:37:34 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
20/9/2014 -- 23:37:34 - <info>-- preallocated 1000 hosts of size 56
20/9/2014 -- 23:37:34 - <info>-- host memory usage: 326144 bytes, maximum: 16777216
20/9/2014 -- 23:37:34 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
20/9/2014 -- 23:37:34 - <info>-- preallocated 10000 flows of size 156
20/9/2014 -- 23:37:34 - <info>-- flow memory usage: 5794304 bytes, maximum: 33554432
20/9/2014 -- 23:37:34 - <info>-- IP reputation disabled
20/9/2014 -- 23:37:34 - <info>-- Added "35" classification types from the classification file
20/9/2014 -- 23:37:34 - <info>-- Added "19" reference types from the reference.config file
20/9/2014 -- 23:37:34 - <info>-- using magic-file /usr/share/misc/magic
20/9/2014 -- 23:37:34 - <info>-- Delayed detect disabled
20/9/2014 -- 23:38:04 - <info>-- 2 rule files processed. 14895 rules successfully loaded, 0 rules failed
20/9/2014 -- 23:38:04 - <info>-- 14896 signatures processed. 912 are IP-only rules, 4166 are inspecting packet payload, 11516 inspect application layer, 0 are decoder event only
20/9/2014 -- 23:38:04 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
20/9/2014 -- 23:38:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
20/9/2014 -- 23:38:35 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
20/9/2014 -- 23:38:46 - <info>-- Threshold config parsed: 1 rule(s) found
20/9/2014 -- 23:38:46 - <info>-- Core dump size is unlimited.
20/9/2014 -- 23:38:46 - <info>-- fast output device (regular) initialized: alerts.log
20/9/2014 -- 23:38:46 - <info>-- Using 1 live device(s).
20/9/2014 -- 23:38:46 - <info>-- using interface nfe0
20/9/2014 -- 23:38:46 - <info>-- Running in 'auto' checksum mode. Detection of interface state will require 1000 packets.
20/9/2014 -- 23:38:46 - <info>-- Found an MTU of 1500 for 'nfe0'
20/9/2014 -- 23:38:46 - <info>-- Set snaplen to 1516 for 'nfe0'
20/9/2014 -- 23:38:46 - <info>-- RunModeIdsPcapAutoFp initialised
20/9/2014 -- 23:38:46 - <info>-- stream "prealloc-sessions": 32768 (per thread)
20/9/2014 -- 23:38:46 - <info>-- stream "memcap": 33554432
20/9/2014 -- 23:38:46 - <info>-- stream "midstream" session pickups: disabled
20/9/2014 -- 23:38:46 - <info>-- stream "async-oneside": disabled
20/9/2014 -- 23:38:46 - <info>-- stream "checksum-validation": disabled
20/9/2014 -- 23:38:46 - <info>-- stream."inline": disabled
20/9/2014 -- 23:38:46 - <info>-- stream "max-synack-queued": 5
20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "memcap": 67108864
20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "depth": 0
20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "toserver-chunk-size": 2610
20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "toclient-chunk-size": 2663
20/9/2014 -- 23:38:46 - <info>-- stream.reassembly.raw: enabled
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 4, prealloc 256
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 16, prealloc 512
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 112, prealloc 512
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 248, prealloc 512
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 512, prealloc 512
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 768, prealloc 1024
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 1448, prealloc 1024
20/9/2014 -- 23:38:46 - <info>-- segment pool: pktsize 65535, prealloc 128
20/9/2014 -- 23:38:46 - <info>-- stream.reassembly "chunk-prealloc": 250
20/9/2014 -- 23:38:46 - <notice>-- all 2 packet processing threads, 1 management threads initialized, engine started.
20/9/2014 -- 23:38:46 - <notice>-- Signal Received.  Stopping engine.
20/9/2014 -- 23:38:46 - <info>-- 0 new flows, 0 established flows were timed out, 0 flows in closed state
20/9/2014 -- 23:38:46 - <info>-- time elapsed 0.125s
20/9/2014 -- 23:38:46 - <info>-- (RxPcapnfe01) Packets 0, bytes 0
20/9/2014 -- 23:38:46 - <info>-- (RxPcapnfe01) Pcap Total:0 Recv:0 Drop:0 (nan%).
20/9/2014 -- 23:38:46 - <info>-- AutoFP - Total flow handler queues - 1
20/9/2014 -- 23:38:46 - <info>-- AutoFP - Queue 0  - pkts: 0            flows: 0           
20/9/2014 -- 23:38:46 - <info>-- Stream TCP processed 0 TCP packets
20/9/2014 -- 23:38:46 - <info>-- Fast log output wrote 0 alerts
20/9/2014 -- 23:38:46 - <info>-- host memory usage: 326144 bytes, maximum: 16777216
20/9/2014 -- 23:38:47 - <info>-- cleaning up signature grouping structure... complete
20/9/2014 -- 23:38:47 - <notice>-- Stats for 'nfe0':  pkts: 0, drop: 0 (nan%), invalid chksum: 0
20/9/2014 -- 23:38:48 - <error>-- [ERRCODE: SC_ERR_DAEMON(87)] - Child died unexpectedly</error></notice></info></info></info></info></info></info></info></info></info></info></notice></notice></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice>
        1 Reply Last reply Reply Quote 0
        • D
          DigitalDeviant
          last edited by

          @DigitalDeviant:

          Well, it was running OK until sometime this AM when it seemingly randomly crashed and fails to restart. The only recent change to the system was enabling the DHCPv6 server, I disabled that again but it did not fix the issue.. Rebooting and reinstalling so far haven't helped. The only relevant error I found is at the end of the suricata.log below

          22/9/2014 -- 10:54:02 - <notice>-- This is Suricata version 2.0.3 RELEASE
22/9/2014 -- 10:54:02 - <info>-- CPUs/cores online: 4
22/9/2014 -- 10:54:02 - <info>-- Live rule reloads enabled
22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
22/9/2014 -- 10:54:02 - <info>-- HTTP memcap: 67108864
22/9/2014 -- 10:54:02 - <info>-- DNS request flood protection level: 500
22/9/2014 -- 10:54:02 - <info>-- DNS per flow memcap (state-memcap): 524288
22/9/2014 -- 10:54:02 - <info>-- DNS global memcap: 16777216
22/9/2014 -- 10:54:02 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24
22/9/2014 -- 10:54:02 - <info>-- preallocated 65535 defrag trackers of size 144
22/9/2014 -- 10:54:02 - <info>-- defrag memory usage: 11009904 bytes, maximum: 33554432
22/9/2014 -- 10:54:02 - <info>-- AutoFP mode using "Active Packets" flow load balancer
22/9/2014 -- 10:54:02 - <info>-- preallocated 1024 packets. Total memory 3508224
22/9/2014 -- 10:54:03 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
22/9/2014 -- 10:54:03 - <info>-- preallocated 1000 hosts of size 88
22/9/2014 -- 10:54:03 - <info>-- host memory usage: 366144 bytes, maximum: 16777216
22/9/2014 -- 10:54:03 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
22/9/2014 -- 10:54:03 - <info>-- preallocated 10000 flows of size 240
22/9/2014 -- 10:54:03 - <info>-- flow memory usage: 6674304 bytes, maximum: 33554432
22/9/2014 -- 10:54:03 - <info>-- IP reputation disabled
22/9/2014 -- 10:54:03 - <info>-- Added "35" classification types from the classification file
22/9/2014 -- 10:54:03 - <info>-- Added "19" reference types from the reference.config file
22/9/2014 -- 10:54:03 - <info>-- using magic-file /usr/share/misc/magic
22/9/2014 -- 10:54:03 - <info>-- Delayed detect disabled
22/9/2014 -- 10:54:06 - <info>-- 2 rule files processed. 3736 rules successfully loaded, 0 rules failed
22/9/2014 -- 10:54:07 - <info>-- 3736 signatures processed. 190 are IP-only rules, 1004 are inspecting packet payload, 2886 inspect application layer, 0 are decoder event only
22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
22/9/2014 -- 10:54:12 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
22/9/2014 -- 10:54:13 - <info>-- Threshold config parsed: 3 rule(s) found
22/9/2014 -- 10:54:13 - <info>-- Core dump size is unlimited.
22/9/2014 -- 10:54:13 - <info>-- alert-pf output device (regular) initialized: block.log
22/9/2014 -- 10:54:13 - <error>-- [ERRCODE: SC_ERR_INVALID_ARGUMENTS(52)] - prefix or user NULL</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice>

          running v2.0.2 on 2.1.5-RELEASE (amd64).

          Actually, my issue ended up being the IPV6 addresses of my DNS servers added to the pass list from "Add WAN DNS servers to the list." I guess it took a day or two to update the pass lists automatically since I re-enabled IPv6 for the first time in a few months a couple days ago and added the DNS servers shortly after.

          1 Reply Last reply Reply Quote 0
          • bmeeksB
            bmeeks
            last edited by

            @DigitalDeviant:

            @DigitalDeviant:

            Well, it was running OK until sometime this AM when it seemingly randomly crashed and fails to restart. The only recent change to the system was enabling the DHCPv6 server, I disabled that again but it did not fix the issue.. Rebooting and reinstalling so far haven't helped. The only relevant error I found is at the end of the suricata.log below

            22/9/2014 -- 10:54:02 - <notice>-- This is Suricata version 2.0.3 RELEASE
22/9/2014 -- 10:54:02 - <info>-- CPUs/cores online: 4
22/9/2014 -- 10:54:02 - <info>-- Live rule reloads enabled
22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
22/9/2014 -- 10:54:02 - <info>-- 'default' server has 'response-body-minimal-inspect-size' set to 33695 and 'response-body-inspect-window' set to 4218 after randomization.
22/9/2014 -- 10:54:02 - <info>-- HTTP memcap: 67108864
22/9/2014 -- 10:54:02 - <info>-- DNS request flood protection level: 500
22/9/2014 -- 10:54:02 - <info>-- DNS per flow memcap (state-memcap): 524288
22/9/2014 -- 10:54:02 - <info>-- DNS global memcap: 16777216
22/9/2014 -- 10:54:02 - <info>-- allocated 1572864 bytes of memory for the defrag hash... 65536 buckets of size 24
22/9/2014 -- 10:54:02 - <info>-- preallocated 65535 defrag trackers of size 144
22/9/2014 -- 10:54:02 - <info>-- defrag memory usage: 11009904 bytes, maximum: 33554432
22/9/2014 -- 10:54:02 - <info>-- AutoFP mode using "Active Packets" flow load balancer
22/9/2014 -- 10:54:02 - <info>-- preallocated 1024 packets. Total memory 3508224
22/9/2014 -- 10:54:03 - <info>-- allocated 262144 bytes of memory for the host hash... 4096 buckets of size 64
22/9/2014 -- 10:54:03 - <info>-- preallocated 1000 hosts of size 88
22/9/2014 -- 10:54:03 - <info>-- host memory usage: 366144 bytes, maximum: 16777216
22/9/2014 -- 10:54:03 - <info>-- allocated 4194304 bytes of memory for the flow hash... 65536 buckets of size 64
22/9/2014 -- 10:54:03 - <info>-- preallocated 10000 flows of size 240
22/9/2014 -- 10:54:03 - <info>-- flow memory usage: 6674304 bytes, maximum: 33554432
22/9/2014 -- 10:54:03 - <info>-- IP reputation disabled
22/9/2014 -- 10:54:03 - <info>-- Added "35" classification types from the classification file
22/9/2014 -- 10:54:03 - <info>-- Added "19" reference types from the reference.config file
22/9/2014 -- 10:54:03 - <info>-- using magic-file /usr/share/misc/magic
22/9/2014 -- 10:54:03 - <info>-- Delayed detect disabled
22/9/2014 -- 10:54:06 - <info>-- 2 rule files processed. 3736 rules successfully loaded, 0 rules failed
22/9/2014 -- 10:54:07 - <info>-- 3736 signatures processed. 190 are IP-only rules, 1004 are inspecting packet payload, 2886 inspect application layer, 0 are decoder event only
22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 1: preprocessing rules... complete
22/9/2014 -- 10:54:07 - <info>-- building signature grouping structure, stage 2: building source address list... complete
22/9/2014 -- 10:54:12 - <info>-- building signature grouping structure, stage 3: building destination address lists... complete
22/9/2014 -- 10:54:13 - <info>-- Threshold config parsed: 3 rule(s) found
22/9/2014 -- 10:54:13 - <info>-- Core dump size is unlimited.
22/9/2014 -- 10:54:13 - <info>-- alert-pf output device (regular) initialized: block.log
22/9/2014 -- 10:54:13 - <error>-- [ERRCODE: SC_ERR_INVALID_ARGUMENTS(52)] - prefix or user NULL</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></notice>

            running v2.0.2 on 2.1.5-RELEASE (amd64).

            Actually, my issue ended up being the IPV6 addresses of my DNS servers added to the pass list from "Add WAN DNS servers to the list." I guess it took a day or two to update the pass lists automatically since I re-enabled IPv6 for the first time in a few months a couple days ago and added the DNS servers shortly after.

            Values in a PASS LIST are only updated with a Suricata restart.  They cannot be dynamically updated due to limitations within the Suricata binary.  So when you make a change to a PASS LIST, you must stop and restart Suricata for those changes to become active.  Could be that an overnight automatic rule update (assuming you have those enabled) finally restarted Suricata and it picked up something in the updated PASS LIST.  The rules update process only restarts Suricata if an actual rules package download and update is necessary (that is, the new rules have been posted by the rules vendor).

            Bill

            1 Reply Last reply Reply Quote 0
            • G
              gogglespisano
              last edited by

              I have a new install of pfSense and Suricata.  This is the only version of Suricata I've used.

              Services: Suricata 2.0.3 pkg v2.0.2 - Intrusion Detection System

              2.1.5-RELEASE (amd64)
              built on Mon Aug 25 07:44:45 EDT 2014
              FreeBSD pfsense.bedrock 8.3-RELEASE-p16 FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014

              I hit the CSV comma bug in block.log that I thought was fixed in the previous release, so I'm confused.

              This is the entry in block.log

              10/09/2014-21:10:46.431197,Block Src,1,2001972,17,ET SCAN Behavioral Unusually fast Terminal Server Traffic, Potential Scan or Infection (Inbound),Misc activity,3,TCP,124.232.152.37,18785

              This is the error in the system log

              php: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:10:46.431197 [1] => Block Src [2] => 1 [3] => 2001972 [4] => 17 [5] => ET SCAN Behavioral Unusually fast Terminal Server Traffic [6] => Potential Scan or Infection (Inbound) [7] => Misc activity [8] => 3 [9] => TCP [10] => 124.232.152.37 [11] => 18785 )

              I do see alerts and blocks in the tabs, but the error keeps getting logged.  I deleted block.log and that fixed it for now, but I thought this wasn't supposed to happen again with this version?

              -Stuart

              1 Reply Last reply Reply Quote 0
              • bmeeksB
                bmeeks
                last edited by

                @gogglespisano:

                I have a new install of pfSense and Suricata.  This is the only version of Suricata I've used.

                Services: Suricata 2.0.3 pkg v2.0.2 - Intrusion Detection System

                2.1.5-RELEASE (amd64)
                built on Mon Aug 25 07:44:45 EDT 2014
                FreeBSD pfsense.bedrock 8.3-RELEASE-p16 FreeBSD 8.3-RELEASE-p16 #0: Mon Aug 25 08:27:11 EDT 2014

                I hit the CSV comma bug in block.log that I thought was fixed in the previous release, so I'm confused.

                This is the entry in block.log

                10/09/2014-21:10:46.431197,Block Src,1,2001972,17,ET SCAN Behavioral Unusually fast Terminal Server Traffic, Potential Scan or Infection (Inbound),Misc activity,3,TCP,124.232.152.37,18785

                This is the error in the system log

                php: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:10:46.431197 [1] => Block Src [2] => 1 [3] => 2001972 [4] => 17 [5] => ET SCAN Behavioral Unusually fast Terminal Server Traffic [6] => Potential Scan or Infection (Inbound) [7] => Misc activity [8] => 3 [9] => TCP [10] => 124.232.152.37 [11] => 18785 )

                I do see alerts and blocks in the tabs, but the error keeps getting logged.  I deleted block.log and that fixed it for now, but I thought this wasn't supposed to happen again with this version?

                -Stuart

                Well, the error specifically fixed was in the ALERTS tab.  The BLOCKED tab gets its data from a log file that is still in CSV format.  I did put a hack in so it should not corrupt the HTML display, and it logs the offending entry to the system log as you see.  I will put this fix on my TODO list.

                Bill

                1 Reply Last reply Reply Quote 0
                • G
                  gogglespisano
                  last edited by

                  Thanks Bill for the correction.  I missed the that distinction.  I now see the difference in the format of alerts.log and block.log.

                  Being new to pfSense and Suricata, I found the learning curve quite short and the product very good, a pleasant surprise for a firewall / IDS!

                  I do have a couple of feature suggestions.

                  It would be nice to be able to set the highest priority level to block.  I ended up suppressing some priority 3 rules so that I could enable blocking.  I'd rather still see/log most of those priority 3 alerts but have only priority 1 and 2 alerts cause blocks to be generated.

                  When installing the Snort VRT rules you need to enter the rules filename.  It took a while to find the filename to use and be confident I had the correct one (which turned out was the same as the GUI example).  In the release description of the change to add the filename field, there was a good explanation of why it was needed, where to get it and how frequently it would change.  It would be nice to capture that info into the GUI.

                  Thanks,
                  -Stuart

                  1 Reply Last reply Reply Quote 0
                  • bmeeksB
                    bmeeks
                    last edited by

                    @gogglespisano:

                    Thanks Bill for the correction.  I missed the that distinction.  I now see the difference in the format of alerts.log and block.log.

                    Being new to pfSense and Suricata, I found the learning curve quite short and the product very good, a pleasant surprise for a firewall / IDS!

                    I do have a couple of feature suggestions.

                    It would be nice to be able to set the highest priority level to block.  I ended up suppressing some priority 3 rules so that I could enable blocking.  I'd rather still see/log most of those priority 3 alerts but have only priority 1 and 2 alerts cause blocks to be generated.

                    When installing the Snort VRT rules you need to enter the rules filename.  It took a while to find the filename to use and be confident I had the correct one (which turned out was the same as the GUI example).  In the release description of the change to add the filename field, there was a good explanation of why it was needed, where to get it and how frequently it would change.  It would be nice to capture that info into the GUI.

                    Thanks,
                    -Stuart

                    I intend to fix the Block log problem.  Might can sneak it into the next update.  If not, it will go in the following one.  That log is written by the custom blocking plugin I created for Suricata, so it's familiar territory for me.

                    I had actually thought about adding an option to block by priority.  If more folks are interested, I could add it.  I will see what I can do about the VRT filename.  The Sourcefire guys messed up a good thing when they discontinued their "edge" file that always contained the latest rule update.  The problem I face on the GUI is somewhat limited space, but maybe there is enough room without making it too cluttered to put a URL link to the rules file or something.

                    Bill

                    1 Reply Last reply Reply Quote 0
                    • F
                      fsansfil
                      last edited by

                      Related to the block.log… I get this only with long IPv6 address : ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...

                      Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:09:57.398584 [1] => Block Dst [2] => 1 [3] => 770058 [4] => 1 [5] => PB-PROTOCOL ICMP for IPv6 RFC 4443 [6] => RFC 4884 [7] => Detection of a Non-Standard Protocol or Event [8] => 2 [9] => IPV6-ICMP [10] => ff02:0000:0000:0000:0000:0000:0000:0001 [11] => 0 )
                      Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry…

                      F.

                      1 Reply Last reply Reply Quote 0
                      • bmeeksB
                        bmeeks
                        last edited by

                        @fsansfil:

                        Related to the block.log… I get this only with long IPv6 address : ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...

                        Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/09/2014-21:09:57.398584 [1] => Block Dst [2] => 1 [3] => 770058 [4] => 1 [5] => PB-PROTOCOL ICMP for IPv6 RFC 4443 [6] => RFC 4884 [7] => Detection of a Non-Standard Protocol or Event [8] => 2 [9] => IPV6-ICMP [10] => ff02:0000:0000:0000:0000:0000:0000:0001 [11] => 0 )
                        Oct 11 10:16:36 php-fpm[44596]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry…

                        F.

                        Can you also post text from the ALERTS and BLOCK log for this exact event?  You can correlate the timestamp to be sure you get the exact event.  I'm working now on coding a fix for this problem on the BLOCK tab.  You can find these two files in the /var/log/suricata subdirectory for the interface, or you can go to the LOGS VIEW tab and select the interface and log file to view.

                        Bill

                        1 Reply Last reply Reply Quote 0
                        • S
                          simby
                          last edited by

                          bmeeks: do we have plane to update to suricata 2.0.4 or 2.1 ? :)

                          1 Reply Last reply Reply Quote 0
                          • G
                            gogglespisano
                            last edited by

                            Bill, can you change the blocks tab to show most recent first like the alerts tab?

                            -Stuart

                            1 Reply Last reply Reply Quote 0
                            • BBcan177B
                              BBcan177 Moderator
                              last edited by

                              @gogglespisano:

                              Bill, can you change the blocks tab to show most recent first like the alerts tab?

                              -Stuart

                              The column headers are sortable. Just click on any one of the headers.

                              "Experience is something you don't get until just after you need it."

                              Website: http://pfBlockerNG.com
                              Twitter: @BBcan177  #pfBlockerNG
                              Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                              1 Reply Last reply Reply Quote 0
                              • G
                                gogglespisano
                                last edited by

                                @BBcan177:

                                @gogglespisano:

                                Bill, can you change the blocks tab to show most recent first like the alerts tab?

                                -Stuart

                                The column headers are sortable. Just click on any one of the headers.

                                There's no "Date" column on Blocks like there is on "Alerts", so nothing to click to get most recent first.

                                I have auto refresh turned on, so having the default would be nice since the column sort doesn't stick through an auto refresh.

                                What is the default sort order of the Blocks?  It seems mostly "recent last", but the first part of the list doesn't quite match that.  Is it ordered by "earliest seen last"?  (I have a 1hr expiration on blocks)

                                1 Reply Last reply Reply Quote 0
                                • F
                                  fsansfil
                                  last edited by

                                  Logs rotated from that event, but heres a new one, same error and this time not from a custom rule.

                                  Cheers.

                                  sys.log

                                  Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/11/2014-21:27:04.782000 [1] => Block Src [2] => 1 [3] => 22 [4] => 1 [5] => FILE pdf claimed [6] => but not pdf [7] => (null) [8] => 3 [9] => TCP [10] => 128.119.103.XXX [11] => 80 )
Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...

                                  alert.log

                                  10/11/2014-21:27:04.782000  [**] [1:22:1] FILE pdf claimed, but not pdf [**] [Classification: (null)] [Priority: 3] {TCP} 128.119.103.XXX:80 -> 192.168.1.XXX:47798

                                  block.log

                                  10/11/2014-21:27:04.782000,Block Src,1,22,1,FILE pdf claimed, but not pdf,(null),3,TCP,128.119.103.XXX,80

                                  rule

                                  alert http any any -> any any (msg:"FILE pdf claimed, but not pdf"; flow:established,to_client; fileext:"pdf"; filemagic:!"PDF document"; filestore; sid:22; rev:1;)
                                  1 Reply Last reply Reply Quote 0
                                  • bmeeksB
                                    bmeeks
                                    last edited by

                                    @fsansfil:

                                    Logs rotated from that event, but heres a new one, same error and this time not from a custom rule.

                                    Cheers.

                                    sys.log

                                    Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] Failed block.log entry fields are: Array ( [0] => 10/11/2014-21:27:04.782000 [1] => Block Src [2] => 1 [3] => 22 [4] => 1 [5] => FILE pdf claimed [6] => but not pdf [7] => (null) [8] => 3 [9] => TCP [10] => 128.119.103.XXX [11] => 80 )
Oct 12 14:10:11	php-fpm[5131]: /suricata/suricata_blocked.php: [suricata] ERROR: block.log entry failed to parse correctly with too many or not enough CSV entities, skipping this entry...

                                    alert.log

                                    10/11/2014-21:27:04.782000  [**] [1:22:1] FILE pdf claimed, but not pdf [**] [Classification: (null)] [Priority: 3] {TCP} 128.119.103.XXX:80 -> 192.168.1.XXX:47798

                                    block.log

                                    10/11/2014-21:27:04.782000,Block Src,1,22,1,FILE pdf claimed, but not pdf,(null),3,TCP,128.119.103.XXX,80

                                    rule

                                    alert http any any -> any any (msg:"FILE pdf claimed, but not pdf"; flow:established,to_client; fileext:"pdf"; filemagic:!"PDF document"; filestore; sid:22; rev:1;)

                                    I'm working on the fix, but this is really just a cosmetic error.  The custom module that inserts a block also writes some of the fields provided by the Suricata alert module to a log file (the block.log file).  To make it easy to parse in PHP, I configured the blocking module to separate those Suricata fields with commas.  PHP has a built-in function for reading a CSV text file and placing each field into an array.  This technique has been used for ages in the Snort package.  However, in Suricata an unforeseen problem popped up because a tiny handful of the text rules have embedded commas in the rule message.  That messes up the CSV parser because it expects the 11 fields of data to be delimited by commas.  When you have extra commas as part of the data of a field, the CSV parser winds up presenting PHP with 12, 13 or possibly more fields.  That messes up the order in which the BLOCK tab code expects the fields to be in.

                                    I have a fix working in my test lab and hope to push it out soon for the pfSense developers to review and hopefully approve.

                                    Bill

                                    1 Reply Last reply Reply Quote 0
                                    • F
                                      fsansfil
                                      last edited by

                                      Hey Bill,

                                      Thanks again. Another question; I dont understand how the SID MGMT works.

                                      Lets say I want to remove the fast_pattern:only from those SID 31944,29895,28807,28406,14081,21474,26722

                                      I then edit the modifysid-sample.conf and add:
                                      31944,29895,28807,28406,14081,21474,26722 "fast_pattern:only; " ""

                                      Then select the proper interface (LAN) in the Interface SID Management File Assignements and select modify-sample.conf under Modify SID File, click save and nothing happens ;(

                                      Please let me know what I doing wrong.

                                      Thanks!

                                      F.

                                      1 Reply Last reply Reply Quote 0
                                      • BBcan177B
                                        BBcan177 Moderator
                                        last edited by

                                        Hi fsansfil,

                                        I would suggest you make a new modify-sid (and enable/disable sid also) because if you use the "Sample" Ones, they will be overwritten when you do any re-install or Package Updates.

                                        Did you click the "Rebuild" Checkboxes beside the Interface Names?

                                        "Experience is something you don't get until just after you need it."

                                        Website: http://pfBlockerNG.com
                                        Twitter: @BBcan177  #pfBlockerNG
                                        Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                        1 Reply Last reply Reply Quote 0
                                        • F
                                          fsansfil
                                          last edited by

                                          Hey there,

                                          Thanks for the quick reply. Just tested it again, this time with a newly created modify.config, did checked rebuild in front of the interface (LAN), still nothing. Seems the check rebuild button dont save, every refresh it goes back to uncheck…

                                          Thanks.

                                          F.

                                          1 Reply Last reply Reply Quote 0
                                          • BBcan177B
                                            BBcan177 Moderator
                                            last edited by

                                            The Rebuild Checkbox does not stay checked after you hit save…

                                            Once you make you Sid Mgmt Changes, Click "Rebuild" then save.

                                            Goto the Interface (ie: LAN) click "Edit"

                                            LAN Rules: and Look for the Rule that you modified to see if the changes were made to the Rule.

                                            "Experience is something you don't get until just after you need it."

                                            Website: http://pfBlockerNG.com
                                            Twitter: @BBcan177  #pfBlockerNG
                                            Reddit: https://www.reddit.com/r/pfBlockerNG/new/

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.