Suricata-5.0.3 Package Update -- Release Notes (pfSense-2.5 DEVEL)



  • pfSense-pkg-suricata-5.0.3
    This updates the Suricata GUI package to version 5.0.3. Included are a number of bug fixes and feature enhancements. The underlying Suricata binary is also upgraded to version 5.0.3. Release Notes for the binary can be found here.

    This version also removes entirely Barnyard2 support from Suricata since that port is no longer maintained and has a number of unpatched security vulnerabilities in its runtime-dependent shared libraries.

    Installation Instructions:
    It is strongly recommended that you install this new version by first removing Suricata from your firewall and then installing the update. This will help ensure the deprecated Barnyard2 package and its vulnerable shared libraries are removed. Your settings will be retained so long as you have the option to keep the existing configuration enabled on the GLOBAL SETTINGS tab of Suricata. That setting is enabled by default. You can verify it is "On" by checking the GLOBAL SETTINGS tab and then scrolling down to the General Settings section near the bottom of the page. See screenshot below.

    Suricata_KeepSettings.png

    Major Functionality Changes:

    1. Remove all Barnyard2 support from the Suricata package. Barnyard2 is no longer actively maintained in FreeBSD ports and uses end-of-life MySQL database libraries with unpatched security vulnerabilities. Suricata upstream is also deprecating the Unified2 binary logging output option required by Barnyard2.

    New Features:

    1. Added support for the Telegraf input.suricata plugin via a UNIX socket. See Redmine Feature Request #10421 for further details.

    2. Added additional EVE output types for RDP, SIP and SNMP to EVE output logging configuration.

    3. Added TLS Extended Fields logging options to EVE output logging configuration.

    4. Added new EVE output for logging Anomalies to EVE output logging configuration.

    5. Added new App-Layer parser options for RDP, SIP, and SNMP to the APP PARSERS tab.

    Bug Fixes:

    1. Modify LOG MGMT cron task code to respect and use any user-specified custom File Store directory.

    2. Add warning in Help Text for syslog export section on INTERFACE SETTINGS tab stating that the current FreeBSD syslog daemon will truncate exported messages to 480 bytes.

    3. In suricata_generate_yaml.php, test for both existence of file and a size greater than zero bytes before adding rules file to suricata.yaml conf file. This will prevent PHP warnings from non-existent files if user has a configured interface but zero selected rules.

    4. Fix lingering issues with Suricata log rotation and remove suricata_post_delete_logs() function. Send SIGHUP to Suricata when logs are rotated so it will reopen the files.

    5. Change exit() to return() in GeoIP database download code so we don't possibly break out of pkg install script on a GeoIP download error and cause a failure to complete the installation of the GUI package.

    6. Remove deprecated "file-log" configuration parameters from GUI and suricata.yaml.

    7. Fix remaining occurences of the constant LOG_WARN in syslog() function calls instead of the proper LOG_WARNING constant. Redmine Issue #10751.

    How To Use the New Features
    All of the new features described below are available on the INTERFACE SETTINGS tab for each configured Suricata interface.

    Suricata performance stats integration with Telegraf
    This update adds the ability for Suricata to send EVE-based performance statistics to a Telegraf instance installed on pfSense. To use this feature, you must configure the input.suricata plugin in Telegraf. Note the location and name of the UNIX socket created by Telgraf. Next, be sure the generation of performance stats is enabled in Suricata for the interface you want to monitor. When stats collection is enabled for an interface, you will see additional options appear for enabling the Telgraf integration. See the screenshot below. Note that you must provide the Telegraf-generated UNIX socket name and path to Suricata.

    User @kiokoman also discovered that you need to bump up the defaults for these system tunables to the values shown:

    sysctl -w net.local.stream.recvspace=16384
    sysctl -w net.local.stream.sendspace=16384
    

    If those system tunable values are not large enough, it seems that the stream data is truncated and telegraf fails to see the end of line character in the stream and thus discards the stats info.

    Suricata_Telegraf_Stats_Socket.png

    Screenshots of new EVE Logging options and APP Parser settings
    This update adds several new EVE logging options for RDP, SIP and SNMP along with TLS Extended Data fields. See the screen shot below. You must enable the EVE output logger in order to use these features.

    Suricata_EVE_RDP_SIP_SNMP_TLS_Ext_Fields_Logging.png

    EVE Anomaly logging
    The ability to log various packet anomalies via the EVE logger has been added. See the screenshot below.

    Suricata_EVE_Anomaly_Logging.png


  • LAYER 8

    @bmeeks you are the best !



  • @bmeeks Thank you for the release



  • This update will show up for pfSense-2.4.5 RELEASE soon. We are giving it a week or two for shakedowns in the 2.5-DEVEL branch before merging it into 2.4.5-RELEASE.



  • Went smoothly ... hooray, no more Bardyard! Thank you @bmeeks !


  • LAYER 8

    sadly the telegraf stuff does not work anymore, i need to investigate.
    the data are sent from suricata to the socket but telegraf does not grab it



  • @kiokoman said in Suricata-5.0.3 Package Update -- Release Notes (pfSense-2.5 DEVEL):

    sadly the telegraf stuff does not work anymore, i need to investigate.
    the data are sent from suricata to the socket but telegraf does not grab it

    I didn't have a telgraf installation to test with. All I did in the PHP code was add code to create the entry in suricata.yaml that generates a separate EVE log with the "unix_stream" output type.

    Let me know what you discover. Perhaps some changes are needed on the Suricata side ???


  • LAYER 8

    [2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/var/run: nc -lkU suricataWAN-stats.sock
    {"timestamp":"2020-07-15T15:59:22.616765+0200","event_type":"stats","stats":{"uptime":43,"capture":{"kernel_packets":549,"kernel_drops":0,"kernel_ifdrops":0},"decoder":{"pkts":549,"bytes":75660,"invalid":0,"ipv4":549,"ipv6":25,"ethernet":0,"raw":0,"null":549,"sll":0,"tcp":16,"udp":480 alot of stuff here ...
    

    but

    [2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/var/run: /usr/local/bin/telegraf --debug -config=/root/telegraf.conf
    2020-07-15T14:03:13Z I! Starting Telegraf 1.13.4
    2020-07-15T14:03:13Z I! Loaded inputs: suricata
    2020-07-15T14:03:13Z I! Loaded aggregators:
    2020-07-15T14:03:13Z I! Loaded processors:
    2020-07-15T14:03:13Z I! Loaded outputs: influxdb
    2020-07-15T14:03:13Z I! Tags enabled: host=pfSense.kiokoman.home
    2020-07-15T14:03:13Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"pfSense.kiokoman.home", Flush Interval:10s
    2020-07-15T14:03:13Z D! [agent] Initializing plugins
    2020-07-15T14:03:13Z D! [agent] Connecting outputs
    2020-07-15T14:03:13Z D! [agent] Attempting connection to [outputs.influxdb]
    2020-07-15T14:03:13Z D! [agent] Successfully connected to outputs.influxdb
    2020-07-15T14:03:13Z D! [agent] Starting service inputs
    2020-07-15T14:03:30Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
    

    i think there is something on telegraf



  • @kiokoman said in Suricata-5.0.3 Package Update -- Release Notes (pfSense-2.5 DEVEL):

    [2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/var/run: nc -lkU suricataWAN-stats.sock
    {"timestamp":"2020-07-15T15:59:22.616765+0200","event_type":"stats","stats":{"uptime":43,"capture":{"kernel_packets":549,"kernel_drops":0,"kernel_ifdrops":0},"decoder":{"pkts":549,"bytes":75660,"invalid":0,"ipv4":549,"ipv6":25,"ethernet":0,"raw":0,"null":549,"sll":0,"tcp":16,"udp":480 alot of stuff here ...
    

    but

    [2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/var/run: /usr/local/bin/telegraf --debug -config=/root/telegraf.conf
    2020-07-15T14:03:13Z I! Starting Telegraf 1.13.4
    2020-07-15T14:03:13Z I! Loaded inputs: suricata
    2020-07-15T14:03:13Z I! Loaded aggregators:
    2020-07-15T14:03:13Z I! Loaded processors:
    2020-07-15T14:03:13Z I! Loaded outputs: influxdb
    2020-07-15T14:03:13Z I! Tags enabled: host=pfSense.kiokoman.home
    2020-07-15T14:03:13Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"pfSense.kiokoman.home", Flush Interval:10s
    2020-07-15T14:03:13Z D! [agent] Initializing plugins
    2020-07-15T14:03:13Z D! [agent] Connecting outputs
    2020-07-15T14:03:13Z D! [agent] Attempting connection to [outputs.influxdb]
    2020-07-15T14:03:13Z D! [agent] Successfully connected to outputs.influxdb
    2020-07-15T14:03:13Z D! [agent] Starting service inputs
    2020-07-15T14:03:30Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
    

    i think there is something on telegraf

    Suricata is certainly spewing stats out through the socket, so that part is working. I agree it is likely on the telegraf side. I'm not familiar with telegraf having never used it. Are there some debugging options to have telegraf log someplace the raw data it is getting from the socket?

    Here is the original Feature Request: https://redmine.pfsense.org/issues/10421#change-47203. The creator of the request claimed to have it working (although he had to manually edit the suricata.yaml file). I duplicated in the PHP code what he did on the Suricata side.

    EDIT: never mind, just realized you were the creator of the Feature Request ... ☺


  • LAYER 8

    yes, i was the one requesting it, it was working at that time, maybe it was before the upgrade to freebsd 12.1-stable
    now telegraf does not work anymore

    from debug i have this

    _umtx_op(0x45d95f8,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    _umtx_op(0x45d95f8,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0xc000079e48) = 0 (0x0)
    write(2,"2020-07-15T16:40:06Z D! [outputs"...,78) = 78 (0x4e)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    thr_kill(100746,SIGURG)                          = 0 (0x0)
    SIGNAL 16 (SIGURG) code=SI_LWP pid=69801 uid=0
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    sigreturn(0xc000087b00)                          EJUSTRETURN
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    compat11.kevent(3,0x0,0,{ },64,{ 3.378182125 })  = 0 (0x0)
    compat11.kevent(3,0x0,0,{ },64,{ 0.000000000 })  = 0 (0x0)
    _umtx_op(0xc00006a4d0,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    _umtx_op(0xc00006a4d0,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0x0) = 0 (0x0)
    compat11.kevent(3,0x0,0,{ },64,{ 0.000000000 })  = 0 (0x0)
    nanosleep({ 0.000003000 })                       = 0 (0x0)
    _umtx_op(0xc000247650,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    _umtx_op(0xc000247650,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0x0) = 0 (0x0)
    _umtx_op(0x45d95f8,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0xc000079e48) ERR#60 'Operation timed out'
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    _umtx_op(0x45d95f8,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0xc000079e48) ERR#60 'Operation timed out'
    compat11.kevent(3,0x0,0,{ },64,{ 1.567486309 })  = 0 (0x0)
    _umtx_op(0xc000246f50,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    compat11.kevent(3,0x0,0,{ },64,{ 0.000000000 })  = 0 (0x0)
    _umtx_op(0xc000246f50,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0x0) = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    _umtx_op(0xc000247650,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    _umtx_op(0xc000247650,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0x0) = 0 (0x0)
    sched_yield()                                    = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    compat11.kevent(3,0x0,0,{ },64,{ 0.000000000 })  = 0 (0x0)
    _umtx_op(0xc00005a768,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    sched_yield()                                    = 0 (0x0)
    _umtx_op(0x45d94f8,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    compat11.kevent(3,0x0,0,{ 9,EVFILT_READ,EV_CLEAR,0,0x2000,0x82b10ce90 },64,{ 4.958977535 }) = 1 (0x1)
    _umtx_op(0x45d95f8,UMTX_OP_WAKE_PRIVATE,0x1,0x0,0x0) = 0 (0x0)
    _umtx_op(0x45d95f8,UMTX_OP_WAIT_UINT_PRIVATE,0x0,0x18,0xc000079e48) = 0 (0x0)
    read(9,"{"timestamp":"2020-07-15T18:40:1"...,10420224) = 8192 (0x2000)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    read(9,0xc00095a000,10412032)                    ERR#35 'Resource temporarily unavailable'
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    nanosleep({ 0.000020000 })                       = 0 (0x0)
    compat11.kevent(3,0x0,0,{ },64,{ 0.000000000 })  = 0 (0x0)
    

    as you can see there is a

    read(9,"{"timestamp":"2020-07-15T18:40:1"...,10420224)
    

    that's part of the output from suricata

    followed by

    read(9,0xc00095a000,10412032)                    ERR#35 'Resource temporarily unavailable'
    

    well, i have no idea, i also tried with Telegraf 1.15.0 but same error/problem



  • FYI. As of today (Thursday, July 16th), barring no major issues showing up the DEVEL branch, the plan is to migrate this 5.0.3 Suricata package update over to pfSense-2.4.5_p1 RELEASE Monday or Tuesday of next week (July 20th or 21st).


  • LAYER 8

    @bmeeks maybe i found something,
    the suricata plugin of telegraf is expecting \n at the end of the line but i think it's never coming,
    it seems to be truncated at the end
    any settings from suricata that can solve this?

    example coming from unix socket

    {"timestamp":"2020-07-17T10:08:18.856758+0200" ......."invalid_checksum":0,"no_flow":0,"syn":1044,"synack":454,"r <- alway end like this
    

    example coming from eve.json

    {"timestamp":"2020-07-17T10:35:00.974605+0200".......<{"expectations":0},"flow":{"memuse":8443384}}}}}  <--- good output
    

  • LAYER 8

    problem solved, i need to increse this

    sysctl -w net.local.stream.recvspace=16384
    sysctl -w net.local.stream.sendspace=16384

    now i see data on my grafana



  • @kiokoman said in Suricata-5.0.3 Package Update -- Release Notes (pfSense-2.5 DEVEL):

    problem solved, i need to increse this

    sysctl -w net.local.stream.recvspace=16384
    sysctl -w net.local.stream.sendspace=16384

    now i see data on my grafana

    Ah-ha! It was truncating the stream. FreeBSD has so darn many system tunables ... 😕



  • @kiokoman said in Suricata-5.0.3 Package Update -- Release Notes (pfSense-2.5 DEVEL):

    problem solved, i need to increse this

    sysctl -w net.local.stream.recvspace=16384
    sysctl -w net.local.stream.sendspace=16384

    now i see data on my grafana

    Added a section to my initial RELEASE NOTES post detailing the need to check and probably adjust these system tuneables. Thanks for the investigation and feedback!


  • LAYER 8

    it will also be added to the README of the plugin https://github.com/influxdata/telegraf/issues/7843




Log in to reply