Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Suricata v4.1.4_7 Package Update Release Notes (pfSense-2.4.4_3)

    Scheduled Pinned Locked Moved
    IDS/IPS
    3
    10
    789
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bmeeksB
      bmeeks
      last edited by bmeeks

      pfSense-pkg-suricata-4.1.4_7

      !! IMPORTANT NOTICE !!
      Do NOT install this update. It contains a nasty bug that will overwrite your most recently created Suricata interface with the settings from your first configured interface. The overwritten interface will be lost. A fix has been posted for the pfSense team to review and merge, but that probably won't happen until Monday, August 26, at the earliest! Look for a new package with version 4.1.4_8 and install that one, but avoid installing 4.1.4_7.

      If you have already installed package version 4.1.4_7 and notice a duplicate interface and your last interface is missing, then you can restore from a previous config.xml backup to bring back the original configuration. You can also simply delete the duplicate interface and create the missing one from scratch.

      This update fixes an issue with displaying the last rules update job status and corrects the spelling of a syslog() PRIORITY constant in the GeoIP2 database update cron task script.

      Formerly the rules update status info was stored in the config.xml file, but that resulted in unnecessary backups of config.xml with each rules update job run. A previous package update removed the call to write_config() that was generating the unnecessary backup, but that prevented the recording of rules update time and status. The rules update execution time and status are now recorded locally in a small file on the firewall.

      New Features:
      None

      Bug Fixes:

      1. A PHP warning message is generated in the crash log due to use of an unknown constant in a call to the syslog() function in the GeoIP2 database update cron task.

      2. Rules update task info (execution time and status) is displaying as either "unknown" or the last package installation date and time.

      1 Reply Last reply Reply Quote 1
      • N
        NRgia
        last edited by NRgia

        Hello @bmeeks,

        Thank you for the release.

        I have a little problem after the update. I tried to uninstall/reinstall also.

        Before the update I had Suricata running on WAN and LAN.

        In Suricata interfaces the WAN interface is doubled now, although you can change it from the drop down box. But if I just select the interface that way, I will loose the selection of the rules for LAN, I had different rules for WAN and LAN.

        suricata interfaces.png

        Also the installation logs, shows that the rules are updated only for WAN interface as we can see:

        Installing Emerging Threats Open rules... done.
        Installing Snort rules... done.
        Updating rules configuration for: WAN ... done.
        Updating rules configuration for: WAN ... done.
        Cleaning up after rules extraction... done.
        The Rules update has finished.
        Generating suricata.yaml configuration file from saved settings.
        Generating YAML configuration file for WAN... done.
        Generating YAML configuration file for WAN... done.
        Finished rebuilding Suricata configuration from saved settings.
        Setting package version in configuration file.
        done.
        Executing custom_php_resync_config_command()...done.
        Menu items... done.
        Services... done.
        Writing configuration... done.

        Please let me know if I can revert to a previous package, or do a workaround, until next release. Thanks

        bmeeksB 1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks @NRgia
          last edited by bmeeks

          @NRgia said in Suricata v4.1.4_7 Package Update Release Notes (pfSense-2.4.4_3):

          Hello @bmeeks,

          Thank you for the release.

          I have a little problem after the update. I tried to uninstall/reinstall also.

          Before the update I had Suricata running on WAN and LAN.

          In Suricata interfaces the WAN interface is doubled now, although you can change it from the drop down box. But if I just select the interface that way, I will loose the selection of the rules for LAN, I had different rules for WAN and LAN.

          suricata interfaces.png

          Also the installation logs, shows that the rules are updated only for WAN interface as we can see:

          Installing Emerging Threats Open rules... done.
          Installing Snort rules... done.
          Updating rules configuration for: WAN ... done.
          Updating rules configuration for: WAN ... done.
          Cleaning up after rules extraction... done.
          The Rules update has finished.
          Generating suricata.yaml configuration file from saved settings.
          Generating YAML configuration file for WAN... done.
          Generating YAML configuration file for WAN... done.
          Finished rebuilding Suricata configuration from saved settings.
          Setting package version in configuration file.
          done.
          Executing custom_php_resync_config_command()...done.
          Menu items... done.
          Services... done.
          Writing configuration... done.

          Please let me know if I can revert to a previous package, or do a workaround, until next release. Thanks

          I know what the problem is. Unfortunately you are going to have to recreate your LAN from scratch or else restore a config.xml backup from before the Suricata update. I fixed this in Snort but did not see it in the Suricata code. There is now a difference apparently in the way PHP treats by-reference array iterators. The bug in the GUI code is part of the post-install script that runs after the package is installed. It essentially overwrites the last entry in the array of configured Suricata interfaces with the first one. For a user with the normal two interfaces (WAN and LAN), this means LAN gets overwritten with WAN's setup.

          I will get a fix posted, but since it is the weekend I doubt the pfSense team can merge it until Monday. So in the meantime, you have three optiions:

          1. Delete that extra WAN interface and recreate LAN from scratch.

          2. Restore a config.xml backup from before you updated Suricata. That should restore the missing interface but leave the Suricata package at the current new version.

          3. Copy and paste the LAN section from an older config.xml into the current file (risky, but doable if you are comfortable reading XML).

          1 Reply Last reply Reply Quote 0
          • bmeeksB
            bmeeks
            last edited by

            The bug present in this version of the Suricata package has been corrected in the latest 4.1.4_8 version of Suricata for pfSense-2.4.4_3. Please install only the 4.1.4_8 or later version of the Suricata package.

            1 Reply Last reply Reply Quote 0
            • B
              btspce
              last edited by btspce

              FYI: We have been running Suricata 4.1.4_5 and when I logged in to the fw this morning both interfaces had changed to WAN suddenly. I do not recall them being both WAN after update to 4.1.4_5.

              Maybe there is another bug similar to this or even older versions of suricata is affected ?

              bmeeksB 1 Reply Last reply Reply Quote 0
              • bmeeksB
                bmeeks @btspce
                last edited by

                @btspce said in Suricata v4.1.4_7 Package Update Release Notes (pfSense-2.4.4_3):

                FYI: We have been running Suricata 4.1.4_5 and when I logged in to the fw this morning both interfaces had changed to WAN suddenly. I do not recall them being both WAN after update to 4.1.4_5.

                Maybe there is another bug similar to this or even older versions of suricata is affected ?

                The potential for the bug exists in the GUI code going back for many versions, but the buggy code exists only in a single PHP file that is only executed during a package upgrade or install. The specific PHP module is called to migrate existing configuration settings to the new version being installed. That code would not be called just "out of the blue" during normal operation.

                I believe the bug was triggered by some recent changes in the way the PHP engine itself handles something called iterators in foreach() loops. The same Suricata PHP code has existed for several years, but something recently made it start "expressing itself". I think that something was a change in how the PHP engine handles array iterators at the end of a loop..

                B 1 Reply Last reply Reply Quote 0
                • B
                  btspce @bmeeks
                  last edited by

                  @bmeeks
                  Hello again. Sorry for waking this thread up but we just noticed that disabling any rules on the WAN interface does nothing. It is still blocked the traffic so I suspect something still is amiss due to the double interface problem before.

                  If I rembember correctly I deleted the extra wan interface and recreated OPT2 from scratch.

                  OPT2 says it logs to:
                  Log File Path: /var/log/suricata/suricata_igb345881/block.log

                  WAN says it logs to:
                  Log File Path: /var/log/suricata/suricata_igb145881/block.log

                  usr/local/etc/suricata: ls -lh
total 200
-rw-r--r--  1 root  wheel   3.1K Nov 17 00:31 classification.config
-rw-r--r--  1 root  wheel   4.1K Oct  1 18:31 classification.config.sample
-rw-r--r--  1 root  wheel    33B Nov 17 00:31 etpro.rules.tar.gz.md5
-rw-r--r--  1 root  wheel   1.2K Nov 17 00:31 reference.config
-rw-r--r--  1 root  wheel   1.3K Oct  1 18:31 reference.config.sample
-rw-r--r--  1 root  wheel    18B Nov 18 00:30 rulesupd_status
-rw-r--r--  1 root  wheel    73K Oct  1 18:31 suricata.yaml
-rw-r--r--  1 root  wheel    73K Oct  1 18:31 suricata.yaml.sample
drwxr-xr-x  3 root  wheel   512B Aug  4 21:07 suricata_39148_igb3
drwxr-xr-x  3 root  wheel   512B Oct  5 17:36 suricata_45881_igb1
drwxr-xr-x  3 root  wheel   512B Oct  5 17:36 suricata_45881_igb3
-rw-r--r--  1 root  wheel   1.6K Oct  1 18:31 threshold.config
-rw-r--r--  1 root  wheel   1.6K Oct  1 18:31 threshold.config.sample

                  So I seems to have three interface folders even though I only use two (WAN and OPT2)
                  (Can I delete this below? (suricata_39148_igb3))

                  /usr/local/etc/suricata/suricata_39148_igb3: ls -lh
total 12
-rw-r--r--  1 root  wheel   3.1K Aug  4 21:07 classification.config
-rw-r--r--  1 root  wheel   1.2K Aug  4 21:07 reference.config
drwxr-xr-x  2 root  wheel   512B Aug  4 21:07 rules

                  Overview of WAN folders

                  /usr/local/etc/suricata/suricata_45881_igb1: ls -lh
total 8068
-rw-r--r--  1 root  wheel   3.1K Nov 18 09:36 classification.config
-rw-r--r--  1 root  wheel   975B Nov 18 09:36 passlist
-rw-r--r--  1 root  wheel   1.2K Nov 18 09:36 reference.config
drwxr-xr-x  2 root  wheel   512B Oct  5 17:36 rules
-rw-r--r--  1 root  wheel   7.8M Nov 18 09:36 sid-msg.map
-rw-r--r--  1 root  wheel    12K Nov 18 09:36 suricata.yaml
-rw-r--r--  1 root  wheel   1.8K Nov 18 09:36 threshold.config

/usr/local/etc/suricata/suricata_45881_igb1/rules: ls -lh
total 31936
-rw-r--r--  1 root  wheel     0B Nov 18 09:36 custom.rules
-rw-r--r--  1 root  wheel     0B Nov 18 09:36 flowbit-required.rules
-rw-r--r--  1 root  wheel    31M Nov 18 09:36 suricata.rules

                  Overview of OPT2 folders

                  /usr/local/etc/suricata/suricata_45881_igb3: ls -lh
total 8068
-rw-r--r--  1 root  wheel   3.1K Nov 18 09:37 classification.config
-rw-r--r--  1 root  wheel   961B Nov 18 09:37 passlist
-rw-r--r--  1 root  wheel   1.2K Nov 18 09:37 reference.config
drwxr-xr-x  2 root  wheel   512B Oct  5 17:36 rules
-rw-r--r--  1 root  wheel   7.8M Nov 18 09:38 sid-msg.map
-rw-r--r--  1 root  wheel    13K Nov 18 09:38 suricata.yaml
-rw-r--r--  1 root  wheel   291B Nov 18 09:37 threshold.config

/usr/local/etc/suricata/suricata_45881_igb3/rules: ls -lh
total 31936
-rw-r--r--  1 root  wheel     0B Nov 18 09:37 custom.rules
-rw-r--r--  1 root  wheel     0B Nov 18 09:37 flowbit-required.rules
-rw-r--r--  1 root  wheel    31M Nov 18 09:37 suricata.rules

                  Can you help me sort this issue out ?

                  bmeeksB 1 Reply Last reply Reply Quote 0
                  • bmeeksB
                    bmeeks @btspce
                    last edited by bmeeks

                    @btspce :

                    If you disabled rules, but still are getting blocks, there are two possible causes:

                    1. You did not also manually clear the previous blocks on the BLOCKED tab. I mention that because many times folks forget about that step. Disabling a rule does not clear any previous blocks from that rule when running in Legacy Blocking Mode.

                    2. You have duplicate instances of Suricata running on the same physical interface. In that scenario, only one of the two instances (the most recently started one) will respond to GUI edits. The other running instance will ignore the GUI edits and keep running with the configuration it had when it started.

                    So assuming you have manually cleared any blocks and are still getting a block on the host, then you need to see if multiple Suricata instances are running by using this command from a shell prompt. First stop all Suricata instances on the INTERFACES tab using the GUI icons. Next, get to a shell prompt and issue this command:

                    ps -ax | grep suricata

                    You should see no running Suricata instances. If you see one in the command's output, then you have a duplicate zombie process. You will need to explicitly kill that instance using this command:

                    kill -9 <pid>

                    where <pid> is the process ID of the duplicate zombie process.

                    It also sounds like you have changed interface cards or deleted and then recreated interfaces if you have multiple UUIDs. To clean that up, do this:

                    1. Be sure "Save Settings on Deinstall" is checked on the GLOBAL SETTINGS tab and then go to SYSTEM > PACKAGE MANAGER and on the Installed Packages tab delete the Suricata package.

                    2. Now get to a shell prompt and run these commands to completely remove all Suricata-related directories.

                    rm -rf /usr/local/etc/suricata
rm -rf /var/log/suricata
rm -f /var/run/suricata*
                    1. Return to the SYSTEM > PACKAGE MANAGER menu and on the Available Packages tab find Suricata and install it again. It will detect your previously saved settings and restore everything including creating the required directories with the correct names per the currently configured UUIDs.
                    B 1 Reply Last reply Reply Quote 0
                    • B
                      btspce @bmeeks
                      last edited by

                      @bmeeks
                      It must have been duplicate processes of suricata running. Didn't think of checking that. Did an uninstall and reinstall from 4.1.5_1 to the latest 4.1.5_2 last night and the rules I previously disabled is not blocked any longer.
                      Will do a uninstall and remove the folders per your instructions later this week to get rid of the extra folders.

                      Thanks for your hard work and fast support Bill !

                      1 Reply Last reply Reply Quote 0
                      • bmeeksB
                        bmeeks
                        last edited by

                        I did forget to mention that when you clear out the /var/log/suricata directory that will wipe out all of the Suricata log files, so if you want those for some reason copy them off before executing the command.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post