Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Snort services stopped after update

    IDS/IPS
    3
    7
    1126
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Sander88 last edited by

      This morning I found one of my PFSense boxes with stopped Snort services. It's running Snort on 3 different interfaces and stopped working when the update script was applying the updated files.

      My syslog (Snort related messages):
      Oct 25 00:03:00 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2983.tar.gz…
      Oct 25 00:04:27 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort VRT rules file update downloaded successfully
      Oct 25 00:04:27 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz…
      Oct 25 00:04:29 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
      Oct 25 00:04:34 pfsense kernel: pid 13602 (snort), uid 0: exited on signal 11
      Oct 25 00:04:34 pfsense kernel: pid 12826 (snort), uid 0: exited on signal 10

      Oct 25 00:04:37 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: INTERF1 …
      Oct 25 00:04:38 pfsense kernel: pid 24601 (snort), uid 0: exited on signal 4
      Oct 25 00:04:38 pfsense kernel: pppoe0: promiscuous mode disabled
      Oct 25 00:04:44 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for INTERF1…
      Oct 25 00:04:45 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: INTERF2 …
      Oct 25 00:04:52 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for INTERF2…
      Oct 25 00:04:53 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: INTERF3 …
      Oct 25 00:05:00 pfsense php: /usr/local/pkg/snort/snort_check_cron_misc.inc: [Snort] Alert pcap file cleanup job removed 1 pcap file(s) from /var/log/snort/snort_em465338/…
      Oct 25 00:05:00 pfsense php: /usr/local/pkg/snort/snort_check_cron_misc.inc: [Snort] Alert pcap file cleanup job removed 1 pcap file(s) from /var/log/snort/snort_pppoe019272/…
      Oct 25 00:05:00 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for INTERF3…
      Oct 25 00:05:01 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
      Oct 25 00:05:03 pfsense check_reload_status: Syncing firewall
      Oct 25 00:05:04 pfsense php-fpm[45441]: /rc.filter_synchronize: Beginning XMLRPC sync to https://1.2.3.4:443.
      Oct 25 00:05:13 pfsense php-fpm[45441]: /rc.filter_synchronize: XMLRPC sync successfully completed with https://1.2.3.4:443.
      Oct 25 00:05:20 pfsense php-fpm[45441]: /rc.filter_synchronize: Filter sync successfully completed with https://1.2.3.4:443.

      Based on this logs and script (/usr/local/pkg/snort/snort_check_for_rule_updates.php) I think this is a bug:
      1. Snort crashes when the rule definition files are replaced by the updater. Snort is probably using/reading them at the same time.
      2. The update scripts only checks if Snort is running after these update have been applied. So it the script will see no running Snort processing and skips the (re)start.

      An easy fix would probably to check the Snort service status a few seconds earlier, before appling the update files and use that result to determine if services need to be (re)started afterwards.

      My box is running 2.3.3-RELEASE (amd64) and Snort 3.2.9.2_16.

      Do I see this correctly? Or any other idea's or possible Snort crash causes?

      1 Reply Last reply Reply Quote 0
      • bmeeks
        bmeeks last edited by

        You may have a hardware problem.  According to the FreeBSD docs, Signal 10 is a memory bus error.  Found this reference on Google –

        
        bus error
        
         <processor>A fatal failure in the execution of a machine language
        instruction resulting from the processor detecting an anomalous condition
        on its bus. Such conditions include invalid address alignment (accessing a
        multi-byte number at an odd address), accessing a physical address that
        does not correspond to any device, or some other device-specific hardware
        error. A bus error triggers a processor-level exception which Unix
        translates into a "SIGBUS" signal which, if not caught, will terminate the
        current process.
        
        This can quite plausibly be caused by hardware error, or memory problems
        in particular.</processor> 
        

        Signal 4 is an "illegal instruction" error.

        I have Snort running on three interfaces on my personal firewall.  I have had no issues with the daily updates.  I use Snort VRT rules which typically update twice weekly, and Emerging Threats Open rules which typically update daily.  My box checks for updates twice daily at 0130 and 1330.

        Bill

        1 Reply Last reply Reply Quote 0
        • Raffi_
          Raffi_ last edited by

          I have the same problem. It doesn't always crash the services after an update, I think it depends on the specific types of rule being updated. Signal 10 may be related to a hardware issue, but why would that issue only come up after snort updates it's rules? Everything else has been rock solid for months.

          I'm running pfsense 2.4.1
          Snort package 3.2.9.5_2

          Oct 27 00:05:04 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2990.tar.gz…
          Oct 27 00:05:16 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort VRT rules file update downloaded successfully
          Oct 27 00:06:09 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort OpenAppID detectors md5 download failed…
          Oct 27 00:06:09 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Server returned error code 503…
          Oct 27 00:06:15 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort OpenAppID RULES detectors posted. Downloading appid_rules.tar.gz…
          Oct 27 00:06:16 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort OpenAppID RULES detectors file update downloaded successfully
          Oct 27 00:06:38 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort GPLv2 Community Rules are up to date…
          Oct 27 00:06:39 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz…
          Oct 27 00:06:40 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
          Oct 27 00:06:48 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: WAN …
          Oct 27 00:06:55 kernel pid 81481 (snort), uid 0: exited on signal 10
          Oct 27 00:06:55 kernel re0: promiscuous mode disabled
          Oct 27 00:06:56 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Enabling any flowbit-required rules for: WAN…
          Oct 27 00:06:57 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for WAN…
          Oct 27 00:07:00 php /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
          Oct 27 00:07:00 check_reload_status Syncing firewall

          1 Reply Last reply Reply Quote 0
          • bmeeks
            bmeeks last edited by

            @Sander88:

            This morning I found one of my PFSense boxes with stopped Snort services. It's running Snort on 3 different interfaces and stopped working when the update script was applying the updated files.

            My syslog (Snort related messages):
            Oct 25 00:03:00 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2983.tar.gz…
            Oct 25 00:04:27 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort VRT rules file update downloaded successfully
            Oct 25 00:04:27 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Emerging Threats Open rules posted. Downloading emerging.rules.tar.gz…
            Oct 25 00:04:29 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules file update downloaded successfully
            Oct 25 00:04:34 pfsense kernel: pid 13602 (snort), uid 0: exited on signal 11
            Oct 25 00:04:34 pfsense kernel: pid 12826 (snort), uid 0: exited on signal 10

            Oct 25 00:04:37 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: INTERF1 …
            Oct 25 00:04:38 pfsense kernel: pid 24601 (snort), uid 0: exited on signal 4
            Oct 25 00:04:38 pfsense kernel: pppoe0: promiscuous mode disabled
            Oct 25 00:04:44 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for INTERF1…
            Oct 25 00:04:45 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: INTERF2 …
            Oct 25 00:04:52 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for INTERF2…
            Oct 25 00:04:53 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Updating rules configuration for: INTERF3 …
            Oct 25 00:05:00 pfsense php: /usr/local/pkg/snort/snort_check_cron_misc.inc: [Snort] Alert pcap file cleanup job removed 1 pcap file(s) from /var/log/snort/snort_em465338/…
            Oct 25 00:05:00 pfsense php: /usr/local/pkg/snort/snort_check_cron_misc.inc: [Snort] Alert pcap file cleanup job removed 1 pcap file(s) from /var/log/snort/snort_pppoe019272/…
            Oct 25 00:05:00 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Building new sid-msg.map file for INTERF3…
            Oct 25 00:05:01 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished.
            Oct 25 00:05:03 pfsense check_reload_status: Syncing firewall
            Oct 25 00:05:04 pfsense php-fpm[45441]: /rc.filter_synchronize: Beginning XMLRPC sync to https://1.2.3.4:443.
            Oct 25 00:05:13 pfsense php-fpm[45441]: /rc.filter_synchronize: XMLRPC sync successfully completed with https://1.2.3.4:443.
            Oct 25 00:05:20 pfsense php-fpm[45441]: /rc.filter_synchronize: Filter sync successfully completed with https://1.2.3.4:443.

            Based on this logs and script (/usr/local/pkg/snort/snort_check_for_rule_updates.php) I think this is a bug:
            1. Snort crashes when the rule definition files are replaced by the updater. Snort is probably using/reading them at the same time.
            2. The update scripts only checks if Snort is running after these update have been applied. So it the script will see no running Snort processing and skips the (re)start.

            An easy fix would probably to check the Snort service status a few seconds earlier, before appling the update files and use that result to determine if services need to be (re)started afterwards.

            My box is running 2.3.3-RELEASE (amd64) and Snort 3.2.9.2_16.

            Do I see this correctly? Or any other idea's or possible Snort crash causes?

            Just noticed this line in your log output –

            
            Oct 25 00:03:00 pfsense php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] There is a new set of Snort VRT rules posted. Downloading snortrules-snapshot-2983.tar.gz...
            
            

            That is the wrong version of Snort rules.  The current version should be snortrules-snapshot-2990.tar.gz.  The only way I can imagine your box trying to download that old version is if your Snort binary is the wrong version.  The precompiled shared-object rules are FreeBSD version specific.  Could be that something is getting tripped up there as the older Snort version might be bringing down FreeBSD 10.x precompiled rules.

            Why don't you try the following steps?

            1.  Remove the Snort package under System…Package Manager.

            2.  Open a CLI session to the firewall and then remove any directories with snort in their name under /usr/local/etc and /usr/local/lib.

            3.  Reinstall Snort from the System…Package Manager menu.

            If you have "Save Settings" selected on the GLOBAL SETTINGS tab, you won't lose any Snort settings.  That option is enabled by default.

            Bill

            1 Reply Last reply Reply Quote 0
            • bmeeks
              bmeeks last edited by

              For both posters in this thread –

              I looked further into this issue and think I may have found a potential breaking point in the Snort GUI code.  It would show up for users of pfSense 2.4.x which is based on FreeBSD 11 instead of FreeBSD 10 as the older pfSense versions are.  I have submitted a fix that can be viewed here:  https://github.com/pfsense/FreeBSD-ports/pull/471.  It should be approved and merged in the near future.

              Bill

              1 Reply Last reply Reply Quote 0
              • Raffi_
                Raffi_ last edited by

                @bmeeks:

                For both posters in this thread –

                I looked further into this issue and think I may have found a potential breaking point in the Snort GUI code.  It would show up for users of pfSense 2.4.x which is based on FreeBSD 11 instead of FreeBSD 10 as the older pfSense versions are.  I have submitted a fix that can be viewed here:  https://github.com/pfsense/FreeBSD-ports/pull/471.  It should be approved and merged in the near future.

                Bill

                Thanks Bill!

                I downloaded the latest snort update which includes this change. I'll let you know how it goes. Although, right now I'm dealing with a different issue so I have snort disabled for the moment.

                Raffi

                1 Reply Last reply Reply Quote 0
                • Raffi_
                  Raffi_ last edited by

                  Bill,

                  Snort has been working fine for the last few weeks. I haven't received any notifications from the watchdog service for snort. I'll post back here if anything comes up again, but it seems solid now.
                  Thanks for the fix!

                  Raffi

                  1 Reply Last reply Reply Quote 0
                  • First post
                    Last post