Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Snort 2.9.2.3 pkg v. 2.5.1 service fails overnight, unable to restart

    Scheduled Pinned Locked Moved pfSense Packages
    65 Posts 14 Posters 22.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      Cino
      last edited by

      @dwood:

      Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off.

      i'll give it a try.. wanted it on to know when CC were being used…

      1 Reply Last reply Reply Quote 0
      • M
        mschiek01
        last edited by

        @Cino:

        @dwood:

        Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off.

        i'll give it a try.. wanted it on to know when CC were being used…

        I am not having any problems with snort stopping on auto updates on 6 different boxes. 3 i386 and 3 amd64.

        The sensitive data preprocessor is set to on, on all the boxes.

        1 thing I should note is that I am still using the "snort_check_for_rule_updates.php" That Fesoj posted.  I have not updated since then.
        https://github.com/bsdperimeter/pfsense-packages/pull/291/files

        1 Reply Last reply Reply Quote 0
        • C
          Cino
          last edited by

          @mschiek01:

          @Cino:

          @dwood:

          Snort is working without issues so far, and updating (auto updating) providing I toggle the sensitive data preprocessor off.

          i'll give it a try.. wanted it on to know when CC were being used…

          I am not having any problems with snort stopping on auto updates on 6 different boxes. 3 i386 and 3 amd64.

          The sensitive data preprocessor is set to on, on all the boxes.

          1 thing I should note is that I am still using the "snort_check_for_rule_updates.php" That Fesoj posted.  I have not updated since then.
          https://github.com/bsdperimeter/pfsense-packages/pull/291/files

          mine is fresh install using both snort and et rules… If that works, can we have ermal pull it in?

          1 Reply Last reply Reply Quote 0
          • F
            Fesoj
            last edited by

            Maybe yes, maybe no.

            What I've suggested is a q&d patch, not a solution. The problem is that alert types depend on the preprocessors invoked and the rule sets, and different rule sets may need different declarations (actually you also need to look at the rules that are enabled). It boils down to do some kind of resource management, or you go into the details, which requires to understand how Snort works.

            If you enable Snort.org AND ET rules, no patching should be required (I have 3 machines running with this config), even if you activate only a few ET rules.

            I am currently thinking about a rather general solution to this type of problem, but this would take some time and I don't know whether I'd like to discuss this in public now as it would separate the published rules from what would go into the configuration of an interface (and I currently cannot present any code).

            1 Reply Last reply Reply Quote 0
            • K
              kilthro
              last edited by

              I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on..  The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

              1 Reply Last reply Reply Quote 0
              • C
                Cino
                last edited by

                @kilthro:

                I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on..  The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

                I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

                I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

                1 Reply Last reply Reply Quote 0
                • M
                  miles267
                  last edited by

                  Same issue as others have reported.  On latest code.

                  • Snort and ET rules enabled
                  • Sensitive data option enabled under preprocessors on both WAN and LAN interface
                  • if I uninstall and reinstall snort from scratch and update rules, both my WAN and LAN interface start MANUALLY without issue
                  • overnight when the auto-update of rules (snort and ET) occurs, snort on both WAN and LAN interfaces stop and cannot be started manually

                  Returns the following FATAL error in log:

                  snort[33033]: FATAL ERROR: /usr/local/etc/snort/snort_60243_em3/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf
                  snort[33033]: FATAL ERROR: /usr/local/etc/snort/snort_60243_em3/preproc_rules/sensitive-data.rules(1) Unknown ClassType: sdf

                  • once sensitive data option is disabled on preprocessors (either WAN or LAN interface), am able to MANUALLY start snort without issue
                  1 Reply Last reply Reply Quote 0
                  • M
                    mschiek01
                    last edited by

                    @Cino:

                    @kilthro:

                    I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on..  The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

                    I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

                    I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

                    Not seeing any memory problems on my boxes. What is your snort memory settings. I am running A/C.  Snort is reloading not starting and stopping.

                    Jul 28 00:32:33 snort[25620]:
                    Jul 28 00:32:33 snort[25620]:
                    Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
                    Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
                    Jul 28 00:32:33 snort[25620]:
                    Jul 28 00:32:33 snort[25620]:
                    Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------
                    Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------

                    Again I am using Fesoj "snort_check_for_rule_updates.php" That Fesoj posted.  I have not updated since then.
                    https://github.com/bsdperimeter/pfsense-packages/pull/291/files

                    Sensitive data preproc is on and using both rule sets.  Not using any SO. rules

                    1 Reply Last reply Reply Quote 0
                    • K
                      kilthro
                      last edited by

                      @Cino:

                      @kilthro:

                      I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on..  The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

                      I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

                      I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

                      From what i can tell, a reload. Not a stop and start.

                      1 Reply Last reply Reply Quote 0
                      • K
                        kilthro
                        last edited by

                        @mschiek01:

                        @Cino:

                        @kilthro:

                        I have both ET and Snort rules enabled and it has been auto updating without any problems for me. I don't have the sensitive data preprocesser on..  The only thing I am noticing is snort memory usage keeps increasing. generally out of 2 gig pfsense generally with snort in the past would sit around 25% used on ram.. Now it starts out at 25% and keeps climbing. Its currently at 40%. If I stop snort and restart it manually it resets back down the starting point. Then it starts climbing daily and during high usage. It seems like whatever cache its using isn't released once done. Any reason why this would be happening with the latest version of snort? I never had this issue before. I have wiped and cleanly installed snort and same problem.

                        I've noticed the same thing this morning. My norm is 24% with snort and 11% without snort. This morning it was up to 28%..

                        I haven't looked at the rule update code since i've been away for the last week. But does it stop and stop snort or is it doing a reload?

                        Not seeing any memory problems on my boxes. What is your snort memory settings. I am running A/C.  Snort is reloading not starting and stopping.

                        Jul 28 00:32:33 snort[25620]:
                        Jul 28 00:32:33 snort[25620]:
                        Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
                        Jul 28 00:32:33 snort[25620]: –== Reload Complete ==--
                        Jul 28 00:32:33 snort[25620]:
                        Jul 28 00:32:33 snort[25620]:
                        Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------
                        Jul 28 00:32:31 snort[25620]: +–--------------------------------------------------------------

                        Again I am using Fesoj "snort_check_for_rule_updates.php" That Fesoj posted.  I have not updated since then.
                        https://github.com/bsdperimeter/pfsense-packages/pull/291/files

                        Sensitive data preproc is on and using both rule sets.  Not using any SO. rules

                        I have always used    AC-BNFA

                        1 Reply Last reply Reply Quote 0
                        • M
                          mschiek01
                          last edited by

                          I used to use ac/bnfa but since the new binary the memory use seems to be under control and I switched to a/c a week or so ago, snort has not stopped since.  BTW I have 4 gig memory in i386 and 8 gig in amd63

                          1 Reply Last reply Reply Quote 0
                          • M
                            MediocreFred
                            last edited by

                            Here's a different error in the log… same symptom though.

                            Jul 27 12:04:37 kernel: pid 44861 (snort), uid 0: exited on signal 11
                            Jul 27 12:04:37 snort[44861]: FATAL ERROR: Character value out of range, try a binary buffer.

                            Happens after a rules update. Has anybody seen this one?

                            1 Reply Last reply Reply Quote 0
                            • C
                              Cino
                              last edited by

                              I use AC-BNFA with ET and Snort rules, also a few .SO rules on 2.1-Dev i386. Using only what is provided by the package only.. This is really the only way to test and to let the developer know what is going on with the code… Now if we can get Fesoj's code updated with what is currently being offered from the package manager, i'll be willing to test it also.

                              PS i386 can't take 4gigs, its a little over 3 gigs. Limitation of 32-bit with memory address allocation.

                              With it reloading, that kinda makes sense. Just have to figure out why is it increasing its usage.

                              1 Reply Last reply Reply Quote 0
                              • F
                                Fesoj
                                last edited by

                                I am currently using a fresh package install on 2 (virtual) machines and I have NOT applied my latest patch. Both systems are running fine for a couple of hours now. My latest patch is not needed if you load the Snort.org rules (unless my second last patch has been removed meanwhile, but a quick check of the sources show that that is not the case). The rule sets will update during the night, so I'll see tomorrow morning if there is an issue (but I doubt that).

                                Alerting, blocking, blacklisting (with squid/squidGuard), reporting (with Sarg), and freeRADIUS (hooked up to a MySQL server) is working smoothly. I couldn't be happier.

                                If others are still seeing problems, I could strip personal info from the images and publish an ova file somewhere. Then we'll see whether the problems persist (maybe some problems could be due to subtle hardware failures ;D).

                                1 Reply Last reply Reply Quote 0
                                • M
                                  miles267
                                  last edited by

                                  Confirming the latest snort code restarts automatically after the update of Snort and ET rules but ONY if the 'Enable Sensitive Data' pre-processor is disabled.

                                  Is anyone else having this issue and is there a way to correct even with a patch until the next code update?

                                  1 Reply Last reply Reply Quote 0
                                  • F
                                    Fesoj
                                    last edited by

                                    Confirming the latest snort code restarts automatically after the update of Snort and ET rules but ONY if the 'Enable Sensitive Data' pre-processor is disabled.

                                    Yes, I disabled the sensitive data preproc for my latest tests as well.

                                    1 Reply Last reply Reply Quote 0
                                    • F
                                      Fesoj
                                      last edited by

                                      miles267,

                                      snort chokes on the missing sdf declaration. For now, https://github.com/bsdperimeter/pfsense-packages/pull/291/files should solve the problem. I haven't looked at the details why the declaration is missing again, but I guess you've enabled only the ET rules (but I could be wrong).

                                      1 Reply Last reply Reply Quote 0
                                      • F
                                        Fesoj
                                        last edited by

                                        One more thing:

                                        kilthro said

                                        … The only thing I am noticing is snort memory usage keeps increasing...

                                        and Cino agreed

                                        … I've noticed the same thing this morning...

                                        I looked at the memory usage of the snort daemon as there were some reports about s.th. like memory leaks and I was worried a bit myself.

                                        When you look at snort.sh, you'll see that there is some intelligence coded into rc_start. Basically if a snort instance is already running there is no restart but rather a reload triggered by sending a HUP signal, otherwise there is a cold start (stop and start). You can clog and grep the system.log file for START to find out what happened on the last restart, either SOFT START for reloading using the HUP signal or just START for a real restart.

                                        If you don't trust the HUP signal you can call rc_stop before the rc_start function for restarting. I've looked at both scenarios for the following test.

                                        I restarted and reloaded snort for 50 times with s.th. like /bin/sh /usr/local/etc/rc.d/snort.sh restart followed by a ps -u for the snort processes. After grabbing the RSS values (real memory usage) and plotting the values for reloading and restarting I got the nice diagram that I've attached.

                                        As expected the memory usage for restarting remains constant. When reloading, the memory initially increased, but only up to a maximum value, then it stays essentially constant with some cyclic deviations. I guess this is just how the OS handles real memory. There is definetely no runaway behavior. BTW, your memory values will typically differ from mine which depends mainly on the rules that are used.

                                        To summarize, there doesn't seem to be a memory leak problem when the daemon reloads. Comments are welcome.

                                        memory.png
                                        memory.png_thumb

                                        1 Reply Last reply Reply Quote 0
                                        • M
                                          miles267
                                          last edited by

                                          @Fesoj:

                                          miles267,

                                          snort chokes on the missing sdf declaration. For now, https://github.com/bsdperimeter/pfsense-packages/pull/291/files should solve the problem. I haven't looked at the details why the declaration is missing again, but I guess you've enabled only the ET rules (but I could be wrong).

                                          Fesoj, I just applied the fix you've added to github.  Will echo back if there are any issues.  Are you using the Sensitive data preprocessor in your config?

                                          For example, I've added the following to the bottom of my Snort suppress list.  Please let me know if this is accurate.  Also, wasn't sure whether I should have so many suppression rules or just certain ones for the HTTP/HTTPS inspect suppress?  Please let me know your thoughts.  Wasn't sure whether this is the proper use of the suppress list.

                                          uppress gen_id 1, sig_id 536
                                          suppress gen_id 1, sig_id 8375
                                          suppress gen_id 1, sig_id 12286
                                          suppress gen_id 1, sig_id 15147
                                          suppress gen_id 1, sig_id 15362
                                          suppress gen_id 1, sig_id 17458
                                          suppress gen_id 1, sig_id 20583
                                          suppress gen_id 1, sig_id 2014819
                                          suppress gen_id 1, sig_id 2014520
                                          suppress gen_id 1, sig_id 201481
                                          suppress gen_id 1, sig_id 2103134
                                          suppress gen_id 1, sig_id 2500056
                                          suppress gen_id 119, sig_id 2
                                          suppress gen_id 119, sig_id 4
                                          suppress gen_id 119, sig_id 14
                                          suppress gen_id 119, sig_id 31
                                          suppress gen_id 119, sig_id 32
                                          suppress gen_id 120, sig_id 2
                                          suppress gen_id 120, sig_id 3
                                          suppress gen_id 120, sig_id 6
                                          suppress gen_id 120, sig_id 8
                                          suppress gen_id 122, sig_id 19
                                          suppress gen_id 122, sig_id 23
                                          suppress gen_id 122, sig_id 26
                                          suppress gen_id 137, sig_id 1

                                          Sensitive Data disable

                                          Credit Card Numbers

                                          #suppress gen_id 138, sig_id 2

                                          U.S. Social Security Numbers (with dashes)

                                          #suppress gen_id 138, sig_id 3

                                          U.S. Social Security Numbers (w/out dashes)

                                          #suppress gen_id 138, sig_id 4

                                          Email Addresses

                                          suppress gen_id 138, sig_id 5

                                          U.S. Phone Numbers

                                          suppress gen_id 138, sig_id 6

                                          1 Reply Last reply Reply Quote 0
                                          • F
                                            Fesoj
                                            last edited by

                                            miles267,

                                            Are you using the Sensitive data preprocessor in your config?

                                            I verified that the error can occur when the sensitive data preproc is enabled.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.