Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Snort 2.9.2.3 pkg v. 2.5.4 killed by World of Tanks

    pfSense Packages
    8
    24
    5960
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      Fesoj last edited by

      Hi,

      Snort crashes on one of by boxes (guest access in a hotel) quite frequently. It seems to be related to one of the patrons playing World of Tanks, i.e. Snort never crashes when World of Tanks sites do not show up in sarg reports for any given day. I am running Snort 2.9.2.3 pkg v. 2.5.4 and the crashes are independent from the "block offenders" settings. So far I have not found anything relevant in the log files.

      Has anybody seen this as well?

      Sorry for asking here, because if it is Snort itself, then this would be the wrong place to ask this type of question.

      1 Reply Last reply Reply Quote 0
      • bmeeks
        bmeeks last edited by

        Couple of possibilities:

        (1)  Make sure you have the latest binary package and the GUI components.  Easiest way to make 100% sure is to remove the package completely and then reinstall it.  Click the "X" to remove, then go to the "Available Packages" tab and install fresh.  DO NOT simply click the XML icon on the "Installed Packages" tab to do a reinstall.  That is not always dependable and can result in a mixed bag of files and problems starting and/or running.  After the reinstall completes, go to the "Updates" tab in Snort and force a download of new rules.

        (2)  My guess is a particular rule is firing and that rule is crashing Snort.  In the past, prior to some changes that were made in the GUI components, there could be issues with the classification.config file not being correct for a given rule set if you are using both ET and VRT rule packages.  A rule that referenced a classification that was not in the file would generate a crash.  This was fixed in changes starting with 2.5.3.  There are also potentially some preprocessor issues that could be at fault, but more often preprocessor issues cause startup failures and not crashes after running succesfully for a while.  The last round of changes scans for disabled preprocessor and rule interdependencies and disables rules that have rule options dependent upon particular disabled preprocessors.

        1 Reply Last reply Reply Quote 0
        • F
          Fesoj last edited by

          I am always running the latest components && I cannot (yet) trace it back to a single rule. If the person in question is online, Snort crashes with a rate of roughly once every 6 hours (and he is always online and playing WoT). So it could also be a stability issue. Maybe a pcap trace gives more info….

          1 Reply Last reply Reply Quote 0
          • bmeeks
            bmeeks last edited by

            Hmm…that is weird.  Any chance of correlating a crash time with the last alert in the logs?  By that I mean can you perhaps isolate the crash to a specific time window that would correlate with an ALERTS tab entry?  Just grasping at straws here, though.

            Still, even matching up times would not necessarily find the errant rule (if that's what the cause is), because Snort could well crash before writing the offending traffic to the logs.

            I would assume that over a period of 6 hours the user would hit the same rules over and over playing the same game, so I'm now inclined to maybe think it's not an errant rule.  Do you see memory utilization creeping up over the 6 hour window?

            Add some performance logging parameters as follows to the Advanced Configuration Passthrough box at the bottom of the Interfaces window in the Snort GUI.

            config profile_rules: print 50, sort total_ticks, filename LAN_rules_perf.txt append
            config profile_preprocs: sort avg_ticks, filename LAN_preprocs_perf.txt append

            This will write some stats out to the /var/log/snort directory.  Restart Snort and let it run for several hours while this user plays his game.  Manually stop and restart Snort again and examine the data.  This will show you which rules are being hit and how much time is being spent processing them.  Same for the preprocessors.  This may lead to some insight as far as what rules are being triggered most often.  Note that Snort will only write this log data upon a successful shutdown (hence the need to stop and restart).  If you let it run all the way to a crash, it probably won't update the performance log files.

            1 Reply Last reply Reply Quote 0
            • F
              Fesoj last edited by

              If the box would be in my office it would be easier to analyze. I'll keep an eye on the issue. I just prepared a virtual machine with pfsense, so I can study the behavior on my desk–--but that implies I have to play WoT...

              1 Reply Last reply Reply Quote 0
              • F
                Fesoj last edited by

                Hi think I found s.th:

                snort[10544]: S5: Session exceeded configured max bytes to queue 1048576 using 1048775 bytes (client queue). 192.168.x.x 49684 –> x.x.x.x 38922 (0) : LWstate 0xe LWFlags 0x406007
                snort[10544]: S5: Session exceeded configured max bytes to queue 1048576 using 1049416 bytes (client queue). 192.168.x.x 55461 –> x.x.x.x 80 (0) : LWstate 0xe LWFlags 0x406007
                snort[10544]: S5: Session exceeded configured max bytes to queue 1048576 using 1049694 bytes (client queue). 192.168.x.x 52025 –> x.x.x.x 80 (0) : LWstate 0xf LWFlags 0x6007

                I'll update the Stream5 parameters and then I'll wait for the "power surfer"…

                My problem seems to be related to a similar problem discussed here some time ago: http://forum.pfsense.org/index.php?topic=25748.0;prev_next=prev

                1 Reply Last reply Reply Quote 0
                • bmeeks
                  bmeeks last edited by

                  You could possibly be running out of memory, but I would expect some more self-evident errors if that were the case.  This stream5 message is more of a "notification" than a "warning" message.  However, you can get rid of it by bumping up the memcap value substantially (assuming you have sufficient RAM in the box).  The trick is to get a high enough memcap number coupled with sufficiently large max_queued_bytes values.  The magic number is determined by your Internet pipe size.

                  One other potential cause for this according to some posts I found from the Snort developers, is asymmetric traffic through Snort such that it does not see all of the 3-way handshake for a session.  If I understood that post correctly, in a situation like that Snort misses the "close" of the session and does not reset the stream5 counters properly.

                  For me, with relatively slow 12 megabit cable service, I have memcap at 33 MB to squash all my stream5 messages.  Another user commented in a recent post he had to take his up to 128 megabytes, I believe, to squash the notifications.  He had a very high speed connection with either cable or maybe it was Verizon FIOS (fiber to the home).

                  Bill

                  1 Reply Last reply Reply Quote 0
                  • F
                    Fesoj last edited by

                    Hi Bill,

                    this all makes sense to me && since under normal conditions less than 20% of the memory gets used I multiplied the stream5 values by 8. If I understood you correctly, the increased values would mean a higher theshold for the warning messages, but the underlying problem does not get solved.

                    This particular patron will show up again on Friday, then I'll see whether this is possibly related to the Snort crashes.

                    1 Reply Last reply Reply Quote 0
                    • bmeeks
                      bmeeks last edited by

                      Correct, that is the way I understand the memcap and stream5 settings.  Essentially you need enough memcap buffer to "keep up with" your incoming packet rate.  The faster your pipe, the more memcap buffer you need to avoid the pruning messages.

                      Still not sure that this is source of your crash, but worth a shot to investigate.

                      Bill

                      1 Reply Last reply Reply Quote 0
                      • F
                        Fesoj last edited by

                        … I am not either. The log files have been cleaned, port morrioring has been set up: I'll be prepared for Friday  ;)

                        1 Reply Last reply Reply Quote 0
                        • F
                          Fesoj last edited by

                          Bill,

                          I got the data from last weekend, and, as you suggested, there is more than the stream5 issue. I need a couple of days to look at the details, read a bit, and check some things, but then I'll report.

                          1 Reply Last reply Reply Quote 0
                          • S
                            Supermule Banned last edited by

                            Sounds good since IDS/IPS is a VITAL part of any firewall and currently Pfsense is having issues with a vital package that does NOT work as intended!

                            1 Reply Last reply Reply Quote 0
                            • F
                              Fesoj last edited by

                              Well, after dozens of more crashes I am now inclined to say that the current Snort version has a basic stability problem. It is not only that "WoT" crashes but other sources, especially sites that stream media can also bring Snort down.

                              I always see something like this:

                              Mar 17 14:07:06	SnortStartup[54733]: Snort STOP For LAN side(55183_em1)...
                              Mar 17 14:03:27	kernel: em1: promiscuous mode disabled
                              Mar 17 14:03:27	kernel: pid 35907 (snort), uid 0: exited on signal 11
                              

                              There are casual S5 messages, but basically there is no hint on what is going on (using the default log settings). Just to check that it is not related to a specific box (Atom based), I set up an 8GB machine (using a 4 core AMD processor) and got the same behavior.

                              I cannot follow most of the other Snort related reported problems here, except that Snort crashes under heavy load. If only 1 or two users are active, my Snort installation seems to be fine.

                              1 Reply Last reply Reply Quote 0
                              • S
                                Supermule Banned last edited by

                                I see this when streaming via VPN pn PFsense…

                                Snort suddenly stops without reason

                                php: /status_services.php: The command '/usr/local/etc/rc.d/snort.sh stop' returned exit code '1', the output was ''
                                Mar 17 21:26:14 SnortStartup[24393]: Snort STOP For Internet(36256_em0)…

                                1 Reply Last reply Reply Quote 0
                                • bmeeks
                                  bmeeks last edited by

                                  A load-based bug is going to perhaps be difficult to find.  Are you up to building a separate FreeBSD 8.1 virtual machine (or actual box) and run the traffic through it in just pure sniffing mode?  Take pfSense out of the mix and maybe just do alerting on a clean FreeBSD 8.1 kernel.  If that crashes, then try the 8.2 kernel.  Other options are the Snort 2.9.4.1 binary on the 8.1 or 8.2 kernels.  There are compilation instructions for Snort on FreeBSD posted on the Snort.org web site (in the documentation section).

                                  Bill

                                  1 Reply Last reply Reply Quote 0
                                  • F
                                    Fesoj last edited by

                                    Yes, exactly. Get the "independent" sniffing mode working and then see what can be done for pfSense.

                                    1 Reply Last reply Reply Quote 0
                                    • E
                                      eri-- last edited by

                                      I updated the binary to snort 2.4.9.1 so try with that.

                                      1 Reply Last reply Reply Quote 0
                                      • F
                                        Fesoj last edited by

                                        I'll wait a couple of hours such that the repository is udpated && then I'll update.

                                        1 Reply Last reply Reply Quote 0
                                        • D
                                          dwood last edited by

                                          Not sure if this is a snort binary update timing issue or not.  The package installation fails on amd64, 2.0.2 This from the install packages window

                                          Beginning package installation for snort…
                                          Downloading package configuration file... done.
                                          Saving updated package information... done.
                                          Downloading snort and its dependencies...
                                          Checking for package installation...
                                          Downloading http://files.pfsense.org/packages/amd64/8/All/mysql-client-5.1.68.tbz ...  could not download from there or http://ftp2.FreeBSD.org/pub/FreeBSD/ports/amd64/packages-8.1-release/All/mysql-client-5.1.68.tbz.
                                          of mysql-client-5.1.68 failed!

                                          Installation aborted.Backing up libraries..

                                          1 Reply Last reply Reply Quote 0
                                          • E
                                            eri-- last edited by

                                            Heh that was fast.
                                            The binaries are being built they will be uploaded shortly.

                                            1 Reply Last reply Reply Quote 0
                                            • G
                                              grandrivers last edited by

                                              having some rules issues

                                              snort[67221]: FATAL ERROR: The dynamic detection library "/usr/local/lib/snort/dynamicrules/web-misc.so" version 1.0 compiled with dynamic engine library version 1.15 isn't compatible with the current dynamic engine library "/usr/local/lib/snort/dynamicengine/libsf_engine.so" version 1.17.

                                              1 Reply Last reply Reply Quote 0
                                              • D
                                                darklogic last edited by

                                                @dwood:

                                                Not sure if this is a snort binary update timing issue or not.  The package installation fails on amd64, 2.0.2 This from the install packages window

                                                Beginning package installation for snort…
                                                Downloading package configuration file... done.
                                                Saving updated package information... done.
                                                Downloading snort and its dependencies...
                                                Checking for package installation...
                                                Downloading http://files.pfsense.org/packages/amd64/8/All/mysql-client-5.1.68.tbz ...  could not download from there or http://ftp2.FreeBSD.org/pub/FreeBSD/ports/amd64/packages-8.1-release/All/mysql-client-5.1.68.tbz.
                                                of mysql-client-5.1.68 failed!

                                                Installation aborted.Backing up libraries..

                                                I am still having the same exact issue with installing/upgrading SNORT.

                                                1 Reply Last reply Reply Quote 0
                                                • R
                                                  rt_rex last edited by

                                                  Don't know if its on propose I looked and the path  http://ftp2.FreeBSD.org/pub/FreeBSD/ports/amd64/packages-8.1-release/ is missing

                                                  In this path http://files.pfsense.org/packages/8/All/mysql-client-5.1.68.tbz the file is missing.

                                                  1 Reply Last reply Reply Quote 0
                                                  • F
                                                    Fesoj last edited by

                                                    I update my snort package to 2.9.4.1 pkg v. 2.5.4.

                                                    It is now running fine for 2 days and hasn't crashed a single time. I am using a limited set of ET rules only, so I don't have the problems related to so rules.

                                                    1 Reply Last reply Reply Quote 0
                                                    • First post
                                                      Last post

                                                    Products

                                                    • Platform Overview
                                                    • TNSR
                                                    • pfSense
                                                    • Appliances

                                                    Services

                                                    • Training
                                                    • Professional Services

                                                    Support

                                                    • Subscription Plans
                                                    • Contact Support
                                                    • Product Lifecycle
                                                    • Documentation

                                                    News

                                                    • Media Coverage
                                                    • Press
                                                    • Events

                                                    Resources

                                                    • Blog
                                                    • FAQ
                                                    • Find a Partner
                                                    • Resource Library
                                                    • Security Information

                                                    Company

                                                    • About Us
                                                    • Careers
                                                    • Partners
                                                    • Contact Us
                                                    • Legal
                                                    Our Mission

                                                    We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

                                                    Subscribe to our Newsletter

                                                    Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

                                                    © 2021 Rubicon Communications, LLC | Privacy Policy