Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Host Attribute Table

    Scheduled Pinned Locked Moved IDS/IPS
    27 Posts 3 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      cTar
      last edited by

      This problem still seems to exist. Is there no fix coming?

      bmeeksB 1 Reply Last reply Reply Quote 0
      • bmeeksB
        bmeeks @cTar
        last edited by

        @cTar said in Host Attribute Table:

        This problem still seems to exist. Is there no fix coming?

        Doubtful. I was hoping the latest 2.9.13 Snort binary would fix it, but it still fails for me and I can't easily determine why. The Snort binary code is very poorly documented, so it's very hard to dig into the C source code to figure out how something works. Thus it's even harder to then troubleshoot and fix it when a feature is broken.

        It very likely could be some supporting library issue caused by an update to that library and not really be a Snort binary problem at all.

        If someone wants to dig into it and submit a patch, I'm open to the idea.

        1 Reply Last reply Reply Quote 0
        • C
          cTar
          last edited by

          I had thought this was a problem with PfSense, not Snort itself. The only way to test would be to try an attribute table on a non-pfsense install of snort.

          1 Reply Last reply Reply Quote 0
          • C
            cTar
            last edited by

            I have this working after finding an error in my host-attribute-table.xml file using the command line program xmllint.

            Specifically, I used the DTD on this page

            xmllint --noout --dtdvalid SnortDTD.dtd snort-attribute-table.xml

            With no output the file is considered good. After correction Snort started up without error. It seems if the file is bad the error output is very general.

            bmeeksB 1 Reply Last reply Reply Quote 0
            • bmeeksB
              bmeeks @cTar
              last edited by

              @cTar said in Host Attribute Table:

              I have this working after finding an error in my host-attribute-table.xml file using the command line program xmllint.

              Specifically, I used the DTD on this page

              xmllint --noout --dtdvalid SnortDTD.dtd snort-attribute-table.xml

              With no output the file is considered good. After correction Snort started up without error. It seems if the file is bad the error output is very general.

              Can you post the specific error in your file? I have been using a file I've used for testing for a few years. That file has not changed that I am aware of. That's why I suspected something within the Snort binary changed (or a supporting library that read the file).

              What exactly did you change in your file to "fix it"? Sharing may help others.

              1 Reply Last reply Reply Quote 0
              • C
                cTar
                last edited by cTar

                My error was embarrassingly simple. One item lacked a closing ">".

                When I fixed that I thought it worked, but alas, it will not continue working, even though it worked for a while INCLUDING some alerts. Now Snort will not start on the LAN again. Will keep working on it.

                1 Reply Last reply Reply Quote 0
                • C
                  cTar
                  last edited by cTar

                  On an Ubuntu based Snort installation, very little configuration done, I was able to load my host-attribute-table.xml just fine. This is a PFSENSE issue.

                  Without any help in the forums from Netgate, the only path forward I see is to directly edit config files, or at least see if it is possible to get the table loaded that way.

                  bmeeksB 1 Reply Last reply Reply Quote 0
                  • bmeeksB
                    bmeeks @cTar
                    last edited by bmeeks

                    @cTar said in Host Attribute Table:

                    On an Ubuntu based Snort installation, very little configuration done, I was able to load my host-attribute-table.xml just fine. This is a PFSENSE issue.

                    Without any help in the forums from Netgate, the only path forward I see is to directly edit config files, or at least see if it is possible to get the table loaded that way.

                    Okay. I will continue to look for the problem. I'm working on another Snort feature now and can look into this issue when I complete the other task. Have you tried your experiment on a FreeBSD machine configured the same as your Ubuntu box? In other words, just plain-vanilla FreeBSD 11.2 or 12.0 without pfSense. I'm wondering if it is potentially a library issue. I have changed absolutely nothing on pfSense in that part of the Snort binary. And it used to work and then mysteriously stopped -- like what you expect if some supporting library got updated/changed and that impacted Snort.

                    1 Reply Last reply Reply Quote 0
                    • C
                      cTar
                      last edited by

                      On FreeBSD 12 in a virtual machine I was able to run the Snort -Tv -c snort.conf command and after chasing down rule errors the command completed with a successful import of the attribute table.

                      Could this be something to do with how php writes to the config?

                      I did find the following in the Snort manual which gave me pause:

                      "config max_attribute_hosts: <hosts>

                      Sets a limit on the maximum number of hosts to read from the attribute table. Minimum value is 32 and the maximum is 524288 (512k). The default is 10000. If the number of hosts in the attribute table exceeds this value, an error is logged and the remainder of the hosts are ignored. This option is only supported with a Host Attribute Table"

                      This implies that if you use config max_attribute_hosts that a MINIMUM of 32 will be necessary.
                      Is this the problem since there is a max hosts setting in the pfSense gui?

                      1 Reply Last reply Reply Quote 0
                      • bmeeksB
                        bmeeks
                        last edited by

                        For your first question, I think the answer is "quite possibly". That will be my first avenue of investigation and why I asked you to try it on vanilla FreeBSD installation.

                        As for the "config max_attribute_hosts" question, I think if that were the issue the error message would be more specific. I will look into this, though, when I work on this issue. I have it logged in my private "Open Bugs" list.

                        Feel free to also open a pfSense Redmine request if you would like. That will create a public tracker for the issue.

                        1 Reply Last reply Reply Quote 0
                        • C
                          cTar
                          last edited by

                          I just tried config max_attribute_hosts: 100 added to the Ubuntu installation and there was no error with less than 32 hosts. I will open that request. Thanks for the help
                          Bill

                          1 Reply Last reply Reply Quote 0
                          • bmeeksB
                            bmeeks
                            last edited by bmeeks

                            I have been troubleshooting this in debug mode for nearly a full day and still am coming up empty-handed in terms of finding the problem. The XML file is correct, but the parser within Snort thinks something is wrong.

                            Debugging is darn near impossible because the actual XML file parsing is done via a nested set of flex/yacc scripts that are used during compile time to produce C source and header files that then get compiled into the Snort executable. I can only trace the function execution a few steps and then lose it within all the unintentional obfuscation created by the nested flex/yacc scripts.

                            It keeps failing with the error message "... check the grammar around line 0 (<tag SNORT_ATTRIBUTES>) ..." and then gives the filename. However, the file validates just fine against the DTD file included with the Snort source code. I've searched for hidden characters within the file, tested with and without DOS line endings versus Linux line endings, and still have come up with nothing to identify the real issue.

                            I am going to put this bug search on pause while I work on completing the new Inline IPS mode feature. When finished with that, I will come back to this bug.

                            1 Reply Last reply Reply Quote 0
                            • bmeeksB
                              bmeeks
                              last edited by

                              I'm happy to report SUCCESS in finding this problem! Turns out it is the choice to use yacc by the FreeBSD poudriere builder system instead of bison when compiling parts of the Snort binary. After some Google research, it seems bison is a newer program that implements the functionality of the older yacc program. bison is backwards compatible with yacc, but the opposite is not 100% true. The use of yacc during the pfSense Snort binary build produces some machine code that fails to properly read the host attributes XML file. When you compile Snort in a different environment, bison is most likely to be the default token parser and so Snort works when importing a host attributes XML file.

                              Look for this to be fixed in the upcoming release of Snort-2.9.13_1 on pfSense-2.5-DEVEL. We are forcing the Snort build on pfSense to use bison instead of defaulting to yacc. I am also going to report this issue to the FreeBSD upstream port maintainer for Snort.

                              1 Reply Last reply Reply Quote 1
                              • C
                                cTar
                                last edited by

                                That is great! Thank you for your efforts to address this. As I have a "production" system I will wait for the stable release.

                                Thanks again

                                Bill

                                bmeeksB 1 Reply Last reply Reply Quote 0
                                • bmeeksB
                                  bmeeks @cTar
                                  last edited by

                                  @cTar said in Host Attribute Table:

                                  That is great! Thank you for your efforts to address this. As I have a "production" system I will wait for the stable release.

                                  Thanks again

                                  Bill

                                  Actually, the FreeBSD Ports maintainer incorporated my fix into the official release, so the pfSense team pulled that change into pfSense-2.4.4-RELEASE this morning. You should see an update for your Snort package in pfSense to version 3.2.9.8_6. That will install Snort binary version 2.9.12_2 which includes the Host Attribute Table fix. Test it and let me know how it works. My testing on pfSense-2.5 was successful.

                                  1 Reply Last reply Reply Quote 0
                                  • C
                                    cTar
                                    last edited by

                                    I can confirm it is working on 2.4.4-RELEASE-p3 (amd64) with the latest Snort package,

                                    Thanks again!

                                    bmeeksB 1 Reply Last reply Reply Quote 0
                                    • bmeeksB
                                      bmeeks @cTar
                                      last edited by

                                      @cTar -- Good deal. Thanks for the feedback confirmation!

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.