Host Attribute Table
-
Hi,
I cannot activate the "Host Attribute Table" in the "Preprocessors and Flow" menu.
I always get the same error:
php-fpm[8446]: /snort/snort_interfaces.php: The command '/usr/local/bin/snort -R 49222 -D -l /var/log/snort/snort_re0_vlan40049222 –pid-path /var/run --nolock-pidfile -G 49222 -c /usr/local/etc/snort/snort_49222_re0_vlan400/snort.conf -i re0_vlan400' returned exit code '1', the output was ''
snort[58728]: FATAL ERROR: /usr/local/etc/snort/snort_49222_re0_vlan400/snort.conf(307) ==> failed to load attribute table from /usr/local/etc/snort/snort_49222_re0_vlan400/host_attributes
snort[58728]: /usr/local/etc/snort/snort_49222_re0_vlan400/snort.conf(307) ==> Invalid Attribute Table specification: '/usr/local/etc/snort/snort_49222_re0_vlan400/host_attributes'. Please verify the grammar at or near line 0 (tag '<snort_attributes>')</snort_attributes>.
But I also tried, the "official" example from the Snort Documentation. (–> https://www.snort.org/documents/1 on Page 170)
Same Error… :(
My mistake? Someone can help me?
Thx!
-
Did you by chance create or upload your Host Attribute table from a Windows machine? Snort might be getting tripped up on the DOS line endings (CR/LF) versus the typical UNIX line endings (LF only).
It's been quite some time since I've looked at or tested that code in Snort, but that also means nothing really should have changed there either since it has not been touched. I know it worked when the feature was first introduced. I might still have the old Host Attribute table file I tested with. If so I can test it again in a VM.
Bill
-
Thx, for reply!
I just tested again, with the example from the snort documentation.
Here the example, I tried:
<snort_attributes><attribute_map><entry><id>1</id> <value>Linux</value></entry> <entry><id>2</id> <value>ssh</value></entry></attribute_map> <attribute_table><host><ip>192.168.1.234</ip> <operating_system><name><attribute_id>1</attribute_id> <confidence>100</confidence></name> <vendor><attribute_value>Red Hat</attribute_value> <confidence>99</confidence></vendor> <version><attribute_value>2.6</attribute_value> <confidence>98</confidence></version> <frag_policy>linux</frag_policy> <stream_policy>linux</stream_policy></operating_system> <services><service><port><attribute_value>22</attribute_value> <confidence>100</confidence></port> <ipproto><attribute_value>tcp</attribute_value> <confidence>100</confidence></ipproto> <protocol><attribute_id>2</attribute_id> <confidence>100</confidence></protocol> <application><attribute_value>OpenSSH</attribute_value> <confidence>100</confidence> <version><attribute_value>3.9p1</attribute_value> <confidence>93</confidence></version></application></service> <service><port><attribute_value>2300</attribute_value> <confidence>100</confidence></port> <ipproto><attribute_value>tcp</attribute_value> <confidence>100</confidence></ipproto> <protocol><attribute_value>telnet</attribute_value> <confidence>100</confidence></protocol> <application><attribute_value>telnet</attribute_value> <confidence>50</confidence></application></service></services> <clients><client><ipproto><attribute_value>tcp</attribute_value> <confidence>100</confidence></ipproto> <protocol><attribute_value>http</attribute_value> <confidence>91</confidence></protocol> <application><attribute_value>IE Http Browser</attribute_value> <confidence>90</confidence> <version><attribute_value>6.0</attribute_value> <confidence>89</confidence></version></application></client></clients></host></attribute_table></snort_attributes>
Yes, the upload was done with a windows machine, but I used "Notepad++" and converted the file to Unix(LF) format, before the upload to snort. –> Same Error. :(
Aug 16 22:37:53 snort[83947]: /usr/local/etc/snort/snort_49222_re0_vlan400/snort.conf(307) ==> Invalid Attribute Table specification: '/usr/local/etc/snort/snort_49222_re0_vlan400/host_attributes'. Please verify the grammar at or near line 0 (tag '<snort_attributes>').</snort_attributes>
-
It looks OK. Give me a little time to test this in a virtual machine. Might be something that got inadvertently messed up way back with the Bootstrap conversion of the GUI code. That was a lot of work done in a hurry, and some insiduous little bugs crept in.
Bill
-
OK, Thx! :)
-
This problem is officially kicking my butt … :'(.
I can't seem to find why it rejects the Host Attribute Table file. I even used the one verbatim from the Snort documentation web site, and it still fails to load it. Still scratching my head trying to find this bug ...
Bill
-
:'(
Hmm… Compilation flag missing?
Note: To use a host attribute table and service information, Snort must be configured with the -enable-targetbased flag.
-
:'(
Hmm… Compilation flag missing?
Note: To use a host attribute table and service information, Snort must be configured with the -enable-targetbased flag.
No, checked that first. If that is not turned on, you get a different error about the feature not being recognized. I made sure the line endings were UNIX – no difference. Tried several slightly different forms of the XML -- no difference. I'm wondering if it is an issue within Snort itself. Wonder if this feature is heavily used? I will try spinning up a plain vanilla Linux machine and running just the Snort binary to see if it also chokes on the Host Attribute Table file.
Bill
-
Still have not found the source of this Host Attribute Table validation error. I went ahead and posted an update with other bug fixes because those needed to get out to users. I will keep looking for the Host Attribute Table problem.
Bill
-
Thank you, for your support! :)
-
This problem still seems to exist. Is there no fix coming?
-
@cTar said in Host Attribute Table:
This problem still seems to exist. Is there no fix coming?
Doubtful. I was hoping the latest 2.9.13 Snort binary would fix it, but it still fails for me and I can't easily determine why. The Snort binary code is very poorly documented, so it's very hard to dig into the C source code to figure out how something works. Thus it's even harder to then troubleshoot and fix it when a feature is broken.
It very likely could be some supporting library issue caused by an update to that library and not really be a Snort binary problem at all.
If someone wants to dig into it and submit a patch, I'm open to the idea.
-
I had thought this was a problem with PfSense, not Snort itself. The only way to test would be to try an attribute table on a non-pfsense install of snort.
-
I have this working after finding an error in my host-attribute-table.xml file using the command line program xmllint.
Specifically, I used the DTD on this page
xmllint --noout --dtdvalid SnortDTD.dtd snort-attribute-table.xml
With no output the file is considered good. After correction Snort started up without error. It seems if the file is bad the error output is very general.
-
@cTar said in Host Attribute Table:
I have this working after finding an error in my host-attribute-table.xml file using the command line program xmllint.
Specifically, I used the DTD on this page
xmllint --noout --dtdvalid SnortDTD.dtd snort-attribute-table.xml
With no output the file is considered good. After correction Snort started up without error. It seems if the file is bad the error output is very general.
Can you post the specific error in your file? I have been using a file I've used for testing for a few years. That file has not changed that I am aware of. That's why I suspected something within the Snort binary changed (or a supporting library that read the file).
What exactly did you change in your file to "fix it"? Sharing may help others.
-
My error was embarrassingly simple. One item lacked a closing ">".
When I fixed that I thought it worked, but alas, it will not continue working, even though it worked for a while INCLUDING some alerts. Now Snort will not start on the LAN again. Will keep working on it.
-
On an Ubuntu based Snort installation, very little configuration done, I was able to load my host-attribute-table.xml just fine. This is a PFSENSE issue.
Without any help in the forums from Netgate, the only path forward I see is to directly edit config files, or at least see if it is possible to get the table loaded that way.
-
@cTar said in Host Attribute Table:
On an Ubuntu based Snort installation, very little configuration done, I was able to load my host-attribute-table.xml just fine. This is a PFSENSE issue.
Without any help in the forums from Netgate, the only path forward I see is to directly edit config files, or at least see if it is possible to get the table loaded that way.
Okay. I will continue to look for the problem. I'm working on another Snort feature now and can look into this issue when I complete the other task. Have you tried your experiment on a FreeBSD machine configured the same as your Ubuntu box? In other words, just plain-vanilla FreeBSD 11.2 or 12.0 without pfSense. I'm wondering if it is potentially a library issue. I have changed absolutely nothing on pfSense in that part of the Snort binary. And it used to work and then mysteriously stopped -- like what you expect if some supporting library got updated/changed and that impacted Snort.
-
On FreeBSD 12 in a virtual machine I was able to run the Snort -Tv -c snort.conf command and after chasing down rule errors the command completed with a successful import of the attribute table.
Could this be something to do with how php writes to the config?
I did find the following in the Snort manual which gave me pause:
"config max_attribute_hosts: <hosts>
Sets a limit on the maximum number of hosts to read from the attribute table. Minimum value is 32 and the maximum is 524288 (512k). The default is 10000. If the number of hosts in the attribute table exceeds this value, an error is logged and the remainder of the hosts are ignored. This option is only supported with a Host Attribute Table"
This implies that if you use config max_attribute_hosts that a MINIMUM of 32 will be necessary.
Is this the problem since there is a max hosts setting in the pfSense gui? -
For your first question, I think the answer is "quite possibly". That will be my first avenue of investigation and why I asked you to try it on vanilla FreeBSD installation.
As for the "config max_attribute_hosts" question, I think if that were the issue the error message would be more specific. I will look into this, though, when I work on this issue. I have it logged in my private "Open Bugs" list.
Feel free to also open a pfSense Redmine request if you would like. That will create a public tracker for the issue.