Suricata Update 4.0.13_9 PHP Warnings
-
@veldkornet said in Suricata Update 4.0.13_9 PHP Warnings:
After I updated to 4.0.13_11, it was all working fine, items are now all back in the services menus etc, but then I noticed today that it wasn’t running.... if I tried to start it, it just stopped again with no errors in the log.
So I rebooted my pfSense, and the following showed up in the startup log:
20:28:54 PHP ERROR: Type: 1, File: /usr/local/pkg/suricata/suricata_migrate_config.php, Line: 241, Message: Uncaught Error: Cannot create references to/from string offsets in /usr/local/pkg/suricata/suricata_migrate_config.php:241 Stack trace: #0 /usr/local/pkg/suricata/suricata_post_install.php(185): include() #1 /etc/inc/pkg-utils.inc(768) : eval()'d code(1): include_once('/usr/local/pkg/...') #2 /etc/inc/pkg-utils.inc(768): eval() #3 /etc/inc/pkg-utils.inc(854): eval_once('include_once("/...') #4 /etc/rc.packages(74): install_package_xml('suricata') #5 {main} thrown
Can you double-check the date and timestamp for those error messages? That's the error that was fixed in the last package update (4.0.13._11). Maybe you are seeing the error messages logged by the old package version. Another reason I say this is that the ONLY time the suricata_migrate_config.php file is loaded and run is during a package installation. You can't get an error from that file unless the package is being installed/updated again. That does not happen with just a plain-vanilla reboot of the firewall.
Next question for you is what kind of hardware are you using. Is it by chance a Netgate SG-3100 appliance?
Finally, look in the suricata.log for the interface and see what's in there. You can find that on the LOGS VIEW tab.
-
@bmeeks said in Suricata Update 4.0.13_9 PHP Warnings:
Can you double-check the date and timestamp for those error messages? That's the error that was fixed in the last package update (4.0.13._11). Maybe you are seeing the error messages logged by the old package version. Another reason I say this is that the ONLY time the suricata_migrate_config.php file is loaded and run is during a package installation. You can't get an error from that file unless the package is being installed/updated again. That does not happen with just a plain-vanilla reboot of the firewall.
Next question for you is what kind of hardware are you using. Is it by chance a Netgate SG-3100 appliance?
Finally, look in the suricata.log for the interface and see what's in there. You can find that on the LOGS VIEW tab.
I think you’re probably right wrt being the old error. Weird that it showed up in my boot log though....
I’m using a PCEngines APU2C4.
I’ve checked the suricata.log but it’s empty... check via the interface and via terminal.
-
@veldkornet
Go to a command-line prompt and execute this command and post back the results if you can --/usr/local/bin/suricata -V
That should print out some version information and exit. See if any error messages appear. If you have an empty suricata.log file for the interface, that would indicate perhaps some needed dependency packages are missing.
-
@bmeeks said in Suricata Update 4.0.13_9 PHP Warnings:
@veldkornet
Go to a command-line prompt and execute this command and post back the results if you can --/usr/local/bin/suricata -V
That should print out some version information and exit. See if any error messages appear. If you have an empty suricata.log file for the interface, that would indicate perhaps some needed dependency packages are missing.
This is Suricata version 4.0.6 RELEASE
-
That's the expected output. Now try this to start Suricata from the shell prompt --
/usr/local/etc/rc.d/suricata.sh start
Run that from the command line and give it a few seconds to startup. That should start Suricata on all of your configured interfaces. You should also see some data show up in the suricata.log file for each configured interface.
If it fails to start, you need to carefully examine the firewall system log and the suricata.log for the interface to see if any messages were logged. It would be extraordinarily rare for Suricata to fail in startup but not log a single message anywhere.
-
It prints out this:
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/ipsec /usr/local/lib/mysql /usr/local/lib/nss /usr/local/lib/perl5/5.26/mach/CORE 32-bit compatibility ldconfig path:
But nothing in the suricata.log
In the system.log, this is all that it shows:
Nov 15 22:43:13 SuricataStartup 5237 Suricata START for WAN(8011_igb1)...
-
Well, at this point the easiest next step is to remove the Suricata package and reinstall it. You won't lose any settings.
Go to the Package Manager screen and click the trashcan icon next to Suricata to remove the Suricata package. When that completes, click the Available Packages tab and find the Suricata package and install it again. Be sure to stay on that screen until the entire process completes. It will log the package installation progress on the screen.
-
I tried a reinstall, same result.
Then I tried a remove, the new install... same result.
Really weird....
-
Please excuse the following questions, but I have to ask them. Your issue is way out of the ordinary, so I need to get a feel for your experience level.
How are you determining that Suricata is not running? What are you basing that on?
Did you go in and configure an interface to use Suricata and save those changes?
What rule sets (vendor rule archives) did you configure for download (Snort VRT and/or Emering Threats)?
When you go to the LOGS VIEW tab, what interface are you selecting in the INTERFACE drop-down? Is it the same interface where you have Suricata configured?
Do you have a /var/log/suricata directory tree? Does the root account have full permissions in that tree?
-
@bmeeks said in Suricata Update 4.0.13_9 PHP Warnings:
Please excuse the following questions, but I have to ask them. Your issue is way out of the ordinary, so I need to get a feel for your experience level.
Like I said, it was working... Then I noticed that it was stopped, and I couldn't start it up again.
How are you determining that Suricata is not running? What are you basing that on?
Did you go in and configure an interface to use Suricata and save those changes?
What rule sets (vendor rule archives) did you configure for download (Snort VRT and/or Emering Threats)?
Note that the only reason they are up to date is because they updated when I re-installed.
When you go to the LOGS VIEW tab, what interface are you selecting in the INTERFACE drop-down? Is it the same interface where you have Suricata configured?
Do you have a /var/log/suricata directory tree? Does the root account have full permissions in that tree?
As extra info, I deleted all of the logs and tried to start suricata again. The suricata.log is re-created, but it's still blank.
-
I've never seen this type of problem before. It appears the Suricata process never even attempts to start up. But the fact you can run it from the command line (at least to print out the version information) would indicate the basic binary is there and installed OK.
The suricata.log file is now blanked by the shell script when Suricata is started (actually, just before the Suricata binary is called, the log is blanked out). This is to prevent the log from growing unabated with each restart. So the empty log file is sort of expected, but only if the binary never even attempts to start. The very first line in that log file during startup should be the version string information. The fact even that is not there indicates to me the binary never gets called. Very weird indeed.
Are you configured to use Legacy Mode blocking or Inline IPS Mode blocking? If Inline IPS Mode, try switching to Legacy Mode. I kind of doubt that's going to make a difference, though. At the very least I would expect to see something printed to the suricata.log file during the startup attempt.
Try this from the command line to actually launch a Suricata session. See what it prints out. To exit the process (if it starts), do a CTRL-C.
/usr/local/bin/suricata -i igb1 -c /usr/local/etc/suricata/suricata_18011_igb1/suricata.yaml
The above command will launch a Suricata instance using the configuration information in the YAML file given. It should start up and print various stuff to the output screen and then seem to freeze. At that point you can CTRL-C to break out. Post back here with anything you think might be helpful in troubleshooting that you see in the output.
-
16/11/2018 -- 00:25:21 - <Error> - [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - Failed to parse configuration file at line 399: did not find expected key
At line 399 though is the following (personality: IDS):
########################################################################### # Configure libhtp. libhtp: default-config: personality: IDS request-body-limit: 4096 double-decode-path: no double-decode-query: no uri-include-all: no
Update:
Fixed those extra spaces in front on "personality". I know yaml's are fussy, so I put it in line with the request-body-limit.Weird that it was like that in the first place though...
However, I see that removing these spaces fixes the issue, and I can start suricata. Although now during my testing, the spaces came back once or twice, and I had to remove them again for it to start.
Also, the log is now littered with errors. Not sure if it's related, or if it's just the first time I've noticed it...
16/11/2018 -- 00:52:31 - <Notice> -- This is Suricata version 4.0.6 RELEASE 16/11/2018 -- 00:52:31 - <Info> -- CPUs/cores online: 4 16/11/2018 -- 00:52:31 - <Info> -- HTTP memcap: 67108864 16/11/2018 -- 00:52:31 - <Notice> -- using flow hash instead of active packets 16/11/2018 -- 00:52:31 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Signature combines packet specific matches (like dsize, flags, ttl) with stream / state matching by matching on app layer proto (like using http_* keywords). 16/11/2018 -- 00:52:31 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $HOME_NET any -> $EXTERNAL_NET 1024:65535 (msg:"MALWARE-CNC Win.Trojan.Fakeavlock variant outbound connection"; flow:to_server,established; dsize:267<>276; content:"User-Agent|3A| Mozilla/5.0 (Windows|3B| U|3B| MSIE 9.0|3B| Windows NT 9.0|3B| en-US)|0D 0A|"; fast_pattern:only; http_header; urilen:159; pcre:"/\x2f[A-F0-9]{158}/U"; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/file/c49f7dbc036ad0a86df02cbbde00cb3b3fbd651d82f6c9c5a98170644374f64f/analysis/; classtype:trojan-activity; sid:25675; rev:7;)" from file /usr/local/etc/suricata/suricata_8011_igb1/rules/suricata.rules at line 52 16/11/2018 -- 00:52:31 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content 16/11/2018 -- 00:52:32 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"MALWARE-OTHER Win.Trojan.Zeus Spam 2013 dated zip/exe HTTP Response - potential malware download"; flow:to_client,established; content:"-2013.zip|0D 0A|"; fast_pattern:only; content:"-2013.zip|0D 0A|"; http_header; content:"-"; within:1; distance:-14; http_header; file_data; content:"-2013.exe"; content:"-"; within:1; distance:-14; metadata:impact_flag red, policy balanced-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.virustotal.com/en/file/2eff3ee6ac7f5bf85e4ebcbe51974d0708cef666581ef1385c628233614b22c0/analysis/; classtype:trojan-activity; sid:26470; rev:1;)" from file /usr/local/etc/suricata/suricata_8011_igb1/rules/suricata.rules at line 73 16/11/2018 -- 00:52:32 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_pattern:only; set. Can't have relative keywords around a fast_pattern only content
And a whole lot more tha tthe forum doesn't let me post because it thinks it's spam.
-
@veldkornet said in Suricata Update 4.0.13_9 PHP Warnings:
16/11/2018 -- 00:25:21 - <Error> - [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - Failed to parse configuration file at line 399: did not find expected key
At line 399 though is the following (personality: IDS):
########################################################################### # Configure libhtp. libhtp: default-config: personality: IDS request-body-limit: 4096 double-decode-path: no double-decode-query: no uri-include-all: no
Update:
Fixed those extra spaces in front on "personality". I know yaml's are fussy, so I put it in line with the request-body-limit.Weird that it was like that in the first place though...
I think you have or had corruption in part of your config.xml file related to Suricata. This item (libhtp) is the same thing that was causing the PHP error message you originally reported. This information is stored in the config.xml file as an array, and that array was not existent in your file when it really should have been there. All my fix did was initialize that array if it was missing before attempting to use it. This YAML config error happening with the same item is the connecting link.
Look carefully on the FLOW/STREAM tab at the configuration engine settings for HTTP. Something is amiss in your setup. Making manual edits to the suricata.yaml file is pointless as that file is overwritten by the GUI code each time you start Suricata from the INTERFACES tab.
The Invalid Signature errors are expected when using the Snort rules with Suricata. See several of my other posts in this forum in recent threads. Suricata does not understand and thus can't process several Snort rule keywords.
-
I have this exact problem with absolutely no errors being logged with 4.0.13_11. I'm on different hardware as well.
-
@rawrsquad said in Suricata Update 4.0.13_9 PHP Warnings:
I have this exact problem with absolutely no errors being logged with 4.0.13_11. I'm on different hardware as well.
Which exact problem do you mean? Is Suricata failing to start with no error, or did you see the PHP error mentioned in the very first post of this thread?
If Suricata is failing to start but logging no error, then run the command I gave in this post, but you will need to adjust the random GUID number and physical interface in the sub-directory path to match the one on your firewall. It will not be the same, but you can easily find yours by looking at the sub-directory names under /usr/local/etc/suricata. See if you also get a suricata.yaml file syntax error.
-
My problem on line 399 is back after updating to pfSense 4.3.3_p1.
Except this time, it’s changing back after I try start the service via the interface...
Any ideas?
-
@veldkornet said in Suricata Update 4.0.13_9 PHP Warnings:
My problem on line 399 is back after updating to pfSense 4.3.3_p1.
Except this time, it’s changing back after I try start the service via the interface...
Any ideas?
You have corruption within the Suricata section of your firewall's config.xml file. Specifically it looks like some funky data exists for the HTTP stream engine. Go to the FLOW/STREAM tab and scroll down to the HTTP engine section. Click to edit any entries there and see if you either get an error or see bogus data (like extra spaces or something).
-
@bmeeks said in Suricata Update 4.0.13_9 PHP Warnings:
@veldkornet said in Suricata Update 4.0.13_9 PHP Warnings:
My problem on line 399 is back after updating to pfSense 4.3.3_p1.
Except this time, it’s changing back after I try start the service via the interface...
Any ideas?
You have corruption within the Suricata section of your firewall's config.xml file. Specifically it looks like some funky data exists for the HTTP stream engine. Go to the FLOW/STREAM tab and scroll down to the HTTP engine section. Click to edit any entries there and see if you either get an error or see bogus data (like extra spaces or something).
So I found that when I click on the “App Parsers” tab, I get the following error:
Fatal error: Uncaught Error: Cannot use string offset as an array in /usr/local/www/suricata/suricata_app_parsers.php:79 Stack trace: #0 {main} thrown in /usr/local/www/suricata/suricata_app_parsers.php on line 79 PHP ERROR: Type: 1, File: /usr/local/www/suricata/suricata_app_parsers.php, Line: 79, Message: Uncaught Error: Cannot use string offset as an array in /usr/local/www/suricata/suricata_app_parsers.php:79 Stack trace: #0 {main} thrown
Could this be causing the issue?
-
Yes, that is another symptom of something being amiss within your config.xml file for the firewall. Specifically within the <installedpackages><suricata></suricata></installedpackages> section. The last bug fix I posted for Suricata was supposed to fix that issue (an uninitialized array), but for some reason it's not working in your case. That's why I suspect something amiss within your configuration file.
Does the APP PARSERS tab display? If so, please post a screenshot of that tab.
-
@bmeeks said in Suricata Update 4.0.13_9 PHP Warnings:
Yes, that is another symptom of something being amiss within your config.xml file for the firewall. Specifically within the <installedpackages><suricata></suricata></installedpackages> section. The last bug fix I posted for Suricata was supposed to fix that issue (an uninitialized array), but for some reason it's not working in your case. That's why I suspect something amiss within your configuration file.
Does the APP PARSERS tab display? If so, please post a screenshot of that tab.
Nope, it doesn’t display. As soon as you click the tab, it throws that error on the screen. White screen, nothing else visible except that error