(SOLVED) Suricata Interfaces have to be manually Restarted

[bmeeks]

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

Hi Bill, thanks for the quick update.
I uninstalled 4.1.4_3 and installed 4.1.4_4. It seemed ok for a while. After some time I saw the same tree null errors and I also noticed one of the alerts that came up did not get blocked. The alert prior to it did come up in the block log however so I wonder if these tree null errors have something to do with alerts not being blocked.

I also ran a force update to see if my theory on the update was right. As soon as I did that, Suricata stopped running on both interfaces and that [ERRCODE: SC_ERR_PCAP_DISPATCH(20)] - error code -2, line came up again.

Let me know if you need any more info.

This is really puzzling. The "tree is null" error should be impossible in the new binary. I'm wondering if you actually received the newest binary. It really sounds like you are still running the "unfixed" binary. Look at the file date and time for this file for me:

/usr/local/bin/suricata

It should yesterday's date. The values on my test machine are June 13, 2019 and the file size is 4340864 bytes.

[kiokoman]

sorry for the intrusion, but , this only apply to pfsense 2.4 ? because i don't see a 4.1.4_4 on my pfsense 2.5, the last available version is still 4.1.4_3

[bmeeks]

@kiokoman said in Suricata Interfaces have to be manually Restarted:

sorry for the intrusion, but , this only apply to pfsense 2.4 ? because i don't see a 4.1.4_4 on my pfsense 2.5, the last available version is still 4.1.4_3

The new version is supposed to be in both places. I'm wondering if something went weird with the package building and distribution in the 2.5-DEVEL tree. I have a 2.5-DEVEL virtual machine that is current and it also can't see the newest posted version of the Suricata package. However, if you browse the actual repository of posted packages the lastest version shows up there.

[Raffi_]

@bmeeks said in Suricata Interfaces have to be manually Restarted:

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

Hi Bill, thanks for the quick update.
I uninstalled 4.1.4_3 and installed 4.1.4_4. It seemed ok for a while. After some time I saw the same tree null errors and I also noticed one of the alerts that came up did not get blocked. The alert prior to it did come up in the block log however so I wonder if these tree null errors have something to do with alerts not being blocked.

I also ran a force update to see if my theory on the update was right. As soon as I did that, Suricata stopped running on both interfaces and that [ERRCODE: SC_ERR_PCAP_DISPATCH(20)] - error code -2, line came up again.

Let me know if you need any more info.

This is really puzzling. The "tree is null" error should be impossible in the new binary. I'm wondering if you actually received the newest binary. It really sounds like you are still running the "unfixed" binary. Look at the file date and time for this file for me:

/usr/local/bin/suricata

It should yesterday's date. The values on my test machine are June 13, 2019 and the file size is 4340864 bytes.

Sorry for the trouble. Here is what mine looks like. It has the same date as yours but the size is a bit bigger.

[bmeeks]

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

@bmeeks said in Suricata Interfaces have to be manually Restarted:

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

Hi Bill, thanks for the quick update.
I uninstalled 4.1.4_3 and installed 4.1.4_4. It seemed ok for a while. After some time I saw the same tree null errors and I also noticed one of the alerts that came up did not get blocked. The alert prior to it did come up in the block log however so I wonder if these tree null errors have something to do with alerts not being blocked.

I also ran a force update to see if my theory on the update was right. As soon as I did that, Suricata stopped running on both interfaces and that [ERRCODE: SC_ERR_PCAP_DISPATCH(20)] - error code -2, line came up again.

Let me know if you need any more info.

This is really puzzling. The "tree is null" error should be impossible in the new binary. I'm wondering if you actually received the newest binary. It really sounds like you are still running the "unfixed" binary. Look at the file date and time for this file for me:

/usr/local/bin/suricata

It should yesterday's date. The values on my test machine are June 13, 2019 and the file size is 4340864 bytes.

Sorry for the trouble. Here is what mine looks like. It has the same date as yours but the size is a bit bigger.

The larger size is puzzling. Is your hardware a Netgate appliance or do you run on a generic Intel AMD64 type machine?

[Raffi_]

@bmeeks said in Suricata Interfaces have to be manually Restarted:

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

@bmeeks said in Suricata Interfaces have to be manually Restarted:

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

Hi Bill, thanks for the quick update.
I uninstalled 4.1.4_3 and installed 4.1.4_4. It seemed ok for a while. After some time I saw the same tree null errors and I also noticed one of the alerts that came up did not get blocked. The alert prior to it did come up in the block log however so I wonder if these tree null errors have something to do with alerts not being blocked.

I also ran a force update to see if my theory on the update was right. As soon as I did that, Suricata stopped running on both interfaces and that [ERRCODE: SC_ERR_PCAP_DISPATCH(20)] - error code -2, line came up again.

Let me know if you need any more info.

This is really puzzling. The "tree is null" error should be impossible in the new binary. I'm wondering if you actually received the newest binary. It really sounds like you are still running the "unfixed" binary. Look at the file date and time for this file for me:

/usr/local/bin/suricata

It should yesterday's date. The values on my test machine are June 13, 2019 and the file size is 4340864 bytes.

Sorry for the trouble. Here is what mine looks like. It has the same date as yours but the size is a bit bigger.

The larger size is puzzling. Is your hardware a Netgate appliance or do you run on a generic Intel AMD64 type machine?

I forgot to include the system info. It's not a netgate appliance, it's a custom build.

[bmeeks]

Let's compare MD5 hashes to see if your binary is the same as mine. Run this command from a shell prompt on the firewall:

md5 -q /usr/local/bin/suricata

The output should be:

cc5200e8369def9268b9e30c0c3f41c6

Let me know what you get.

[Raffi_]

@bmeeks said in Suricata Interfaces have to be manually Restarted:

Let's compare MD5 hashes to see if your binary is the same as mine. Run this command from a shell prompt on the firewall:

md5 -q /usr/local/bin/suricata

The output should be:

cc5200e8369def9268b9e30c0c3f41c6

Let me know what you get.

Here is what I got.

c962d5d995867c5baf3136035a34fac7

[bmeeks]

Okay. Thanks for the information. I've sent an email to my pfSense developer contact who works with me on the Suricata and Snort packages asking him to investigate. It's obvious the binary files are different. I would expect them to be the same.

I will wait to hear from him before taking any further action. The bug is fixed on my end so far as I can tell. I definitely saw the problem in the source code and fixed it there. So I'm wondering why the fix seems to be missing for users.

[Raffi_]

Thanks for the help Bill. I'll keep any eye out for an update on this. In the meantime, I'll try uninstalling and installing again just for the sake of being persistent. If anything changes on my end I'll let you know.

[bmeeks]

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

Thanks for the help Bill. I'll keep any eye out for an update on this. In the meantime, I'll try uninstalling and installing again just for the sake of being persistent. If anything changes on my end I'll let you know.

I doubt that will make any difference so long as it still downloads and installs the same binary. It really looks like to me that the binary built as suricata-4.1.4_2 is not "really" my new 4.1.4_2. It appears to still have the old version of my custom blocking plugin patch. That's the only way I can explain the "tree is null" error. I very explicitly fixed that error ... .

[Raffi_]

@bmeeks said in Suricata Interfaces have to be manually Restarted:

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

Thanks for the help Bill. I'll keep any eye out for an update on this. In the meantime, I'll try uninstalling and installing again just for the sake of being persistent. If anything changes on my end I'll let you know.

I doubt that will make any difference so long as it still downloads and installs the same binary. It really looks like to me that the binary built as suricata-4.1.4_2 is not "really" my new 4.1.4_2. It appears to still have the old version of my custom blocking plugin patch. That's the only way I can explain the "tree is null" error. I very explicitly fixed that error ... .

Yup, uninstall and install of 4.1.4_4 gave me the same md5 results as before.

[kiokoman]

the update appeared to me now on my pf 2.5

[2.5.0-DEVELOPMENT][root@pfSense.localdomain]/root: md5 -q /usr/local/bin/suricata
cc5200e8369def9268b9e30c0c3f41c6

[bose301s]

I am seeing odd behavior, Suricata was running fine for awhile but when I got home from work around 6 I noticed the WAN instance had stopped, when I restarted it I lost all internet connectivity on all the devices on my network, once I stopped it everything immediately worked again, wondering if this is related to these bugs.

[bmeeks]

@bose301s said in Suricata Interfaces have to be manually Restarted:

I am seeing odd behavior, Suricata was running fine for awhile but when I got home from work around 6 I noticed the WAN instance had stopped, when I restarted it I lost all internet connectivity on all the devices on my network, once I stopped it everything immediately worked again, wondering if this is related to these bugs.

Yes, if you actually still have the non-patched binary running it will cause a crash. As I just posted in another thread, something unusual happened during the posting of the update and the new binary was actually missing my bug fix. You should get the MD5 hash I posted up above (and the same one user @kiokoman just posted about for pfSense-2.5.

[bose301s]

@bmeeks OK, I got the same md5 as Raffi, should I reinstall, I already tried that but obviously didn't work as I still get the wrong md5. I don't want to go to 2.5 devel, I am on 2.4.4 p3.

[bmeeks]

Just sit tight and try again tomorrow. I reported the issue to the pfSense developer who posts my Snort and Suricata packages to the repository. Since the new file showed up in pfSense-2.5 DEVEL a couple of hours ago, I suspect the correct file will get posted to 2.4.4 RELEASE pretty soon.

[bmeeks]

Is this problem with no blocking and/or having to manually restart Suricata still occurring? I have been unable to reproduce this, and I've tried in both a 2.4.4 RELEASE virtual machine and a 2.5 DEVEL virtual machine. The blocking module now correctly senses and registers firewall interface IP changes without generating a "tree is null" error and without those invalid kernel IP message errors.

The MD5 hash values for the Suricata binary are different for pfSense 2.4.4 versus 2.5. This is due to the differences in the compiling libraries, but both versions do contain my fix for the problems above.

[Raffi_]

@bmeeks said in Suricata Interfaces have to be manually Restarted:

Is this problem with no blocking and/or having to manually restart Suricata still occurring? I have been unable to reproduce this, and I've tried in both a 2.4.4 RELEASE virtual machine and a 2.5 DEVEL virtual machine. The blocking module now correctly senses and registers firewall interface IP changes without generating a "tree is null" error and without those invalid kernel IP message errors.

The MD5 hash values for the Suricata binary are different for pfSense 2.4.4 versus 2.5. This is due to the differences in the compiling libraries, but both versions do contain my fix for the problems above.

Hi @bmeeks, unfortunately the problem is still around for me. I tried uninstall and reinstall this morning to see if anything changed, but the md5 is still the incorrect one I posted before. The behavior is still the same as well. I even tried to un/reinstall just before posting this to be sure nothing has changed since earlier this morning. The md5 is still not correct for me.

[bmeeks]

@Raffi_ said in Suricata Interfaces have to be manually Restarted:

@bmeeks said in Suricata Interfaces have to be manually Restarted:

Is this problem with no blocking and/or having to manually restart Suricata still occurring? I have been unable to reproduce this, and I've tried in both a 2.4.4 RELEASE virtual machine and a 2.5 DEVEL virtual machine. The blocking module now correctly senses and registers firewall interface IP changes without generating a "tree is null" error and without those invalid kernel IP message errors.

The MD5 hash values for the Suricata binary are different for pfSense 2.4.4 versus 2.5. This is due to the differences in the compiling libraries, but both versions do contain my fix for the problems above.

Hi @bmeeks, unfortunately the problem is still around for me. I tried uninstall and reinstall this morning to see if anything changed, but the md5 is still the incorrect one I posted before. The behavior is still the same as well. I even tried to un/reinstall just before posting this to be sure nothing has changed since earlier this morning. The md5 is still not correct for me.

The MD5 hash you posted last Friday (the one ending in fac7) is the same one I have on my 2.4.4 virtual machine, and that's the version I can't see the problem with. So we need to determine what is different between your setup and my virtual machine.

For starters, is your WAN IP a PPPoE interface or is it a DHCP or static IP setup?