suricata update your ruleset causes unbound to restart

jpgpi250

Please bear with me, long story.

TLDR; the cron job that automatically updates the ruleset causes unbound on a remote (behind the firewall) machine to restart and causes query errors for IPv6 only.

I have suricata configured to updated the ruleset at 05:29. The data on the update tab indicates this is working as expected.

I was looking at the unbound log on the remote machine (raspberry pi, latest OS, unbound latest version 1.13.1) and noticed the log contained an entry:

May 07 05:29:21 unbound[4456:0] info: service stopped (unbound 1.13.1).

no apparent reason logged for this.

Since unbound automatically restarts, and the rule update happens early morning, I never noticed this before, this may have been going on for a while.

Looking at the logs (what happens after this restart) I noticed a lot (100+) of unbound log entries, immediately after the restart:

May 07 05:29:22 unbound[4456:0] info: error sending query to auth server 2001:7fd::1 port 53

Looking at these log entries, I noticed all of the entries refer an IPv6 address, there are no IPv4 entries.

Sometimes, not always, multiple unbound stop messages appear in the log, followed by an additional series of query errors

Eventually, without any intervention, the errors stop and normal unbound operation is resumed.

Further info

I verified the suricata rule update is the cause, by forcing a rule update. The unbound log shows the same results (errors).
latest pfsense 2.5.1 CE
latest suricata 6.0.0_10
using 'track interface' to provide IPv6 adresses to the devices
Suricata is only configured on the WAN interface

What is causing unbound (on the remote machine) to restart?
Why are only IPv6 addresses affected, no IPv4 errors in the log?

Thank you for your time and effort.

bmeeks

Are you using Inline IPS Mode in Suricata? If so, that mode uses the netmap kernel device in FreeBSD, and each time Suricata stops and restarts for the rules update, it will tear down and then rebuild the netmap connection. When the kernel netmap driver is told to "stop" and "start" on an interface, it physically takes the network interface down and then brings it back up. Unbound is not going to like that.

There is an option on the GLOBAL SETTINGS tab in Suricata to enable "Live Reload" of the rules. When you enable that option, Suricata will not stop and restart upon a rules update. Instead, it will use a different process to load the new rules into RAM and then switch over to using them. The downside of this approach is that during the update period two complete copies of the rules and the Suricata configuration must reside in RAM. So you need enough free RAM to accomodate this.

jpgpi250

@bmeeks Thank you for your reply

Yes, using IPS

Aparently, I do have sufficient memory available

dmesg | grep memory
real memory  = 8589934592 (8192 MB)
avail memory = 8125104128 (7748 MB)
real memory  = 8589934592 (8192 MB)
avail memory = 8125104128 (7748 MB)

Found the option and enabled it.
forced an update of the rules

No more unbound restart, No more unbound errors. Will keep monitoring this for a few days, but given your unambiguous answer, very confident this is solved. Hope some other users will benefit from this to.

Thank you so much for providing this solution. As always, thanks for your time and effort.