DNS won't start

HeMan321

Hello,
We have an urgent DNS issue. During the night last night (when no one was even using the system, let alone making changes) the DNS service logged the following:
Sep 15 00:42:28 unbound 6054:0 notice: init module 0: validator
Sep 15 00:42:28 unbound 6054:0 error: failed to read /root.key
Sep 15 00:42:28 unbound 6054:0 error: error reading auto-trust-anchor-file: /var/unbound/root.key
Sep 15 00:42:28 unbound 6054:0 error: validator: error in trustanchors config
Sep 15 00:42:28 unbound 6054:0 error: validator: could not apply configuration settings.
Sep 15 00:42:28 unbound 6054:0 error: module init for module validator failed
Sep 15 00:42:28 unbound 6054:0 fatal error: failed to setup modules
Sep 15 00:43:14 unbound 18640:0 notice: init module 0: validator
Sep 15 00:43:14 unbound 18640:0 error: failed to read /root.key
Sep 15 00:43:14 unbound 18640:0 error: error reading auto-trust-anchor-file: /var/unbound/root.key
Sep 15 00:43:14 unbound 18640:0 error: validator: error in trustanchors config
Sep 15 00:43:14 unbound 18640:0 error: validator: could not apply configuration settings.
Sep 15 00:43:14 unbound 18640:0 error: module init for module validator failed
Sep 15 00:43:14 unbound 18640:0 fatal error: failed to setup modules

Now whenever we try and re-start the DNS service we get:
Sep 15 11:55:48 unbound 96592:0 notice: init module 0: validator
Sep 15 11:55:48 unbound 96592:0 error: failed to read /root.key
Sep 15 11:55:48 unbound 96592:0 error: error reading auto-trust-anchor-file: /var/unbound/root.key
Sep 15 11:55:48 unbound 96592:0 error: validator: error in trustanchors config
Sep 15 11:55:48 unbound 96592:0 error: validator: could not apply configuration settings.
Sep 15 11:55:48 unbound 96592:0 error: module init for module validator failed
Sep 15 11:55:48 unbound 96592:0 fatal error: failed to setup modules

And it does not start! We found that the root.key file was empty and zero bytes. I found (here https://forum.netgate.com/topic/78531/unbound-cannot-start-in-2-2-release/6?_=1600168816524) that you can do this to re-create the file:
rm /var/unbound/root.key
unbound-anchor -a /var/unbound/root.key
chown unbound /var/unbound/root.key

And this does create a new file but when you try and re-start the DNS service exactly the same errors are logged and the service fails and the root.key file is wiped (back to zero bytes).

Please help as we are completely down until this is fixed!

Thanks

HeMan321

OK, problem solved! I noticed that the disk was at 100% It seems the Suricata logs had filled the drive, so I enabled the hard limit for their log size, disk usage dropped to 56% and DNS now starts :o)

Maybe a more obvious warning if the disk fills up or more useful logging for the DNS service would be a useful addition in the future?