24.11 - KEA DHCP/DNS Logging customization?

keyser

I migrated to the new KEA DHCP server (v4) with 24.11 as I was really looking forward to the integrated client DNS registration.

Everything seems to work beautifully - including the DNS registration of DHCP clients in Unbound.

BUT: I'm missing some logging configuration options:
First I really need to log DHCP activity (MAC -> IP tracking) like ISC did.
Second: The log is REALLY flooded with kea2unbound entries for client registrations because most clients are mobile and thus sleeps (reaffirms DHCP IP quite often) - causing A LOT of DNS registration entries.

Is there any way to finetune the logging setup in KEA?

Gertjan

@keyser

I wanted to wait to upgrade to 24.11, but you are making me really curious now.
After all, I was waiting for the

Everything seems to work beautifully - including the DNS registration of DHCP clients in Unbound.

several years now.

@keyser said in 24.11 - KEA DHCP/DNS Logging customization?:

Second: The log is REALLY flooded with kea2unbound entries

I've got a job for you. Do something about it ?!

You've said it yourself :

Is there any way to finetune the logging setup in KEA?

If you have a process that spams the log with 'useless' lines, there is a solution.
Easiest solution : if "kea2unbound " is documented : are there log options ? I haven't found the source of kea2unbound yet myself in the pfSense repository ...
Plan B : pfSense uses syslogd.
Look at the config file that syslogd uses :

[24.03-RELEASE][root@pfSense.bhf.tld]/root: ps aux | grep 'syslog'
......
root    80599   0.0  0.1  12824   3000  -  Ss   Tue07      0:03.27 /usr/sbin/syslogd -O rfc5424 -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /etc/syslog.conf -b 192.168.1.1
.....

/etc/syslogd.conf shows :

# Automatically generated, do not edit!
# Place configuration files in /var/etc/syslog.d
!*

include						/var/etc/syslog.d
# /* Manually added files with non-conflicting names will not be automatically removed */

So, there you have it : /var/etc/syslog.d doesn't exist, but if you make create this yourself ?

FreeBSD Manual Pages syslogd.conf

There are filter options !!

On one of my debian systems, where I use also syslogd, I have this :

:msg, contains, "connect from localhost.localdomain[127.0.0.1]" ~

Which means : If the log line contains "connect from localhost.localdomain[127.0.0.1]" then discard the line.
Can you try this ?

keyser

@Gertjan I'm not a huge a fan of manual edits and following modifications to make them stick, so I'll just live with the current log spam (I have a huge SSD, so wear death is not an issue).
The missing DHCP interactions (MAC -> IP) is however another issue - I hope it can be enabled to log that like ISC did.

Gertjan

@keyser said in 24.11 - KEA DHCP/DNS Logging customization?:

I'm not a huge a fan of manual edits and following modifications to make them stick

I fully agree.
Nothing will stick here.
Take note : there is no 'pfSense files to edit' in this case.
Just adding a file "/var/etc/syslog.d" with extra (yours !) syslogd instructions.
During pfSense install/var/ will be empty to begin with anyway.
In case of issues : delete the file and done ^^

Like you, I have a big SSD (4100 max) so no space issues.
But ... imho, log files are part of the overall security aspect of pfSense, and if one get spammed, the important info will get hidden, like the "security be obscurity" concept, but this time the other way around.

10.0.0.0.1

Regarding the customization of KEA DHCP/DNS logging, you can explore a few aspects based on your specific requirements. The KEA DHCP server, along with its DNS service (if you are using KEA with DNS), provides flexible logging options that can be customized to suit your needs.

Here’s how you can approach it:

Log Levels: KEA allows you to configure different log levels for various subsystems (e.g., ERROR, INFO, DEBUG). The log level is set in the kea-dhcp4.conf (for DHCPv4) or kea-dhcp6.conf (for DHCPv6) configuration files. By adjusting the loggers section in the config file, you can control the verbosity and detail of the logs.

Log Format: The log format can also be customized. By default, KEA uses JSON for logging, but you can adjust it to a different format or modify the existing structure if needed. You can specify additional fields such as timestamps, log levels, and other custom attributes relevant to your network's needs.

Log File Location: KEA logs can be written to specific files, and you can configure where those log files are stored. This is useful for archival purposes and easier troubleshooting. The file option within the loggers configuration section helps to specify the destination.

Custom Loggers: KEA allows the addition of custom loggers, meaning you can write logs to different outputs, such as syslog, a remote server, or even to a custom application for further analysis. This is particularly useful for centralized logging in large network environments.

DNS Logging: If you are also using KEA with DNS, DNS query logs can be customized in the kea-dns.conf file. You can enable detailed logging for DNS queries, including the source IP address, query type, and response status. This can help in debugging DNS-related issues.

Log Rotation: It’s essential to configure log rotation to avoid your log files growing too large over time. This can be set up outside of KEA through system-level log rotation mechanisms, such as log rotation in Linux systems. [SPAM]
Integration with External Tools: KEA logs can be integrated with external log management and analysis tools such as Splunk, ELK stack, or Prometheus for advanced analysis and alerting.

You may want to start by defining what kind of log information you need to capture and decide the level of detail required for each type of event. Adjusting the configurations based on your environment will help you achieve the best results.

Let me know if you need further details on configuring any of these options or if there's a specific logging use case you're working with!

Gertjan

@10-0-0-0-1

Read again :

Second: The log is REALLY flooded with kea2unbound entries

kea2unbound is an 'in house' (by Netgate) developed tool to get DHCP leases from KEA and sends it to unbound.
So : this is not kea related.
No man file avaible.
No documentation.

And what is 100001.su ? What language ?

stephenw10

ChatGPT has helped spammers a lot!

luckman212

Resurfacing this...

Does anyone know of a custom (JSON?) option for Kea to enable more detailed logging of LAN DHCP lease assignments (MAC → IP) the way ISC did?

dennypage

@luckman212 FWIW, there’s a pfSense package pending, ANDwatch, that allows you to keep and query a history of IPv4/IPv6 to MAC address mappings. It’s not particular to dhcp, but works with any form of assignment, static, dhcp, IPv6 assisted, etc. Redmine is here.

luckman212

@dennypage That looks great! I will keep an eye out for it. Looks like you're just waiting for @marcosm to merge.

In the meantime, I dug a bit more, posted this about enabling debug logging in Kea. But for some reason it isn't having the expected effect. Maybe something with the build that's shipped with pfSense.

Amarand

I'm curious, does anyone have any solutions for getting KEA to display requests like the old DHCP server used to do?

I'm sure KEA is fantastic, but as a network administrator, being able to see who's requesting what in real-time in the logs is an essential DHCP/firewall requirement.

KEA isn't a good DHCP server if it can't do that natively.

Amarand

I just noticed that KEA used to provide this detail. The last logs I see where I get the normal DHCP back and forth is Aug 5. Was there an update to pfSense or KEA that removed this functionality, or changed log levels?

I have a Netgate 6100 (if that matters), and I'm curious if KEA is the only option for my pfSense for a DHCP server? I was actually pretty happy with the previous DHCP server, whatever it was called, and migrated to KEA because it was the only supported DHCP server, I guess?

keyser

@Amarand See this thread. That solution works perfectly and requires no "behind the scenes trickery"
https://forum.netgate.com/topic/196313/how-to-change-kea-dhcp-log-level/13?_=1756913953287

Amarand

@keyser

Fantastic, thank you! Yeah, I ended up getting to the JSON settings before I saw your reply, and I had DEBUG instead of just INFO and the logs were going crazy!

I think, with as active as my network is, and as chatty as the DHCP devices are, I'm going to ignore the web GUI, and just tail the logs over SSH. That way I can grep and sed to my heart's content.

I also set-up log rotation using the built-in method, so that's good.

Every once in a while I have these bursts of pfSense learning.