PfSense 2.3 Check_mk working with xinetd
-
As my company uses check_mk for monitoring pfsense, it is frustrating that the package for ckeck_mk has been removed. To be honest, the old package didn't work without manual modification, so it is understandable that is was removed.
I've managed to get check_mk working with the pfsense 2.3 implementation of xinetd, which survives reboots, and allows config changed without rebooting the firewall..
As this is a manual installation, you are advised that following the information below at your own risk.
Before you install check_mk, you first need to understand how xinetd has been implemented in pfsense 2.3.
The config file stored in /var/etc/xinetd.conf is regenerated every time you perform a filter reload. This means everytime you change a NAT rule, a filewall rule, change the TFTP config, etc… the xinetd.conf file is deleted, regenerated and then xinetd is restarted. The consequences of this is any manual change to the xinetd.conf get wiped out.
My intention is to try and write a package for check_mk that makes the changes below automatically, but it does depend on whether pfsense will change the way it has implemented xinetd in 2.3
So, to allow us to make make changes to the xinetd.conf file, we need to modify the filter reload process.
The way I have chosen to do this is to modify the function, in /etc/inc/filter.inc that generates the xinetd.conf file, to also include xinetd config files from another directory. This will allow config changes to individual xinetd services, without having to modify the /etc/inc/filter.inc everytime.
- First we need to make sure the check_mk agent can run without modifying the agent executable each time; for this we need to install bash.
pkg install bash
- Next we need to create a location to store the check_mk_agent executable, and a location to store the xinetd config. I've chosen to use /opt as it is less likely to be deleted during an upgrade.
mkdir -p /opt/bin mkdir -p /opt/etc/xinetd.d
- Download the latest version of the check_mk_agent.freebsd from the check_mk public git.
Place it is /opt/bin/check_mk_agent
- Make it executable
chmod +x /opt/bin/check_mk_agent
- At this point you should be able to run the agent and see the check_mk output
/opt/bin/check_mk_agent
- To able to use xinetd we need to create a new config file with the service details.
Please edit this file to suit your needs.
vi /opt/etc/xinetd.d/check_mk
# +------------------------------------------------------------------+ # | ____ _ _ __ __ _ __ | # | / ___| |__ ___ ___| | __ | \/ | |/ / | # | | | | '_ \ / _ \/ __| |/ / | |\/| | ' / | # | | |___| | | | __/ (__| < | | | | . \ | # | \____|_| |_|\___|\___|_|\_\___|_| |_|_|\_\ | # | | # | Copyright Mathias Kettner 2014 mk@mathias-kettner.de | # +------------------------------------------------------------------+ # # This file is part of Check_MK. # The official homepage is at http://mathias-kettner.de/check_mk. # # check_mk is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation in version 2. check_mk is distributed # in the hope that it will be useful, but WITHOUT ANY WARRANTY; with- # out even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. See the GNU General Public License for more de- # ails. You should have received a copy of the GNU General Public # License along with GNU Make; see the file COPYING. If not, write # to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, # Boston, MA 02110-1301 USA. service check_mk { type = UNLISTED port = 6556 socket_type = stream protocol = tcp wait = no user = root server = /opt/bin/check_mk_agent # If you use fully redundant monitoring and poll the client # from more then one monitoring servers in parallel you might # want to use the agent cache wrapper: #server = /usr/bin/check_mk_caching_agent # configure the IP address(es) of your Nagios server here: #only_from = 127.0.0.1 10.0.20.1 10.0.20.2 # Don't be too verbose. Don't log every check. This might be # commented out for debugging. If this option is commented out # the default options will be used for this service. log_on_success = disable = no }
- Now we need to modify the filter reload function to include our config files.
vi /etc/inc/filter.inc
You need to find the line which begins fclose($xinetd_fd);
This is the last part of the xinetd.conf file generation function. We need to insert our changes just above this line, it will need to look like this:fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d"); fclose($xinetd_fd); // Close file handle
- That should be it to get xinetd to pick up the changes, you now need to manually initiate a filter reload.
Status / Filter Reload / Reload Filters
You can check the System Logs to see if xinetd has picked up the new service; you should see something like below.
May 9 11:41:57 check_reload_status Reloading filter May 9 11:41:58 xinetd 8327 Starting reconfiguration May 9 11:41:58 xinetd 8327 Swapping defaults May 9 11:41:58 xinetd 8327 readjusting service 6969-udp May 9 11:41:58 xinetd 8327 readjusting service check_mk May 9 11:41:58 xinetd 8327 Reconfigured: new=0 old=2 dropped=0 (services)
- By default your LAN interface will allow access to port 6556, if you need access on your WAN, you can create a firewall or NAT rule and pass 6556 to localhost.
Do let me know if you have any feedback about this, or if it doesn't work.
Joe
-
It works very well!
-
Thanks, good to know.
-
Hi joeclifford,
I had problem with step 5, I could not install libstatgrab.
The pfsense not find the package.
Would you help me? -
The check_mk_agent will just output an empty statgrab section is libstatgrab isn't installed.
By default pfsense 2.3 doesn't pull packages from the FreeBSD repo.
You can install it manually if you want to; you can download the latest build from the FreeBSD Ports Monitoring System.
http://portsmon.freebsd.org/portoverview.py?category=devel&portname=libstatgrab
Just grab the package for your processor type. E.g, for pfsense 2.3 x64;
curl --remote-name "http://pkg.freebsd.org/freebsd:10:x86:64/latest/All/libstatgrab-0.91.txz" pkg install libstatgrab-0.91.txz
You might have to restart the firewall once you've installed statgrab to be able to use it.
-
Worked perfectly.
Thanks for the help there, man.
I will continue the following steps …:D :D :D
-
Like previous post, it works so good
Thx -
Tested on 2.3.1
works !! :)Thanks
-
All worked then I came back to work the next week and found the agent is unreachable. I went through the steps againt and found that the filter file changed back to default settings and is missing the line:
fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");If I re-add the line again, the agent works again. How do I make that filter file save the changes persistently through a reboot?
-
Thank you, worked like a charm.
-
After a update to 2.3.1_1 the filter.inc have to edit again.
and a filter reload. -
Make it update-persistent
Problem: After an update the filter.inc has to be modified again.
Solution: Modify the /etc/inc/filter.inc-file automatically via cron-Script.A) Create the cron-script
vi /opt/filter_check_mk_cron
#!/bin/sh
grep includedir /etc/inc/filter.inc
if [ $? -eq 0 ]
then
exit 0
else
awk '/Close file handle/{print "fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");"}1' /etc/inc/filter.inc > /etc/inc/filter.inc.temp
mv /etc/inc/filter.inc.temp /etc/inc/filter.inc
fi
exit 0B) Make it executable
chmod +x /opt/filter_check_mk_cron
C) Create the cron-job
In the web-ui go to
System -> Package Manager
If cron is not installed -> installThen open Services -> Cron and click on Add
*/15 * * * * root /opt/filter_check_mk_cron
This will check the config-file every 15 Minutes
Somebody might want to create an official package by now? :)
-
FJerusalem,
Good idea to cron job a script to add in the needed line to the filter.inc. It might be worth modifying your script so it finds the line based on the fclose statement, rather than the comment. It's unlikely, but if and update adds an identical comment to the filter.inc file it could mess it up. I've used the below script.
#!/bin/sh grep includedir /etc/inc/filter.inc if [ $? -eq 0 ] then exit 0 else awk '/fclose\(\$xinetd_fd\)\;/{print "fwrite($xinetd_fd, \"includedir /opt/etc/xinetd.d\");"}1' /etc/inc/filter.inc > /etc/inc/filter.inc.temp mv /etc/inc/filter.inc.temp /etc/inc/filter.inc fi exit 0
-
All worked then I came back to work the next week and found the agent is unreachable. I went through the steps againt and found that the filter file changed back to default settings and is missing the line:
fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");If I re-add the line again, the agent works again. How do I make that filter file save the changes persistently through a reboot?
Hmm, I haven't yet had the filter.inc file revert on just a reboot, it normally only happens when there is an update that overrides the filter.inc file. Like this bug fix introduced in 2.3.1 . https://redmine.pfsense.org/issues/6307
Are you running the nano version of pfsense? Pre 2.3.1 the filesystem is set to read only by default.
Thankfully, FJerusalem's script will make the changes for you.
-
Thank you very much for the manual solution. Any intentions to continue to develop the package?
-
Trying to get this running on pfSense 2.3.3 for external monitoring and I am running into a problem.
When I modify filter.inc and then reload filter I get this message in the System Log
"Unable to read included directory: /opt/etc/xinetd.d [file=/var/etc/xinetd.conf] [line=1]"The /var/etc/xinetd.conf files contains
includedir /opt/etc/xinetd.dI am not sure if I have made an error, or something otherwise is happening.
-
Thanks for a working Check_MK Agent on pfSense!
I've written a simple local check for Check_MK to check whether a new version of pfSense is available. Maybe someone is interested in it.
Create a new file (I've named it "check_version.php") in your $LOCALDIR of Check_MK (which is "$MK_LIBDIR/local") containing the following content:
#! /usr/bin/env php include "pkg-utils.inc"; $info = get_system_pkg_version(); if ($info["installed_version"] == $info["version"]) { echo "0 pfSense_Version - No new update available\n"; } else { echo "1 pfSense_Version - Version " . $info["version"] . " available\n"; }
Make it executable: chmod +x check_version.php
You probably don't want to check for a new version on every run of check_mk_agent, so move the script into a subfolder containing the run interval as the name. I've saved the script to $LOCALDIR/3600/check_update.php so it will be executed every hour.
-
Thank you for the guide!
Everything it still works for me under 2.4rc - just remember to grab a new libstatgrab-0.91.txz for Freebsd11.
Best, Bernd
-
I've also updated pfSense (in my case the final release of version 2.4) and can confirm it still works. I didn't had to grab a new libstatgrab, it just works for me out of the box (I only had to add the additional include in /etc/inc/filter.inc just like on every update).
-
Hey guys
I just updated to 2.4 and started getting these errors:
**EDIT / SOLVED
Needed to update libstatgrab
fetch http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/libstatgrab-0.91.txz && pkg install libstatgrab-0.91.txz -
After an upgrade to 2.4(.1) I seem to be getting errors now with the Memory Used plugin: UNKNOWN - check failed - please submit a crash report!
(The crash report says: KeyError: 'MemTotal')Agent output:
Relevant Check_MK output:<<<mem>>> SwapTotal: 782336 kB SwapFree: 782336 kB</mem> ```(No MemTotal line) My copy of the agent includes a <<<mem>>> section:
Memory Usage
currently we'll need sysutils/muse for this.
if [ -x /usr/local/bin/muse ]
then
echo '<<<mem>>>'yes, i don't know sed well.
muse -k 2>/dev/null | sed 's/Total/MemTotal/' | sed 's/Free/MemFree/'
swapinfo -k 1K | tail -n 1 | awk '{ print "SwapTotal: "$2" kB\nSwapFree: "$4" kB" }'
fi</mem>I've installed muse with 'fetch http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/muse-0.2.txz && pkg install muse-0.2.txz' But 'muse -k' outputs:
kvm_open: kvm_nlist: No such file or directory
kvm_open: kvm_nlist: No such file or directory
kvm_open: kvm_nlist: No such file or directory
kvm_open: kvm_nlist: No such file or directory
kvm_nlist failedPossibly the <<<mem>>> code no longer works? It looks like it used to be in check_mk_agent.freebsd but is not there now. EDIT: I realised we are using an old, customised version of the agent. I'm not willing to update wholesale at this point, but this seems to work now, from a less old version of the agent. It also means we don't have to install muse now.
Memory Usage. First we try statgrab, since
muse does not seem to support >4GB
if [ -x /usr/local/bin/statgrab ] ; then
echo '<<<mem>>>'
statgrab -K mem.total mem.free swap.total swap.free |
awk '{gsub(/swap./,"Swap");}{gsub(/mem./,"Mem");}{gsub(/tot/,"Tot");}{gsub(/free/,"Free");} { print $1": "$3" kB"}'
elif [ -x /usr/local/bin/muse ] ; then
echo '<<<mem>>>'
muse -k 2>/dev/null | sed 's/Total/MemTotal/' | sed 's/Free/MemFree/'
swapinfo -k 1K | tail -n 1 | awk '{ print "SwapTotal: "$2" kB\nSwapFree: "$4" kB" }'
fi</mem></mem> -
Just update /opt/bin/check_mk_agent with the newest version available from http://git.mathias-kettner.de/git/?p=check_mk.git;a=blob_plain;f=agents/check_mk_agent.freebsd;hb=HEAD and install libstatgrab for the memory display to work (fetch http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/libstatgrab-0.91.txz && pkg install libstatgrab-0.91.txz)
-
Didn't work for me (pfSense 2.4.2-RELEASE-p1 (amd64)). I did all steps without any problems, but after reloading the filters, nothing happens. Nothing with xinetd or check_mk in the system logs and I can't to "nc localhost 6556" or "telnet localhost 6556".
-
Didn't work for me (pfSense 2.4.2-RELEASE-p1 (amd64)). I did all steps without any problems, but after reloading the filters, nothing happens. Nothing with xinetd or check_mk in the system logs and I can't to "nc localhost 6556" or "telnet localhost 6556".
Does the Check_MK Agent work if called from the terminal?
Try to execute "/opt/bin/check_mk_agent" on your pfSense host.
And what exact error are you getting if you try to reach port 6556? Connection refused? Connection timed out?
A filter reload should show nothing special (i.e. does not contain anything with check_mk). But you should see something like "readjusting service check_mk" in "Status -> System Logs -> System -> General" after reloading the filters.
After a filter reload, the file "/var/etc/xinetd.conf" should contain a line "includedir /opt/etc/xinetd.d". /opt/etc/xinetd.d should be a directory containing your xinetd config file for Check_MK.
Also make sure, xinetd is running: "ps ax | grep xinetd" should return something like "/usr/local/sbin/xinetd -syslog daemon -f /var/etc/xinetd.conf -pidfile /var/run/xinetd.pid"
-
Worked perfectly for me on 2.4.2!
Really appreciate the instructions!
Thank you!!!!!!!
-
Didn't work for me (pfSense 2.4.2-RELEASE-p1 (amd64)). I did all steps without any problems, but after reloading the filters, nothing happens. Nothing with xinetd or check_mk in the system logs and I can't to "nc localhost 6556" or "telnet localhost 6556".
Does the Check_MK Agent work if called from the terminal?
Try to execute "/opt/bin/check_mk_agent" on your pfSense host.
And what exact error are you getting if you try to reach port 6556? Connection refused? Connection timed out?
A filter reload should show nothing special (i.e. does not contain anything with check_mk). But you should see something like "readjusting service check_mk" in "Status -> System Logs -> System -> General" after reloading the filters.
After a filter reload, the file "/var/etc/xinetd.conf" should contain a line "includedir /opt/etc/xinetd.d". /opt/etc/xinetd.d should be a directory containing your xinetd config file for Check_MK.
Also make sure, xinetd is running: "ps ax | grep xinetd" should return something like "/usr/local/sbin/xinetd -syslog daemon -f /var/etc/xinetd.conf -pidfile /var/run/xinetd.pid"
HI,
I have also a problem with check_mk and xinetd.check_mk works from the terminal. I cannot find a xinetd-process on my box. Also, after reloading the filters (with the manually added line to include /opt/etc/xinetd.d) I only find
Feb 24 16:58:45 check_reload_status Reloading filter
in the logs. /var/etc/xinetd.conf is emptyps doesn't show a xinetd.
How do I start xinetd on pfsense? Shouldn*t it be running by default?
Any help appreciated,
Karl -
For those interested, I've consolidated the instructions into a new thread, and added a line in the update-persistence script to automatically reload the filters after making the change so the check_mk service is started.
https://forum.netgate.com/topic/131967/check_mk-on-pfsense-2-4-w-update-persistence
-
Has anybody had success running check_mk 2.0 agent on pfSense 21.02?
Do steps look exactly as described here:
https://forum.netgate.com/topic/131967/check_mk-on-pfsense-2-4-w-update-persistence
?
That post is almost 3 years old.