PfSense 2.3 Check_mk working with xinetd

joeclifford

As my company uses check_mk for monitoring pfsense, it is frustrating that the package for ckeck_mk has been removed. To be honest, the old package didn't work without manual modification, so it is understandable that is was removed.

I've managed to get check_mk working with the pfsense 2.3 implementation of xinetd, which survives reboots, and allows config changed without rebooting the firewall..

As this is a manual installation, you are advised that following the information below at your own risk.

Before you install check_mk, you first need to understand how xinetd has been implemented in pfsense 2.3.

The config file stored in /var/etc/xinetd.conf is regenerated every time you perform a filter reload. This means everytime you change a NAT rule, a filewall rule, change the TFTP config, etc… the xinetd.conf file is deleted, regenerated and then xinetd is restarted. The consequences of this is any manual change to the xinetd.conf get wiped out.

My intention is to try and write a package for check_mk that makes the changes below automatically, but it does depend on whether pfsense will change the way it has implemented xinetd in 2.3

So, to allow us to make make changes to the xinetd.conf file, we need to modify the filter reload process.

The way I have chosen to do this is to modify the function, in /etc/inc/filter.inc that generates the xinetd.conf file, to also include xinetd config files from another directory. This will allow config changes to individual xinetd services, without having to modify the /etc/inc/filter.inc everytime.

First we need to make sure the check_mk agent can run without modifying the agent executable each time; for this we need to install bash.

pkg install bash

Next we need to create a location to store the check_mk_agent executable, and a location to store the xinetd config. I've chosen to use /opt as it is less likely to be deleted during an upgrade.


mkdir -p /opt/bin
mkdir -p /opt/etc/xinetd.d

Download the latest version of the check_mk_agent.freebsd from the check_mk public git.

Place it is /opt/bin/check_mk_agent

http://git.mathias-kettner.de/git/?p=check_mk.git;a=blob_plain;f=agents/check_mk_agent.freebsd;hb=HEAD

Make it executable

chmod +x /opt/bin/check_mk_agent

At this point you should be able to run the agent and see the check_mk output

/opt/bin/check_mk_agent

To able to use xinetd we need to create a new config file with the service details.

Please edit this file to suit your needs.

vi /opt/etc/xinetd.d/check_mk


# +------------------------------------------------------------------+
# |             ____ _               _        __  __ _  __           |
# |            / ___| |__   ___  ___| | __   |  \/  | |/ /           |
# |           | |   | '_ \ / _ \/ __| |/ /   | |\/| | ' /            |
# |           | |___| | | |  __/ (__|   <    | |  | | . \            |
# |            \____|_| |_|\___|\___|_|\_\___|_|  |_|_|\_\           |
# |                                                                  |
# | Copyright Mathias Kettner 2014             mk@mathias-kettner.de |
# +------------------------------------------------------------------+
#
# This file is part of Check_MK.
# The official homepage is at http://mathias-kettner.de/check_mk.
#
# check_mk is free software;  you can redistribute it and/or modify it
# under the  terms of the  GNU General Public License  as published by
# the Free Software Foundation in version 2.  check_mk is  distributed
# in the hope that it will be useful, but WITHOUT ANY WARRANTY;  with-
# out even the implied warranty of  MERCHANTABILITY  or  FITNESS FOR A
# PARTICULAR PURPOSE. See the  GNU General Public License for more de-
# ails.  You should have  received  a copy of the  GNU  General Public
# License along with GNU Make; see the file  COPYING.  If  not,  write
# to the Free Software Foundation, Inc., 51 Franklin St,  Fifth Floor,
# Boston, MA 02110-1301 USA.

service check_mk
{
	type           = UNLISTED
	port           = 6556
	socket_type    = stream
	protocol       = tcp
	wait           = no
	user           = root
	server         = /opt/bin/check_mk_agent

	# If you use fully redundant monitoring and poll the client
	# from more then one monitoring servers in parallel you might
	# want to use the agent cache wrapper:

	#server         = /usr/bin/check_mk_caching_agent

	# configure the IP address(es) of your Nagios server here:
	#only_from      = 127.0.0.1 10.0.20.1 10.0.20.2

	# Don't be too verbose. Don't log every check. This might be
	# commented out for debugging. If this option is commented out
	# the default options will be used for this service.
	log_on_success =

	disable        = no
}

Now we need to modify the filter reload function to include our config files.

vi /etc/inc/filter.inc

You need to find the line which begins fclose($xinetd_fd);
This is the last part of the xinetd.conf file generation function. We need to insert our changes just above this line, it will need to look like this:


        fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");

        fclose($xinetd_fd);             // Close file handle

That should be it to get xinetd to pick up the changes, you now need to manually initiate a filter reload.

Status / Filter Reload / Reload Filters

You can check the System Logs to see if xinetd has picked up the new service; you should see something like below.


May 9 11:41:57	check_reload_status		Reloading filter
May 9 11:41:58	xinetd	8327	Starting reconfiguration
May 9 11:41:58	xinetd	8327	Swapping defaults
May 9 11:41:58	xinetd	8327	readjusting service 6969-udp
May 9 11:41:58	xinetd	8327	readjusting service check_mk
May 9 11:41:58	xinetd	8327	Reconfigured: new=0 old=2 dropped=0 (services)

By default your LAN interface will allow access to port 6556, if you need access on your WAN, you can create a firewall or NAT rule and pass 6556 to localhost.

Do let me know if you have any feedback about this, or if it doesn't work.

Joe

jcouillardboreal-is.com

It works very well!

joeclifford

Thanks, good to know.

cordeiroef

Hi joeclifford,
I had problem with step 5, I could not install libstatgrab.
The pfsense not find the package.
Would you help me?

joeclifford

The check_mk_agent will just output an empty statgrab section is libstatgrab isn't installed.

By default pfsense 2.3 doesn't pull packages from the FreeBSD repo.

You can install it manually if you want to; you can download the latest build from the FreeBSD Ports Monitoring System.

http://portsmon.freebsd.org/portoverview.py?category=devel&portname=libstatgrab

Just grab the package for your processor type. E.g, for pfsense 2.3 x64;


curl --remote-name "http://pkg.freebsd.org/freebsd:10:x86:64/latest/All/libstatgrab-0.91.txz"
pkg install libstatgrab-0.91.txz

You might have to restart the firewall once you've installed statgrab to be able to use it.

cordeiroef

Worked perfectly.
Thanks for the help there, man.
I will continue the following steps …

:D :D :D

cedlerouge

Like previous post, it works so good
Thx

z80

Tested on 2.3.1
works !! :)

Thanks

MontanaIce

All worked then I came back to work the next week and found the agent is unreachable. I went through the steps againt and found that the filter file changed back to default settings and is missing the line:
fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");

If I re-add the line again, the agent works again. How do I make that filter file save the changes persistently through a reboot?

azekiel

Thank you, worked like a charm.

z80

After a update to 2.3.1_1 the filter.inc have to edit again.
and a filter reload.

FJerusalem

Make it update-persistent

Problem: After an update the filter.inc has to be modified again.
Solution: Modify the /etc/inc/filter.inc-file automatically via cron-Script.

A) Create the cron-script

vi /opt/filter_check_mk_cron

#!/bin/sh

grep includedir /etc/inc/filter.inc
if [ $? -eq 0 ]
then
exit 0
else
awk '/Close file handle/{print "fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");"}1' /etc/inc/filter.inc > /etc/inc/filter.inc.temp
mv /etc/inc/filter.inc.temp /etc/inc/filter.inc
fi
exit 0

B) Make it executable

chmod +x /opt/filter_check_mk_cron

C) Create the cron-job

In the web-ui go to
System -> Package Manager
If cron is not installed -> install

Then open Services -> Cron and click on Add

*/15 * * * * root /opt/filter_check_mk_cron

This will check the config-file every 15 Minutes

Somebody might want to create an official package by now? :)

joeclifford

FJerusalem,

Good idea to cron job a script to add in the needed line to the filter.inc. It might be worth modifying your script so it finds the line based on the fclose statement, rather than the comment. It's unlikely, but if and update adds an identical comment to the filter.inc file it could mess it up. I've used the below script.

#!/bin/sh

grep includedir /etc/inc/filter.inc
if [ $? -eq 0 ]
then
        exit 0
else
        awk '/fclose\(\$xinetd_fd\)\;/{print "fwrite($xinetd_fd, \"includedir /opt/etc/xinetd.d\");"}1' /etc/inc/filter.inc > /etc/inc/filter.inc.temp
        mv /etc/inc/filter.inc.temp /etc/inc/filter.inc
fi
exit 0

joeclifford

@MontanaIce:

All worked then I came back to work the next week and found the agent is unreachable. I went through the steps againt and found that the filter file changed back to default settings and is missing the line:
fwrite($xinetd_fd, "includedir /opt/etc/xinetd.d");

If I re-add the line again, the agent works again. How do I make that filter file save the changes persistently through a reboot?

Hmm, I haven't yet had the filter.inc file revert on just a reboot, it normally only happens when there is an update that overrides the filter.inc file. Like this bug fix introduced in 2.3.1 . https://redmine.pfsense.org/issues/6307

Are you running the nano version of pfsense? Pre 2.3.1 the filesystem is set to read only by default.

Thankfully, FJerusalem's script will make the changes for you.

azekiel

Thank you very much for the manual solution. Any intentions to continue to develop the package?

toddh

Trying to get this running on pfSense 2.3.3 for external monitoring and I am running into a problem.

When I modify filter.inc and then reload filter I get this message in the System Log
"Unable to read included directory: /opt/etc/xinetd.d [file=/var/etc/xinetd.conf] [line=1]"

The /var/etc/xinetd.conf files contains
includedir /opt/etc/xinetd.d

I am not sure if I have made an error, or something otherwise is happening.

Programie

Thanks for a working Check_MK Agent on pfSense!

I've written a simple local check for Check_MK to check whether a new version of pfSense is available. Maybe someone is interested in it.

Create a new file (I've named it "check_version.php") in your $LOCALDIR of Check_MK (which is "$MK_LIBDIR/local") containing the following content:

#! /usr/bin/env php
include "pkg-utils.inc";

$info = get_system_pkg_version();

if ($info["installed_version"] == $info["version"]) {
    echo "0 pfSense_Version - No new update available\n";
} else {
    echo "1 pfSense_Version - Version " . $info["version"] . " available\n";
}

Make it executable: chmod +x check_version.php

You probably don't want to check for a new version on every run of check_mk_agent, so move the script into a subfolder containing the run interval as the name. I've saved the script to $LOCALDIR/3600/check_update.php so it will be executed every hour.

lebernd

Thank you for the guide!

Everything it still works for me under 2.4rc - just remember to grab a new libstatgrab-0.91.txz for Freebsd11.

Best, Bernd

Programie

I've also updated pfSense (in my case the final release of version 2.4) and can confirm it still works. I didn't had to grab a new libstatgrab, it just works for me out of the box (I only had to add the additional include in /etc/inc/filter.inc just like on every update).

steini

Hey guys

I just updated to 2.4 and started getting these errors:

**EDIT / SOLVED

Needed to update libstatgrab
fetch http://pkg.freebsd.org/FreeBSD:11:amd64/latest/All/libstatgrab-0.91.txz && pkg install libstatgrab-0.91.txz