Installing RSYSLOGD on pfSense [WIP]



  • Hi All,

    This is one of my first few posts here but I've been a long time pfSense user and want to start giving back to the community a bit. I'm currently doing a lot of work with syslog and am having the issue with the included syslogd not sending the pfSense system's hostname in the syslog packet so am working to get rsyslogd running.

    Please, any feedback (even to tell me I'm crazy) is appreciated (especially if i'm posting in the wrong forum). This thread is a WIP that I will update as I progress this into an installable package. These instructions are a loose set of suggestions of the commands to run ONLY.

    ISSUES:

    • GUI logs and configuration are definitely BROKEN until I get a chance to go through the rules, match them up to the log view pages and then rewrite the configuration pages to correctly write a /etc/syslog.conf file understood by rsyslogd.
    • May not be persistent on reboot - this is a semi live system that i'm experimenting on at the moment so I haven't been able to see how it handles boot

    WORKING:

    • rsyslogd running and accepting system, firewall and Squid logs as well as forwarding to remote syslog-ng host (including sending correct hostname)

    1. Get console access
    2. Install packages:

    
    pkg_add -i http://ftpmirror.your.org/pub/FreeBSD-Unofficial-Packages/81amd64-default/Latest/rsyslog.tbz
    pkg_add -i http://ftpmirror.your.org/pub/FreeBSD-Unofficial-Packages/81amd64-default/Latest/json-c.tbz
    pkg_add -i http://ftpmirror.your.org/pub/FreeBSD-Unofficial-Packages/81amd64-default/Latest/libee.tbz
    pkg_add -i http://ftpmirror.your.org/pub/FreeBSD-Unofficial-Packages/81amd64-default/Latest/libestr.tbz
    pkg_add -i http://ftpmirror.your.org/pub/FreeBSD-Unofficial-Packages/81amd64-default/Latest/libsysinfo.tbz
    pkg_add -i http://ftpmirror.your.org/pub/FreeBSD-Unofficial-Packages/81amd64-default/Latest/e2fsprogs-libuuid.tbz
    
    

    3. Modify /usr/local/etc/rc.d/rsyslogd with:

    # PROVIDE: rsyslogd
    # REQUIRE: mountcritremote cleanvar newsyslog ldconfig
    # BEFORE:  SERVERS
    
    . /etc/rc.subr
    
    name=rsyslogd
    rcvar=rsyslogd_enable
    command="/usr/local/sbin/${name}"
    load_rc_config $name
    : ${rsyslogd_enable:="YES"}: ${rsyslogd_pidfile:="/var/run/syslog.pid"}
    : ${rsyslogd_config:="/etc/syslog.conf"}
    pidfile="${rsyslogd_pidfile}"
    command_args="-i ${pidfile} -f ${rsyslogd_config}"
    required_files="${rsyslogd_config}"
    extra_commands="reload"
    
    run_rc_command "$1"
    
    

    4. Overwrite the contents of /etc/syslog.conf with:

    
    $ModLoad immark.so   # provides --MARK-- message capability
    $ModLoad imuxsock.so # provides support for local system logging
    $ModLoad imklog.so   # kernel logging
    
    # Specify forwarding host here. If none, please delete this line
    *.*   @@127.0.0.1
    # if you need to forward to other systems as well, just
    
    # Log anything (except mail) of level info or higher.
    
    # Don't log private authentication messages!
    
    *.info;mail.none;authpriv.none;cron.none      /var/log/messages
    
    # The authpriv file has restricted access.
    
    authpriv.*                                    /var/log/secure
    
    # Log all the mail messages in one place.
    
    mail.*                                        /var/log/maillog
    
    # Log cron stuff
    
    cron.*                                        /var/log/cron
    
    # Everybody gets emergency messages
    
    *.emerg                                       *
    
    # Save news errors of level crit and higher in a special file.
    
    uucp,news.crit                                /var/log/spooler
    
    # Save boot messages also to boot.log
    
    local7.*                                      /var/log/boot.log
    
    

    5. Stop old syslog daemon: /etc/rc.d/syslogd stop
    6. Disable / enable daemons: /etc/rc.conf

    
    syslogd_enable="NO"
    rsyslogd_enable="YES"
    
    

  • Moderator

    Hi Navillus,

    Have you tried this patch to get pfSense syslog to output into one line?

    https://forum.pfsense.org/index.php/topic,69544.msg394631.html#msg394631



  • I had a look at that but it appears to only change the source address. I want the actual hostname sent in the syslog packet. From my research it seems to be a limitation with *BSD's syslogd implementation.

    https://groups.google.com/forum/#!topic/fa.openbsd.tech/Owd2TmxmFz0 <- see first post

    https://forums.freebsd.org/viewtopic.php?&t=41703


  • Moderator

    I checked my installation and its only sending the Host IP address to ELSA.

    Is there a particular reason why you need the hostname. The Routers are usually a Static address anyways? Are you using Base?



  • Yep that's correct, there are no issues with syslogd sending an IP address and / or letting the far end resolve it.

    My particular requirements are to monitor many pfSense instances deployed on ADSL / 3G connections which all have dynamic IPs and I need to be able to have a consistent identifier for each box. It's not feasible time-wise to configure separate monitoring ports on my collector for each far end device hence why I need hostname identification - the collector can read the hostname then put the log item into the correct table.

    Please excuse my lack of knowledge on this - this is my first time doing pfSense core development - how do you mean "Base"? If you're referring to BSD version / pfSense edition, I'm running stock, non-dev 2.1:

    
    2.1-RELEASE (amd64) 
    built on Wed Sep 11 18:17:48 EDT 2013 
    FreeBSD 8.3-RELEASE-p11
    
    

  • Moderator

    I read in the Forum that you had referenced in your previous links, where they were talking about Base. Look like your not using Base as your collector. Not sure if it even has LOG management functionality?      http://base.professionallyevil.com/

    I am using "Security Onion" which has ELSA as its Log Management program. I have 5 pfSense Routers all pointing its logs to one listening address and port. This all happens on the LAN side and as each pfSense LAN ip is unique, ELSA can pivot on any information relevant to a particular pfSense Box.

    From your previous email it looks like your sending your logs thru the WAN side (ADSL / 3G connections which all have dynamic IPs) . If that is the case, you should look at creating a VPN tunnel from each pfSense box and sending syslogs thru a more protected environment.  This will also give you the static LAN address for your logs.



  • Ah, apologies for not making what I'm doing clearer. Basically the collector is an instance of syslog-ng with scripting and filtering to place the logs into database tables basically providing a queryable profile of each device. These devices are on VERY marginal internet connections generally with up to 20% packet loss considered normal. Keeping a VPN up and running is practically impossible and the timeouts seem to do strange things to Squid's syslog module hence why it needs to be all kept very simple with UDP packets.
    Security isn't a HUGE concern here as the only thing being monitored is HTTP traffic on what are essentially public wifi networks (what the pfSense boxes are serving)



  • I follow your post to install rsyslog on pfsense .
    But while I am restarting my machine the /etc/syslog.conf file restored to previous file that is one before installation of rsyslog.