A System To Continuously Log Traffic

[focalguy]

First let me say I'm not really looking to do this on the pfSense box and that's why I put this in the General Discussion. I am hoping to find an open source application that I can put on another box and then connect to a port mirroring port on the switch. The purpose is to keep a record of all traffic for about a week in case I want to go back and open the packets in Wireshark to examine what was going on.

I just want something I can leave alone and let it capture packets continuously (in a loop for a set number of days) until I want to use it. Any ideas?

[Cry Havok]

Off the shelf, and free, no.  However there are a number of open source packages for performing packet capture (tcpdump and tshark are the best known) and ISTR that at least some of them support rolling files over based upon size or duration.  It would be pretty easy to then stick in a cron job to remove files older then X days and let it rip.

[focalguy]

I didn't know about tshark. Yes, it does support creating new files after the file reaches a certain size. I didn't think of a cron job for just deleting the old files. That sounds simple enough and maybe that's why there is no dedicated application for this. Thanks for the ideas!

[Cry Havok]

NP.

If you want to get really fancy you could create a script that's run from cron that looks for a certain amount of free space in the (dedicated) partition and removes the oldest files to bring it down below that threshold.  I'd probably do it something like:

#!/path/to/bash
SPACE=1024          # MB of space to have free in...
PARTITION=/pcap/    # The storage partition
CHUNK=10            # Process this many files at a time

LEFT=`df -m ${PARTITION} | awk '{ print $4 }'`
while [ ${LEFT} -lt ${SPACE} ]; then
    ls -ltr ${PARTITION} | egrep "^-" | head -n ${CHUNK} | awk '{ print $NF }' | while read FILE
        do
        rm ${FILE}
        LEFT=`df -m ${PARTITION} | awk '{ print $4 }'`
        if [ ${LEFT} -ge ${SPACE} ]; then
            break 2
        fi
    done
done

This, if I've got my logic right, will check to see that there is enough free space.  While there isn't it'll take the oldest ${CHUNK} files and iterate over them until either there's enough space or it's deleted all 10 files, at which point it'll move on to the next 10.  Overkill, but hey ;)

To work out the value of ${SPACE} you need to work out the maximum packet capture in the time window between runs of the script, and add a reasonable fudge factor (I'd go for a simple double or triple the original value).  That way you make the best use of your storage space and rather than having a fixed window you have a variable window that adjusts according to your traffic volumes.

[focalguy]

That looks great! I'm not very experienced with scripting but I think I understand most of your script there. I'm not sure when I'll get a chance to implement this but when I do I'll try out the script and report back what I find. Thanks again for the help!