Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    A System To Continuously Log Traffic

    Scheduled Pinned Locked Moved Off-Topic & Non-Support Discussion
    5 Posts 2 Posters 3.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      focalguy
      last edited by

      First let me say I'm not really looking to do this on the pfSense box and that's why I put this in the General Discussion. I am hoping to find an open source application that I can put on another box and then connect to a port mirroring port on the switch. The purpose is to keep a record of all traffic for about a week in case I want to go back and open the packets in Wireshark to examine what was going on.

      I just want something I can leave alone and let it capture packets continuously (in a loop for a set number of days) until I want to use it. Any ideas?

      1 Reply Last reply Reply Quote 0
      • Cry HavokC
        Cry Havok
        last edited by

        Off the shelf, and free, no.  However there are a number of open source packages for performing packet capture (tcpdump and tshark are the best known) and ISTR that at least some of them support rolling files over based upon size or duration.  It would be pretty easy to then stick in a cron job to remove files older then X days and let it rip.

        1 Reply Last reply Reply Quote 0
        • F
          focalguy
          last edited by

          I didn't know about tshark. Yes, it does support creating new files after the file reaches a certain size. I didn't think of a cron job for just deleting the old files. That sounds simple enough and maybe that's why there is no dedicated application for this. Thanks for the ideas!

          1 Reply Last reply Reply Quote 0
          • Cry HavokC
            Cry Havok
            last edited by

            NP.

            If you want to get really fancy you could create a script that's run from cron that looks for a certain amount of free space in the (dedicated) partition and removes the oldest files to bring it down below that threshold.  I'd probably do it something like:

            #!/path/to/bash
            SPACE=1024          # MB of space to have free in...
            PARTITION=/pcap/    # The storage partition
            CHUNK=10            # Process this many files at a time
            
            LEFT=`df -m ${PARTITION} | awk '{ print $4 }'`
            while [ ${LEFT} -lt ${SPACE} ]; then
                ls -ltr ${PARTITION} | egrep "^-" | head -n ${CHUNK} | awk '{ print $NF }' | while read FILE
                    do
                    rm ${FILE}
                    LEFT=`df -m ${PARTITION} | awk '{ print $4 }'`
                    if [ ${LEFT} -ge ${SPACE} ]; then
                        break 2
                    fi
                done
            done
            
            

            This, if I've got my logic right, will check to see that there is enough free space.  While there isn't it'll take the oldest ${CHUNK} files and iterate over them until either there's enough space or it's deleted all 10 files, at which point it'll move on to the next 10.  Overkill, but hey ;)

            To work out the value of ${SPACE} you need to work out the maximum packet capture in the time window between runs of the script, and add a reasonable fudge factor (I'd go for a simple double or triple the original value).  That way you make the best use of your storage space and rather than having a fixed window you have a variable window that adjusts according to your traffic volumes.

            1 Reply Last reply Reply Quote 0
            • F
              focalguy
              last edited by

              That looks great! I'm not very experienced with scripting but I think I understand most of your script there. I'm not sure when I'll get a chance to implement this but when I do I'll try out the script and report back what I find. Thanks again for the help!

              1 Reply Last reply Reply Quote 0
              • First post
                Last post
              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.