Capturing IP of every passed packet.



  • Hi

    I would like to track every IP address that manages to pass a packet into my network.
    I'm not interested in the individual packets although the protocol would be useful.

    I've found this posting which asks a very similar question:

    https://forum.pfsense.org/index.php?topic=47865.msg252055#msg252055

    I'm already logging to syslog but don't really understand the flags and messages enough to extract what I want and I doubt I'm actually logging what I need to. Some lines say:  'passed in' some say 'blocked' but many don't indicate either to me.

    So my problem is two fold:

    1. I don't know how to get this data from psense (what to turn on and how to extract the specific data).
    2. I don't understand the data ie flags and states enough to be sure I'm not including or excluding the wrong things pre or post dump.

    I'm thinking a continuous dump of the state table would give me what I want, because is easily understood.
    But I don't know how to set that up or whether that's the best approach.

    If I could extract the data I can obviously grep for 'established' but I don't know if 'established' is all I'm looking for.
    I don't know if some packets pass through without an 'established' state (icmp)?

    I'm using pfsense 2.1.

    Any advise greatly appreciated.

    Thanks in advance

    Charlie101

    Update:

    OK so the only progress I've made so far is to switch packet logging on for one of the internal interfaces.
    This only gives me packets reaching that interface but that is good enough at the moment.  I can now grep out the IPs and remove the duplicates. It's not very elegants as its being done at the packet level rather than the connection level.
    I Still can't see how to do that.

    Charlie101



  • You've already done everything I could think of.  Why do you need to log every incoming packet?



  • Hi Kom

    Thanks for replying.

    I don't need to log any packets per se. I want the IP addresses not any of the packet details.

    I have a Drupal site and I want to analyse the acccess logs. OK, so I can do this with SQL but things are just made harder by the amount of rubbish coming from various bots etc. And of course there are the security issues associated with all this probing traffic.

    I only trade in the uK so I have started blocking non uk subnets. I have automated this by collecting the IPs which access my drupal site, doing a whois on them and if they are outside the uK I run a script to find the largest subnet I can safely block and add that to my block list.

    This works fine but it only deals with 'visitors' accessing drupal ie port 80 on one of my machines. Lots of probing is going on against other  machines and other ports and  if they don't touch my drupal site they wont make it onto my block list.
    Hence I want to generalize this process for any 'visitor' accessing anything on my network.

    Hence I need to collect all IP's managing to traverse my router.

    I have pfblocker installed but I find the country block lists are not accurate enough for me and don't give me the option to allow certain subnets through. Maybe I don't know enough about pfblocker. Althouigh I am using it for my aliase list and for regularly retrieving my updated block list.

    So what I need is an easy way of obtain that IP list or precise rules for isolating that data from the syslog feed.

    Regards

    Charlie101


  • LAYER 8 Netgate

    Depends on how much data it is.  Enabling logging on our pass rules will get the list of IPs to your syslog server.  Then it's a matter of parsing the data.  If this is a temporary project or it's just in the lab and you can get by with the limitations of the free version, look at www.splunk.com.



  • Hi  Derelict

    Thanks for replying

    "Enabling logging on our pass rules" That sounds exactly like what I need.

    This is a live project but I'm a self employeed Database Training/Consultant offering live training over the web.
    The site isn't making any real money as yet so I'm forced down the D.I.Y road.

    How do I enable logging on the pass rules? Is it in the GUI or do I need to do it in the shell?

    What am I looking at on the Splunk site?

    Thanks

    Charlie101


  • LAYER 8 Netgate

    You just enable logging on the pass rules, probably on WAN.  You need to be sending your logs to a syslog server (or splunk, etc).  They'll wrap pretty quickly in pfSense.

    Splunk is a tool to help you make sense of your data.  I'm just pointing you in that direction.  There are also products like SolarWinds LEM that try to do the same thing.  Using it to get what you want out of it is probably best directed at a splunk support forum.

    ![Screen Shot 2014-09-07 at 4.08.33 PM.png](/public/imported_attachments/1/Screen Shot 2014-09-07 at 4.08.33 PM.png)
    ![Screen Shot 2014-09-07 at 4.08.33 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2014-09-07 at 4.08.33 PM.png_thumb)



  • Hi Derelict

    I've checked and I've already enabled logging on my pfblocker rules. So my stream of data via syslog most already contain that information. I can't say it's overly clear to me. as I'm getting  lines denoting the access details followed by lines stating a 'pass in' or a 'match/block'.

    However they are not always adjacent, there is sometime a few seconds between the timestamps and I can't find a shared identifier to link each access line with its corresponding 'action' line.

    However, the whole process is a lot clearer so thanks for your help.
    I'll spend some more time on the syslog data and see if I can't figure it out. I'll also see what I can garner from splunk.

    Thanks again.

    Regards

    Charlie101


  • Moderator

    Here is a thread about remote syslogs also:

    https://forum.pfsense.org/index.php?topic=80154.msg438558#msg438558



  • Hi BBcan177

    Thanks for the link. It looks really useful.

    Regards

    Charlie101.


Log in to reply