Grok Filter OpenVPN and Snort for logstash with Json Dashboard

[killmasta93]

Hi,
I Have been trying for many months and no luck on getting this to work to filter the logs from pfSense and Send them to ELK. I am not sure what the price could be so message me and we can negotiate the price, The payment will be though paypal.

Thank you

[killmasta93]

BUMP?

[Noebas]

I have this running with this dashboard http://imgur.com/5gaJ7ZY

Showing failed logins and geo locations of logins. But one could build any dashboard once the data is in.
If this would be the thing you need i am willing to share, do not have to have anything for it just a thank you.

[Noebas]

I do use Kibana 4, i would upgrade to it.

Do you use syslog? It fist strips the date and time from the rest, changes it to the correct kibana time stamp. Then i filter the filter log and openvpn.
I am working on the 2.3 gateway log. The Filter log you need the patten for and for geo you need the geo datafile

This is My code:

filter { 
#Date time translation
  if [program] == "syslog" {
    grok {
      match => [ "message", "(?<datetime>(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?<prog>.?): (?<msg>.)" ]
    }
    mutate {
      gsub => ["datetime","  "," "]
    }
    date {
      match => [ "datetime", "MMM dd HH:mm:ss" ]
    }
    mutate {
      replace => [ "message", "%{msg}" ]
    }
    mutate {
      remove_field => [ "msg", "datetime" ]
    }
}
#Filterlog
if "filterlog" in [prog] { 
    grok {
      patterns_dir => "/etc/logstash/conf.d/patterns"
      match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}",
        "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ]
    }
    mutate {
      lowercase => [ 'proto' ]
    }
  }
#OpenVPN
if "openvpn" in [prog] {
  grok {
    match => [ "message", "user '%{WORD:openvpn_user}'" ]
    match => [ "message", "%{WORD:openvpn_user}/%{IP:openvpn_scr_ip}:%{INT:openvpn_scr_port} MULTI_sva: pool returned IPv4=%{IP:openvpn_ip}" ]
  }
  #GEO DATA
    geoip {
    source => "openvpn_scr_ip"
    database => "/etc/logstash/GeoLiteCity.dat"
      }
}
}</msg></prog></datetime>