NAT Logs

michmoor

@kprovost Copy that. Ok i'll make adjustments to the collector.

michmoor

@kprovost @stephenw10

Can confirm i am seeing the additional fields using IPFix.

mcury

@michmoor said in NAT Logs:

@kprovost @stephenw10

Can confirm i am seeing the additional fields using IPFix.

According to RFC 8158, the fields you need are:

postNATSourceIPv4Address
postNAPTSourceTransportPort
postNATDestinationIPv4Address
postNAPTDestinationTransportPort

I'm also playing with it right now

michmoor

@mcury Let me know what you come up with in Graylog.

Its parsed automatically which is great but not sure how to utilize this data. Reporting?

mcury

@michmoor said in NAT Logs:

@mcury Let me know what you come up with in Graylog.

Its parsed automatically which is great but not sure how to utilize this data. Reporting?

pfSense is sending the data correctly, I'll try to create an extractor for those fields..

michmoor

@mcury

Its already parsed for me

mcury

@michmoor

So you got it.
Weird, I don't have those here.. Any tip of what I should do ?

michmoor

@mcury Graylog did it for me. For the input i selected IPFIX UDP

mcury

@michmoor said in NAT Logs:

@mcury Graylog did it for me. For the input i selected IPFIX UDP

Also using that input.

Did you enable tracking on the firewall rule that is associated with the NAT rule ?

Ohh, disregard that.
It is working here..

michmoor

@mcury The only piece i dont know how to do is actually display data to show top talkers.
each IPfix record shows bytes which is great but i would like a table that shows all the fields plus the amount of data seen cumulative.

mcury

@michmoor said in NAT Logs:

The only piece i dont know how to do is actually display data to show top talkers.
each IPfix record shows bytes which is great but i would like a table that shows all the fields plus the amount of data seen cumulative.

hmm, as I see it, you would nee to create a widget with the search field:

Example if the destination host is in the 192.168.255.0/24 subnet.
postNATSourceIPv4Address:192.168.255.* OR postNATDestinationIPv4Address:192.168.255.*

Then, as per picture above, filter by destination or source (this will give you the external users data), use SUM and Data Table.

michmoor

@mcury Ok yeah thats the way to go BUT....i have to use NetFlow as netflow already has bytes parsed in the output while IPFix does not.

Still figuring out how to normalize the nf_bytes column to megabytes

mcury

@michmoor said in NAT Logs:

Still figuring out how to normalize the nf_bytes column to megabytes

Try this pipeline rule:

rule "arithmetic"
when
has_field("octetDeltaCount")
then
let size_kb = to_double($message.octetDeltaCount);
set_field("size_bytes", size_kb / to_double(value:"1048576"));
end

This will give you something like this:

michmoor

@mcury Not sure if its working to be honest.
There are no errors showing up in the pipeline so messages are being processed.

Message:

mcury

@michmoor said in NAT Logs:

@mcury Not sure if its working to be honest.
There are no errors showing up in the pipeline so messages are being processed.

Did you edit the pipeline connections and assigned to the ipfix stream ?

michmoor

@mcury
Im on Graylog 6.0.5

Yes my pipeline is connected to my stream. Im using Netflow data just for testing.

mcury

@michmoor I'm on Graylog 6.1.5, but it should work on 6.0.5, no problems with that.

Is Configurations tab, Manage processors, is pipeline active ?

michmoor

@mcury
Yep its active.
I have other pipelines in effect so this is all working. The new pipeline for Netflow is the latest addition. I guess I'm trying to figure out how to prove its working
With my other pipelines, I'm extracting fields into Grok Patterns so i can tell immediately if its working.

Considering i see no errors in the pipeline i have to imagine its working.

mcury

@michmoor said in NAT Logs:

Considering i see no errors in the pipeline i have to imagine its working.

But do you see messages being processed in the Throughput ?

michmoor

@mcury