Pfflowd and duplicate flow records
-
Though pfflowd is involved I have posted this topic in this forum since the possibility of duplicate flow records being passed to pfflowd may be of more general interest, particularly to those people who have reported significant anomolies in various places where byte counts are reported.
I use pfflowd to send flow records to a Linux system on which I use the flow-tools package to capture and analyse those records. Some time ago I noticed duplicate flow records. I'll give an example later. Thinking it might be a quirk of pfflowd I went and looked at the options available through the web GUI and saw pf rule direction restriction - Restrict creation of flow records to states matching a certain direction (in, out, or any). and noted it was set to Any. The explanation wasn't particularly helpful to understanding what setting I should choose nor was the corresponding FreeBSD man page so I decided to change to in which seemed to stop the duplicate records. However big downloads did not show up in the records. In particular, a 12MB download from 72.21.194.22 (from states reported by pftop) seemed to have been lost from the records (in the following reports flow-cat concatenates a number of flow files, removing file headers and flow-nfilter selects specified flow record, in these examples selecting a specific IP address and flow-print displays the selected flow records):
flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=72.21.194.22 | flow-print -f 1
Sif SrcIPaddress DIf DstIPaddress Pr SrcP DstP Pkts Octets
StartTime EndTime Active B/Pk Ts Fl000c 192.168.211.241 000a 72.21.194.22 06 d6b6 50 4 180
0317.07:24:02.498 0317.07:25:43.498 101.000 45 00 00flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=72.21.194.22 | flow-print -f 1
Sif SrcIPaddress DIf DstIPaddress Pr SrcP DstP Pkts Octets
StartTime EndTime Active B/Pk Ts Fl000a 72.21.194.22 000c 192.168.211.241 06 50 d6b6 2 92
0317.07:24:02.498 0317.07:25:43.498 101.000 46 00 00I changed the pfflowd rule direction restriction to Out and downloaded a (different) 12MB file from 63.173.70.10 (again, IP address taken from state shown by pftop) and this time saw:
flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=63.173.70.10 | flow-print -f 1
Sif SrcIPaddress DIf DstIPaddress Pr SrcP DstP Pkts Octets
StartTime EndTime Active B/Pk Ts Fl000c 192.168.211.241 000a 63.173.70.10 06 b052 50 5843 314821
0317.07:42:18.614 0317.07:48:14.614 356.000 53 00 00flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=63.173.70.10 | flow-print -f 1
Sif SrcIPaddress DIf DstIPaddress Pr SrcP DstP Pkts Octets
StartTime EndTime Active B/Pk Ts Fl000a 63.173.70.10 000c 192.168.211.241 06 50 b052 9933 14410509
0317.07:42:18.614 0317.07:48:14.614 356.000 1450 00 00Next I changed the pfflowd rule direction restriction to Any and downloaded the same file (though this time pftop reported it coming from 80.239.224.51) and saw duplicate flow records:
flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=80.239.224.51 | flow-print -f 1
Sif SrcIPaddress DIf DstIPaddress Pr SrcP DstP Pkts Octets
StartTime EndTime Active B/Pk Ts Fl000c 192.168.211.241 000a 80.239.224.51 06 8cc4 50 5864 314565
0317.08:02:48.917 0317.08:10:53.917 485.000 53 00 00000c 192.168.211.241 000a 80.239.224.51 06 8cc4 50 5864 314565
0317.08:02:48.917 0317.08:10:53.917 485.000 53 00 00flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=80.239.224.51 | flow-print -f 1
Sif SrcIPaddress DIf DstIPaddress Pr SrcP DstP Pkts Octets
StartTime EndTime Active B/Pk Ts Fl000a 80.239.224.51 000c 192.168.211.241 06 50 8cc4 9949 14440741
0317.08:02:48.917 0317.08:10:53.917 485.000 1451 00 00000a 80.239.224.51 000c 192.168.211.241 06 50 8cc4 9949 14440741
0317.08:02:48.917 0317.08:10:53.917 485.000 1451 00 001. What is the meaning of each of the settings of the pfflowd rule direction restriction? (It seems to need a fair bit of knowledge of firewall rule generation hidden by the web GUI and the kernel pf component to know the meaning of the different settings.)
2. What is the correct setting of of the pfflowd rule direction restriction to record at least the flows through the WAN interface? (Any seems to mean more than both in and out).
3. Does the apparent duplicate flow reporting have any implications for other packages interested in byte usage? (I don't know if pfflowd is generating the duplicate or its just passing on a duplicate reported by the kernel.)