Pfflowd and duplicate flow records

[wallabybob]

Though pfflowd is involved I have posted this topic in this forum since the possibility of duplicate flow records being passed to pfflowd may be of more general interest, particularly to those people who have reported significant anomolies in various places where byte counts are reported.

I use pfflowd to send flow records to a Linux system on which I use the flow-tools package to capture and analyse those records. Some time ago I noticed duplicate flow records. I'll give an example later. Thinking it might be a quirk of pfflowd I went and looked at the options available through the web GUI and saw pf rule direction restriction - Restrict creation of flow records to states matching a certain direction (in, out, or any). and noted it was set to Any. The explanation wasn't particularly helpful to understanding what setting I should choose nor was the corresponding FreeBSD man page so I decided to change to in which seemed to stop the duplicate records. However big downloads did not show up in the records. In particular, a 12MB download from 72.21.194.22 (from states reported by pftop) seemed to have been lost from the records (in the following reports flow-cat concatenates a number of flow files, removing file headers and flow-nfilter selects specified flow record, in these examples selecting a specific IP address and flow-print displays the selected flow records):

flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=72.21.194.22 | flow-print -f 1

Sif  SrcIPaddress    DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
StartTime          EndTime            Active  B/Pk Ts Fl

000c 192.168.211.241  000a 72.21.194.22      06 d6b6 50    4          180     
0317.07:24:02.498  0317.07:25:43.498    101.000 45  00 00

flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=72.21.194.22 | flow-print -f 1

Sif  SrcIPaddress    DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
StartTime          EndTime            Active  B/Pk Ts Fl

000a 72.21.194.22    000c 192.168.211.241  06 50  d6b6  2          92       
0317.07:24:02.498  0317.07:25:43.498    101.000 46  00 00

I changed the pfflowd rule direction restriction to Out and downloaded a (different) 12MB file from 63.173.70.10 (again, IP address taken from state shown by pftop) and this time saw:

flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=63.173.70.10 | flow-print -f 1

Sif  SrcIPaddress    DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
StartTime          EndTime            Active  B/Pk Ts Fl

000c 192.168.211.241  000a 63.173.70.10      06 b052 50    5843      314821   
0317.07:42:18.614  0317.07:48:14.614    356.000 53  00 00

flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=63.173.70.10 | flow-print -f 1

Sif  SrcIPaddress    DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
StartTime          EndTime            Active  B/Pk Ts Fl

000a 63.173.70.10    000c 192.168.211.241  06 50  b052  9933      14410509 
0317.07:42:18.614  0317.07:48:14.614    356.000 1450 00 00

Next I changed the pfflowd rule direction restriction to Any and downloaded the same file (though this time pftop reported it coming from 80.239.224.51) and saw duplicate flow records:

flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-dst-addr -v ADDR=80.239.224.51 | flow-print -f 1

Sif  SrcIPaddress    DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
StartTime          EndTime            Active  B/Pk Ts Fl

000c 192.168.211.241  000a 80.239.224.51    06 8cc4 50    5864      314565   
0317.08:02:48.917  0317.08:10:53.917    485.000 53  00 00

000c 192.168.211.241  000a 80.239.224.51    06 8cc4 50    5864      314565   
0317.08:02:48.917  0317.08:10:53.917    485.000 53  00 00

flow-cat ft-v05.2012-03-17.* | flow-nfilter -F ip-src-addr -v ADDR=80.239.224.51 | flow-print -f 1

Sif  SrcIPaddress    DIf  DstIPaddress      Pr SrcP DstP  Pkts  Octets
StartTime          EndTime            Active  B/Pk Ts Fl

000a 80.239.224.51    000c 192.168.211.241  06 50  8cc4  9949      14440741 
0317.08:02:48.917  0317.08:10:53.917    485.000 1451 00 00

000a 80.239.224.51    000c 192.168.211.241  06 50  8cc4  9949      14440741 
0317.08:02:48.917  0317.08:10:53.917    485.000 1451 00 00

1. What is the meaning of each of the settings of the pfflowd rule direction restriction? (It seems to need a fair bit of knowledge of firewall rule generation hidden by the web GUI and the kernel pf component to know the meaning of the different settings.)
2. What is the correct setting of of the pfflowd rule direction restriction to record at least the flows through the WAN interface? (Any seems to mean more than both in and out).
3. Does the apparent duplicate flow reporting have any implications for other packages interested in byte usage? (I don't know if pfflowd is generating the duplicate or its just passing on a duplicate reported by the kernel.)