PBR gets wrong after filter reload



  • The later snapshots are pretty stable – the state table survives many days without reset. When pf reloads, however, it does it incorrectly that some rules no longer apply, which sometimes causes troubles.

    For example, I'm testing pfsense in the following config:

    LANs –-- router ----- pfsense ----- primary Internet link (OPT3 = em3)
                               |
                               +------- 2 secondary Internet links (OPT1 = em4, OPT2 = em5)

    The router is used to monitor pfsense and to fail-over Internet traffic to other (tertiary) links. It checks pfsense-to-Internet connectivity by ping-ing a well-known server, namely 8.8.8.8. The pfsense is configured (see rule @101) to forward the ping via a gateway group with:
    – OPT3 as primary link (Tier 1)
    -- OPT1 and OPT2 as secondary links (Tier 2).

    When all the three links are up rules apply normally. When OPT3 is down, however, pfsense stops forwarding the ping to Internet, the router signifies total loss of Internet connectivity, and statistics of rule @51 through @55 show zeros.

    The following is my complete rule list.

    When all links up:

    @0 scrub in on em0 all max-mss 1460 fragment reassemble
      [ Evaluations: 1636269   Packets: 1232      Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @1 scrub in on em1 all max-mss 1460 fragment reassemble
      [ Evaluations: 1481685   Packets: 405138    Bytes: 93240273    States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @2 scrub in on em4 all max-mss 1460 fragment reassemble
      [ Evaluations: 720530    Packets: 38165     Bytes: 1453694     States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @3 scrub in on em5 all max-mss 1460 fragment reassemble
      [ Evaluations: 637314    Packets: 113811    Bytes: 1830960     States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @4 scrub in on em3 all max-mss 1460 fragment reassemble
      [ Evaluations: 395591    Packets: 177534    Bytes: 39857361    States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @0 anchor "relayd/*" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @1 anchor "firewallrules" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @2 block drop in all label "Default deny rule"
      [ Evaluations: 136099    Packets: 425       Bytes: 140538      States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @3 block drop out all label "Default deny rule"
      [ Evaluations: 136099    Packets: 31        Bytes: 8494        States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @4 block drop in quick inet6 all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @5 block drop out quick inet6 all
      [ Evaluations: 67851     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @6 block drop quick proto tcp from any port = 0 to any
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @7 block drop quick proto tcp from any to any port = 0
      [ Evaluations: 3580      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @8 block drop quick proto udp from any port = 0 to any
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @9 block drop quick proto udp from any to any port = 0
      [ Evaluations: 132512    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @10 block drop quick from <snort2c:0> to any label "Block snort2c hosts"
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @11 block drop quick from any to <snort2c:0> label "Block snort2c hosts"
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @12 anchor "packageearly" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @13 anchor "carp" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @14 block drop in log quick proto tcp from <sshlockout:0> to any port = ssh label "sshlockout"
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @15 block drop in quick from <virusprot:0> to any label "virusprot overload table"
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @16 block drop in on ! em0 inet from 10.0.0.0/24 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @17 block drop in inet from 10.0.0.3 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @18 block drop in on ! em1 inet from 192.168.0.72/29 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @19 block drop in inet from 192.168.0.74 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @20 block drop in on em0 inet6 from fe80::20c:29ff:fe45:2054 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @21 block drop in on em1 inet6 from fe80::20c:29ff:fe45:205e to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @22 anchor "dhcpserverLAN" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @23 pass in on em1 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @24 pass in on em1 inet proto udp from any port = bootpc to 192.168.0.74 port = bootps keep state label "allow access to DHCP server"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @25 pass out on em1 inet proto udp from 192.168.0.74 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
      [ Evaluations: 66300     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @26 block drop in on ! em4 inet from 192.168.0.64/30 to any
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @27 block drop in inet from 192.168.0.66 to any
      [ Evaluations: 68773     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @28 block drop in on ! em5 inet from 192.168.0.68/30 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @29 block drop in inet from 192.168.0.70 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @30 block drop in on ! em3 inet from 192.168.0.80/29 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @31 block drop in inet from 192.168.0.82 to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @32 block drop in on em4 inet6 from fe80::20c:29ff:fe45:207c to any
      [ Evaluations: 68248     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @33 block drop in on em5 inet6 from fe80::20c:29ff:fe45:2086 to any
      [ Evaluations: 47865     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @34 block drop in on em3 inet6 from fe80::20c:29ff:fe45:2072 to any
      [ Evaluations: 28607     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @35 anchor "spoofing" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @36 anchor "loopback" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @37 pass in on lo0 all flags S/SA keep state label "pass loopback"
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @38 pass out on lo0 all flags S/SA keep state label "pass loopback"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @39 anchor "firewallout" all
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @40 pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @41 pass out route-to (em0 10.0.0.2) inet from 10.0.0.3 to ! 10.0.0.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 67851     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @42 pass out route-to (em1 192.168.0.75) inet from 192.168.0.74 to ! 192.168.0.72/29 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 67851     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @43 pass out route-to (em4 192.168.0.65) inet from 192.168.0.66 to ! 192.168.0.64/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 67851     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @44 pass out route-to (em5 192.168.0.69) inet from 192.168.0.70 to ! 192.168.0.68/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 67851     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @45 pass out route-to (em3 192.168.0.81) inet from 192.168.0.82 to ! 192.168.0.80/29 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 67851     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @46 pass in log quick on em0 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 136099    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @47 pass in log quick on em1 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 136094    Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @48 pass in log quick on em4 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 68047     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @49 pass in log quick on em5 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 47139     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @50 pass in log quick on em3 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 27356     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @51 pass out quick on em0 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 67852     Packets: 1766      Bytes: 113024      States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @52 pass out quick on em1 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 67847     Packets: 1766      Bytes: 113024      States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @53 pass out quick on em4 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 6527      Packets: 1766      Bytes: 113024      States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @54 pass out quick on em5 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 6002      Packets: 1766      Bytes: 113024      States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @55 pass out quick on em3 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 5477      Packets: 2209      Bytes: 139592      States: 2     ]
      [ Inserted: uid 0 pid 58704 ]
    @56 pass out all flags S/SA keep state label "USER_RULE: Penalty Box" queue qP2P
      [ Evaluations: 136092    Packets: 154881    Bytes: 105236038   States: 142   ]
      [ Inserted: uid 0 pid 58704 ]
    @57 pass out proto tcp from any to any port = 3389 flags S/SA keep state label "USER_RULE: m_Other MSRDP outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 67845     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @58 pass out proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: m_Other HTTP outbound" queue(qOthersDefault, qACK)
      [ Evaluations: 1587      Packets: 43676     Bytes: 35049807    States: 3     ]
      [ Inserted: uid 0 pid 58704 ]
    @59 pass out proto tcp from any to any port = https flags S/SA keep state label "USER_RULE: m_Other HTTPS outbound" queue(qOthersDefault, qACK)
      [ Evaluations: 1587      Packets: 1073      Bytes: 527454      States: 2     ]
      [ Inserted: uid 0 pid 58704 ]
    @60 pass out proto tcp from any to any port = smtp flags S/SA keep state label "USER_RULE: m_Other SMTP outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 131       Bytes: 15390       States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @61 pass out proto tcp from any to any port = smtps flags S/SA keep state label "USER_RULE: m_Other SMTP/S outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @62 pass out proto tcp from any to any port = pop3 flags S/SA keep state label "USER_RULE: m_Other POP3 outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @63 pass out proto tcp from any to any port = pop3s flags S/SA keep state label "USER_RULE: m_Other POP3/S outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @64 pass out proto tcp from any to any port = imap flags S/SA keep state label "USER_RULE: m_Other IMAP outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 212       Bytes: 21251       States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @65 pass out proto tcp from any to any port = imaps flags S/SA keep state label "USER_RULE: m_Other IMAP/S outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @66 pass out proto tcp from any to any port = domain flags S/SA keep state label "USER_RULE: m_Other DNS1 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @67 pass out proto udp from any to any port = domain keep state label "USER_RULE: m_Other DNS2 outbound" queue qOthersHigh
      [ Evaluations: 66258     Packets: 1654      Bytes: 174891      States: 19    ]
      [ Inserted: uid 0 pid 58704 ]
    @68 pass out proto tcp from any to any port = microsoft-ds flags S/SA keep state label "USER_RULE: m_Other SMB1 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 67845     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @69 pass out proto tcp from any to any port 136 >< 140 flags S/SA keep state label "USER_RULE: m_Other SMB2 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @70 pass out proto tcp from any to any port = nntp flags S/SA keep state label "USER_RULE: m_Other NNTP1 outbound" queue(qOthersLow, qACK)
      [ Evaluations: 1587      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @71 pass out proto udp from any to any port = nntp keep state label "USER_RULE: m_Other NNTP2 outbound" queue qOthersLow
      [ Evaluations: 66258     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @72 pass out proto udp from any to any port = ntp keep state label "USER_RULE: m_Other NTP outbound" queue qVoIP
      [ Evaluations: 67845     Packets: 276789    Bytes: 24379680    States: 1690  ]
      [ Inserted: uid 0 pid 58704 ]
    @73 pass out proto tcp from any to any port = 30443 flags S/SA keep state label "USER_RULE: m_Other FW Control outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 67845     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @74 pass in quick proto tcp from any to <interfaces:8> port = 31443 flags S/SA keep state label "USER_RULE: m_Other FW Control 1 inbound" queue(qOthersHigh, qACK)
      [ Evaluations: 69835     Packets: 21533     Bytes: 19525924    States: 4     ]
      [ Inserted: uid 0 pid 58704 ]
    @75 pass out proto tcp from any to any port = 32443 flags S/SA keep state label "USER_RULE: m_Other FW Control 2 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 3569      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @76 pass in quick proto tcp from any to <interfaces:8> port = ssh flags S/SA keep state label "USER_RULE: m_Other SSH inbound" queue(qOthersHigh, qACK)
      [ Evaluations: 3569      Packets: 1162      Bytes: 153937      States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @77 pass in quick on em3 reply-to (em3 192.168.0.81) inet all flags S/SA keep state label "USER_RULE: Pass all in OPT3"
      [ Evaluations: 134494    Packets: 110390    Bytes: 23601381    States: 410   ]
      [ Inserted: uid 0 pid 58704 ]
    @78 pass in quick on em5 reply-to (em5 192.168.0.69) inet all flags S/SA keep state label "USER_RULE: Pass in on VNPT9 all"
      [ Evaluations: 108603    Packets: 247931    Bytes: 21942648    States: 678   ]
      [ Inserted: uid 0 pid 58704 ]
    @79 pass in quick on em4 reply-to (em4 192.168.0.65) inet all flags S/SA keep state label "USER_RULE: Pass in on VNPT8 all"
      [ Evaluations: 89001     Packets: 72509     Bytes: 5526584     States: 590   ]
      [ Inserted: uid 0 pid 58704 ]
    @80 pass in quick on em1 reply-to (em1 192.168.0.75) inet from any to 192.168.0.64/29 flags S/SA keep state label "USER_RULE: pass all to vnpt8-vnpt9 splitters"
      [ Evaluations: 68230     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @81 pass in quick on em1 reply-to (em1 192.168.0.75) inet from any to 192.168.0.80/29 flags S/SA keep state label "USER_RULE: pass all to viettel splitter"
      [ Evaluations: 6716      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @82 pass in quick on em1 reply-to (em1 192.168.0.75) inet proto tcp from any to any port = ftp flags S/SA keep state label "USER_RULE: pass FTP via default gateway"
      [ Evaluations: 6716      Packets: 12        Bytes: 720         States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @83 pass in log quick on em1 inet proto tcp from 192.168.12.23 to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1730      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @84 pass in log quick on em1 route-to (em4 192.168.0.65) inet proto tcp from 192.168.12.23 to any port = smtp flags S/SA keep state label "USER_RULE: mx1.intereal.vn, VNPT8 only"
      [ Evaluations: 25        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @85 pass in log quick on em1 inet proto tcp from 192.168.12.3 to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1730      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @86 pass in log quick on em1 route-to (em5 192.168.0.69) inet proto tcp from 192.168.12.3 to any port = smtp flags S/SA keep state label "USER_RULE: mail.khangthong.vn, VNPT9 only"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @87 pass in log quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1730      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @88 pass in log quick on em1 route-to (em0 10.0.0.2) inet proto tcp from any to any port = smtp flags S/SA keep state label "USER_RULE: other SMTP servers out, WAN only"
      [ Evaluations: 1730      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @89 pass in quick on em1 proto tcp from <netcservers:4> to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1730      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @90 pass in quick on em1 proto udp from <netcservers:4> to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 4982      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @91 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from <netcservers:4> to any port = domain flags S/SA keep state label "USER_RULE: critical DNS servers out, VNPT first"
      [ Evaluations: 948       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @92 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto udp from <netcservers:4> to any port = domain keep state label "USER_RULE: critical DNS servers out, VNPT first"
      [ Evaluations: 921       Packets: 1654      Bytes: 174891      States: 19    ]
      [ Inserted: uid 0 pid 58704 ]
    @93 pass in quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5890      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @94 pass in quick on em1 proto udp from any to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 4160      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @95 pass in quick on em1 route-to (em0 10.0.0.2) inet proto tcp from any to any port = domain flags S/SA keep state label "USER_RULE: other DNS clients out, WAN only"
      [ Evaluations: 5890      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @96 pass in quick on em1 route-to (em0 10.0.0.2) inet proto udp from any to any port = domain keep state label "USER_RULE: other DNS clients out, WAN only"
      [ Evaluations: 4160      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @97 pass in quick on em1 proto tcp from <netcservers:4> to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5890      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @98 pass in quick on em1 proto udp from <netcservers:4> to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 4160      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @99 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto tcp from <netcservers:4> to any port = ntp flags S/SA keep state label "USER_RULE: critical NTP clients out, VietTel first"
      [ Evaluations: 126       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @100 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto udp from <netcservers:4> to any port = ntp keep state label "USER_RULE: critical NTP clients out, VietTel first"
      [ Evaluations: 99        Packets: 182       Bytes: 13812       States: 3     ]
      [ Inserted: uid 0 pid 58704 ]
    @101 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto icmp from any to 8.8.8.8 keep state label "USER_RULE: Test Internet connectivity, VietTelfirst"
      [ Evaluations: 5792      Packets: 446       Bytes: 26760       States: 1     ]
      [ Inserted: uid 0 pid 58704 ]
    @102 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto tcp from any to <vn01:56> port = http flags S/SA keep state label "USER_RULE: HTTP domestic1 out, VietTel first"
      [ Evaluations: 5791      Packets: 30092     Bytes: 26290434    States: 3     ]
      [ Inserted: uid 0 pid 58704 ]
    @103 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto tcp from any to <vn02:76> port = http flags S/SA keep state label "USER_RULE: HTTP domestic2 out, VietTel first"
      [ Evaluations: 1410      Packets: 7955      Bytes: 5623706     States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @104 pass in quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1487      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @105 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: HTTP abroad out VNPT first"
      [ Evaluations: 1487      Packets: 5629      Bytes: 3135667     States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @106 pass in quick on em1 inet proto tcp from 192.168.0.0/20 to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1261      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @107 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto tcp from 192.168.0.0/20 to any port = mmcc flags S/SA keep state label "USER_RULE: YIM, VietTel first"
      [ Evaluations: 1246      Packets: 136       Bytes: 31598       States: 3     ]
      [ Inserted: uid 0 pid 58704 ]
    @108 pass in quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 1258      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @109 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto tcp all flags S/SA keep state label "USER_RULE: TCP out, VietTel first"
      [ Evaluations: 1258      Packets: 96066     Bytes: 66566193    States: 83    ]
      [ Inserted: uid 0 pid 58704 ]
    @110 pass in quick on em1 proto udp from any to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 4249      Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @111 pass in quick on em1 route-to { (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet proto udp all keep state label "USER_RULE: UDP out, VietTel first"
      [ Evaluations: 4061      Packets: 51249     Bytes: 31057685    States: 69    ]
      [ Inserted: uid 0 pid 58704 ]
    @112 pass in quick on em1 from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 188       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @113 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81), (em3 192.168.0.81) } round-robin inet all flags S/SA keep state label "USER_RULE: pass others out via any WAN"
      [ Evaluations: 188       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @114 anchor "packagelate" all
      [ Evaluations: 68270     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @115 anchor "tftp-proxy/*" all
      [ Evaluations: 68270     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @116 anchor "limitingesr" all
      [ Evaluations: 68270     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]
    @117 anchor "miniupnpd" all
      [ Evaluations: 68270     Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 58704 ]</vpns:*></vpns:*></vpns:*></vpns:*></vpns:*></vn02:76></vn01:56></netcservers:4></netcservers:4></vpns:*></netcservers:4></vpns:*></netcservers:4></vpns:*></vpns:*></netcservers:4></netcservers:4></vpns:*></netcservers:4></vpns:*></netcservers:4></vpns:*></vpns:*></vpns:*></interfaces:8></interfaces:8></interfaces:8></interfaces:8></interfaces:8></interfaces:8></interfaces:8></virusprot:0></sshlockout:0></snort2c:0></snort2c:0>
    


  • When OPT3 (i.e. em3) is down:

    
    @0 scrub in on em0 all max-mss 1460 fragment reassemble
      [ Evaluations: 8425      Packets: 10        Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @1 scrub in on em1 all max-mss 1460 fragment reassemble
      [ Evaluations: 7768      Packets: 2210      Bytes: 785073      States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @2 scrub in on em4 all max-mss 1460 fragment reassemble
      [ Evaluations: 3829      Packets: 865       Bytes: 86051       States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @3 scrub in on em5 all max-mss 1460 fragment reassemble
      [ Evaluations: 1857      Packets: 796       Bytes: 7562        States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @4 scrub in on em3 all max-mss 1460 fragment reassemble
      [ Evaluations: 49        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @0 anchor "relayd/*" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @1 anchor "firewallrules" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @2 block drop in all label "Default deny rule"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @3 block drop out all label "Default deny rule"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @4 block drop in quick inet6 all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @5 block drop out quick inet6 all
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @6 block drop quick proto tcp from any port = 0 to any
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @7 block drop quick proto tcp from any to any port = 0
      [ Evaluations: 15        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @8 block drop quick proto udp from any port = 0 to any
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @9 block drop quick proto udp from any to any port = 0
      [ Evaluations: 868       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @10 block drop quick from <snort2c:0> to any label "Block snort2c hosts"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @11 block drop quick from any to <snort2c:0> label "Block snort2c hosts"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @12 anchor "packageearly" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @13 anchor "carp" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @14 block drop in log quick proto tcp from <sshlockout:0> to any port = ssh label "sshlockout"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @15 block drop in quick from <virusprot:0> to any label "virusprot overload table"
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @16 block drop in on ! em0 inet from 10.0.0.0/24 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @17 block drop in inet from 10.0.0.3 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @18 block drop in on ! em1 inet from 192.168.0.72/29 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @19 block drop in inet from 192.168.0.74 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @20 block drop in on em0 inet6 from fe80::20c:29ff:fe45:2054 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @21 block drop in on em1 inet6 from fe80::20c:29ff:fe45:205e to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @22 anchor "dhcpserverLAN" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @23 pass in on em1 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @24 pass in on em1 inet proto udp from any port = bootpc to 192.168.0.74 port = bootps keep state label "allow access to DHCP server"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @25 pass out on em1 inet proto udp from 192.168.0.74 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
      [ Evaluations: 434       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @26 block drop in on ! em4 inet from 192.168.0.64/30 to any
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @27 block drop in inet from 192.168.0.66 to any
      [ Evaluations: 469       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @28 block drop in on ! em5 inet from 192.168.0.68/30 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @29 block drop in inet from 192.168.0.70 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @30 block drop in on ! em3 inet from 192.168.0.80/29 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @31 block drop in inet from 192.168.0.82 to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @32 block drop in on em4 inet6 from fe80::20c:29ff:fe45:207c to any
      [ Evaluations: 444       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @33 block drop in on em5 inet6 from fe80::20c:29ff:fe45:2086 to any
      [ Evaluations: 256       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @34 block drop in on em3 inet6 from fe80::20c:29ff:fe45:2072 to any
      [ Evaluations: 52        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @35 anchor "spoofing" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @36 anchor "loopback" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @37 pass in on lo0 all flags S/SA keep state label "pass loopback"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @38 pass out on lo0 all flags S/SA keep state label "pass loopback"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @39 anchor "firewallout" all
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @40 pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @41 pass out route-to (em0 10.0.0.2) inet from 10.0.0.3 to ! 10.0.0.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @42 pass out route-to (em1 192.168.0.75) inet from 192.168.0.74 to ! 192.168.0.72/29 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @43 pass out route-to (em4 192.168.0.65) inet from 192.168.0.66 to ! 192.168.0.64/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @44 pass out route-to (em5 192.168.0.69) inet from 192.168.0.70 to ! 192.168.0.68/30 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @45 pass out route-to (em3 192.168.0.81) inet from 192.168.0.82 to ! 192.168.0.80/29 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @46 pass in log quick on em0 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @47 pass in log quick on em1 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 883       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @48 pass in log quick on em4 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @49 pass in log quick on em5 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 226       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @50 pass in log quick on em3 inet proto icmp from any to <interfaces:8> keep state label "USER_RULE: m_Other ICMP inbound" queue qOthersHigh
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @51 pass out quick on em0 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @52 pass out quick on em1 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @53 pass out quick on em4 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 47        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @54 pass out quick on em5 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 22        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @55 pass out quick on em3 inet proto icmp all keep state label "USER_RULE: m_Other ICMP outbound" queue qOthersHigh
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @56 pass out all flags S/SA keep state label "USER_RULE: Penalty Box" queue qP2P
      [ Evaluations: 883       Packets: 94        Bytes: 12743       States: 41    ]
      [ Inserted: uid 0 pid 57460 ]
    @57 pass out proto tcp from any to any port = 3389 flags S/SA keep state label "USER_RULE: m_Other MSRDP outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @58 pass out proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: m_Other HTTP outbound" queue(qOthersDefault, qACK)
      [ Evaluations: 5         Packets: 12        Bytes: 3425        States: 1     ]
      [ Inserted: uid 0 pid 57460 ]
    @59 pass out proto tcp from any to any port = https flags S/SA keep state label "USER_RULE: m_Other HTTPS outbound" queue(qOthersDefault, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @60 pass out proto tcp from any to any port = smtp flags S/SA keep state label "USER_RULE: m_Other SMTP outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @61 pass out proto tcp from any to any port = smtps flags S/SA keep state label "USER_RULE: m_Other SMTP/S outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @62 pass out proto tcp from any to any port = pop3 flags S/SA keep state label "USER_RULE: m_Other POP3 outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @63 pass out proto tcp from any to any port = pop3s flags S/SA keep state label "USER_RULE: m_Other POP3/S outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @64 pass out proto tcp from any to any port = imap flags S/SA keep state label "USER_RULE: m_Other IMAP outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @65 pass out proto tcp from any to any port = imaps flags S/SA keep state label "USER_RULE: m_Other IMAP/S outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @66 pass out proto tcp from any to any port = domain flags S/SA keep state label "USER_RULE: m_Other DNS1 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @67 pass out proto udp from any to any port = domain keep state label "USER_RULE: m_Other DNS2 outbound" queue qOthersHigh
      [ Evaluations: 434       Packets: 10        Bytes: 880         States: 5     ]
      [ Inserted: uid 0 pid 57460 ]
    @68 pass out proto tcp from any to any port = microsoft-ds flags S/SA keep state label "USER_RULE: m_Other SMB1 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @69 pass out proto tcp from any to any port 136 >< 140 flags S/SA keep state label "USER_RULE: m_Other SMB2 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @70 pass out proto tcp from any to any port = nntp flags S/SA keep state label "USER_RULE: m_Other NNTP1 outbound" queue(qOthersLow, qACK)
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @71 pass out proto udp from any to any port = nntp keep state label "USER_RULE: m_Other NNTP2 outbound" queue qOthersLow
      [ Evaluations: 434       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @72 pass out proto udp from any to any port = ntp keep state label "USER_RULE: m_Other NTP outbound" queue qVoIP
      [ Evaluations: 439       Packets: 868       Bytes: 66008       States: 392   ]
      [ Inserted: uid 0 pid 57460 ]
    @73 pass out proto tcp from any to any port = 30443 flags S/SA keep state label "USER_RULE: m_Other FW Control outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @74 pass in quick proto tcp from any to <interfaces:8> port = 31443 flags S/SA keep state label "USER_RULE: m_Other FW Control 1 inbound" queue(qOthersHigh, qACK)
      [ Evaluations: 449       Packets: 61        Bytes: 26348       States: 5     ]
      [ Inserted: uid 0 pid 57460 ]
    @75 pass out proto tcp from any to any port = 32443 flags S/SA keep state label "USER_RULE: m_Other FW Control 2 outbound" queue(qOthersHigh, qACK)
      [ Evaluations: 10        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @76 pass in quick proto tcp from any to <interfaces:8> port = ssh flags S/SA keep state label "USER_RULE: m_Other SSH inbound" queue(qOthersHigh, qACK)
      [ Evaluations: 10        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @77 pass in quick on em3 reply-to (em3 192.168.0.81) inet all flags S/SA keep state label "USER_RULE: Pass all in OPT3"
      [ Evaluations: 873       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @78 pass in quick on em5 reply-to (em5 192.168.0.69) inet all flags S/SA keep state label "USER_RULE: Pass in on VNPT9 all"
      [ Evaluations: 873       Packets: 696       Bytes: 52896       States: 204   ]
      [ Inserted: uid 0 pid 57460 ]
    @79 pass in quick on em4 reply-to (em4 192.168.0.65) inet all flags S/SA keep state label "USER_RULE: Pass in on VNPT8 all"
      [ Evaluations: 649       Packets: 606       Bytes: 46096       States: 188   ]
      [ Inserted: uid 0 pid 57460 ]
    @80 pass in quick on em1 reply-to (em1 192.168.0.75) inet from any to 192.168.0.64/29 flags S/SA keep state label "USER_RULE: pass all to vnpt8-vnpt9 splitters"
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @81 pass in quick on em1 reply-to (em1 192.168.0.75) inet from any to 192.168.0.80/29 flags S/SA keep state label "USER_RULE: pass all to viettel splitter"
      [ Evaluations: 47        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @82 pass in quick on em1 reply-to (em1 192.168.0.75) inet proto tcp from any to any port = ftp flags S/SA keep state label "USER_RULE: pass FTP via default gateway"
      [ Evaluations: 47        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @83 pass in log quick on em1 inet proto tcp from 192.168.12.23 to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @84 pass in log quick on em1 route-to (em4 192.168.0.65) inet proto tcp from 192.168.12.23 to any port = smtp flags S/SA keep state label "USER_RULE: mx1.intereal.vn, VNPT8 only"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @85 pass in log quick on em1 inet proto tcp from 192.168.12.3 to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @86 pass in log quick on em1 route-to (em5 192.168.0.69) inet proto tcp from 192.168.12.3 to any port = smtp flags S/SA keep state label "USER_RULE: mail.khangthong.vn, VNPT9 only"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @87 pass in log quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @88 pass in log quick on em1 route-to (em0 10.0.0.2) inet proto tcp from any to any port = smtp flags S/SA keep state label "USER_RULE: other SMTP servers out, WAN only"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @89 pass in quick on em1 proto tcp from <netcservers:4> to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @90 pass in quick on em1 proto udp from <netcservers:4> to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 42        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @91 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from <netcservers:4> to any port = domain flags S/SA keep state label "USER_RULE: critical DNS servers out, VNPT first"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @92 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto udp from <netcservers:4> to any port = domain keep state label "USER_RULE: critical DNS servers out, VNPT first"
      [ Evaluations: 5         Packets: 10        Bytes: 880         States: 5     ]
      [ Inserted: uid 0 pid 57460 ]
    @93 pass in quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 42        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @94 pass in quick on em1 proto udp from any to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 37        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @95 pass in quick on em1 route-to (em0 10.0.0.2) inet proto tcp from any to any port = domain flags S/SA keep state label "USER_RULE: other DNS clients out, WAN only"
      [ Evaluations: 42        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @96 pass in quick on em1 route-to (em0 10.0.0.2) inet proto udp from any to any port = domain keep state label "USER_RULE: other DNS clients out, WAN only"
      [ Evaluations: 37        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @97 pass in quick on em1 proto tcp from <netcservers:4> to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 42        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @98 pass in quick on em1 proto udp from <netcservers:4> to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 37        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @99 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from <netcservers:4> to any port = ntp flags S/SA keep state label "USER_RULE: critical NTP clients out, VietTel first"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @100 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto udp from <netcservers:4> to any port = ntp keep state label "USER_RULE: critical NTP clients out, VietTel first"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @101 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto icmp from any to 8.8.8.8 keep state label "USER_RULE: Test Internet connectivity, VietTelfirst"
      [ Evaluations: 42        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @102 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from any to <vn01:56> port = http flags S/SA keep state label "USER_RULE: HTTP domestic1 out, VietTel first"
      [ Evaluations: 42        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @103 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from any to <vn02:76> port = http flags S/SA keep state label "USER_RULE: HTTP domestic2 out, VietTel first"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @104 pass in quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 5         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @105 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from any to any port = http flags S/SA keep state label "USER_RULE: HTTP abroad out VNPT first"
      [ Evaluations: 5         Packets: 12        Bytes: 3425        States: 1     ]
      [ Inserted: uid 0 pid 57460 ]
    @106 pass in quick on em1 inet proto tcp from 192.168.0.0/20 to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 4         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @107 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp from 192.168.0.0/20 to any port = mmcc flags S/SA keep state label "USER_RULE: YIM, VietTel first"
      [ Evaluations: 4         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @108 pass in quick on em1 proto tcp from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 4         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @109 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto tcp all flags S/SA keep state label "USER_RULE: TCP out, VietTel first"
      [ Evaluations: 4         Packets: 32        Bytes: 2253        States: 4     ]
      [ Inserted: uid 0 pid 57460 ]
    @110 pass in quick on em1 proto udp from any to <vpns:*> keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 37        Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @111 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet proto udp all keep state label "USER_RULE: UDP out, VietTel first"
      [ Evaluations: 37        Packets: 62        Bytes: 10490       States: 37    ]
      [ Inserted: uid 0 pid 57460 ]
    @112 pass in quick on em1 from any to <vpns:*> flags S/SA keep state label "NEGATE_ROUTE: Negate policy route for vpn(s)"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @113 pass in quick on em1 route-to { (em4 192.168.0.65), (em4 192.168.0.65), (em5 192.168.0.69), (em5 192.168.0.69) } round-robin inet all flags S/SA keep state label "USER_RULE: pass others out via any WAN"
      [ Evaluations: 0         Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @114 anchor "packagelate" all
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @115 anchor "tftp-proxy/*" all
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @116 anchor "limitingesr" all
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]
    @117 anchor "miniupnpd" all
      [ Evaluations: 439       Packets: 0         Bytes: 0           States: 0     ]
      [ Inserted: uid 0 pid 57460 ]</vpns:*></vpns:*></vpns:*></vpns:*></vpns:*></vn02:76></vn01:56></netcservers:4></netcservers:4></vpns:*></netcservers:4></vpns:*></netcservers:4></vpns:*></vpns:*></netcservers:4></netcservers:4></vpns:*></netcservers:4></vpns:*></netcservers:4></vpns:*></vpns:*></vpns:*></interfaces:8></interfaces:8></interfaces:8></interfaces:8></interfaces:8></interfaces:8></interfaces:8></virusprot:0></sshlockout:0></snort2c:0></snort2c:0>
    

    I'm on May 23rd snapshot. But the problem is not specific to that snapshot. I've observed it earlier.



  • You are usign the pfSense monitoring right?!

    Interesting would be the routing table statistics and pcaps of such traffic.



  • @ermal:

    You are usign the pfSense monitoring right?!

    What is the "pfSense monitoring"?

    My pfSense does not monitor anything. Actually it is monitored by routers around it, namely, m0n0walls and Fortigates.

    @ermal:

    Interesting would be the routing table statistics and pcaps of such traffic.

    I sent it. Pls check mail @pfsense.org.



  • I would like to ask devs about status of this issue. Is it covered by some bug ticket already? If no, I would like to open a new ticket.

    The bug remains even on Sat Nov 20 19:22:47 EST 2010 snapshot.



  • Can you resend the data.
    With the latest image this should not be present there!


Log in to reply