Default rule - fail

[BenKenobe]

It would appear that a default block rule is targetting my outgoing LAN connection - why ?

block
Aug 30 09:10:40 LAN 192.168.1.8:45473 74.125.24.138:443 TCP:FA
block
Aug 30 09:10:24 LAN 192.168.1.8:45473 74.125.24.138:443 TCP:FA
block
Aug 30 09:10:15 LAN 192.168.1.8:45473 74.125.24.138:443 TCP:FA

this is a legitimate connection to a secure server from a mobile device. These are my LAN rules - pretty basic

• BTVision * * * OPT1 none
• LAN net * * * * none   Default allow LAN to any rule

The first rule contains the IP address of my BT Vision box so that it is 'routed' to the appropriate line (I have two - one business, the WAN, and one private - OPT1), the second rule 'should' pass all other traffic without interference to the 'default' gateway. I have both WAN and OPT1 combined as a load balancing group i.e. both WAN and OPT1 are Tier 1, sticky connections are enabled.

I can find no logical or valid reason for this 'blocking' behaviour - but then it isn't possible to see the 'default block any' rule

[panz]

cat /tmp/rules.debug

and post the results.

[kejianshi]

http://forum.pfsense.org/index.php?topic=35400.0

[BenKenobe]

Yup I read that but it is inconclusive and since sticky connections are enabled that should not happen - once a session is established it should remain on whatever gateway the session was established on, there is no way for remote session holders to know both my domain IP's for a single session.

While both networks are from the same provider one is a business i.e. fixed IP etc, and one is private any old IP.

Here is the rule.debug … notice the redirects are sending to 127.0.0.1 when this is NOT the appropriate NAT destination - 127.0.0.1 will fail since it doesn't have any 443 capable service running - it's pfSense !!  - I'd expect a NAT redirect to get turned back to the appropriate internal address.

Not withstanding that the connection that's getting blocked is actually the wifes phone trying to connect to GMail which is external, there are no blocks whatsoever on traffic from the LAN subnet outbound - but still pfSense is blocking port attempts 443 from her phone.

set limit tables 3000
set optimization normal
set limit states 198000
set limit src-nodes 198000

#System aliases

loopback = "{ lo0 }"
WAN = "{ pppoe0 }"
LAN = "{ re0 }"
OPT1 = "{ pppoe1 }"

#SSH Lockout Table
table <sshlockout>persist
table <webconfiguratorlockout>persist
#Snort tables
table <snort2c>table <virusprot># User Aliases
table <blockdrs>{  14.148.131.59  23.25.216.129  24.123.56.246  24.199.42.34  37.209.31.239  46.246.119.139  50.121.152.110  50.34.10.50  50.39.90.242  50.84.168.222  62.49.22.147  63.252.106.18  64.52.155.10  64.82.225.246  65.171.64.218  66.183.52.139  66.64.240.218  66.64.6.154  68.213.103.27  69.86.213.68  70.184.122.160  70.43.109.131  70.80.28.38  71.46.210.226  72.89.191.60  74.11.126.243  74.84.111.214  74.95.89.172  75.127.236.194  75.149.2.246  75.151.241.229  75.181.131.19  78.111.75.125  78.55.254.111  79.129.19.99  79.144.190.144  80.13.177.2  80.177.69.146  80.33.151.18  81.70.233.60  82.165.134.70  83.136.86.135  83.175.212.125  83.223.112.138  83.223.112.142  87.23.197.245  87.28.147.41  88.149.180.8  88.2.247.204  88.91.75.223  89.87.130.233  90.220.107.13  91.135.4.116  93.64.20.6  94.80.4.82  94.89.253.73  94.91.131.100  95.224.107.100  95.225.148.31  95.230.52.125  95.231.96.15  95.240.32.27  98.174.235.103  108.162.17.130  108.64.133.67  113.78.39.61  114.42.129.55  114.42.130.32  114.44.101.116  114.44.101.166  116.23.198.153  116.246.22.38  120.146.193.153  134.255.242.243  142.59.240.51  151.78.252.4  168.188.35.248  173.162.251.81  183.236.40.118  183.57.193.149  187.65.74.210  188.229.7.200  189.13.198.57  190.188.202.39  190.224.126.164  195.228.228.53  200.68.86.253  201.42.103.181  201.49.69.250  201.72.166.242  202.64.64.68  203.147.88.10  203.45.114.24  203.45.134.40  211.25.222.226  212.235.31.158  212.92.23.168  213.153.47.1  213.82.200.130  216.1.42.19  217.159.181.170  217.40.3.237  220.165.5.7  222.231.33.164  69.162.123.36  31.101.203.142 }
BlockDRS = "<blockdrs>"
table <blockranges>{  114.43.5.0/24  114.42.0.0/12  14.222.0.0/12  220.128.0.0/16  186.18.128.0/18  202.104.251.200/27 }
BlockRanges = "<blockranges>"
table <btipranges>{  178.79.195.0/24  213.248.117.0/24  195.59.54.0/24  80.239.171.0/24  193.113.8.0/24  66.193.112.0/24  86.151.173.0/24 }
BTIPRanges = "<btipranges>"
table <btvision>{  192.168.1.64  192.168.1.252 }
BTVision = "<btvision>"
table <easyruleblockhostsopt1>persist
EasyRuleBlockHostsOPT1 = "<easyruleblockhostsopt1>"
table <easyruleblockhostswan>persist
EasyRuleBlockHostsWAN = "<easyruleblockhostswan>"
table <edf>{  195.59.168.0/24 }
EDF = "<edf>"
table <gbserver>{  192.168.1.253 }
GBServer = "<gbserver>"
table <office>{  192.168.1.250 }
Office = "<office>"

Gateways

GWWAN = " route-to ( pppoe0 85.139.96.6 ) "
GWOPT1 = " route-to ( pppoe1 212.33.142.8 ) "
GWLoadBalance = "  route-to { ( pppoe0 85.139.96.6 ) ( pppoe1 212.33.142.8 )  }  round-robin  sticky-address  "

set loginterface re0

set skip on pfsync0

no nat proto carp
no rdr proto carp
nat-anchor "natearly/"
nat-anchor "natrules/"

Outbound NAT rules

Subnets to NAT

tonatsubnets = "{ 192.168.1.0/24 127.0.0.0/8  }"
nat on $WAN  from $tonatsubnets port 500 to any port 500 -> 172.135.178.20/32 port 500 
nat on $WAN  from $tonatsubnets to any -> 172.135.178.20/32 port 1024:65535

nat on $OPT1  from $tonatsubnets port 500 to any port 500 -> 219.57.132.72/32 port 500 
nat on $OPT1  from $tonatsubnets to any -> 219.57.132.72/32 port 1024:65535

Load balancing anchor

rdr-anchor "relayd/*"

TFTP proxy

rdr-anchor "tftp-proxy/*"
rdr pass on pppoe0 proto udp from any to any port tftp -> 127.0.0.1 port 6969
rdr pass on re0 proto udp from any to any port tftp -> 127.0.0.1 port 6969
table <negate_networks>{ 172.135.178.20/32 192.168.1.0/24 219.57.132.72/32 }

NAT Inbound Redirects

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 80 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 80 tag PFREFLECT -> 127.0.0.1 port 19000

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 443 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 443 tag PFREFLECT -> 127.0.0.1 port 19001

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 21 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 21 tag PFREFLECT -> 127.0.0.1 port 19002

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 110 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 110 tag PFREFLECT -> 127.0.0.1 port 19003

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 995 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 995 tag PFREFLECT -> 127.0.0.1 port 19004

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 25 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 25 tag PFREFLECT -> 127.0.0.1 port 19005

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 465 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 465 tag PFREFLECT -> 127.0.0.1 port 19006

UPnPd rdr anchor

rdr-anchor "miniupnpd"

anchor "relayd/*"
#---------------------------------------------------------------------------

default deny rules

#---------------------------------------------------------------------------
block in log all label "Default deny rule"
block out log all label "Default deny rule"

We use the mighty pf, we cannot be fooled.

block quick proto { tcp, udp } from any port = 0 to any
block quick proto { tcp, udp } from any to any port = 0

Block all IPv6

block in quick inet6 all
block out quick inet6 all

Snort package

block quick from <snort2c>to any label "Block snort2c hosts"
block quick from any to <snort2c>label "Block snort2c hosts"

SSH lockout

block in log quick proto tcp from <sshlockout>to any port 22 label "sshlockout"

webConfigurator lockout

block in log quick proto tcp from <webconfiguratorlockout>to any port 80 label "webConfiguratorlockout"
block in quick from <virusprot>to any label "virusprot overload table"
table <bogons>persist file "/etc/bogons"

block bogon networks

http://www.cymru.com/Documents/bogon-bn-nonagg.txt

block in log quick on $WAN from <bogons>to any label "block bogon networks from WAN"
antispoof for pppoe0

block anything from private networks on interfaces with the option set

antispoof for $WAN
block in log quick on $WAN from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
block in log quick on $WAN from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
block in log quick on $WAN from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
block in log quick on $WAN from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
antispoof for re0

block bogon networks

http://www.cymru.com/Documents/bogon-bn-nonagg.txt

block in log quick on $OPT1 from <bogons>to any label "block bogon networks from OPT1"
antispoof for pppoe1

block anything from private networks on interfaces with the option set

antispoof for $OPT1
block in log quick on $OPT1 from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
block in log quick on $OPT1 from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
block in log quick on $OPT1 from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
block in log quick on $OPT1 from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"

loopback

pass in on $loopback all label "pass loopback"
pass out on $loopback all label "pass loopback"

let out anything from the firewall host itself and decrypted IPsec traffic

pass out all keep state allow-opts label "let out anything from firewall host itself"
pass out route-to ( pppoe0 85.139.96.6 ) from 172.135.178.20 to !172.135.178.20/32 keep state allow-opts label "let out anything from firewall host itself"
pass out route-to ( pppoe1 212.33.142.8 ) from 219.57.132.72 to !219.57.132.72/32 keep state allow-opts label "let out anything from firewall host itself"

make sure the user cannot lock himself out of the webConfigurator or SSH

pass in quick on re0 proto tcp from any to (re0) port { 80 } keep state label "anti-lockout rule"

NAT Reflection rules

pass in inet tagged PFREFLECT keep state label "NAT REFLECT: Allow traffic to localhost"

User-defined rules follow

anchor "userrules/*"
block  on {  pppoe0  pppoe1  }  from  $BlockDRS to any  label "USER_RULE"
block  on {  pppoe0  pppoe1  }  from  $BlockRanges to any  label "USER_RULE"
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 80  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 443  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 21  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 110  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 995  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 25  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto tcp  from any to  $GBServer port 465  flags S/SA keep state  label "USER_RULE: NAT "
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  proto igmp  from any to 172.135.178.20 keep state  label "USER_RULE"
pass  in  quick  on $WAN reply-to ( pppoe0 85.139.96.6 )  inet proto icmp  from any to 172.135.178.20 keep state  label "USER_RULE"
pass  in log  quick  on $LAN  from  $BTVision  to <negate_networks>keep state  label "NEGATE_ROUTE: Negate policy routing for destination"
pass  in log  quick  on $LAN  $GWOPT1  from  $BTVision to any keep state  label "USER_RULE"
pass  in  quick  on $LAN  from 192.168.1.0/24  to <negate_networks>keep state  label "NEGATE_ROUTE: Negate policy routing for destination"
pass  in  quick  on $LAN  $GWLoadBalance  from 192.168.1.0/24 to any keep state  label "USER_RULE: Default allow LAN to any rule"
pass  in  quick  on $OPT1 reply-to ( pppoe1 212.33.142.8 )  proto igmp  from any to 219.57.132.72 keep state  label "USER_RULE"
pass  in  quick  on $OPT1 reply-to ( pppoe1 212.33.142.8 )  inet proto icmp  from any to 219.57.132.72 keep state  label "USER_RULE"
pass  in  quick  on $OPT1 reply-to ( pppoe1 212.33.142.8 )  proto igmp  from  212.33.142.8 to  224.0.0.1 keep state  label "USER_RULE: Easy Rule: Passed from Firewall Log View"

VPN Rules

anchor "tftp-proxy/*"</negate_networks></negate_networks></bogons></bogons></bogons></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></negate_networks></office></office></gbserver></gbserver></edf></edf></easyruleblockhostswan></easyruleblockhostswan></easyruleblockhostsopt1></easyruleblockhostsopt1></btvision></btvision></btipranges></btipranges></blockranges></blockranges></blockdrs></blockdrs></virusprot></snort2c></webconfiguratorlockout></sshlockout>

[kejianshi]

Thats an epic novel you posted there.

Not sure whats up, but seeing as how you seem to have two connections there and this is usually cause when connections don't enter and exit along the same routes that they should, my guess is that your connections are not near as sticky as you would like to believe.

Maybe set up manual outbound NAT with a outbound route per interface?

[doktornotor]

@kejianshi:

my guess is that your connections are not near as sticky as you would like to believe.

Yeah, bingo.

[BenKenobe]

If they aren't sticky then the option in pfSense for sticky connections doesn't work correctly - this is supposed to be a 'stateful' firewall.

The ONLY place the stickiness can fail is outbound - it can't fail inbound because the remote session is utterly unaware of the 'other' IP - meaning the only place that's common is pfSense not applying 'states' correctly and ensuring that a session started on WAN stays on WAN and the load balancing should respect that 'state'.

I can't possibly start a bunch of outgoing NAT - makes a nonsense of load balancing to do so - besides I'd need different nat for different devices to either WAN or OPT1 - I might as well run two seperate pfSense boxes if that's the case. If states aren't being respected then that's a bug IMHO.

[BenKenobe]

Anyone know how to log which packets are going via which 'network' - packet capture can only work on one at a time.

I checked the state tables and it isn't indicated 'which' network the state belongs to - merely that a state exists. - I take that back - it calls it router -  :o

I can't see the incoming 'router' though only the outgoing …

Thinking about this it still doesn't explain WHY an attempt to reach GMail via port 443 from the LAN is being blocked - it should be permitted - there ARE no block rules in place outbound on LAN, WAN or OPT1 .... inbound yes but there are no user defined OUTBOUND blocks.

[panz]

Can't see the NAT for port 443

[iamzam]

This has nothing to do with NAT or Load Balancing.  It is the normal blocking of the final session packets FIN ACK because the firewall has already closed the connection due to receiving a RST from the destination or has otherwise closed the session

http://doc.pfsense.org/index.php/Logs_show_%22blocked%22_for_traffic_from_a_legitimate_connection,_why%3F

http://forum.pfsense.org/index.php?topic=58827.0

[kejianshi]

Hmmmm - If thats the case, then am I to understand that there has been no actual call drops or offline states being cause?  Only some log noise?
Everything on the network works fine then?

[iamzam]

BenKenobe, on the mobile device are you actually being blocked from accessing gmail, or is it that you just see these blocked packets in the logs?

If you are actually being blocked and you just happened to post the log snippet that only contains the blocked FIN ACK packets, then you probably have a real problem, otherwise you are likely seeing the session reset packets only being blocked.

In your syslog do you see any other packets blocked by this default rule with flags different than FA such as SYN, SYN/ACK, etc.?

[kejianshi]

Yeah - I have a log full of these:

127.0.0.1:3128 TCP:FA

I ignore them.  Everything is working fine.

Wouldn't it be nice if we could enter in a setting some place things to not log?
Like TCP:FA, TCP:FPA etc, etc….  It would make the logs more meaningful.

[BenKenobe]

I never asked her and she doesn't complain, I asked and her response is that mostly it is OK, occasionally it times out but not all the time.

If these 'blocks' are normal behaviour (which I appreciate) why log them at all - as has been said it would be nice to be able to clean up what is and isn't reported.

I'm just a little perplexed why a stateful firewall would block ANY outgoing packets unless explicitly told to do so, incoming I can buy into but outgoing - just doesn't seem right - why would the other side of the connection close the session - surely that's the session initiators job ?

[kejianshi]

Yeah….  and...  Did I mention...

I have a log full of these:

127.0.0.1:3128    TCP:FA

I ignore them.  Everything is working fine.

Wouldn't it be nice if we could enter in a setting some place things to not log?
Like TCP:FA, TCP:FPA etc, etc....  It would make the logs more meaningful.

Maybe a regular expression filter as a package?  Devs?  Anyone...

(I hear crickets chirping...)