Default rule - fail

BenKenobe

Yup I read that but it is inconclusive and since sticky connections are enabled that should not happen - once a session is established it should remain on whatever gateway the session was established on, there is no way for remote session holders to know both my domain IP's for a single session.

While both networks are from the same provider one is a business i.e. fixed IP etc, and one is private any old IP.

Here is the rule.debug … notice the redirects are sending to 127.0.0.1 when this is NOT the appropriate NAT destination - 127.0.0.1 will fail since it doesn't have any 443 capable service running - it's pfSense !! - I'd expect a NAT redirect to get turned back to the appropriate internal address.

Not withstanding that the connection that's getting blocked is actually the wifes phone trying to connect to GMail which is external, there are no blocks whatsoever on traffic from the LAN subnet outbound - but still pfSense is blocking port attempts 443 from her phone.

set limit tables 3000
set optimization normal
set limit states 198000
set limit src-nodes 198000

#System aliases

loopback = "{ lo0 }"
WAN = "{ pppoe0 }"
LAN = "{ re0 }"
OPT1 = "{ pppoe1 }"

#SSH Lockout Table
table <sshlockout>persist
table <webconfiguratorlockout>persist
#Snort tables
table <snort2c>table <virusprot># User Aliases
table <blockdrs>{ 14.148.131.59 23.25.216.129 24.123.56.246 24.199.42.34 37.209.31.239 46.246.119.139 50.121.152.110 50.34.10.50 50.39.90.242 50.84.168.222 62.49.22.147 63.252.106.18 64.52.155.10 64.82.225.246 65.171.64.218 66.183.52.139 66.64.240.218 66.64.6.154 68.213.103.27 69.86.213.68 70.184.122.160 70.43.109.131 70.80.28.38 71.46.210.226 72.89.191.60 74.11.126.243 74.84.111.214 74.95.89.172 75.127.236.194 75.149.2.246 75.151.241.229 75.181.131.19 78.111.75.125 78.55.254.111 79.129.19.99 79.144.190.144 80.13.177.2 80.177.69.146 80.33.151.18 81.70.233.60 82.165.134.70 83.136.86.135 83.175.212.125 83.223.112.138 83.223.112.142 87.23.197.245 87.28.147.41 88.149.180.8 88.2.247.204 88.91.75.223 89.87.130.233 90.220.107.13 91.135.4.116 93.64.20.6 94.80.4.82 94.89.253.73 94.91.131.100 95.224.107.100 95.225.148.31 95.230.52.125 95.231.96.15 95.240.32.27 98.174.235.103 108.162.17.130 108.64.133.67 113.78.39.61 114.42.129.55 114.42.130.32 114.44.101.116 114.44.101.166 116.23.198.153 116.246.22.38 120.146.193.153 134.255.242.243 142.59.240.51 151.78.252.4 168.188.35.248 173.162.251.81 183.236.40.118 183.57.193.149 187.65.74.210 188.229.7.200 189.13.198.57 190.188.202.39 190.224.126.164 195.228.228.53 200.68.86.253 201.42.103.181 201.49.69.250 201.72.166.242 202.64.64.68 203.147.88.10 203.45.114.24 203.45.134.40 211.25.222.226 212.235.31.158 212.92.23.168 213.153.47.1 213.82.200.130 216.1.42.19 217.159.181.170 217.40.3.237 220.165.5.7 222.231.33.164 69.162.123.36 31.101.203.142 }
BlockDRS = "<blockdrs>"
table <blockranges>{ 114.43.5.0/24 114.42.0.0/12 14.222.0.0/12 220.128.0.0/16 186.18.128.0/18 202.104.251.200/27 }
BlockRanges = "<blockranges>"
table <btipranges>{ 178.79.195.0/24 213.248.117.0/24 195.59.54.0/24 80.239.171.0/24 193.113.8.0/24 66.193.112.0/24 86.151.173.0/24 }
BTIPRanges = "<btipranges>"
table <btvision>{ 192.168.1.64 192.168.1.252 }
BTVision = "<btvision>"
table <easyruleblockhostsopt1>persist
EasyRuleBlockHostsOPT1 = "<easyruleblockhostsopt1>"
table <easyruleblockhostswan>persist
EasyRuleBlockHostsWAN = "<easyruleblockhostswan>"
table <edf>{ 195.59.168.0/24 }
EDF = "<edf>"
table <gbserver>{ 192.168.1.253 }
GBServer = "<gbserver>"
table <office>{ 192.168.1.250 }
Office = "<office>"

Gateways

GWWAN = " route-to ( pppoe0 85.139.96.6 ) "
GWOPT1 = " route-to ( pppoe1 212.33.142.8 ) "
GWLoadBalance = " route-to { ( pppoe0 85.139.96.6 ) ( pppoe1 212.33.142.8 ) } round-robin sticky-address "

set loginterface re0

set skip on pfsync0

no nat proto carp
no rdr proto carp
nat-anchor "natearly/"
nat-anchor "natrules/"

Outbound NAT rules

Subnets to NAT

tonatsubnets = "{ 192.168.1.0/24 127.0.0.0/8 }"
nat on $WAN from $tonatsubnets port 500 to any port 500 -> 172.135.178.20/32 port 500
nat on $WAN from $tonatsubnets to any -> 172.135.178.20/32 port 1024:65535

nat on $OPT1 from $tonatsubnets port 500 to any port 500 -> 219.57.132.72/32 port 500
nat on $OPT1 from $tonatsubnets to any -> 219.57.132.72/32 port 1024:65535

Load balancing anchor

rdr-anchor "relayd/*"

TFTP proxy

rdr-anchor "tftp-proxy/*"
rdr pass on pppoe0 proto udp from any to any port tftp -> 127.0.0.1 port 6969
rdr pass on re0 proto udp from any to any port tftp -> 127.0.0.1 port 6969
table <negate_networks>{ 172.135.178.20/32 192.168.1.0/24 219.57.132.72/32 }

NAT Inbound Redirects

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 80 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 80 tag PFREFLECT -> 127.0.0.1 port 19000

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 443 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 443 tag PFREFLECT -> 127.0.0.1 port 19001

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 21 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 21 tag PFREFLECT -> 127.0.0.1 port 19002

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 110 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 110 tag PFREFLECT -> 127.0.0.1 port 19003

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 995 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 995 tag PFREFLECT -> 127.0.0.1 port 19004

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 25 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 25 tag PFREFLECT -> 127.0.0.1 port 19005

rdr on pppoe0 proto tcp from any to 172.135.178.20 port 465 -> $GBServer

Reflection redirects

rdr on re0 proto tcp from any to 172.135.178.20 port 465 tag PFREFLECT -> 127.0.0.1 port 19006

UPnPd rdr anchor

rdr-anchor "miniupnpd"

anchor "relayd/*"
#---------------------------------------------------------------------------

default deny rules

#---------------------------------------------------------------------------
block in log all label "Default deny rule"
block out log all label "Default deny rule"

We use the mighty pf, we cannot be fooled.

block quick proto { tcp, udp } from any port = 0 to any
block quick proto { tcp, udp } from any to any port = 0

Block all IPv6

block in quick inet6 all
block out quick inet6 all

Snort package

block quick from <snort2c>to any label "Block snort2c hosts"
block quick from any to <snort2c>label "Block snort2c hosts"

SSH lockout

block in log quick proto tcp from <sshlockout>to any port 22 label "sshlockout"

webConfigurator lockout

block in log quick proto tcp from <webconfiguratorlockout>to any port 80 label "webConfiguratorlockout"
block in quick from <virusprot>to any label "virusprot overload table"
table <bogons>persist file "/etc/bogons"

block bogon networks

http://www.cymru.com/Documents/bogon-bn-nonagg.txt

block in log quick on $WAN from <bogons>to any label "block bogon networks from WAN"
antispoof for pppoe0

block anything from private networks on interfaces with the option set

antispoof for $WAN
block in log quick on $WAN from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
block in log quick on $WAN from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
block in log quick on $WAN from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
block in log quick on $WAN from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
antispoof for re0

block bogon networks

http://www.cymru.com/Documents/bogon-bn-nonagg.txt

block in log quick on $OPT1 from <bogons>to any label "block bogon networks from OPT1"
antispoof for pppoe1

block anything from private networks on interfaces with the option set

antispoof for $OPT1
block in log quick on $OPT1 from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
block in log quick on $OPT1 from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
block in log quick on $OPT1 from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
block in log quick on $OPT1 from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"

loopback

pass in on $loopback all label "pass loopback"
pass out on $loopback all label "pass loopback"

let out anything from the firewall host itself and decrypted IPsec traffic

pass out all keep state allow-opts label "let out anything from firewall host itself"
pass out route-to ( pppoe0 85.139.96.6 ) from 172.135.178.20 to !172.135.178.20/32 keep state allow-opts label "let out anything from firewall host itself"
pass out route-to ( pppoe1 212.33.142.8 ) from 219.57.132.72 to !219.57.132.72/32 keep state allow-opts label "let out anything from firewall host itself"

make sure the user cannot lock himself out of the webConfigurator or SSH

pass in quick on re0 proto tcp from any to (re0) port { 80 } keep state label "anti-lockout rule"

NAT Reflection rules

pass in inet tagged PFREFLECT keep state label "NAT REFLECT: Allow traffic to localhost"

User-defined rules follow

anchor "userrules/*"
block on { pppoe0 pppoe1 } from $BlockDRS to any label "USER_RULE"
block on { pppoe0 pppoe1 } from $BlockRanges to any label "USER_RULE"
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 80 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 443 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 21 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 110 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 995 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 25 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto tcp from any to $GBServer port 465 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) proto igmp from any to 172.135.178.20 keep state label "USER_RULE"
pass in quick on $WAN reply-to ( pppoe0 85.139.96.6 ) inet proto icmp from any to 172.135.178.20 keep state label "USER_RULE"
pass in log quick on $LAN from $BTVision to <negate_networks>keep state label "NEGATE_ROUTE: Negate policy routing for destination"
pass in log quick on $LAN $GWOPT1 from $BTVision to any keep state label "USER_RULE"
pass in quick on $LAN from 192.168.1.0/24 to <negate_networks>keep state label "NEGATE_ROUTE: Negate policy routing for destination"
pass in quick on $LAN $GWLoadBalance from 192.168.1.0/24 to any keep state label "USER_RULE: Default allow LAN to any rule"
pass in quick on $OPT1 reply-to ( pppoe1 212.33.142.8 ) proto igmp from any to 219.57.132.72 keep state label "USER_RULE"
pass in quick on $OPT1 reply-to ( pppoe1 212.33.142.8 ) inet proto icmp from any to 219.57.132.72 keep state label "USER_RULE"
pass in quick on $OPT1 reply-to ( pppoe1 212.33.142.8 ) proto igmp from 212.33.142.8 to 224.0.0.1 keep state label "USER_RULE: Easy Rule: Passed from Firewall Log View"

VPN Rules

anchor "tftp-proxy/*"</negate_networks></negate_networks></bogons></bogons></bogons></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></negate_networks></office></office></gbserver></gbserver></edf></edf></easyruleblockhostswan></easyruleblockhostswan></easyruleblockhostsopt1></easyruleblockhostsopt1></btvision></btvision></btipranges></btipranges></blockranges></blockranges></blockdrs></blockdrs></virusprot></snort2c></webconfiguratorlockout></sshlockout>

kejianshi

Thats an epic novel you posted there.

Not sure whats up, but seeing as how you seem to have two connections there and this is usually cause when connections don't enter and exit along the same routes that they should, my guess is that your connections are not near as sticky as you would like to believe.

Maybe set up manual outbound NAT with a outbound route per interface?

doktornotor

@kejianshi:

my guess is that your connections are not near as sticky as you would like to believe.

Yeah, bingo.

BenKenobe

If they aren't sticky then the option in pfSense for sticky connections doesn't work correctly - this is supposed to be a 'stateful' firewall.

The ONLY place the stickiness can fail is outbound - it can't fail inbound because the remote session is utterly unaware of the 'other' IP - meaning the only place that's common is pfSense not applying 'states' correctly and ensuring that a session started on WAN stays on WAN and the load balancing should respect that 'state'.

I can't possibly start a bunch of outgoing NAT - makes a nonsense of load balancing to do so - besides I'd need different nat for different devices to either WAN or OPT1 - I might as well run two seperate pfSense boxes if that's the case. If states aren't being respected then that's a bug IMHO.

BenKenobe

Anyone know how to log which packets are going via which 'network' - packet capture can only work on one at a time.

I checked the state tables and it isn't indicated 'which' network the state belongs to - merely that a state exists. - I take that back - it calls it router - :o

I can't see the incoming 'router' though only the outgoing …

Thinking about this it still doesn't explain WHY an attempt to reach GMail via port 443 from the LAN is being blocked - it should be permitted - there ARE no block rules in place outbound on LAN, WAN or OPT1 .... inbound yes but there are no user defined OUTBOUND blocks.

panz

Can't see the NAT for port 443

iamzam

This has nothing to do with NAT or Load Balancing. It is the normal blocking of the final session packets FIN ACK because the firewall has already closed the connection due to receiving a RST from the destination or has otherwise closed the session

http://doc.pfsense.org/index.php/Logs_show_%22blocked%22_for_traffic_from_a_legitimate_connection,_why%3F

http://forum.pfsense.org/index.php?topic=58827.0

kejianshi

Hmmmm - If thats the case, then am I to understand that there has been no actual call drops or offline states being cause? Only some log noise?
Everything on the network works fine then?

iamzam

BenKenobe, on the mobile device are you actually being blocked from accessing gmail, or is it that you just see these blocked packets in the logs?

If you are actually being blocked and you just happened to post the log snippet that only contains the blocked FIN ACK packets, then you probably have a real problem, otherwise you are likely seeing the session reset packets only being blocked.

In your syslog do you see any other packets blocked by this default rule with flags different than FA such as SYN, SYN/ACK, etc.?

kejianshi

Yeah - I have a log full of these:

127.0.0.1:3128 TCP:FA

I ignore them. Everything is working fine.

Wouldn't it be nice if we could enter in a setting some place things to not log?
Like TCP:FA, TCP:FPA etc, etc…. It would make the logs more meaningful.

BenKenobe

I never asked her and she doesn't complain, I asked and her response is that mostly it is OK, occasionally it times out but not all the time.

If these 'blocks' are normal behaviour (which I appreciate) why log them at all - as has been said it would be nice to be able to clean up what is and isn't reported.

I'm just a little perplexed why a stateful firewall would block ANY outgoing packets unless explicitly told to do so, incoming I can buy into but outgoing - just doesn't seem right - why would the other side of the connection close the session - surely that's the session initiators job ?

kejianshi

Yeah…. and... Did I mention...

I have a log full of these:

127.0.0.1:3128 TCP:FA

I ignore them. Everything is working fine.

Wouldn't it be nice if we could enter in a setting some place things to not log?
Like TCP:FA, TCP:FPA etc, etc.... It would make the logs more meaningful.

Maybe a regular expression filter as a package? Devs? Anyone...

(I hear crickets chirping...)