Blocked my internal lan but I dont see where

KOM

You have Allow All rules on both LAN and LABNET, so nothing should be getting blocked. It is likely just out of state traffic.

https://doc.pfsense.org/index.php/Why_do_my_logs_show_%22blocked%22_for_traffic_from_a_legitimate_connection

Are you having any connectivity problems, or you just saw the log data and wondered what was going on?

poshpaws

Yes Traffic is getting blocked , ssh , web etc all seem to be blocked by the default deny rule (1000000103)
but as far as i can tell should not be.

/tmp/rules.debug follows , maybe someone spots something daft …

set limit table-entries 2000000
set optimization normal
set limit states 404000
set limit src-nodes 404000

#System aliases

loopback = "{ lo0 }"
WAN = "{ igb2 }"
LAN = "{ igb0_vlan1 }"
LABNET = "{ igb0_vlan4 }"
WLAN = "{ ath0_wlan0 }"
OpenVPN = "{ openvpn }"

#SSH Lockout Table
table <sshlockout>persist
table <webconfiguratorlockout>persist
#Snort tables
table <snort2c>table <virusprot>table <bogons>persist file "/etc/bogons"
table <vpn_networks>{ 10.8.0.0/24 }
table <negate_networks>{ 10.8.0.0/24 }

User Aliases

table <pfb_top_v4>persist file "/var/db/aliastables/pfB_Top_v4.txt"
pfB_Top_v4 = "<pfb_top_v4>"

Gateways

GWWAN_DHCP = " route-to ( igb2 178.83.128.1 ) "
GWWAN_DHCP6 = " route-to ( igb2 178.83.128.1 ) "
GWLAN_DHCP6 = " "

set loginterface igb0_vlan1

set skip on pfsync0

scrub on $WAN all fragment reassemble
scrub on $LAN all fragment reassemble
scrub on $LABNET all fragment reassemble
scrub on $WLAN all fragment reassemble

no nat proto carp
no rdr proto carp
nat-anchor "natearly/"
nat-anchor "natrules/"

Outbound NAT rules (automatic)

Subnets to NAT

tonatsubnets = "{ 127.0.0.0/8 192.168.1.0/24 192.168.0.0/24 10.8.0.0/24 }"
nat on $WAN from $tonatsubnets to any port 500 -> 178.83.131.6/32 static-port
nat on $WAN from $tonatsubnets to any -> 178.83.131.6/32 port 1024:65535

Load balancing anchor

rdr-anchor "relayd/*"

TFTP proxy

rdr-anchor "tftp-proxy/*"

UPnPd rdr anchor

rdr-anchor "miniupnpd"

anchor "relayd/"
anchor "openvpn/"
anchor "ipsec/*"

block IPv4 link-local. Per RFC 3927, link local "MUST NOT" be forwarded by a routing device,

and clients "MUST NOT" send such packets to a router. FreeBSD won't route 169.254./16, but

route-to can override that, causing problems such as in redmine #2073

block in log quick from 169.254.0.0/16 to any tracker 1000000101 label "Block IPv4 link-local"
block in log quick from any to 169.254.0.0/16 tracker 1000000102 label "Block IPv4 link-local"
#---------------------------------------------------------------------------

default deny rules

#---------------------------------------------------------------------------
block in log inet all tracker 1000000103 label "Default deny rule IPv4"
block out log inet all tracker 1000000104 label "Default deny rule IPv4"
block in log inet6 all tracker 1000000105 label "Default deny rule IPv6"
block out log inet6 all tracker 1000000106 label "Default deny rule IPv6"

IPv6 ICMP is not auxilary, it is required for operation

See man icmp6(4)

1 unreach Destination unreachable

2 toobig Packet too big

128 echoreq Echo service request

129 echorep Echo service reply

133 routersol Router solicitation

134 routeradv Router advertisement

135 neighbrsol Neighbor solicitation

136 neighbradv Neighbor advertisement

pass quick inet6 proto ipv6-icmp from any to any icmp6-type {1,2,135,136} tracker 1000000107 keep state

Allow only bare essential icmpv6 packets (NS, NA, and RA, echoreq, echorep)

pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {129,133,134,135,136} tracker 1000000108 keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {129,133,134,135,136} tracker 1000000109 keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000110 keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000111 keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {128,133,134,135,136} tracker 1000000112 keep state

We use the mighty pf, we cannot be fooled.

block log quick inet proto { tcp, udp } from any port = 0 to any tracker 1000000113 label "Block traffic from port 0"
block log quick inet proto { tcp, udp } from any to any port = 0 tracker 1000000114 label "Block traffic to port 0"
block log quick inet6 proto { tcp, udp } from any port = 0 to any tracker 1000000115 label "Block traffic from port 0"
block log quick inet6 proto { tcp, udp } from any to any port = 0 tracker 1000000116 label "Block traffic to port 0"

Snort package

block log quick from <snort2c>to any tracker 1000000117 label "Block snort2c hosts"
block log quick from any to <snort2c>tracker 1000000118 label "Block snort2c hosts"

SSH lockout

block in log quick proto tcp from <sshlockout>to (self) port 22 tracker 1000000301 label "sshlockout"

webConfigurator lockout

block in log quick proto tcp from <webconfiguratorlockout>to (self) port 80 tracker 1000000351 label "webConfiguratorlockout"
block in log quick from <virusprot>to any tracker 1000000400 label "virusprot overload table"

allow our DHCPv6 client out to the WAN

pass in quick on $WAN proto udp from fe80::/10 port = 546 to fe80::/10 port = 546 tracker 1000000561 label "allow dhcpv6 client in WAN"
pass in quick on $WAN proto udp from any port = 547 to any port = 546 tracker 1000000562 label "allow dhcpv6 client in WAN"
pass out quick on $WAN proto udp from any port = 546 to any port = 547 tracker 1000000563 label "allow dhcpv6 client out WAN"
antispoof log for $WAN tracker 1000001570

allow our DHCP client out to the WAN

pass in on $WAN proto udp from any port = 67 to any port = 68 tracker 1000001591 label "allow dhcp client out WAN"
pass out on $WAN proto udp from any port = 68 to any port = 67 tracker 1000001592 label "allow dhcp client out WAN"

Not installing DHCP server firewall rules for WAN which is configured for DHCP.

allow our DHCPv6 client out to the LAN

pass in quick on $LAN proto udp from fe80::/10 port = 546 to fe80::/10 port = 546 tracker 1000001611 label "allow dhcpv6 client in LAN"
pass in quick on $LAN proto udp from any port = 547 to any port = 546 tracker 1000001612 label "allow dhcpv6 client in LAN"
pass out quick on $LAN proto udp from any port = 546 to any port = 547 tracker 1000001613 label "allow dhcpv6 client out LAN"

allow access to DHCP server on LAN

pass in quick on $LAN proto udp from any port = 68 to 255.255.255.255 port = 67 tracker 1000002641 label "allow access to DHCP server"
pass in quick on $LAN proto udp from any port = 68 to 192.168.1.1 port = 67 tracker 1000002642 label "allow access to DHCP server"
pass out quick on $LAN proto udp from 192.168.1.1 port = 67 to any port = 68 tracker 1000002643 label "allow access to DHCP server"
antispoof log for $LABNET tracker 1000003670

allow access to DHCP server on LABNET

pass in quick on $LABNET proto udp from any port = 68 to 255.255.255.255 port = 67 tracker 1000003691 label "allow access to DHCP server"
pass in quick on $LABNET proto udp from any port = 68 to 192.168.0.1 port = 67 tracker 1000003692 label "allow access to DHCP server"
pass out quick on $LABNET proto udp from 192.168.0.1 port = 67 to any port = 68 tracker 1000003693 label "allow access to DHCP server"

loopback

pass in on $loopback inet all tracker 1000005811 label "pass IPv4 loopback"
pass out on $loopback inet all tracker 1000005812 label "pass IPv4 loopback"
pass in on $loopback inet6 all tracker 1000005813 label "pass IPv6 loopback"
pass out on $loopback inet6 all tracker 1000005814 label "pass IPv6 loopback"

let out anything from the firewall host itself and decrypted IPsec traffic

pass out inet all keep state allow-opts tracker 1000005815 label "let out anything IPv4 from firewall host itself"
pass out inet6 all keep state allow-opts tracker 1000005816 label "let out anything IPv6 from firewall host itself"
pass out route-to ( igb2 178.83.128.1 ) from 178.83.131.6 to !178.83.128.0/21 tracker 1000005911 keep state allow-opts label "let out anything from firewall host itself"

make sure the user cannot lock himself out of the webConfigurator or SSH

pass in quick on igb0_vlan1 proto tcp from any to (igb0_vlan1) port { 80 22 } tracker 10000 keep state label "anti-lockout rule"

User-defined rules follow

anchor "userrules/*"
pass in quick on $OpenVPN from any to any keep state label "USER_RULE: OpenVPN redgate wizard"
block in log quick on $WAN reply-to ( igb2 178.83.128.1 ) inet from $pfB_Top_v4 to any tracker 1770009106 label "USER_RULE: pfB_Top_v4 auto rule"
pass in quick on $WAN reply-to ( igb2 178.83.128.1 ) proto udp from any to 178.83.131.6 port 1194 keep state label "USER_RULE: OpenVPN redgate wizard"
pass in quick on $LAN inet from 192.168.1.0/24 to any tracker 0100000101 keep state label "USER_RULE: Default allow LAN to any rule"

at the break! label "USER_RULE: Default allow LAN IPv6 to any rule"

pass in quick on $LABNET inet from any to any tracker 1476796414 keep state label "USER_RULE"

VPN Rules

anchor "tftp-proxy/*"</virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></pfb_top_v4></pfb_top_v4></negate_networks></vpn_networks></bogons></virusprot></snort2c></webconfiguratorlockout></sshlockout>

KOM

Disable any IDS/IPS packages like Snort, Suricata, pfBlocker and try again. Any time there is weirdness that belies the rules, I make sure these specific packages are disabled.

btw it's much easier for us to read screenshots of your rules & logs than the raw dumps.

poshpaws

Hi , all IDS tools were uninstalled and the device rebooted once or twice since so there should be nothing held over from pfblockerng install

and thanks for the screenshot tip :)

KOM

I'm not sure why it's blocking traffic from 192.168.1.230 to 192.168.1.109 since it's local and shouldn't even hit the firewall. The client and server are on the same VLAN? There isn't something weird going on with your setup?

poshpaws

So it appears I must redesign my network somewhat as i am getting some asymmetric routing , which is why i guess pfsense blocks it.

:-[

johnpoz

" as i am getting some asymmetric routing"

How is that? Do you have downstream routers without using transit networks?

poshpaws

No , no more downstream routers left

ISP===> cablemodem ig2 => pfsense appliance ig1 => cisco 3750 (vlans tagged on switch , no routing) vlan 3= > esx host
ig0 => vlan 4=> esx host

gateway is set to be the pf box.
anything that does not go near the esx seems to be fine , direct subnets no issues , vm to vm traffic gets blocked same vlan or not

if i pcap my outgoing lan networks , I see packets from my esx vm's , these are then blocked by default rules.

So slight change , I do see where it's getting blocked , I have yet to figure out why ….

johnpoz

Well if no downstream routing how would you be getting asymmetrical.. How you would you possible have more than 1 route to somewhere? Do you have multihomed devices that have nic in more than 1 vlan?

"vm to vm traffic gets blocked same vlan or not"

How exactly would traffic on the same vlan get blocked? Vlan traffic to same vlan has ZERO to do with pfsense. If I had to guess your prob running multiple layer 3 over the same layer 2, which is just BORKED, etc..

Do a real drawing of your network - break out the crayons if you have to and label what networks/vlans and what is plugged into what at layer 1. And your switch config for these ports..

poshpaws

Ok point taken about the diagram , attached should be a slight improvement

I would largely agree traffic on the same vlan should have zero to do with pfsense , yet the logs show the block and the traffic is well .. blocked.

L1.jpg_thumb

Derelict

Your issue is in ESXi somewhere.

Traffic between 192.168.0.3 and 192.168.0.230 should all be handled in the vswitch and should never be sent to the router.

Your problem is not in the firewall.

johnpoz

if you have devices in esxi on the same vswitch that can not talk to each other and your seeing this traffic go to pfsense, my guess would be you have a mask wrong somewhere on one of the vms. So the 1 vm sends him syn and he answers back syn,ack to pfsense because from his mask the IP talking to him is on a different network.