Firewall blocks traffic in spite of ANY rule

Armin_

Hello,

I have a problem with the firewall or a firewall rule and need advice.

I connected 2 Raspberry PIs to my network. Both in the same subnet. My PC is also in the same subnet 192.168.40.1/24. I use a remote desktop connection or putty to access the PIs from my PC. After a while this stopps working. I checked the pfsense logs and found that the firewall is blocking traffic although I addedd pass rules.

I cross checked with pfctl -d that the firewall causes the problems. If disabled, everything works fine.

Here the firewall rules:

And here the log:

I tried "easy rules" and "any" rules. With and without "Options Enabled". The FW is still blocking.

What am I doing wrong? Any help appreciated.

Thanks

Armin

System PC Engines APU2
Serial: xxxx
Netgate Unique ID: yyyy
BIOS Vendor: coreboot
Version: 88a4f96
Release Date: 03/11/2016
Version 2.3.4-RELEASE (amd64)
built on Wed May 03 15:13:29 CDT 2017
FreeBSD 10.3-RELEASE-p19

viragomann

Seems to be a sort of an asymmetric routing issue. All the blocked packets are out of state.
Look here: https://doc.pfsense.org/index.php/Asymmetric_Routing_and_Firewall_Rules

This happens when response packets take another way than request packets.

BTW: Your second VLANDATA rule allows any access and so the following ones are wasted. But maybe this is just for testing.

Armin_

Hello,

first of all I would like to thank you. I could resolve the issue with your hint.

After reading https://doc.pfsense.org/index.php/Asymmetric_Routing_and_Firewall_Rules
I am using the following solution now:

I created 2 "floating" rules, allowing any traffic (any direction, in and out) for any protocol (yes, not only tcp. I did not tick the "any flags" option) on the corresponding interface/VLAN. One rule with source "IP Address of PC" and destination "IP Address of RPI", the other rule vice versa.

I verified that it does of course not work to have the rules in the dedicated VLAN section only, since actually the "OUT" direction is missing. Next I created 2 floating rules with "out" direction only. This worked together with the same rules (in) for the respective VLAN. Because I did not want to have the rules scattered over 2 locations, I deleted the rules for the VLAN and modified the floating rules to "any" direction. Now Putty and RDP can communicate with the RPi.

The firewall logs were helping me and misleading me at the same time. Without Logs, I would not have found the solution. But since there are still Firewal Log Entries, I was also misguided. After reading https://doc.pfsense.org/index.php/Why_do_my_logs_show_%22blocked%22_for_traffic_from_a_legitimate_connection
I can understand now.

I read the documentation and found only little information about "sloppy". Beeing a network beginner, I am wondering, if parts of the traffic are handled by the firewall and the rest by the Switch on L2. Maybe this causes the problem. I have no static routes and both devices are connected to the same switch and belong to the same subnet/VLAN. The - "Bypass firewall rules for traffic on the same interface" did of course no help, because I do not have "static routes"

I was a little bit astonished that PUTTY or a simple connection of 2 network devices can create such a problem. But since I do not know, what putty is doing or how SSH works in detail, I can only guess.

Maybe somebody can enlighten me. All other additional advices are also highly appreciated.

Thank you again for your help.

Armin

johnpoz

"I created 2 "floating" rules, allowing any traffic (any direction, in and out) for any protocol (yes, not only tcp. I did not tick the "any flags" option)"

that clearly is NOT a fix to asymmetrical routing - the FIX is to correct the asymmetrical routing.. I can not think of reason why you would ever desire asymmetrical routing - it should be avoided at all costs!!

You have multiple vlans it seems… How are you seeing asymmetrical?? All devices in a vlan should be using pfsense IP address that vlan as their gateway. Do you have any sort of downstream routers or networks? You should correct the bad routing vs allowing out of state traffic!!

Armin_

Hello,

I do fully agree. I do not want to have asymmetric routing. No intention at all. I would like to avoid it. I do not even think I have it, do I? Not sure . . .

The problem was that putty/rdp showed network problems and stopped working during sessions with my RPi (same VLAN, same Subnet). Reproducable. Why this happens I do not know.

viragomann pointed me to this "sloppy" state handling setting. I did what was described above. That resolved the issue. And, to be honest, I do not know in detail why.

The PC and the RPi are connected to the same Switch and are in the same VLAN (Subnet 192.168.40.1/24). In the beginning I expected that a lot of things would be handled on L2. But obviously putty and rdp are doing some additional stuff.

I checked the gateway address of the PC with ipconfig. Because of IGMP proxy and DLNA problems I have 2 IP addresses and a separate gateway address for each IP address for my PC. I used WIN7 route comand to set the target IP and gateway addresses correctly (hopefully). See https://forum.pfsense.org/index.php?topic=129372.0

PC, IPconfig:

Ethernet-Adapter LAN-Verbindung 5:

Verbindungsspezifisches DNS-Suffix: localdomain
Verbindungslokale IPv6-Adresse . : fe80::85ec:32f8:8f55:b329%31
IPv4-Adresse . . . . . . . . . . : 192.168.60.130
Subnetzmaske . . . . . . . . . . : 255.255.255.0
Standardgateway . . . . . . . . . : 192.168.60.1

Ethernet-Adapter LAN-Verbindung 4:

Verbindungsspezifisches DNS-Suffix: localdomain
Verbindungslokale IPv6-Adresse . : fe80::f115:dda8:a1e8:6eb7%30
IPv4-Adresse . . . . . . . . . . : 192.168.40.128
Subnetzmaske . . . . . . . . . . : 255.255.255.0
Standardgateway . . . . . . . . . : 192.168.40.1

ROUTE PRINT -4

Ständige Routen:
Netzwerkadresse Netzmaske Gatewayadresse Metrik
192.168.10.0 255.255.255.0 192.168.40.1 11
192.168.30.0 255.255.255.0 192.168.40.1 11
192.168.40.0 255.255.255.0 192.168.40.1 11
192.168.50.0 255.255.255.0 192.168.40.1 11
192.168.60.0 255.255.255.0 192.168.60.1 11
192.168.70.0 255.255.255.0 192.168.40.1 11
192.168.80.0 255.255.255.0 192.168.40.1 11

RPI

pi@raspberrypi ~ $ ip addr
1: lo: <loopback,up,lower_up>mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <broadcast,multicast,up,lower_up>mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether b8:27:eb:ad:e6:82 brd ff:ff:ff:ff:ff:ff
inet 192.168.40.6/24 brd 192.168.40.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::901b:7fd7:23ac:2763/64 scope link
valid_lft forever preferred_lft forever
pi@raspberrypi ~ $ route -nee
Kernel-IP-Routentabelle
Ziel Gateway Maske Flags Metric Ref Benutzer Iface MSS Fenster irtt
0.0.0.0 192.168.40.1 0.0.0.0 UG 202 0 0 eth0 0 0 0
192.168.40.0 0.0.0.0 255.255.255.0 U 202 0 0 eth0 0 0 0

Can you help me with another idea? Or do you have an explanation for this behavior? Or a different proposal how to solve the problem?

Any advise is higly appreciated.

Thanks

Armin</broadcast,multicast,up,lower_up></loopback,up,lower_up>

johnpoz

"same VLAN, same Subnet"

Devices on the same network/vlan would not ever talk to pfsense - ever! Other than maybe to ask it for dns of some fqdn you requested if pfsense was set as your dhcp server.

"Because of IGMP proxy and DLNA problems I have 2 IP addresses and a separate gateway address for each IP address for my PC"

Yeah that is BORKED!!! So your trying to run multiple layer 3 networks over the same layer 2?

If you want to run another network on your PC for L2 connectivity to run some nonsense DLNA, then there should be NO routes to this at all.. It should not have a gateway even on this interface.. And it needs to be isolated as different L2 on your switch.

What is your default route on your windows machine? Looks like you have 2 default gateways set.. So yeah your going to have asymmetrical routing all day long with this machine!!

Armin_

Hi,

yes "same VLAN, same Subnet". I also was baffled. Why on heavens earth does the pfsense firewall block something. Yes, my VLAN uses the pfsense DHCP Server, but thats all. I expected no involvement of the firwall at all. So I started with some tests.

I used the console command "pfctl -d" and disabled everything. Putty and RDP worked immediately
Then I renabled the firewall (pfctl -e) and Putty and RDP displayed some network error after some seconds and stopped working
The firewall logs showed some blocked packets. From my PC (192.168.40.128) to my Pi (192.168.40.6). See above.
I tried "easy rules" and "any rules" without success
viragomann's post helped me to find a solution using floating rules and "sloppy" state types
And yes, I was wondering about "asymmetric routes"

For my PC. No, I think I do not run multiple layer 3 networks over the same layer 2. Sorry, I was not very precise. Since pfsense has an issue with DLNA, I was looking for a solution. My 3 Panasonic TVs and Home Theater System are running in one VLAN: VLANMEDIA, 192.168.60.1/24. So I can look Netflix, Amazon Fire or Youtube without having multicast traffic all over the place. Works fine. But then. I wanted to look some photos stored on my PC via my TV. Using DLNA. My PC is in VLANDATA, 192.168.40.1/24. So I wanted pfsense to route DLNA/Multicast traffic over subnets. This does not work. pfsense (IGMP Proxy) has a bug. This will be fixed in version 2.4.

I solved the issue with creating 2 VLANS on my PCs NIC. VLANDATA 192.168.40.1/24, Gateway 192.168.40.1 and VLANMEDIA 192.168.60.1/24, Gateway 192.168.60.1. It is connected via a trunk port with the switch. The switch then distributes the packets to the corresponding access ports and vice versa. In my opinion this is a valid setup, but maybe that this is not correct. However. It works perfectly. In order to tell the PC, which VLAN and what gateway it should use, I added permanent routes on the PC. Everything with destination IP in the VLANMEDIA range will go to there and use the gateway of the VLAN Media. The rest shall go to whereever and use gateway 192.168.40.1. So, yes, each VLAN on my windows machine does have its own default gateway.

Again, I thought that this is OK. But maybe this really created something assymetric. And, to be very honest, I do still not understand why. Because I thought, that packets for 192.168.40.1/24 should be sent over VLANDATA to the RPI. How can there be a different route to the PI or back?

Sorry for bothering you, but could you perhaps again explain me why?

Thank you for your support.

Armin

johnpoz

"In my opinion this is a valid setup"

Sure you can have multiple vlans via tagging on your PC.. This is "valid" - but what is not valid is your routing. And brings about questions of security ;)

You have 2 default gateway setup.. So yeah your going to have asymmetrical routing if you do not correctly route, and all your other devices in their answers also take the same route, etc.

Why do you have a gateway set on vlamedia if all you want this for is L2 access? Remove your gateway and all your BS routes you have setup via route.. And your asymmetrical issues will go away.

"How can there be a different route to the PI or back?"

You didn't show your full route table.. While packets should never go out your different interface when it has an interface on 192.168.40 (directly connected) Its possible you do not have your tagging setup correctly. Did you actually sniff the packets and validate they are tagged? Windows quite often does not tag correrctly..

Do a simple sniff validate the traffic is tagged…

"192.168.60.1/24. So I can look Netflix, Amazon Fire or Youtube without having multicast traffic all over the place."

So you turned microsoft off from sending multicast out its 192.168.40 interface? Windows out of the box is a broadcast/multicast noise machine ;) Did you block multicast as your switch for the vlandata? And left it open for the vlanmedia?

tag-notag.png_thumb