Network firewall rules not doing what I expect



  • I feel like this is one of those questions that end up getting asked a lot, but let's do it yet again.

    I have an interface set up on a VLAN, connected to a Unifi AP broadcasting an SSID on the VLAN. My laptop is able to connect to the SSID and obtain a DHCP address on the correct network (192.168.30.0/24). However, all other traffic is blocked. With no firewall rules (sans pfBlockerNG) on the interface, there is absolutely no connectivity, even within the same subnet (additionally with WAN as well). I am even unable to ping the pfSense box at 192.168.30.1. Of course this may also be due to no firewall rules. So I add one which should pass traffic through the subnet but to no avail.
    rules
    For all my other interfaces I have skirted around the issue by setting source and destination to any but I would like to change this ASAP.
    The interface does indeed see the traffic and pinging from pfSense to the device works as expected.

    18:44:47.122214 IP 192.168.30.100.63438 > 192.168.30.1.domain: 19580+ A? ap.spotify.com. (32)
    18:44:49.356631 IP 192.168.30.100.64416 > 192.168.30.1.domain: 40732+ A? 48-courier.push.apple.com. (43)
    18:44:49.384666 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 0, length 64
    18:44:50.391544 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 1, length 64
    18:44:50.555514 IP 192.168.30.100.51241 > 192.168.30.1.domain: 25649+ A? bag.itunes.apple.com. (38)
    18:44:51.399414 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 2, length 64
    18:44:52.116178 IP 192.168.30.100.63870 > 192.168.30.1.domain: 42418+ A? api.apple-cloudkit.com. (40)
    18:44:52.116184 IP 192.168.30.100.50602 > 192.168.30.1.domain: 14258+ A? cl2.apple.com. (31)
    18:44:52.389959 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 3, length 64
    18:44:52.966850 IP 192.168.30.100.50602 > 192.168.30.1.domain: 14258+ A? cl2.apple.com. (31)
    18:44:53.450264 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 4, length 64
    18:44:54.494938 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 5, length 64
    

    Meanwhile a tcpdump with everything set to all on a different network looks like:

    18:48:51.263909 IP 192.168.20.103 > 192.168.20.1: ICMP echo request, id 1, seq 23, length 40
    18:48:51.263946 IP 192.168.20.1 > 192.168.20.103: ICMP echo reply, id 1, seq 23, length 40
    18:48:52.264663 IP 192.168.20.103 > 192.168.20.1: ICMP echo request, id 1, seq 24, length 40
    18:48:52.264684 IP 192.168.20.1 > 192.168.20.103: ICMP echo reply, id 1, seq 24, length 40
    18:48:53.265910 IP 192.168.20.103 > 192.168.20.1: ICMP echo request, id 1, seq 25, length 40
    18:48:53.265931 IP 192.168.20.1 > 192.168.20.103: ICMP echo reply, id 1, seq 25, length 40
    18:48:54.267224 IP 192.168.20.103 > 192.168.20.1: ICMP echo request, id 1, seq 26, length 40
    18:48:54.267245 IP 192.168.20.1 > 192.168.20.103: ICMP echo reply, id 1, seq 26, length 40
    

    Is there anything I'm doing horribly wrong? I can manually enter a rule for 192.168.30.0/24 and have it work but why does GUEST net not work?

    If it helps at all here are my interfaces:
    interfaces
    BRIDGE0 is a bridge of MLX0 and MLX1, which is my connectx-4 card, but nothing is connected to it at the moment and I doubt this would affect anything on VLAN 20 or 30.



  • Do you have the Unifi AP configured in guest mode / isolation?



  • No everything is configured as a standard network. I experienced the exact same issue with wired clients plugged directly into the NIC on the pfSense machine.

    Unifi settings here:
    settings



  • @incertia said in Network firewall rules not doing what I expect:

    even within the same subnet

    @incertia said in Network firewall rules not doing what I expect:

    I experienced the exact same issue with wired clients plugged directly into the NIC on the pfSense machine.

    Redo the tests.

    When devices on a subnet all have an IP in the same /24 (your 92.168.30.0/24, so pfSense's DHCP did it's job) then you could even disconnect pfSense and still have "subnet" communication.

    Btw :

    You said :
    acf0686c-ae42-47f5-aacf-ee0728546ab0-image.png

    but carefully hide the actual interface LAN name. I presume it's "GUEST"
    GUEST is set up as 192.168.30.0/24 - as you told.

    Consider again your second firewall rule.
    and (example) :

    18:44:47.122214 IP 192.168.30.100.63438 > 192.168.30.1.domain: 19580+ A? ap.spotify.com. (32)
    

    The counters in front of the second rule don't count == do not match - so, no PASS.
    This means that "192.168.30.100.63438" is not part of "GUEST net".
    Or "192.168.30.100.63438" already blocked up front.

    Btw : these :

    18:44:47.122214 IP 192.168.30.100.63438 > 192.168.30.1.domain: 19580+ A? ap.spotify.com. (32)
    18:44:49.356631 IP 192.168.30.100.64416 > 192.168.30.1.domain: 40732+ A? 48-courier.push.apple.com. (43)
    18:44:49.384666 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 0, length 64
    18:44:50.391544 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 1, length 64
    18:44:50.555514 IP 192.168.30.100.51241 > 192.168.30.1.domain: 25649+ A? bag.itunes.apple.com. (38)
    18:44:51.399414 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 2, length 64
    18:44:52.116178 IP 192.168.30.100.63870 > 192.168.30.1.domain: 42418+ A? api.apple-cloudkit.com. (40)
    18:44:52.116184 IP 192.168.30.100.50602 > 192.168.30.1.domain: 14258+ A? cl2.apple.com. (31)
    18:44:52.389959 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 3, length 64
    18:44:52.966850 IP 192.168.30.100.50602 > 192.168.30.1.domain: 14258+ A? cl2.apple.com. (31)
    18:44:53.450264 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 4, length 64
    18:44:54.494938 IP 192.168.30.100 > 192.168.30.1: ICMP echo request, id 37140, seq 5, length 64
    

    are all packets coming from "192.168.30.100" == a device (?) to pfSense (== 192.168.30.100).
    NOT

    @incertia said in Network firewall rules not doing what I expect:

    the traffic and pinging from pfSense to the device works as expected.

    which is the other way around.
    Can you check for incoming and outgoing traffic ?

    I advise toy to :
    Go basic first : undo VLAN's - undo bridging.
    Plain old physical LAN's.

    Then add one 'thing' at the time : A VLAN, and test.
    undo the VLAN, a make a bridge, and test.
    Still there ?
    Add a bridged VLAN ** - and test.

    A soon as things go broke, you know what not to do - and you're up and running.

    ** why would people actually do such things ?



  • @Gertjan said in Network firewall rules not doing what I expect:

    but carefully hide the actual interface LAN name. I presume it's "GUEST"

    It is indeed GUEST. Let us, for example, look at MGMT instead. Here is the interface:
    MGMT
    It has the following DHCP server:
    DHCP
    I have completely disabled any firewall rules for that interface:
    firewall
    @Gertjan said in Network firewall rules not doing what I expect:

    Redo the tests.

    Here, we see the tcpdump.

    [2.4.4-RELEASE][root@pfsense.localdomain]/root: tcpdump -i igb1.1 -n icmp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on igb1.1, link-type EN10MB (Ethernet), capture size 262144 bytes
    11:16:15.869867 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1315, length 64
    11:16:16.883375 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1316, length 64
    11:16:17.896794 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1317, length 64
    11:16:18.910131 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1318, length 64
    11:16:19.923128 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1319, length 64
    11:16:20.936858 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1320, length 64
    11:16:21.950238 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1321, length 64
    11:16:22.963285 IP 192.168.1.104 > 192.168.1.1: ICMP echo request, id 183, seq 1322, length 64
    ^C
    8 packets captured
    14 packets received by filter
    0 packets dropped by kernel
    

    As you can see there are no replies being sent. The same is true if we enable MGMT net to MGMT net. I have checked the firewall logs, it seems to be falling under the default IPv4 deny rule.
    log
    I have checked the pf rules, the second rule does not even appear.

    [2.4.4-RELEASE][root@pfsense.localdomain]/root: cat /tmp/rules.debug | grep MGMT
    MGMT = "{ igb1.1 }"
    scrub on $MGMT all    fragment reassemble
    antispoof log for $MGMT tracker 1000004720
    # allow access to DHCP server on MGMT
    pass in  quick on $MGMT proto udp from any port = 68 to 255.255.255.255 port = 67 tracker 1000004741 label "allow access to DHCP server"
    pass in  quick on $MGMT proto udp from any port = 68 to 192.168.1.1 port = 67 tracker 1000004742 label "allow access to DHCP server"
    pass out  quick on $MGMT proto udp from 192.168.1.1 port = 67 to any port = 68 tracker 1000004743 label "allow access to DHCP server"
    block return  in log  quick  on $MGMT inet from any to $pfB_PRI1_v4 tracker 1770009590  label "USER_RULE: pfB_PRI1_v4"
    [2.4.4-RELEASE][root@pfsense.localdomain]/root: pfctl -sa | grep 'igb1\.1'
    rdr pass on igb1.1 inet proto tcp from any to 10.10.10.1 port = http -> 127.0.0.1 port 8081
    rdr pass on igb1.1 inet proto tcp from any to 10.10.10.1 port = https -> 127.0.0.1 port 9443
    scrub on igb1.1 all fragment reassemble
    block drop in log on ! igb1.1 inet from 192.168.1.0/24 to any
    block drop in log on ! igb1.1 inet from 10.10.10.1 to any
    block drop in log on igb1.1 inet6 from fe80::6eb3:11ff:fe1c:b299 to any
    pass in quick on igb1.1 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
    pass in quick on igb1.1 inet proto udp from any port = bootpc to 192.168.1.1 port = bootps keep state label "allow access to DHCP server"
    pass out quick on igb1.1 inet proto udp from 192.168.1.1 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
    block return in log quick on igb1.1 inet from any to <pfB_PRI1_v4> label "USER_RULE: pfB_PRI1_v4"
    

    Even with pfctl -vvsa, grepping for the tracking ID yields no results.
    Rules enabled, ping from pfSense to MGMT device:

    [2.4.4-RELEASE][root@pfsense.localdomain]/root: tcpdump -i igb1.1 -n icmp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on igb1.1, link-type EN10MB (Ethernet), capture size 262144 bytes
    11:36:58.104312 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 15, length 64
    11:36:58.105255 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 15, length 64
    11:36:59.110783 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 16, length 64
    11:36:59.111181 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 16, length 64
    11:37:00.130726 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 17, length 64
    11:37:00.131092 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 17, length 64
    11:37:01.152722 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 18, length 64
    11:37:01.153531 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 18, length 64
    11:37:02.166115 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 19, length 64
    11:37:02.166366 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 19, length 64
    11:37:03.183370 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 20, length 64
    11:37:03.183620 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 20, length 64
    11:37:04.197545 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 21, length 64
    11:37:04.198122 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 21, length 64
    11:37:05.202766 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 22, length 64
    11:37:05.203309 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 22, length 64
    11:37:06.213229 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 23, length 64
    11:37:06.214589 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 23, length 64
    11:37:07.216295 IP 192.168.1.1 > 192.168.1.104: ICMP echo request, id 34357, seq 24, length 64
    11:37:07.216718 IP 192.168.1.104 > 192.168.1.1: ICMP echo reply, id 34357, seq 24, length 64
    ^C
    20 packets captured
    21 packets received by filter
    0 packets dropped by kernel
    

    @Gertjan said in Network firewall rules not doing what I expect:

    Plain old physical LAN's.

    I enabled a regular LAN on igb1 with address 192.168.88.1/24 with DHCP enabled. I also connected my laptop directly via ethernet to the interface. After confirming a DHCP address, ping 192.168.88.1 -I <laptop interface> times out as well. It is the same issue as before.



  • @incertia said in Network firewall rules not doing what I expect:

    onnected my laptop directly via ethernet to the interface

    Why the -I parameter ?

    ipconfig /all
    

    or

    ifconfig
    

    tells what ? IP ok ? Gateway ok ? DNS ok ?

    @incertia said in Network firewall rules not doing what I expect:

    [2.4.4-RELEASE][root@pfsense.localdomain]/root: tcpdump -i igb1.1 -n icmp

    Strange.
    My turn.
    I started this - em1 is my LAN interface :

    [2.4.5-RC][admin@pfsense.brit-hotel-fumel.net]/root: tcpdump -i em1 -n icmp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on em1, link-type EN10MB (Ethernet), capture size 262144 bytes
    .......
    

    and on a Windows PC I started

    ping 192.168.1.1.
    

    This is what I saw on the pfSense console :

    19:11:34.892523 IP 192.168.1.6 > 192.168.1.1: ICMP echo request, id 1, seq 43, length 40
    19:11:34.892573 IP 192.168.1.1 > 192.168.1.6: ICMP echo reply, id 1, seq 43, length 40
    19:11:35.893070 IP 192.168.1.6 > 192.168.1.1: ICMP echo request, id 1, seq 44, length 40
    19:11:35.893099 IP 192.168.1.1 > 192.168.1.6: ICMP echo reply, id 1, seq 44, length 40
    19:11:36.893243 IP 192.168.1.6 > 192.168.1.1: ICMP echo request, id 1, seq 45, length 40
    19:11:36.893277 IP 192.168.1.1 > 192.168.1.6: ICMP echo reply, id 1, seq 45, length 40
    19:11:37.893290 IP 192.168.1.6 > 192.168.1.1: ICMP echo request, id 1, seq 46, length 40
    19:11:37.893316 IP 192.168.1.1 > 192.168.1.6: ICMP echo reply, id 1, seq 46, length 40
    

    I have a request (from the Windows device) and the reply (from pfSense).

    Btw : "igb1.1" is a VLAN interface.

    Is your LAN a VLAN ?
    Test with 'Plain old physical LAN's' - no VLAN.

    Another check :
    Save your config - this permits you to get back into this "non workable" situation right away.
    Reset pfSEnse to default.
    Set up a WAN - if needed.
    You wind up having one LAN - no VLAN's - nothing.
    Do nothing else.

    Do the ping test.
    If no ok, ditch your system.
    If ok, you know it's your setup .... (config).



  • Aha, I figured out why the MGMT net rules were not working. I had edited my config.xml and renamed my opt interfaces. After switching the naming convention back to opt1, opt2, ..., we achieve normal operation.


Log in to reply