Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Default deny overriding pass rule on one interface

    Scheduled Pinned Locked Moved Firewalling
    23 Posts 3 Posters 1.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      amorimpermissus @Griffo
      last edited by amorimpermissus

      @griffo Yeah I'll do the ping test in a bit. But it's definitely in arp
      Screenshot_20210214-034218__01.jpg

      Here's ifconfig

      flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
      	options=80003<RXCSUM,TXCSUM,LINKSTATE>
      	ether 40:62:31:03:e5:f9
      	inet6 fe80::4262:31ff:fe03:e5f9%re1.30 prefixlen 64 scopeid 0x9
      	inet 192.168.30.1 netmask 0xffffff00 broadcast 192.168.30.255
      	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
      	media: Ethernet autoselect (1000baseT <full-duplex>)
      	status: active
      	vlan: 30 vlanpcp: 0 parent interface: re1
      	groups: vlan
      
      1 Reply Last reply Reply Quote 0
      • A
        amorimpermissus @Griffo
        last edited by

        @griffo here's the pcap for ping test. I just used the built in ping from diagnostics. FYI I disabled windows firewall before I ran the ping with netsh advfirewall set allprofiles state off

        PING 192.168.30.100 (192.168.30.100): 56 data bytes
        
        --- 192.168.30.100 ping statistics ---
        3 packets transmitted, 0 packets received, 100.0% packet loss
        
        03:59:28.743718 40:62:31:03:e5:f9 > 54:ee:75:3b:ab:f9, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 135, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->bc6c)!)
            192.168.30.1 > 192.168.30.100: ICMP echo request, id 62039, seq 0, length 64
        03:59:29.745060 40:62:31:03:e5:f9 > 54:ee:75:3b:ab:f9, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 30776, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->44bb)!)
            192.168.30.1 > 192.168.30.100: ICMP echo request, id 62039, seq 1, length 64
        03:59:30.746068 40:62:31:03:e5:f9 > 54:ee:75:3b:ab:f9, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 27366, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->520d)!)
            192.168.30.1 > 192.168.30.100: ICMP echo request, id 62039, seq 2, length 64
        03:59:31.314871 54:ee:75:3b:ab:f9 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.30.1 tell 192.168.30.100, length 46
        03:59:31.314885 40:62:31:03:e5:f9 > 54:ee:75:3b:ab:f9, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.30.1 is-at 40:62:31:03:e5:f9, length 28
        03:59:34.840063 c8:d1:2a:c3:b0:93 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 56: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.1.254 tell 192.168.10.5, length 42
        03:59:37.644328 54:ee:75:3b:ab:f9 > 40:62:31:03:e5:f9, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.30.1 (40:62:31:03:e5:f9) tell 192.168.30.100, length 46
        03:59:37.644341 40:62:31:03:e5:f9 > 54:ee:75:3b:ab:f9, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Reply 192.168.30.1 is-at 40:62:31:03:e5:f9, length 28
        
        johnpozJ 1 Reply Last reply Reply Quote 0
        • johnpozJ
          johnpoz LAYER 8 Global Moderator @amorimpermissus
          last edited by

          This is showing an outbound rule

          outbound.png

          That means the traffic was blocked on the outbound direction.

          You can only place outbound rules in floating.

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.8, 24.11

          A 2 Replies Last reply Reply Quote 0
          • A
            amorimpermissus @johnpoz
            last edited by

            @johnpoz Hi and thanks so much for your response. I will add a floating rule and test, but I am also confused. As shown, I don't have any floating rules in existence. I created LAN30 (re1.30) the exact same way as LAN10 (re1.10). The only differences between the two are their VLAN and subnet. What did I misconfigure / forget that makes the LAN30 traffic behave differently than the LAN10 traffic (which has no problems getting to 192.168.10.1)?

            johnpozJ 1 Reply Last reply Reply Quote 0
            • johnpozJ
              johnpoz LAYER 8 Global Moderator @amorimpermissus
              last edited by

              I have no idea what you could of have done. I am just saying that the log is showing that is blocked on the outbound

              And your traffic would be outbound as well that is your SA back to your client 30.100.. This would be outbound traffic from pfsense..

              An intelligent man is sometimes forced to be drunk to spend time with his fools
              If you get confused: Listen to the Music Play
              Please don't Chat/PM me for help, unless mod related
              SG-4860 24.11 | Lab VMs 2.8, 24.11

              1 Reply Last reply Reply Quote 0
              • A
                amorimpermissus @johnpoz
                last edited by

                @johnpoz OK, just for the record, I created this floating rule:
                Screenshot_20210214-094022.jpg

                I then waited for filters to reload and reset states. This is (still) the result when attempting to access 192.168.30.1 from 192.168.30.100:
                Screenshot_20210214-094845__01.jpg

                (Apologies for not being able to zoom out more, doing the log viewing from my cell phone)

                johnpozJ 1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator @amorimpermissus
                  last edited by

                  You got something really odd going on that is for sure.

                  Something is clearly messed up in you rules.. Since there would not be a default rule on the outbound side of any interface.

                  I would start over ;)

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  A 1 Reply Last reply Reply Quote 0
                  • A
                    amorimpermissus @johnpoz
                    last edited by

                    @johnpoz I've already completely blown away the interface config and recreated from scratch.

                    This is the full output of pfctl -sr (specific WAN address and explicit port rules redacted for security):
                    https://pastebin.com/VByW2jet

                    If you (or anyone!) is willing to look through it, I would be immensely grateful. I would like to avoid re-configuring the entire system, especially since I might just go through everything only to have the problem continue.

                    Thanks everyone so much for your time, even if you aren't able or willing!

                    My main concern with reinstall and reconfig is I have configured (but completely disabled) @marcelloc 's (you rock!) amazing E2Guardian package. I can't find any clear details about what files / folders to make an external backup of if I want to export and re-import the settings. By default, it bundles its backups into the auto-config backup settings, so I wouldn't be able to restore the E2Guardian settings without also restoring any (presumably-corrupted) rules, too. Not asking for anyone to solve that problem here, because I know it's not an official package. I'm just explaining why I'm so reluctant to blow the entire install away and start over =)

                    johnpozJ 1 Reply Last reply Reply Quote 0
                    • johnpozJ
                      johnpoz LAYER 8 Global Moderator @amorimpermissus
                      last edited by

                      There is a default block out rule, but that should only come into play where there is no state.

                      Off the top, guess its possible that if traffic is being seen inbound on different interface than the vlan30 interface. Trying to send out different interface would be stopped by the default deny..

                      When you ping pfsense 30.1 from 30.100

                      What interface are you seeing the state create on. Or when you try and hit the gui, which is that 443 SA traffic your seeing..

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                      A 1 Reply Last reply Reply Quote 0
                      • A
                        amorimpermissus @johnpoz
                        last edited by

                        @johnpoz Thanks for looking, I will check the states, shortly when trying to access the web gui on .30.1. I don't think it creates a state for ICMP though does it?

                        johnpozJ 1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator @amorimpermissus
                          last edited by johnpoz

                          yeah you will see a "state" created for icmp

                          pingstate.png

                          I started a constant ping to pfsense from my pc, then looking in the state table I see it.

                          edit: Off the top of my head that only thing that is coming to mind where the default block out deny rule would come into play. Is if there is no state for answer, which I would think could only happen when your trying to answer via a different interface than the traffic came in on.

                          So you have IP set on the native interface your re1, and only have vlans 10 for your lan and 30 for your other vlan sitting on this physical interface.

                          If traffic came into re1 without a tag, and source IP was 30.x then pfsense would try and answer via its vlan30 interface. But would be denied because no state on that interface - that is a GUESS to possible scenario that could maybe cause what your seeing..

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                          A 1 Reply Last reply Reply Quote 0
                          • A
                            amorimpermissus @johnpoz
                            last edited by

                            @johnpoz Gotcha. Regardless, I started a pcap on the firewall, listening on the LAN30 interface for 192.168.30.100, along with opening up the states page in another tab and filtering on all interfaces to 192.168.30.100. Then I opened https://192.168.30.1 on my laptop. I saw it creating a TCP connection on 443 in Windows perfmon. On the pcap, both the pings and the https traffic showed up, but in the states page, it stayed at "No states were found that match the current filter," no matter how many times I refreshed.

                            If you don't see anything misconfigured in the pastebin of my firewall rules, I'm leaning toward thinking some system files (or disk sectors) are corrupted. It's just the only logical explanation at this point. Plus, I used UFS when I built this box, so it doesn't have the same protection from bitrot that ZFS does, even when you aren't using ECC memory.

                            I'm going to see if I can figure out what files/folders the E2Guardian package lands in, and copy those off. Then I'll wipe and reinstall. I hope it's not something inherently wrong with my hardware, but I'll update here once I reconfigure everything. That way you all will know whether it worked or not.

                            Thanks again, everyone, for your help!

                            johnpozJ A 2 Replies Last reply Reply Quote 0
                            • johnpozJ
                              johnpoz LAYER 8 Global Moderator @amorimpermissus
                              last edited by

                              Well a state can not be created unless allowed.

                              I didn't go through all your rules.. But if your saying your seeing traffic hit the firewall

                              But nothing logged on the inbound?

                              Your sure the traffic was tagged with vlan30? You can do a tcpdump on pfsense with the -e to view vlan info..

                              example

                              I did a tcpdump -i igb2 -e

                              Where igb2 is parent interface for multiple vlans - you can see here traffic from vlan4

                              11:48:12.539018 00:08:a2:0c:e6:20 (oui Unknown) > 68:57:2d:98:36:83 (oui Unknown), ethertype 802.1Q (0x8100), length 58: vlan 4, p 0, ethertype IPv4, ec2-54-187-112-38.us-west-2.compute.amazonaws.com.https > 192.168.4.202.53966: Flags [F.], seq 717, ack 851, win 3752, length 0
                              11:48:12.541636 68:57:2d:98:36:83 (oui Unknown) > 00:08:a2:0c:e6:20 (oui Unknown), ethertype 802.1Q (0x8100), length 60: vlan 4, p 0, ethertype IPv4, 192.168.4.202.53966 > ec2-54-187-112-38.us-west-2.compute.amazonaws.com.https: Flags [.], ack 718, win 3663, length 0
                              

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.8, 24.11

                              1 Reply Last reply Reply Quote 0
                              • A
                                amorimpermissus @amorimpermissus
                                last edited by amorimpermissus

                                @johnpoz Yeah, the log data I posted the screenshots of are, consistently, the only thing appearing in the firewall logs, period, for traffic on that interface/subnet. Even when I do a continual ping from .30.100 to .30.1, the ICMP traffic never shows up in the firewall logs at all, but it's definitely showing up in tcpdumps:

                                https://pastebin.com/rR36L1QJ

                                And a tcpdump on re1 shows up as tagged for that MAC:

                                [2.4.5-RELEASE]/:  tcpdump -i re1 -e | grep 54:ee:75:3b:ab:f9
                                listening on re1, link-type EN10MB (Ethernet), capture size 262144 bytes
                                12:58:40.912971 54:ee:75:3b:ab:f9 (oui Unknown) > 40:62:31:03:e5:f9 (oui Unknown), ethertype 802.1Q (0x8100), length 78: vlan 30, p 0, ethertype IPv4, SUPPORT3.demasci.intra > 192.168.30.1: ICMP echo request, id 1, seq 192, length 40
                                12:58:45.914341 54:ee:75:3b:ab:f9 (oui Unknown) > 40:62:31:03:e5:f9 (oui Unknown), ethertype 802.1Q (0x8100), length 78: vlan 30, p 0, ethertype IPv4, SUPPORT3.demasci.intra > 192.168.30.1: ICMP echo request, id 1, seq 193, length 40
                                

                                This is also confirmed by the fact that my laptop always receives an IP address from the DHCP server listening on LAN30. It's so. Freaking. Weird.

                                johnpozJ 1 Reply Last reply Reply Quote 0
                                • johnpozJ
                                  johnpoz LAYER 8 Global Moderator @amorimpermissus
                                  last edited by

                                  @amorimpermissus

                                  What rules do you have on port forwards, those are evaluated before.. You don't see them with your -sr

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                                  A 1 Reply Last reply Reply Quote 1
                                  • A
                                    amorimpermissus @johnpoz
                                    last edited by

                                    @johnpoz Eh, don't remember exactly-- blew the whole install away and reinstalled from scratch. After setup, same problem as before.

                                    Even tried a different vlan again-- 50 instead of 30. Made no difference.

                                    Moved the port configuration to a different switchport on my managed switch... it worked on VLAN 50.

                                    Deleted the re1.50, recreated it as re1.30. Created the allow all firewall rule.

                                    Traffic now working without any issues. I think I may have just put myself through hell for nothing, reinstalling and everything when it was something strange on my switch. But now my firewall is running ZFS instead of UFS, so that's good at least.

                                    THE MORAL OF THE STORY IS THAT, EVEN IF:
                                    -There are no errors in ethtool
                                    -There is every indication that the packets are uncorrupted based on packet captures
                                    -You have switched out all of your cables

                                    IT COULD STILL BE SOMETHING DUMB ON YOUR SWITCH.

                                    For anyone else who might be experiencing this, I am using a Comtrend GS-7408. Maybe all of this information contained herein could help someone else in the future >_<

                                    johnpozJ 1 Reply Last reply Reply Quote 0
                                    • johnpozJ
                                      johnpoz LAYER 8 Global Moderator @amorimpermissus
                                      last edited by

                                      What doesn't make any sense with the switch being the problem is that you were seeing stuff tagged with vlan 30, but it was being blocked outbound..

                                      Some piece of the puzzle is missing to what was actually going on.. But at least you got it working. And zfs is plus coming out of it. I am waiting for 2.5 to drop and then will do a clean install myself to go to zfs on my sg4860.

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                                      G 1 Reply Last reply Reply Quote 0
                                      • G
                                        Griffo @johnpoz
                                        last edited by

                                        In one of your earlier posts, the dump shows some "Bad Checksum" packets at the beginning. That indicates to me a hardware or switchport config issue, like the vlan tagging was not right leading to framing issues. I tried to post that last night but was still in draft on my phone this morning :-( Glad you fixed the issue.

                                        A 1 Reply Last reply Reply Quote 1
                                        • A
                                          amorimpermissus @Griffo
                                          last edited by

                                          @griffo Doh! Oh well, better late than never lol

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.