Default deny rule IPv4 (1000000103) except ICMP



  • Hello pfSense Forum,

    Thanks for any help and suggestions in advance... a bit of a weird problem I'm encountering here that seems to have been encountered before (but not necessarily a solution provided - more an explanation of why the firewall does what it does with this rule). In my case, it's breaking routing it seems, though.

    I'm getting Default deny rule IPv4 (1000000103) on all packets coming from another LAN subnet (10.0.8.0/24)... except ICMP (ie: I can ping all hosts and firewall interfaces).

    I'll elaborate further though as to why I have this setup (maybe you can make a suggestion that will work better).

    I have an ISP that allows me 2x Dynamic IP's, both in different subnet ranges that are usable on my WAN(s).

    So I have one physical pfSense Firewall running everything:

    pfSense-1:

    WAN (Internet/NAT for whole house)
    LAN (192.168.1.0/24 VLAN1 - main subnet for whole house)
    Guest (192.168.69.0/24 VLAN69 - guest subnet)
    Physical Link to VLAN99 (192.168.99.2/24)

    I have a second (virtual, proxmox VM) pfSense Firewall running as a secondary back-door to my network... in case of primary pfsense-1 failure, or my internal DNS/main servers fail, or something:

    pfSense-2:

    WAN (Internet/NAT for nothing but backdoor access and OpenVPN)
    Backdoor VLAN (192.168.99.0/24 VLAN99)
    OpenVPN (10.0.8.0/24 this is a Subnet set up by OpenVPN for client/tunnel)
    Physical Link to VLAN1 (192.168.1.2/24)

    I've got what I believe to be appropriate rules in place for routing and gateway's, static routes (ie: pfSense-1 has a LAN gateway set up as 192.168.99.1... which is used for Static Route to 10.0.8.0/24 for the OpenVPN pfSense Subnet on pfSense-2) and such (all firewall rules are set for all LAN's to allow to and from other LAN's for VLAN1, VLAN99, 10.0.8.0 OpenVPN Subnet).

    The issue I have is from the VLAN99 and OpenVPN Subnets on pfSense-2... clients cannot access clients on VLAN1 on pfSense-1 (ie: visit GUI / browser). But can however ping them just fine (ICMP flows ok).

    For any traffic other than ICMP... I get the "Default deny rule IPv4 (1000000103)" as a firewall rule deny in logs.

    Things I have tried to remedy the situation:

    1. As I've seen in other threads... maybe this is expected behaviour... but I see it as what should be basic routing, shouldn't it be? I see the following referenced, but am not sure it applies in this case: https://docs.netgate.com/pfsense/en/latest/firewall/troubleshooting-blocked-log-entries-for-legitimate-connection-packets.html

    2. I tried the suggestions in the subsequent link in the above page (both automatic using "Bypass firewall rules for traffic on the same interface option located under System > Advanced on the Firewall/NAT tab" and manual using floating rules with TCP Flags and Sloppy State set): https://docs.netgate.com/pfsense/en/latest/firewall/troubleshooting-blocked-log-entries-due-to-asymmetric-routing.html

    3. I tried adding "quick" to the rules above as per this post... but no avail... and my traceroutes from the OpenVPN Subnet, or VLAN99 backdoor VLAN go direct (not through a WAN) https://jrs-s.net/2020/03/20/when-static-routes-on-pfsense-are-ignored/

    Any ideas on what I should do here?

    I do understand the likely cause is Asymmetric routing... but I thought the workarounds above would allow it to be worked around. If you've got any specific guidance on how to avoid it... I'm happy to put networks in between, or whatever I need to.

    The idea of using OpenVPN on the backdoor pfSense is to have sort of an out-of-band access and hopping VLAN's in case my primary pfSense or DNS fails while away for work (once the pandemic is over). I've got a dedicated management port on some of the servers and can reboot things if necessary (yes... there is still a primary failure point of the ISP Modem... but that's an easy thing to get my wife to reboot if necessary).

    All of the VLAN's, Switching and such is set up just fine (pings fine)... it seems to be the firewall specifically blocking traffic that is not ICMP (or maybe just TCP? I haven't checked a UDP-only service).

    Hoping someone can point me in the right direction... maybe I am going about this the wrong way.

    Best Regards,

    dg6464



  • @dg6464 said in Default deny rule IPv4 (1000000103) except ICMP:

    1000000103

    pfSense is a router and firewall.
    Have a look at all the rules that are loaded into the firewall as of right now : Look at /tmp/rules.debug

    You'll find the 4 basic "block all rule" :

    ....
    #---------------------------------------------------------------------------
    # default deny rules
    #---------------------------------------------------------------------------
    block in  inet all tracker 1000000103 label "Default deny rule IPv4"
    block out  inet all tracker 1000000104 label "Default deny rule IPv4"
    block in  inet6 all tracker 1000000105 label "Default deny rule IPv6"
    block out  inet6 all tracker 1000000106 label "Default deny rule IPv6"
    ...
    

    Surprise, the firewall rules listed in the GUI are there also ... and many more.

    You see the "00000103" rule (number) ?

    These 4 terminate the rule list for every existing interface.
    Which means that any interface created, will initially block all incoming traffic.

    That's why WAN is a total block from the start.
    As are all the OPTx interfaces.
    Exception : LAN : it has a pass rule in the GUI that will take precedence.

    Read more and you'll see that DHCP traffic is allowed on any interface ;)

    If you see firewall log messages like "Default deny rule IPv4 (1000000103)" then you have checked :

    cdd53462-c67f-42eb-9f56-d26cac7cd930-image.png

    Normally, you shouldn't check that one - except if you are debugging your GUI rules

    Seeing these message means that traffic is coming into an interface and there was no pass rule that machtes that traffic : so it gets blocked at the and by our 4 default block rules.

    Up to you to see what interface this is.
    What traffic.
    An if blocking is ok - or not. It probably is, because if not, you would have put a pass rule in the GUI for that interface ^^.



  • @Gertjan thanks for your reply.

    I do understand the default behavior and the setup of pfSense.

    The difficulty here... is that I do, in fact have permit rules for the traffic in question, on all of the appropriate interfaces.

    The interfaces rules are set to pass “any” traffic type (TCP, UDP, ICMP), but all I can get is ICMP to pass... TCP gets denied by the default rule.

    Trying to figure out how to fix that... any ideas?

    Is there a particular output that would help if I post here?

    Did you want an output of the file you mentioned, the firewall block message, or a screenshot of the interfaces on each and the rules on each interface?

    Would adding a true /30 transit network as say “VLAN33” between the firewalls fix the issue?

    I have a feeling that this is potentially being caused by the hosts being on VLAN99 (pfSense-2) and VLAN1 (pfSense-1)... but one of those also has to contain the “gateway” for the static route to 10.0.8.0 (the OpenVPN client network on pfSense-2)... meaning they are directly attached/adjacent to one another, which may cause traffic to be sent to one interface, but return via a different one (asymmetric).

    I also have a feeling that having to use a gateway and static route on pfSense-1 means that if it fails... hosts won’t have a route to 10.0.8.0 (their default route / pfSense-1’s static route)... which means I would have to potentially take a different approach to this if I want it to work and use something like CARP to share the VLAN1 default gateway, but that will break the WAN (since they are on 2 different subnets (and dynamic), as for proper CARP I ideally need 3 static IP’s on the same WAN subnet).

    Ugh... at a bit of a loss on how to do this. Any guidance appreciated. Even if it’s “you can’t”, I guess. But it all seems to come down to the permits not working for non-ICMP traffic... but working fine for ICMP. Odd isn’t it?

    Maybe I just attach the second public WAN IP directly to a “jumpbox” server machine with an interface on VLAN1 and forget pfSense-2 altogether? I just need to have an OpenVPN Server and DuckDNS on there as well... which really lends itself to running pfSense :).

    Thanks!

    Best Regards,

    dg6464



  • Just to update the thread... the asymmetrical stuff just doesn't allow me to do what I wanted to do.

    So until the day comes when we see formal asymmetrical support on pfSense, I set my Backdoor system as follows:

    Windows VM, running dual interfaces:

    1 with a direct Public WAN IP set with a metric of 1
    1 with a Private IP hooked to my 192.168.1.0 LAN with a metric of 100

    I set up OpenVPN on the server on the Windows machine and set up Windows Advanced Firewall to only allow 1194 inbound.

    I have also set up a pfSense Transparent Firewall (bridge), running a bridge interface that denies all except UDP Port 67 for WAN DHCP, and 1194 for OpenVPN.

    Both Windows and pfSense running on proxmox.

    Plans to set up Suricata and/or Snort.

    Unfortunately pfBlockerNG-devel doesn't support bridge mode yet :(.

    Hope this helps someone who happens to have a second WAN IP from their ISP and wants to create some sort of back door to their network in case their primary pfSense fails.

    If anyone finds a way around the asymmetrical routing issue - I'd be happy to try a workaround.

    Thanks!

    Best Regards,

    dg6464


  • LAYER 8 Global Moderator

    @dg6464 said in Default deny rule IPv4 (1000000103) except ICMP:

    If anyone finds a way around the asymmetrical routing issue

    Yeah don't do asymmetrical.. You already mentioned the fix..

    Would adding a true /30 transit network as say “VLAN33” between the firewalls fix the issue?

    If your going to have a downstream router.. Then yes it would and should be connected to the upstream router via a transit network (no hosts on this network)

    If your going to put hosts on what is your transit network.. The network between 2 routers is always a transit network.. Hosts don't belong on a transit network.. If your going to do that - then you need to use host routing, ie routes on these hosts to tell them which router to use to get to what network. Or you have to nat all downstream networks via the IP of the downstream router in the transit network.. So this host on the transit doesn't know about them, nor does the upstream router..



  • @johnpoz said in Default deny rule IPv4 (1000000103) except ICMP:

    @dg6464 said in Default deny rule IPv4 (1000000103) except ICMP:

    If anyone finds a way around the asymmetrical routing issue

    Yeah don't do asymmetrical.. You already mentioned the fix..

    Would adding a true /30 transit network as say “VLAN33” between the firewalls fix the issue?

    If your going to have a downstream router.. Then yes it would and should be connected to the upstream router via a transit network (no hosts on this network)

    If your going to put hosts on what is your transit network.. The network between 2 routers is always a transit network.. Hosts don't belong on a transit network.. If your going to do that - then you need to use host routing, ie routes on these hosts to tell them which router to use to get to what network. Or you have to nat all downstream networks via the IP of the downstream router in the transit network.. So this host on the transit doesn't know about them, nor does the upstream router..

    I think it was youhere who gave me the same advice once. It was the best thing I've ever learned. There are so many possibilities and zero NAT problems. <3 Thanks!


Log in to reply