Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Failover gateway in a multi-WAN setup has odd packets

    Routing and Multi WAN
    2
    8
    430
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mig
      last edited by

      I run 2.4.4-RELEASE-p3 with two WANs: an unlimited cable connection (cheap, "main-WAN") and a MiFi to a local mobile tower (expensive, "WAN-via-MiFi"). Failover works OK but each packet via WAN-via-MiFi costs me something so when the main-WAN connection is up I need the firewall to only (infrequently) send packets to the WAN-via-MiFi "Monitor IP" (it uses 1.1.1.1). I can dial up the probe interval to reduce this cost.

      However, when I capture packets via "WAN-via-MiFi" gateway (192.168.8.101) I see quite a lot of unexpected traffic to IP=9.9.9.9 which is the DNS server for the main-WAN connection specified in the "System/General Setup":

      13:00:40.687525 IP 192.168.8.101 > 1.1.1.1: ICMP echo request, id 39212, seq 10888, length 8
13:00:40.708176 IP 1.1.1.1 > 192.168.8.101: ICMP echo reply, id 39212, seq 10888, length 8
13:00:41.188981 IP 192.168.8.101 > 1.1.1.1: ICMP echo request, id 39212, seq 10889, length 8
13:00:41.207163 IP 1.1.1.1 > 192.168.8.101: ICMP echo reply, id 39212, seq 10889, length 8
13:00:41.359524 IP 192.168.8.101.50727 > 9.9.9.9.853: tcp 0
13:00:41.387279 IP 9.9.9.9.853 > 192.168.8.101.50727: tcp 0
13:00:41.387326 IP 192.168.8.101.50727 > 9.9.9.9.853: tcp 0
13:00:41.387354 IP 192.168.8.101.50727 > 9.9.9.9.853: tcp 311
13:00:41.420094 IP 9.9.9.9.853 > 192.168.8.101.50727: tcp 0
13:00:41.422391 IP 9.9.9.9.853 > 192.168.8.101.50727: tcp 1350
13:00:41.422438 IP 192.168.8.101.50727 > 9.9.9.9.853: tcp 0
13:00:41.422847 IP 9.9.9.9.853 > 192.168.8.101.50727: tcp 1350
13:00:41.422875 IP 192.168.8.101.50727 > 9.9.9.9.853: tcp 0
13:00:41.422878 IP 9.9.9.9.853 > 192.168.8.101.50727: tcp 174
13:00:41.422901 IP 192.168.8.101.50727 > 9.9.9.9.853: tcp 0

      What puzzles me is that the WAN-via-MiFi has a separate DNS server set via "System/General Setup"(and it is NOT 9.9.9.9). FWIW I have "DNS Resolver" enabled (with "DNSSEC" and "Forwarding mode" both on) and "Outgoing Network Interfaces" include both main-WAN and WAN-via-MiFi.

      I would appreciate all ideas about preventing this "9.9.9.9" traffic going into my costly failover link while main-WAN is OK (obviously the solution has to keep the DNS working)

      Thanks in advance!

      1 Reply Last reply Reply Quote 0
      • dotdashD
        dotdash
        last edited by

        That's doing TLS to Quad9. Do you have the resolver enabled with forwarding?

        1 Reply Last reply Reply Quote 0
        • M
          mig
          last edited by

          Yes, I do have DNSSEC enabled in the resolver and Forwarding is also enabled. Are you saying disabling Forwarding should remove the unwanted traffic while maintaining DNSSEC working during failover? (Can't test it now as this would upset all the users!)

          Come to think of it a good way to additionally reduce expensive traffic over WAN-via-MiFi would be by switching to plain DNS when main-WAN is down. Is it possible with pfSense? If so - how?

          1 Reply Last reply Reply Quote 0
          • dotdashD
            dotdash
            last edited by

            I'm just saying the traffic you are seeing is coming from unbound. If you have 'all' selected for outgoing network interfaces, I think it uses both of your wans for queries. I'm not familiar enough with unbound to know if you can make it use the WAN only and fail over to the secondary.

            1 Reply Last reply Reply Quote 1
            • M
              mig
              last edited by

              Thanks, I'll poke around DNS resolver to see if I will be able to fix it without breaking DNS. Need to schedule a maintenance...

              Do you think it is worth posting a question in DNS section of the forum about switching from DNSSEC to plain DNS when failover occurs?

              1 Reply Last reply Reply Quote 0
              • dotdashD
                dotdash
                last edited by

                Sure, can't hurt to ask. I think what you'd need would be unbound querying via the failover gateway instead of using all available interfaces. It's still going to send queries even if it's not using DNSSEC.

                M 1 Reply Last reply Reply Quote 0
                • M
                  mig @dotdash
                  last edited by

                  Switching from DNSSEC to plain DNS when a failover occurs is apparently not possible:
                  [https://forum.netgate.com/topic/150106/switching-from-dnssec-to-plain-dns-in-multi-wan-configuration]

                  1 Reply Last reply Reply Quote 0
                  • M
                    mig
                    last edited by

                    I was able to find a solution to this problem with the help of Netgate admin expert (@stephenw10) advice, the trick was to force Unbound to use "localhost" for outgoing queries. Unbound, when it is configured to operate via localhost outbound gateway would use only the gateway currently chosen by the system (i.e. dictated by the failover rules). This allowed me to completely eliminate unwanted unbound DNS/DNSSEC traffic via an expensive SIM-based failover link.

                    See more details in the following thread:

                    https://forum.netgate.com/topic/150176/php-shell-has-gatewaystatus-but-why-does-it-report-all-gateways-as-status-none

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.