Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Found a quirk w/ pfSense on EC2.. Hope this helps someone else

    Scheduled Pinned Locked Moved General pfSense Questions
    4 Posts 2 Posters 682 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • W
      whosmatt
      last edited by whosmatt

      It appears that the DNS servers assigned by DHCP in EC2 get an entry in the route table that puts them on the local interface (the mac address) as shown by 'netstat -r'. These servers are in a different subnet than pfSense and as such are accessible by the default route. Bad behavior only manifested itself when routing traffic through an IPSEC tunnel terminated on the EC2 end by pfSense. Any traffic destined for those DNS server IP addresses (which, in my case, also happen to be Active Directory DCs) coming over the IPSEC tunnel never exits the single interface of the EC2 instance. Instead, all I see is pfSense ARP requests for the IP of DNS server.

      Local access is fine; pfSense itself can ping and otherwise access the DNS servers in question; it's only traffic coming from the other side of the tunnel that has an issue.

      Fix is to delete those routes. After that, pfSense routes traffic from the other side of the tunnel to those specific IPs just fine. For a fix that survives a reboot, manually assign the DNS servers and assign them the default route in the pfSense GUI.

      IMO this seems like a bug.

      If there's a better way to report this, let me know.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, so the DNS servers supplied are in a different subnet?

        Can we see any example netstat output?

        Steve

        W 1 Reply Last reply Reply Quote 0
        • W
          whosmatt @stephenw10
          last edited by whosmatt

          @stephenw10 Sure. Here's the current state of a second one I just deployed in another region that has the same VPC layout:

          Output of of netstat -r

          Internet:
          Destination        Gateway            Flags     Netif Expire
          default            10.251.253.33      UGS         xn0
          10.251.251.67      06:06:92:93:60:92  UHS         xn0
          10.251.251.252     06:06:92:93:60:92  UHS         xn0
          10.251.253.32/27   link#5             U           xn0
          10.251.253.55      link#5             UHS         lo0
          10.252.252.116     10.251.253.33      UGHS        xn0
          10.252.252.245     10.251.253.33      UGHS        xn0
          104.43.216.101     10.251.253.33      UGHS        xn0
          localhost          link#2             UH          lo0
          172.19.0.1         link#2             UH          lo0
          

          10.251.251.67 and 10.251.251.252 are the DNS servers.
          For reference:

          xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                  options=3<RXCSUM,TXCSUM>
                  ether 06:06:92:93:60:92
                  hwaddr 06:06:92:93:60:92
                  inet6 fe80::406:92ff:fe93:6092%xn0 prefixlen 64 scopeid 0x5
                  inet 10.251.253.55 netmask 0xffffffe0 broadcast 10.251.253.63
                  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                  media: Ethernet manual
                  status: active
          
          1 Reply Last reply Reply Quote 0
          • W
            whosmatt
            last edited by

            And here's what the routes look like after manually assiging the DNS servers in the GUI, assigning them the default gateway, and unchecking "Allow DNS server list to be overridden by DHCP/PPP on WAN"

            Internet:
            Destination        Gateway            Flags     Netif Expire
            default            10.251.253.33      UGS         xn0
            10.251.251.67      10.251.253.33      UGHS        xn0
            10.251.251.252     10.251.253.33      UGHS        xn0
            10.251.253.32/27   link#5             U           xn0
            10.251.253.55      link#5             UHS         lo0
            10.252.252.245     10.251.253.33      UGHS        xn0
            104.43.216.101     10.251.253.33      UGHS        xn0
            localhost          link#2             UH          lo0
            172.19.0.1         link#2             UH          lo0
            

            Now traffic from the other side of an IPSEC tunnel can reach the DNS server IP addresses.

            1 Reply Last reply Reply Quote 0
            • First post
              Last post
            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.