Found a quirk w/ pfSense on EC2.. Hope this helps someone else



  • It appears that the DNS servers assigned by DHCP in EC2 get an entry in the route table that puts them on the local interface (the mac address) as shown by 'netstat -r'. These servers are in a different subnet than pfSense and as such are accessible by the default route. Bad behavior only manifested itself when routing traffic through an IPSEC tunnel terminated on the EC2 end by pfSense. Any traffic destined for those DNS server IP addresses (which, in my case, also happen to be Active Directory DCs) coming over the IPSEC tunnel never exits the single interface of the EC2 instance. Instead, all I see is pfSense ARP requests for the IP of DNS server.

    Local access is fine; pfSense itself can ping and otherwise access the DNS servers in question; it's only traffic coming from the other side of the tunnel that has an issue.

    Fix is to delete those routes. After that, pfSense routes traffic from the other side of the tunnel to those specific IPs just fine. For a fix that survives a reboot, manually assign the DNS servers and assign them the default route in the pfSense GUI.

    IMO this seems like a bug.

    If there's a better way to report this, let me know.


  • Netgate Administrator

    Hmm, so the DNS servers supplied are in a different subnet?

    Can we see any example netstat output?

    Steve



  • @stephenw10 Sure. Here's the current state of a second one I just deployed in another region that has the same VPC layout:

    Output of of netstat -r

    Internet:
    Destination        Gateway            Flags     Netif Expire
    default            10.251.253.33      UGS         xn0
    10.251.251.67      06:06:92:93:60:92  UHS         xn0
    10.251.251.252     06:06:92:93:60:92  UHS         xn0
    10.251.253.32/27   link#5             U           xn0
    10.251.253.55      link#5             UHS         lo0
    10.252.252.116     10.251.253.33      UGHS        xn0
    10.252.252.245     10.251.253.33      UGHS        xn0
    104.43.216.101     10.251.253.33      UGHS        xn0
    localhost          link#2             UH          lo0
    172.19.0.1         link#2             UH          lo0
    

    10.251.251.67 and 10.251.251.252 are the DNS servers.
    For reference:

    xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=3<RXCSUM,TXCSUM>
            ether 06:06:92:93:60:92
            hwaddr 06:06:92:93:60:92
            inet6 fe80::406:92ff:fe93:6092%xn0 prefixlen 64 scopeid 0x5
            inet 10.251.253.55 netmask 0xffffffe0 broadcast 10.251.253.63
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            media: Ethernet manual
            status: active
    


  • And here's what the routes look like after manually assiging the DNS servers in the GUI, assigning them the default gateway, and unchecking "Allow DNS server list to be overridden by DHCP/PPP on WAN"

    Internet:
    Destination        Gateway            Flags     Netif Expire
    default            10.251.253.33      UGS         xn0
    10.251.251.67      10.251.253.33      UGHS        xn0
    10.251.251.252     10.251.253.33      UGHS        xn0
    10.251.253.32/27   link#5             U           xn0
    10.251.253.55      link#5             UHS         lo0
    10.252.252.245     10.251.253.33      UGHS        xn0
    104.43.216.101     10.251.253.33      UGHS        xn0
    localhost          link#2             UH          lo0
    172.19.0.1         link#2             UH          lo0
    

    Now traffic from the other side of an IPSEC tunnel can reach the DNS server IP addresses.


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy