• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Going nuts with DNS over OpenVPN

Scheduled Pinned Locked Moved General pfSense Questions
16 Posts 3 Posters 1.4k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • G
    gelcom
    last edited by gelcom Mar 3, 2022, 1:40 PM Mar 3, 2022, 1:38 PM

    Dear all, I think this problem is very common but I could not find a way to avoid it.

    I have a local pfSense running 2 OpenVPN clients to another countries.

    My goal is:

    • Interface "A" - route all packets through OpenVPN client "A"
    • Interface "B" - route all packets through OpenVPN client "B"
    • All other interfaces - route all packets through WAN

    What have I done so far:
    Setup OpenVPN client "A" and client "B" in my local pfSense as follows:

    TUN
    IPv4 tunnel: blank
    IPv4 remote network: 0.0.0.0/0
    Don't pull routes: not checked
    Don't add/remove routes: not checked
    Pull DNS: not checked
    

    OpenVPN server "A" and "B" in the other end:

    port 1194
    proto udp
    dev tun
    user nobody
    group nogroup
    persist-key
    persist-tun
    keepalive 10 120
    topology subnet
    server XX.XXX.XXX.0 255.255.255.0
    ifconfig-pool-persist ipp.txt
    push "dhcp-option DNS 1.1.1.1"
    dh none
    ecdh-curve prime256v1
    tls-crypt tls-crypt.key
    crl-verify crl.pem
    ca ca.crt
    cert server.crt
    key server.key
    auth SHA256
    cipher AES-128-GCM
    ncp-ciphers AES-128-GCM
    tls-server
    tls-version-min 1.2
    tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
    client-config-dir /etc/openvpn/ccd
    status /var/log/openvpn/status.log
    verb 3
    

    Then I assigned each OpenVPN client to an interface: "OpenVPN_A" and "OpenVPN_B".

    No firewal rules on these interfaces.
    No firewall rules on "master" OpenVPN interface either.

    Then I created GW_A and GW_B on System/Routing/Gatways with Disable Gateway Monitoring Action checked so my connection won't "leak" to default WAN if openvpn interface is down.

    Then I created NAT_OpenVPN_A on Firewall/NAT/Outbound with interface "OpenVPN_A". Source is "any", source port "", destination "", NAT Address set to "OpenVPN_A address", Nat port "*". I have the same to NAT_OpenVPN_B, changing interface accordenly.

    Then I went to interfaces where I want my VMs to be routed through these VPNs:

    Firewall/NAT/Outbound with interface "Interface_VMs_through_A". Source is "any", source port "", destination "", NAT Address set to "Interface_VMs_through_A address", Nat port "*". I have the same to Interface_VMs_through_B, changing interface accordenly.

    Then:
    Services/DHCP Server/Interface_VMs_through_A:

    Enable DHCP Server
    DNS servers: 1.1.1.1
    NTP server: pool.ntp.org
    

    Same for Interface_VMs_through_B.

    Then added this ONLY rule to interfaces to assure everything goes through VPN tunnel:

    Firewall/Rules/Interface_VMs_through_A:

    Protocol: IPv4
    Source: *
    Port: *
    Destination: *
    Advanced Options/Gateway: OpenVPN_A
    

    Same for Interface_VMs_through_B but with Advanced Options/Gateway: OpenVPN_B instead.

    Then I set a port forward to intercept unwanted DNS queries to other servers:

    Firewall/NAT/Port Forward:

    Interface: Interface_VMs_through_A
    Protocol: TCP/UDP
    Source address: *
    Source ports: *
    Dest address: ! 1.1.1.1
    Dest port: 53 (DNS)
    NAT IP: 1.1.1.1
    NAT Port: 53 (DNS)
    

    Interface: Interface_VMs_through_B

    Protocol: TCP/UDP
    Source address: *
    Source ports: *
    Dest address: ! 1.1.1.1
    Dest port: 53 (DNS)
    NAT IP: 1.1.1.1
    NAT Port: 53 (DNS)
    

    On System/General Setup/DNS Server Settings I have:

    Servers setting: 1.1.1.1 and 1.0.0.1
    DNS Server Override: unchecked
    DNS Resolution Behavior: Use local DNS (127.0.0.1), fall back to remote DNS Servers (Default)
    

    On Services/DNS Resolver/General Settings I have

    Enable DNS resolver: checked
    Enable SSL/TLS Service: checked
    SSL/TLS Certificate: my certificate
    SSL/TLS Listen Port:853
    Network Interfaces: All
    Outgoing Network Interfaces: All
    Strict Outgoing Network Interface Binding: unchecked
    System Domain Local Zone Type: Transparent
    DNSSEC: unchecked
    Python Module: unchecked
    DNS Query Forwarding:
    Enable Forwarding Mode: checked
    Use SSL/TLS for outgoing DNS Queries to Forwarding Servers: checked
    Register DHCP leases in the DNS Resolver: checked
    Register DHCP static mappings in the DNS Resolver: checked
    Register connected OpenVPN clients in the DNS Resolver: checked.
    

    Configs done.

    When I go to any machine on Interface_VMs_through_A and try to access "whatsmyip" it correctly shows IP Address from OpenVPN_A server. GREAT!!!

    When I go to any machine on Interface_VMs_through_B and try to access "whatsmyip" it correctly shows IP Address from OpenVPN_B server. GREAT!!!

    When I go to www.dnsleaktest.com it shows the IP address of my local WAN connection and my actual location and not remote IP and remote location as it should. Something is "leaking" from openVPN tunnel.

    What am I missing here?

    kind regards

    1 Reply Last reply Reply Quote 0
    • G
      gelcom
      last edited by Mar 3, 2022, 2:06 PM

      Strange is I see a lot of these messages on my interface logs passing traffic:

      NEGATE_ROUTE: Negate policy routing for destination (10000001)

      Maybe something is creating hidden rules I don't control?

      1 Reply Last reply Reply Quote 0
      • D
        dma_pf
        last edited by Mar 3, 2022, 2:20 PM

        Are your routing your DNS requests out through your VPN provider in the DNS Resolver settings ( Services/DNS Resolver/General Settings) like this?
        7ec4baa9-3787-487d-8e46-5ff492488c3c-image.png

        G 1 Reply Last reply Mar 3, 2022, 2:33 PM Reply Quote 0
        • G
          gelcom @dma_pf
          last edited by Mar 3, 2022, 2:33 PM

          @dma_pf said in Going nuts with DNS over OpenVPN:

          Are your routing your DNS requests out through your VPN provider in the DNS Resolver settings ( Services/DNS Resolver/General Settings) like this?

          It's set to ALL.

          D 1 Reply Last reply Mar 3, 2022, 2:39 PM Reply Quote 0
          • D
            dma_pf @gelcom
            last edited by Mar 3, 2022, 2:39 PM

            @gelcom When you do an "extended" dns leak test in this site what are the results? https://www.dnsleaktest.com

            G 1 Reply Last reply Mar 3, 2022, 2:52 PM Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by Mar 3, 2022, 2:46 PM

              Do you control the server end of these tunnels?
              If not they are almost certainly pushing a default route to connecting clients and it you don't reject it the pfSense default route will be changed.

              Never use 'any' as the source in Outbound NAT rules if you can possibly avoid it (and you can). It can catch unexpected traffic, like traffic from the firewall itself.

              You shouldn't have outbound NAT rules on the internal interfaces.

              Consider using a different DNS serer locally. By using 1.1.1.1 everywhere you are creating routing conflicts and making it far more difficult to diagnose the issues.
              You want clients on those interfaces to use only 1.1.1.1 for DNS and to route those queries over VPN.

              Disabling gateway monitoring action will not prevent traffic leaking to WAN if VPN goes down and the gateway disappears entirely.
              In Sys > Adv > Misc set 'Skip rules when gateway is down'.
              That way if the gateway goes down there will be no pass rules and traffic will simply be blocked.

              Steve

              1 Reply Last reply Reply Quote 0
              • G
                gelcom @dma_pf
                last edited by Mar 3, 2022, 2:52 PM

                @dma_pf said in Going nuts with DNS over OpenVPN:

                When you do an "extended" dns leak test in this site what are the results? https://www.dnsleaktest.com

                6da7789c-44ce-454a-9212-9183a8282d8d-image.png

                Kindly note "São Paulo, Brazil" is local pf and not remote VPN.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by Mar 3, 2022, 3:13 PM

                  Can we see a screenshot of the firewall rules on Interface_VMs_through_A. Assuming that is where you're testing from.

                  Steve

                  G 1 Reply Last reply Mar 3, 2022, 3:41 PM Reply Quote 0
                  • G
                    gelcom @stephenw10
                    last edited by Mar 3, 2022, 3:41 PM

                    @stephenw10 said in Going nuts with DNS over OpenVPN:

                    Do you control the server end of these tunnels?
                    If not they are almost certainly pushing a default route to connecting clients and it you don't reject it the pfSense default route will be changed.

                    Yes, I do. OpenVPN server is a VM on my Oracle Cloud.

                    Pls note my openvpn server.conf:

                    port 1194
                    proto udp
                    dev tun
                    user nobody
                    group nogroup
                    persist-key
                    persist-tun
                    keepalive 10 120
                    topology subnet
                    server XX.XXX.XXX.0 255.255.255.0
                    ifconfig-pool-persist ipp.txt
                    push "dhcp-option DNS 1.1.1.1"
                    dh none
                    ecdh-curve prime256v1
                    tls-crypt tls-crypt.key
                    crl-verify crl.pem
                    ca ca.crt
                    cert server.crt
                    key server.key
                    auth SHA256
                    cipher AES-128-GCM
                    ncp-ciphers AES-128-GCM
                    tls-server
                    tls-version-min 1.2
                    tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
                    client-config-dir /etc/openvpn/ccd
                    status /var/log/openvpn/status.log
                    verb 3
                    

                    Never use 'any' as the source in Outbound NAT rules if you can possibly avoid it (and you can). It can catch unexpected traffic, like traffic from the firewall itself.
                    You shouldn't have outbound NAT rules on the internal interfaces.

                    Ok! Corrected source to Interface_VMs_through_A subnet

                    Consider using a different DNS serer locally. By using 1.1.1.1 everywhere you are creating routing conflicts and making it far more difficult to diagnose the issues.
                    You want clients on those interfaces to use only 1.1.1.1 for DNS and to route those queries over VPN.

                    Indeed. I changed local pfsense DNS servers to google (8.8.8.8 and 8.8.4.4) and I kept openVPN to Cloudfare (1.1.1.1).

                    With all these corrections problems persists:

                    When I run https://www.dnsleaktest.com on local machine it points correctly to google DNS and local location (São Paulo).

                    When I run the test on "Interface_VMs_through_A" machine it shows the same WAN address (incorrect) with Cloudfare DNS (correct).

                    Still leaking.

                    Can we see a screenshot of the firewall rules on Interface_VMs_through_A. Assuming that is where you're testing from.

                    Sure. Here we go

                    cb9858c2-b76a-4212-bf36-dfb991b702e2-image.png

                    Please note I keep seeing logs in this interface pointing to 1.1.1.1 with a rule name I don't have: NEGATE_ROUTE: Negate policy routing for destination (10000001) .

                    Maybe this "hidden" rule is leaking my DNS?

                    3509cd68-07e9-4cde-b830-5ba237270c28-image.png

                    kind regards

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by Mar 3, 2022, 3:57 PM

                      @gelcom said in Going nuts with DNS over OpenVPN:

                      NEGATE_ROUTE: Negate policy routing for destination (10000001) .

                      Ah, it's because you have or had set 1.1.1.1 as a DNS server for pfSense with a gateway and that adds a static route to it creating the conflict triggering the negate rules. If you reboot removing that route it will probably go away but to be sure (or to not reboot) go to Sys > Adv > Firewall&NAT and set 'Disable Negate rules'.

                      Steve

                      G 1 Reply Last reply Mar 3, 2022, 4:06 PM Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by Mar 3, 2022, 4:04 PM

                        Actually, ironically, it's probably because you're pushing 1.1.1.1 as a DNS server from the OpenVPN server which means it is included in the VPN networks alias.
                        Disabling negate networks should prevent it anyway.

                        Steve

                        1 Reply Last reply Reply Quote 0
                        • G
                          gelcom @stephenw10
                          last edited by Mar 3, 2022, 4:06 PM

                          @stephenw10 said in Going nuts with DNS over OpenVPN:

                          Ah, it's because you have or had set 1.1.1.1 as a DNS server for pfSense with a gateway and that adds a static route to it creating the conflict triggering the negate rules. If you reboot removing that route it will probably go away but to be sure (or to not reboot) go to Sys > Adv > Firewall&NAT and set 'Disable Negate rules'.

                          Rebooting pf did not the trick but checking Disable Negate rules did.

                          Now leakdns shows remote openvpn ip and remote location. GREAT!!!

                          Is this an expected behavior on pf?

                          With this option turned on should I expect any other changes on pfSense?

                          Isn't possible to remove that "hidden" static route instead of enabling this option?

                          kind regards

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by Mar 3, 2022, 4:22 PM

                            Check the routing table to see where 1.1.1.1 is added.
                            But it's probably because your servers are configured to pass is to the clients and your clients are NOT configured to reject routes.

                            But in a setup like yours you should disable negate networks anyway since you are relying on policy routing for isolation.

                            Steve

                            G 1 Reply Last reply Mar 3, 2022, 5:25 PM Reply Quote 0
                            • G
                              gelcom @stephenw10
                              last edited by Mar 3, 2022, 5:25 PM

                              @stephenw10 said in Going nuts with DNS over OpenVPN:

                              Check the routing table to see where 1.1.1.1 is added.

                              No entries for 1.1.1.1 ....

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by Mar 3, 2022, 6:11 PM

                                Might be used in an IPSec tunnel if you have any defined. It pretty much has to be a subnet used somewhere in a VPN config to hit that rule.

                                Steve

                                G 1 Reply Last reply Mar 3, 2022, 11:24 PM Reply Quote 0
                                • G
                                  gelcom @stephenw10
                                  last edited by Mar 3, 2022, 11:24 PM

                                  I don’t know. I don’t have any other vpns setup.

                                  Anyway…

                                  Thanks for the support.

                                  @stephenw10: I owe you a beer!!! :-)

                                  Problem is solved.

                                  Kind regards

                                  1 Reply Last reply Reply Quote 1
                                  1 out of 16
                                  • First post
                                    1/16
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                    This community forum collects and processes your personal information.
                                    consent.not_received