Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Going nuts with DNS over OpenVPN

    Scheduled Pinned Locked Moved General pfSense Questions
    16 Posts 3 Posters 1.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G
      gelcom
      last edited by gelcom

      Dear all, I think this problem is very common but I could not find a way to avoid it.

      I have a local pfSense running 2 OpenVPN clients to another countries.

      My goal is:

      • Interface "A" - route all packets through OpenVPN client "A"
      • Interface "B" - route all packets through OpenVPN client "B"
      • All other interfaces - route all packets through WAN

      What have I done so far:
      Setup OpenVPN client "A" and client "B" in my local pfSense as follows:

      TUN
IPv4 tunnel: blank
IPv4 remote network: 0.0.0.0/0
Don't pull routes: not checked
Don't add/remove routes: not checked
Pull DNS: not checked

      OpenVPN server "A" and "B" in the other end:

      port 1194
proto udp
dev tun
user nobody
group nogroup
persist-key
persist-tun
keepalive 10 120
topology subnet
server XX.XXX.XXX.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 1.1.1.1"
dh none
ecdh-curve prime256v1
tls-crypt tls-crypt.key
crl-verify crl.pem
ca ca.crt
cert server.crt
key server.key
auth SHA256
cipher AES-128-GCM
ncp-ciphers AES-128-GCM
tls-server
tls-version-min 1.2
tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
client-config-dir /etc/openvpn/ccd
status /var/log/openvpn/status.log
verb 3

      Then I assigned each OpenVPN client to an interface: "OpenVPN_A" and "OpenVPN_B".

      No firewal rules on these interfaces.
      No firewall rules on "master" OpenVPN interface either.

      Then I created GW_A and GW_B on System/Routing/Gatways with Disable Gateway Monitoring Action checked so my connection won't "leak" to default WAN if openvpn interface is down.

      Then I created NAT_OpenVPN_A on Firewall/NAT/Outbound with interface "OpenVPN_A". Source is "any", source port "", destination "", NAT Address set to "OpenVPN_A address", Nat port "*". I have the same to NAT_OpenVPN_B, changing interface accordenly.

      Then I went to interfaces where I want my VMs to be routed through these VPNs:

      Firewall/NAT/Outbound with interface "Interface_VMs_through_A". Source is "any", source port "", destination "", NAT Address set to "Interface_VMs_through_A address", Nat port "*". I have the same to Interface_VMs_through_B, changing interface accordenly.

      Then:
      Services/DHCP Server/Interface_VMs_through_A:

      Enable DHCP Server
DNS servers: 1.1.1.1
NTP server: pool.ntp.org

      Same for Interface_VMs_through_B.

      Then added this ONLY rule to interfaces to assure everything goes through VPN tunnel:

      Firewall/Rules/Interface_VMs_through_A:

      Protocol: IPv4
Source: *
Port: *
Destination: *
Advanced Options/Gateway: OpenVPN_A

      Same for Interface_VMs_through_B but with Advanced Options/Gateway: OpenVPN_B instead.

      Then I set a port forward to intercept unwanted DNS queries to other servers:

      Firewall/NAT/Port Forward:

      Interface: Interface_VMs_through_A
Protocol: TCP/UDP
Source address: *
Source ports: *
Dest address: ! 1.1.1.1
Dest port: 53 (DNS)
NAT IP: 1.1.1.1
NAT Port: 53 (DNS)

      Interface: Interface_VMs_through_B

      Protocol: TCP/UDP
Source address: *
Source ports: *
Dest address: ! 1.1.1.1
Dest port: 53 (DNS)
NAT IP: 1.1.1.1
NAT Port: 53 (DNS)

      On System/General Setup/DNS Server Settings I have:

      Servers setting: 1.1.1.1 and 1.0.0.1
DNS Server Override: unchecked
DNS Resolution Behavior: Use local DNS (127.0.0.1), fall back to remote DNS Servers (Default)

      On Services/DNS Resolver/General Settings I have

      Enable DNS resolver: checked
Enable SSL/TLS Service: checked
SSL/TLS Certificate: my certificate
SSL/TLS Listen Port:853
Network Interfaces: All
Outgoing Network Interfaces: All
Strict Outgoing Network Interface Binding: unchecked
System Domain Local Zone Type: Transparent
DNSSEC: unchecked
Python Module: unchecked
DNS Query Forwarding:
Enable Forwarding Mode: checked
Use SSL/TLS for outgoing DNS Queries to Forwarding Servers: checked
Register DHCP leases in the DNS Resolver: checked
Register DHCP static mappings in the DNS Resolver: checked
Register connected OpenVPN clients in the DNS Resolver: checked.

      Configs done.

      When I go to any machine on Interface_VMs_through_A and try to access "whatsmyip" it correctly shows IP Address from OpenVPN_A server. GREAT!!!

      When I go to any machine on Interface_VMs_through_B and try to access "whatsmyip" it correctly shows IP Address from OpenVPN_B server. GREAT!!!

      When I go to www.dnsleaktest.com it shows the IP address of my local WAN connection and my actual location and not remote IP and remote location as it should. Something is "leaking" from openVPN tunnel.

      What am I missing here?

      kind regards

      1 Reply Last reply Reply Quote 0
      • G
        gelcom
        last edited by

        Strange is I see a lot of these messages on my interface logs passing traffic:

        NEGATE_ROUTE: Negate policy routing for destination (10000001)

        Maybe something is creating hidden rules I don't control?

        1 Reply Last reply Reply Quote 0
        • D
          dma_pf
          last edited by

          Are your routing your DNS requests out through your VPN provider in the DNS Resolver settings ( Services/DNS Resolver/General Settings) like this?
          7ec4baa9-3787-487d-8e46-5ff492488c3c-image.png

          G 1 Reply Last reply Reply Quote 0
          • G
            gelcom @dma_pf
            last edited by

            @dma_pf said in Going nuts with DNS over OpenVPN:

            Are your routing your DNS requests out through your VPN provider in the DNS Resolver settings ( Services/DNS Resolver/General Settings) like this?

            It's set to ALL.

            D 1 Reply Last reply Reply Quote 0
            • D
              dma_pf @gelcom
              last edited by

              @gelcom When you do an "extended" dns leak test in this site what are the results? https://www.dnsleaktest.com

              G 1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Do you control the server end of these tunnels?
                If not they are almost certainly pushing a default route to connecting clients and it you don't reject it the pfSense default route will be changed.

                Never use 'any' as the source in Outbound NAT rules if you can possibly avoid it (and you can). It can catch unexpected traffic, like traffic from the firewall itself.

                You shouldn't have outbound NAT rules on the internal interfaces.

                Consider using a different DNS serer locally. By using 1.1.1.1 everywhere you are creating routing conflicts and making it far more difficult to diagnose the issues.
                You want clients on those interfaces to use only 1.1.1.1 for DNS and to route those queries over VPN.

                Disabling gateway monitoring action will not prevent traffic leaking to WAN if VPN goes down and the gateway disappears entirely.
                In Sys > Adv > Misc set 'Skip rules when gateway is down'.
                That way if the gateway goes down there will be no pass rules and traffic will simply be blocked.

                Steve

                1 Reply Last reply Reply Quote 0
                • G
                  gelcom @dma_pf
                  last edited by

                  @dma_pf said in Going nuts with DNS over OpenVPN:

                  When you do an "extended" dns leak test in this site what are the results? https://www.dnsleaktest.com

                  6da7789c-44ce-454a-9212-9183a8282d8d-image.png

                  Kindly note "São Paulo, Brazil" is local pf and not remote VPN.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Can we see a screenshot of the firewall rules on Interface_VMs_through_A. Assuming that is where you're testing from.

                    Steve

                    G 1 Reply Last reply Reply Quote 0
                    • G
                      gelcom @stephenw10
                      last edited by

                      @stephenw10 said in Going nuts with DNS over OpenVPN:

                      Do you control the server end of these tunnels?
                      If not they are almost certainly pushing a default route to connecting clients and it you don't reject it the pfSense default route will be changed.

                      Yes, I do. OpenVPN server is a VM on my Oracle Cloud.

                      Pls note my openvpn server.conf:

                      port 1194
proto udp
dev tun
user nobody
group nogroup
persist-key
persist-tun
keepalive 10 120
topology subnet
server XX.XXX.XXX.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 1.1.1.1"
dh none
ecdh-curve prime256v1
tls-crypt tls-crypt.key
crl-verify crl.pem
ca ca.crt
cert server.crt
key server.key
auth SHA256
cipher AES-128-GCM
ncp-ciphers AES-128-GCM
tls-server
tls-version-min 1.2
tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
client-config-dir /etc/openvpn/ccd
status /var/log/openvpn/status.log
verb 3

                      Never use 'any' as the source in Outbound NAT rules if you can possibly avoid it (and you can). It can catch unexpected traffic, like traffic from the firewall itself.
                      You shouldn't have outbound NAT rules on the internal interfaces.

                      Ok! Corrected source to Interface_VMs_through_A subnet

                      Consider using a different DNS serer locally. By using 1.1.1.1 everywhere you are creating routing conflicts and making it far more difficult to diagnose the issues.
                      You want clients on those interfaces to use only 1.1.1.1 for DNS and to route those queries over VPN.

                      Indeed. I changed local pfsense DNS servers to google (8.8.8.8 and 8.8.4.4) and I kept openVPN to Cloudfare (1.1.1.1).

                      With all these corrections problems persists:

                      When I run https://www.dnsleaktest.com on local machine it points correctly to google DNS and local location (São Paulo).

                      When I run the test on "Interface_VMs_through_A" machine it shows the same WAN address (incorrect) with Cloudfare DNS (correct).

                      Still leaking.

                      Can we see a screenshot of the firewall rules on Interface_VMs_through_A. Assuming that is where you're testing from.

                      Sure. Here we go

                      cb9858c2-b76a-4212-bf36-dfb991b702e2-image.png

                      Please note I keep seeing logs in this interface pointing to 1.1.1.1 with a rule name I don't have: NEGATE_ROUTE: Negate policy routing for destination (10000001) .

                      Maybe this "hidden" rule is leaking my DNS?

                      3509cd68-07e9-4cde-b830-5ba237270c28-image.png

                      kind regards

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        @gelcom said in Going nuts with DNS over OpenVPN:

                        NEGATE_ROUTE: Negate policy routing for destination (10000001) .

                        Ah, it's because you have or had set 1.1.1.1 as a DNS server for pfSense with a gateway and that adds a static route to it creating the conflict triggering the negate rules. If you reboot removing that route it will probably go away but to be sure (or to not reboot) go to Sys > Adv > Firewall&NAT and set 'Disable Negate rules'.

                        Steve

                        G 1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Actually, ironically, it's probably because you're pushing 1.1.1.1 as a DNS server from the OpenVPN server which means it is included in the VPN networks alias.
                          Disabling negate networks should prevent it anyway.

                          Steve

                          1 Reply Last reply Reply Quote 0
                          • G
                            gelcom @stephenw10
                            last edited by

                            @stephenw10 said in Going nuts with DNS over OpenVPN:

                            Ah, it's because you have or had set 1.1.1.1 as a DNS server for pfSense with a gateway and that adds a static route to it creating the conflict triggering the negate rules. If you reboot removing that route it will probably go away but to be sure (or to not reboot) go to Sys > Adv > Firewall&NAT and set 'Disable Negate rules'.

                            Rebooting pf did not the trick but checking Disable Negate rules did.

                            Now leakdns shows remote openvpn ip and remote location. GREAT!!!

                            Is this an expected behavior on pf?

                            With this option turned on should I expect any other changes on pfSense?

                            Isn't possible to remove that "hidden" static route instead of enabling this option?

                            kind regards

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Check the routing table to see where 1.1.1.1 is added.
                              But it's probably because your servers are configured to pass is to the clients and your clients are NOT configured to reject routes.

                              But in a setup like yours you should disable negate networks anyway since you are relying on policy routing for isolation.

                              Steve

                              G 1 Reply Last reply Reply Quote 0
                              • G
                                gelcom @stephenw10
                                last edited by

                                @stephenw10 said in Going nuts with DNS over OpenVPN:

                                Check the routing table to see where 1.1.1.1 is added.

                                No entries for 1.1.1.1 ....

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Might be used in an IPSec tunnel if you have any defined. It pretty much has to be a subnet used somewhere in a VPN config to hit that rule.

                                  Steve

                                  G 1 Reply Last reply Reply Quote 0
                                  • G
                                    gelcom @stephenw10
                                    last edited by

                                    I don’t know. I don’t have any other vpns setup.

                                    Anyway…

                                    Thanks for the support.

                                    @stephenw10: I owe you a beer!!! :-)

                                    Problem is solved.

                                    Kind regards

                                    1 Reply Last reply Reply Quote 1
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.