Going nuts with DNS over OpenVPN
-
Dear all, I think this problem is very common but I could not find a way to avoid it.
I have a local pfSense running 2 OpenVPN clients to another countries.
My goal is:
- Interface "A" - route all packets through OpenVPN client "A"
- Interface "B" - route all packets through OpenVPN client "B"
- All other interfaces - route all packets through WAN
What have I done so far:
Setup OpenVPN client "A" and client "B" in my local pfSense as follows:TUN IPv4 tunnel: blank IPv4 remote network: 0.0.0.0/0 Don't pull routes: not checked Don't add/remove routes: not checked Pull DNS: not checked
OpenVPN server "A" and "B" in the other end:
port 1194 proto udp dev tun user nobody group nogroup persist-key persist-tun keepalive 10 120 topology subnet server XX.XXX.XXX.0 255.255.255.0 ifconfig-pool-persist ipp.txt push "dhcp-option DNS 1.1.1.1" dh none ecdh-curve prime256v1 tls-crypt tls-crypt.key crl-verify crl.pem ca ca.crt cert server.crt key server.key auth SHA256 cipher AES-128-GCM ncp-ciphers AES-128-GCM tls-server tls-version-min 1.2 tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256 client-config-dir /etc/openvpn/ccd status /var/log/openvpn/status.log verb 3
Then I assigned each OpenVPN client to an interface: "OpenVPN_A" and "OpenVPN_B".
No firewal rules on these interfaces.
No firewall rules on "master" OpenVPN interface either.Then I created GW_A and GW_B on System/Routing/Gatways with Disable Gateway Monitoring Action checked so my connection won't "leak" to default WAN if openvpn interface is down.
Then I created NAT_OpenVPN_A on Firewall/NAT/Outbound with interface "OpenVPN_A". Source is "any", source port "", destination "", NAT Address set to "OpenVPN_A address", Nat port "*". I have the same to NAT_OpenVPN_B, changing interface accordenly.
Then I went to interfaces where I want my VMs to be routed through these VPNs:
Firewall/NAT/Outbound with interface "Interface_VMs_through_A". Source is "any", source port "", destination "", NAT Address set to "Interface_VMs_through_A address", Nat port "*". I have the same to Interface_VMs_through_B, changing interface accordenly.
Then:
Services/DHCP Server/Interface_VMs_through_A:Enable DHCP Server DNS servers: 1.1.1.1 NTP server: pool.ntp.org
Same for Interface_VMs_through_B.
Then added this ONLY rule to interfaces to assure everything goes through VPN tunnel:
Firewall/Rules/Interface_VMs_through_A:
Protocol: IPv4 Source: * Port: * Destination: * Advanced Options/Gateway: OpenVPN_A
Same for Interface_VMs_through_B but with Advanced Options/Gateway: OpenVPN_B instead.
Then I set a port forward to intercept unwanted DNS queries to other servers:
Firewall/NAT/Port Forward:
Interface: Interface_VMs_through_A Protocol: TCP/UDP Source address: * Source ports: * Dest address: ! 1.1.1.1 Dest port: 53 (DNS) NAT IP: 1.1.1.1 NAT Port: 53 (DNS)
Interface: Interface_VMs_through_B
Protocol: TCP/UDP Source address: * Source ports: * Dest address: ! 1.1.1.1 Dest port: 53 (DNS) NAT IP: 1.1.1.1 NAT Port: 53 (DNS)
On System/General Setup/DNS Server Settings I have:
Servers setting: 1.1.1.1 and 1.0.0.1 DNS Server Override: unchecked DNS Resolution Behavior: Use local DNS (127.0.0.1), fall back to remote DNS Servers (Default)
On Services/DNS Resolver/General Settings I have
Enable DNS resolver: checked Enable SSL/TLS Service: checked SSL/TLS Certificate: my certificate SSL/TLS Listen Port:853 Network Interfaces: All Outgoing Network Interfaces: All Strict Outgoing Network Interface Binding: unchecked System Domain Local Zone Type: Transparent DNSSEC: unchecked Python Module: unchecked DNS Query Forwarding: Enable Forwarding Mode: checked Use SSL/TLS for outgoing DNS Queries to Forwarding Servers: checked Register DHCP leases in the DNS Resolver: checked Register DHCP static mappings in the DNS Resolver: checked Register connected OpenVPN clients in the DNS Resolver: checked.
Configs done.
When I go to any machine on Interface_VMs_through_A and try to access "whatsmyip" it correctly shows IP Address from OpenVPN_A server. GREAT!!!
When I go to any machine on Interface_VMs_through_B and try to access "whatsmyip" it correctly shows IP Address from OpenVPN_B server. GREAT!!!
When I go to www.dnsleaktest.com it shows the IP address of my local WAN connection and my actual location and not remote IP and remote location as it should. Something is "leaking" from openVPN tunnel.
What am I missing here?
kind regards
-
Strange is I see a lot of these messages on my interface logs passing traffic:
NEGATE_ROUTE: Negate policy routing for destination (10000001)
Maybe something is creating hidden rules I don't control?
-
Are your routing your DNS requests out through your VPN provider in the DNS Resolver settings ( Services/DNS Resolver/General Settings) like this?
-
@dma_pf said in Going nuts with DNS over OpenVPN:
Are your routing your DNS requests out through your VPN provider in the DNS Resolver settings ( Services/DNS Resolver/General Settings) like this?
It's set to ALL.
-
@gelcom When you do an "extended" dns leak test in this site what are the results? https://www.dnsleaktest.com
-
Do you control the server end of these tunnels?
If not they are almost certainly pushing a default route to connecting clients and it you don't reject it the pfSense default route will be changed.Never use 'any' as the source in Outbound NAT rules if you can possibly avoid it (and you can). It can catch unexpected traffic, like traffic from the firewall itself.
You shouldn't have outbound NAT rules on the internal interfaces.
Consider using a different DNS serer locally. By using 1.1.1.1 everywhere you are creating routing conflicts and making it far more difficult to diagnose the issues.
You want clients on those interfaces to use only 1.1.1.1 for DNS and to route those queries over VPN.Disabling gateway monitoring action will not prevent traffic leaking to WAN if VPN goes down and the gateway disappears entirely.
In Sys > Adv > Misc set 'Skip rules when gateway is down'.
That way if the gateway goes down there will be no pass rules and traffic will simply be blocked.Steve
-
@dma_pf said in Going nuts with DNS over OpenVPN:
When you do an "extended" dns leak test in this site what are the results? https://www.dnsleaktest.com
Kindly note "São Paulo, Brazil" is local pf and not remote VPN.
-
Can we see a screenshot of the firewall rules on
Interface_VMs_through_A
. Assuming that is where you're testing from.Steve
-
@stephenw10 said in Going nuts with DNS over OpenVPN:
Do you control the server end of these tunnels?
If not they are almost certainly pushing a default route to connecting clients and it you don't reject it the pfSense default route will be changed.Yes, I do. OpenVPN server is a VM on my Oracle Cloud.
Pls note my openvpn server.conf:
port 1194 proto udp dev tun user nobody group nogroup persist-key persist-tun keepalive 10 120 topology subnet server XX.XXX.XXX.0 255.255.255.0 ifconfig-pool-persist ipp.txt push "dhcp-option DNS 1.1.1.1" dh none ecdh-curve prime256v1 tls-crypt tls-crypt.key crl-verify crl.pem ca ca.crt cert server.crt key server.key auth SHA256 cipher AES-128-GCM ncp-ciphers AES-128-GCM tls-server tls-version-min 1.2 tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256 client-config-dir /etc/openvpn/ccd status /var/log/openvpn/status.log verb 3
Never use 'any' as the source in Outbound NAT rules if you can possibly avoid it (and you can). It can catch unexpected traffic, like traffic from the firewall itself.
You shouldn't have outbound NAT rules on the internal interfaces.Ok! Corrected source to Interface_VMs_through_A subnet
Consider using a different DNS serer locally. By using 1.1.1.1 everywhere you are creating routing conflicts and making it far more difficult to diagnose the issues.
You want clients on those interfaces to use only 1.1.1.1 for DNS and to route those queries over VPN.Indeed. I changed local pfsense DNS servers to google (8.8.8.8 and 8.8.4.4) and I kept openVPN to Cloudfare (1.1.1.1).
With all these corrections problems persists:
When I run https://www.dnsleaktest.com on local machine it points correctly to google DNS and local location (São Paulo).
When I run the test on "Interface_VMs_through_A" machine it shows the same WAN address (incorrect) with Cloudfare DNS (correct).
Still leaking.
Can we see a screenshot of the firewall rules on Interface_VMs_through_A. Assuming that is where you're testing from.
Sure. Here we go
Please note I keep seeing logs in this interface pointing to 1.1.1.1 with a rule name I don't have: NEGATE_ROUTE: Negate policy routing for destination (10000001) .
Maybe this "hidden" rule is leaking my DNS?
kind regards
-
@gelcom said in Going nuts with DNS over OpenVPN:
NEGATE_ROUTE: Negate policy routing for destination (10000001) .
Ah, it's because you have or had set 1.1.1.1 as a DNS server for pfSense with a gateway and that adds a static route to it creating the conflict triggering the negate rules. If you reboot removing that route it will probably go away but to be sure (or to not reboot) go to Sys > Adv > Firewall&NAT and set 'Disable Negate rules'.
Steve
-
Actually, ironically, it's probably because you're pushing 1.1.1.1 as a DNS server from the OpenVPN server which means it is included in the VPN networks alias.
Disabling negate networks should prevent it anyway.Steve
-
@stephenw10 said in Going nuts with DNS over OpenVPN:
Ah, it's because you have or had set 1.1.1.1 as a DNS server for pfSense with a gateway and that adds a static route to it creating the conflict triggering the negate rules. If you reboot removing that route it will probably go away but to be sure (or to not reboot) go to Sys > Adv > Firewall&NAT and set 'Disable Negate rules'.
Rebooting pf did not the trick but checking Disable Negate rules did.
Now leakdns shows remote openvpn ip and remote location. GREAT!!!
Is this an expected behavior on pf?
With this option turned on should I expect any other changes on pfSense?
Isn't possible to remove that "hidden" static route instead of enabling this option?
kind regards
-
Check the routing table to see where 1.1.1.1 is added.
But it's probably because your servers are configured to pass is to the clients and your clients are NOT configured to reject routes.But in a setup like yours you should disable negate networks anyway since you are relying on policy routing for isolation.
Steve
-
@stephenw10 said in Going nuts with DNS over OpenVPN:
Check the routing table to see where 1.1.1.1 is added.
No entries for 1.1.1.1 ....
-
Might be used in an IPSec tunnel if you have any defined. It pretty much has to be a subnet used somewhere in a VPN config to hit that rule.
Steve
-
I don’t know. I don’t have any other vpns setup.
Anyway…
Thanks for the support.
@stephenw10: I owe you a beer!!! :-)
Problem is solved.
Kind regards