DNS not working over VPN

[whoeveriam]

I am needing help getting split dns queries working. My set up is a small network behind a pfSense firewall with dynamically assigned public ip. I use a dynamic DNS to keep the public ip accessible. I have set up VPN access for several of our team that require access to internal network resources. This works great with one exception. I can not for the life of me get hostnames to correctly resolve for clients logged into the VPN. When on our local network DNS queries resolve correctly on the same laptops.

To illustrate my problem, I'll summarize my setup (network and server names changed to protect the innocent).

############################################
pfSense 2.4.4 box (mynetwork.org)
----> external IP address 68.XX.XX.XX -----> internal address 10.0.11.1 ------> internal subnets
DNS Resolver, VPN server, DHCP Server, Firewall
I have VPN server provide a DNS Server list to clients
I have VPN server provide a default domain (mynetwork.org) to clients

Internal Server (myserver.mynetwork.org)
IP 10.0.11.3

Remote Laptop
connected either by WiFi in the office or VPN outside
Macbook Pro running MacOS 10.15
############################################

When client laptop is on the local office network DNS queries work fine

>dig myserver.mynetwork.org 
;; ANSWER SECTION:
myserver.mynetwork.org.	3600	IN	A	10.0.11.3

However when I connect from outside the office via the VPN, this is what I get

>dig myserver.mynetwork.org
; <<>> DiG 9.10.6 <<>> myserver.mynetwork.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31397
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;myserver.mynetwork.org.		IN	A

;; ANSWER SECTION:
myserver.mynetwork.org.	0	IN	A	92.242.140.2

;; Query time: 88 msec
;; SERVER: 68.105.28.11#53(68.105.28.11)
;; WHEN: Wed Feb 26 13:33:04 CST 2020
;; MSG SIZE  rcvd: 66

This makes no sense, as a reverse lookup of the returned ip address is 2.140.242.92.in-addr.arpa domain name pointer unallocated.barefruit.co.uk which has nothing to do in any respect with my networking setup.

But that mystery aside, the problem I'm trying to fix is why does the DNS query for myserver.mynetwork.org not go the the DNS Resolver on my pfSense box (10.0.11.1) and instead gets forwarded to the namserver for the external network I'm on?

If I specify the pfSense box as the nameserver I get:

>dig @10.0.11.1 myserver.mynetwork.org 
;; ANSWER SECTION:
myserver.mynetwork.org.	3600	IN	A	10.0.11.3

so, this should work if the laptop used the correct nameserver for the mynetwork.org domain.

When I query the MacOS dns settings using scutil --dns I get:

DNS configuration (for scoped queries)
resolver #1
nameserver[0] : 68.105.28.11
nameserver[1] : 68.105.29.11
nameserver[2] : 68.105.28.12
if_index : 6 (en0)
flags : Scoped, Request A records
reach : 0x00000002 (Reachable)

resolver #2
search domain[0] : mynetwork.org
nameserver[0] : 10.0.11.1
nameserver[1] : 68.105.28.14
nameserver[2] : 68.105.28.13
if_index : 21 (ipsec0)
flags : Scoped, Request A records
reach : 0x00000002 (Reachable)

So the laptop is receiving the default domain and nameservers from the VPN server, so why don't queries for the mynetwork.org domain go to 10.0.11.1?

Any help would be appreciated. It's not the end of the world to have to use ip addresses instead of server names, but it would help me if I could get it working.

Cheers,
Ben

[nkaminski]

@whoeveriam What type of VPN (IPsec, OpenVPN, etc) are you using in this scenario?

Your VPN connected client is trying to resolve your internal hostnames via public DNS servers, not your local DNS resolver, which explains why this doesn't work.

This also exposes you to a slight security risk, as the operator of the network that your VPN client is on can influence the IPs returned by said DNS queries.

[whoeveriam]

@nkaminski
I am using IPsec. I understand that my remote client (a MacBook Pro) is not using the correct DNS servers for queries. I can query the pfSense box explicitly and get the correct answer with dig:

>dig @10.0.11.1 myserver.mynetwork.org 
;; ANSWER SECTION:
myserver.mynetwork.org.	3600	IN	A	10.0.11.3

And when I query the laptops DNS settings with scuitl --dns I get:

DNS configuration (for scoped queries)
resolver #1
nameserver[0] : 68.105.28.11
nameserver[1] : 68.105.29.11
nameserver[2] : 68.105.28.12
if_index : 6 (en0)
flags : Scoped, Request A records
reach : 0x00000002 (Reachable)

resolver #2
search domain[0] : mynetwork.org
nameserver[0] : 10.0.11.1
nameserver[1] : 68.105.28.14
nameserver[2] : 68.105.28.13
if_index : 21 (ipsec0)
flags : Scoped, Request A records
reach : 0x00000002 (Reachable)

So the nameservers and default domain provided by the VPN server to the Mac are correct, but the Mac isn't using them to correctly direct queries for that domain to the correct name server.

I am baffled. Any ideas?

Cheers, Ben

[ciphergeek]

@whoeveriam

I'm seeing the same thing on my Mac. I've got an IPSEC vpn that is providing a DNS server list but the DNS server is being added as secondary to the hosts configured DNS server. Any updates?

[whoeveriam]

@ciphergeek Unfortunately I've had no further luck. I wonder if this isn't a Mac OS bug in the way it handles DNS lookups for VPN connections. I haven't had time to run this down any further. If you find something please post here as well.

Cheers!

[andrelung]

Same thing is happening to me. Any clues or workarounds how to deal with that? Currently I am handing out IPs...

[ciphergeek]

I had to roll out this script to users. I haven't tested against Big Sur yet though to see if this is fixed. I guess we should submit to apple at some point.

tell application "Terminal"
try
do shell script "mkdir /etc/resolver" with administrator privileges
do shell script "touch /etc/resolver/secure" with administrator privileges
do shell script "echo nameserver 10.10.10.15 >> /etc/resolver/secure" with administrator privileges
say "Should be working now!!"
on error
say "uh oh, we have a problem"
end try
end tell

[andrelung]

@ciphergeek Thanks for your answer! I manually added the file and in fact it instantly starts showing up:

$ scutil --dns

[...]
resolver #8
  domain   : secure
  nameserver[0] : 192.168.88.1
  flags    : Request A records, Request AAAA records
  reach    : 0x00000003 (Reachable,Transient Connection)

However, the name is still not resolvable without specifically requesting the nameserver:
login-to-view

Let me know if you see anything odd here. Otherwise I'll keep trying to figure this out...

[ciphergeek]

Hi. In my case, .secure is an internal domain name I created. Are you trying to get general DNS working or an internal DNS zone?

[andrelung]

Right, just the internal name resolution is problematic! I mv'ed the "secure"-file to our own internal domain name. Same results.

General DNS is provided by the "host"-connection, which is fine I guess?

$ nslookup google.com
Server:		172.20.10.1
Address:	172.20.10.1#53

Non-authoritative answer:
Name:	google.com
Address: 216.58.206.14

$ nslookup google.com 192.168.88.1
Server:		192.168.88.1
Address:	192.168.88.1#53

Non-authoritative answer:
Name:	google.com
Address: 142.250.185.142

[ciphergeek]

What's running "scuitl --dns" show

[andrelung]

$ scutil --dns
DNS configuration

resolver #1
  nameserver[0] : fe80::c82:13eb:d00f:5f5d%en0
  nameserver[1] : 172.20.10.1
  if_index : 6 (en0)
  flags    : Request A records, Request AAAA records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #4
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300400

resolver #5
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300600

resolver #6
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 300800

resolver #7
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records, Request AAAA records
  reach    : 0x00000000 (Not Reachable)
  order    : 301000

resolver #8
  domain   : mydomain
  nameserver[0] : 192.168.88.1
  flags    : Request A records, Request AAAA records
  reach    : 0x00000002 (Reachable)

DNS configuration (for scoped queries)

resolver #1
  nameserver[0] : fe80::c82:13eb:d00f:5f5d%en0
  nameserver[1] : 172.20.10.1
  if_index : 6 (en0)
  flags    : Scoped, Request A records, Request AAAA records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)

[ciphergeek]

Sorry man. That's weird. No idea why that isn't working that looks right if your internal dns is 172.20.10.1

[andrelung]

Actually I am testing the VPN through an iPhone hotspot and that's the DNS/Gateway of the iPhone connection.