OpenVPN hub and spoke with AD/DNS on spoke

[micro8765]

I've had a pfsense-based OpenVPN site-to-site hub and spoke topology for several years.

The hub is a cloud-based pfsense Vultr-PF. It has two OpenVPN servers, one on 1194 for site-to-site connections from 5 physical sites, and another on 1195 for road warriors.

Of note, the Windows DC/AD/DNS server is at a phyiscal location, behind Deakin-PF. Because Windows must be the DNS server to join, access shares, etc., clients on remote subnets resolve DNS queries over two OpenVPN hops: up to Vultr-PF, down to Deakin-PF, then back again.

Normally this works fine. I can join new windows clients to the domain, for example, from any remote subnet.

However I just installed TRUENAS behind Murrumbateman-PF and it will not resolve DNS to the Windows server for some reason. An identically configured Win10 laptop (DHCP reservation with DNS 192.168.0.2 only) can resolve DNS no problem.

On Win10:

C:\>ping 192.168.0.2

Pinging 192.168.0.2 with 32 bytes of data:
Reply from 192.168.0.2: bytes=32 time=29ms TTL=125
Reply from 192.168.0.2: bytes=32 time=29ms TTL=125
Reply from 192.168.0.2: bytes=32 time=29ms TTL=125
Reply from 192.168.0.2: bytes=32 time=29ms TTL=125

Ping statistics for 192.168.0.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 29ms, Maximum = 29ms, Average = 29ms

C:\>tracert 192.168.0.2

Tracing route to bhscpdc [192.168.0.2]
over a maximum of 30 hops:

  1    16 ms    16 ms    17 ms  172.27.120.1
  2    28 ms    30 ms    28 ms  172.27.120.5
  3    30 ms    29 ms    28 ms  bhscpdc [192.168.0.2]

Trace complete.

On TRUENAS:

root@truenas-cs[~]# cat /etc/resolv.conf
# Generated by resolvconf
search BHSC.local
nameserver 192.168.0.2

root@truenas-cs[~]# ping google.com
ping: cannot resolve google.com: Host name lookup failure
root@truenas-cs[~]#
root@truenas-cs[~]#
root@truenas-cs[~]# nslookup
> google.com
Server:         192.168.0.2
Address:        192.168.0.2#53

Non-authoritative answer:
Name:   google.com
Address: 172.217.24.46
Name:   google.com
Address: 2404:6800:4006:812::200e
> exit

root@truenas-cs[~]# ping 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
92 bytes from 172.27.120.1: Redirect Host(New addr: 0.0.0.0)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 7948   0 0000  3f  01 76ef 192.168.10.31  192.168.0.2

64 bytes from 192.168.0.2: icmp_seq=0 ttl=125 time=28.402 ms
92 bytes from 172.27.120.1: Redirect Host(New addr: 0.0.0.0)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 de21   0 0000  3f  01 1216 192.168.10.31  192.168.0.2

64 bytes from 192.168.0.2: icmp_seq=1 ttl=125 time=28.243 ms
92 bytes from 172.27.120.1: Redirect Host(New addr: 0.0.0.0)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 e439   0 0000  3f  01 0bfe 192.168.10.31  192.168.0.2

64 bytes from 192.168.0.2: icmp_seq=2 ttl=125 time=27.435 ms
^C
--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 27.435/28.027/28.402/0.423 ms
root@truenas-cs[~]#
root@truenas-cs[~]#
root@truenas-cs[~]#
root@truenas-cs[~]# traceroute 192.168.0.2
traceroute to 192.168.0.2 (192.168.0.2), 64 hops max, 40 byte packets
 1  172.27.120.1 (172.27.120.1)  15.608 ms  15.907 ms  15.453 ms
 2  172.27.120.5 (172.27.120.5)  27.358 ms  27.255 ms  26.850 ms
 3  192.168.0.2 (192.168.0.2)  32.078 ms *  28.784 ms

From the above, nslookup works yet DNS does not. So perhaps this is strictly a TRUENAS issue. TRUENAS is based on FreeBSD.

Note also that the ping from TRUENAS has a 'Redirect Host(New addr: 0.0.0.0)' funniness about it. I thought that might indicate a routing issue.

So at this point, I don't really know whether the DNS resolution failure is due to TRUENAS, DNS or OpenVPN config. Perhaps my OpenVPN config is most suspect as it took a lot of trial and error to get everything working with routable subnets.

Therefore I'm including a basic network diagram and a bunch of OpenVPN-related screenshots, some of which are annotated.

I know this is complex for someone to look at and my expectations are low, but hey if I don't provide information then I definitely won't get answers. My config might help others to set up a similar config as well.

Sorry for the huge post!

[micro8765]

After further testing, it appears this issue is limited to FREEBSD v12.

I installed fresh instances of FREEBSD 12.2 & 12.3 and neither would resolve DNS over OpenVPN.

I then installed FREEBSD 13 and DNS worked no problem.

I still don't know why my hub's routing table looks like it does, with all remote OpenVPN subnets pointing to 172.27.120.2, but hey routing to all subnets seems to work so I guess I'll just ignore that.