Cannot resolve DNS - OpenVPN issues
I have pfsense 2.4.5-RELEASE-p1 running as a virtual firewall.
This was my first firewall and I've been testing it for a year now with OpenVPN with success!
I basically have a terminal server setup at home where daily, myself and other persons connect to the OpenVPN Server setup in my pfsense virtual firewall with no issues to access my terminal server, until yesterday.
I have what I could consider basic setup in the virtual context (Please see attached network diagram below, I have also attached raw file):
WAN->FibreInternetModem/Router ->Virtual PFSENSE BOX-> Virtual LAN ->Virtual Terminal Server
My Virtual PFSense Box is a VM on Oracle VM Virtual Box assigned to physical Intel network adapter.
My Virtual Lan is simply an "Internal Network" as defined by Oracle VM Virtual Box, and also on this Virtual Lan is my Virtual Windows Terminal Server and another VM use for testing purposes.
It must be noted that my Firewall is in monitoring mode only - I have not locked down the firewall and the only additional rules is for what OpenVPN needs to work. I'm using UDP mode only on default port 1194.
I have setup the PFSense with IPV4 only.
The problem started when persons could not VPN in via the OpenVPN client installed on their machines.
I verified that the pfsense box was working, as well as my VMs. The VMs did not have access to the internet, as in, I could not ping a website via DNS or browse to any website. I can however ping the Fibre Modem/Router. Tests from within the pfsense are also the same: with inability to ping websites or tracert to them, but pings to the ip address of Fibre Modem/Router works. I also checked Dynamic DNS under Services menu -> and I got that red IP Address meaning that that is also not working
At this point, I tried the following:
restored firewall with pfsense backup file of full configuration -> no success
restored a backup of the pfsense VM of a working version -> no success
Then I said, let me just reset the pfsense to factory defaults, and here is where I had some success -
the VMs were able to access the internet and resolve DNS etc (with both IPV4 and IPV6 enabled together). At this point I thought that perhaps by resetting to factory defaults, it refreshed any packages or modules etc. Perhaps If at this point I imported a working backup file with a working config (IPV4 only) ,it would work! But alas, it did not work.
Now here's the interesting part -> I did alter my restored PFSENSE config at this point to get it to work partially. If I reassign my WAN address via DCHP, I am able to resolve DNS and browse. Dynamic DNS status is green to that service is working. (I also did revert to HTTP for being the WebConfigurator protocol when it asked via terminal screen of pfsense).
At this point, all I need to do is connect a client to the OpenVPN from one of my branch locations/sites and see, and I am not having success with these results from OpenVPN Client status window:
Thu May 06 07:53:43 2021 OpenVPN 2.4.7 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] built on Feb 21 2019
Thu May 06 07:53:43 2021 Windows version 6.2 (Windows 8 or greater) 64bit
Thu May 06 07:53:43 2021 library versions: OpenSSL 1.1.0j 20 Nov 2018, LZO 2.10
Thu May 06 07:53:47 2021 TCP/UDP: Preserving recently used remote address: [AF_INET]22.214.171.124:1194
Thu May 06 07:53:47 2021 UDPv4 link local (bound): [AF_INET][undef]:0
Thu May 06 07:53:47 2021 UDPv4 link remote: [AF_INET]1xx.118.37.12x:1194 (note: I hid the IP address from this post for my protection)
Thu May 06 07:54:47 2021 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Thu May 06 07:54:47 2021 TLS Error: TLS handshake failed
Thu May 06 07:54:47 2021 SIGUSR1[soft,tls-error] received, process restarting
This error means that the VPN client is not able to connect with the server. Okay
First I checked to see if my OpenVPN server was running, and I did see it listed as okay, confirmed with the green checkmark and mouseover message saying "openvpn Service is Running".
I checked the Dynamic IP address to make sure it is current from the provider and it's okay.
Then I thought I'm supposed to alter my Port Forwarding Rules on my FibreModem/Router to match the new IP address assigned to the WAN side of the PFSENSE. Then comes another fishy problem into the mix. Even though I have altered my port forwarding/mapping configuration, I cannot connect with same Openclient VPN status.
The FibreModem/Router has a build in firewall that was already enabled. So I enabled logging.
Here's the weird part I don't understand.
My new PFSense IP address on it's wan port from the FibreModem/Router is 192.168.100.228.
When I retry the client openvpn connection at a branch location, I can see where it attempts to go through the FibreModem/Router's firewall with the correct source ip (at branch location) and ports, correct destination port but incorrect destination ip (192.168.100.116).
Here's that log entry:
2021-05-05 16:20:24 [Notice] IN=ONT-HSI OUT=LocalNetwork Direction=Public->Private Action=Permit SRC=200.7.xx.1xx (hid that ip :)) DST=192.168.100.116 PROTO=UDP SPT=49970 DPT=1194
But why would the Destination ip end in 116 and not 228?
I wondered if perhaps, this Destination ip address is perhaps encoded in the VPN Client configuration but I do not know this to be so, even after researching the matter and rechecking server and client configurations, unless I missed something!
So I told myself -ok, just set the Static IP address of the WAN interface of PFSense to 192.168.100.116 and give it what it wants - and I did, and then the DNS resolution of the pf sense box, as well as the VMs exhibited the original problems of not resolving (but pinging ip address of FibreModem/Router is fine!). I thought that if I let the router assign the 116 address via MAC address, it would possibly help, it did not.
When I switch WAN interface of PFSense IPV4 address via static to the 228, it resolves DNS. If I switch it to end in 110, it does not work. I confirmed that's a free ip address. So for some IPs it works (dns resolution) and some it does not? I double checked my subnet masks etc.
At this point, here are my current thoughts and here's what I am going to do:
-Unless I missed something, I beleive my FibreModem/Router device is at fault as because it's not working as it should. There are other things I've tested that casts doubt on the device. What I am going to do is setup an intermediary router between that and the pfsense box (double port forward setup) and see if that helps
-Futher doubt is cast on the FibreModem/Router device as I can't believe after restoring pfsense VM to a backup copy and 2 working config files, it still gives issues?
Please share your thoughts on the matter.
I must say that am learning quite alot from this and I am grateful! I needed to move from a novice in pfsense to understand deeper.
I am also referencing 3 sources in my research of these errors that brought comfort that my current setup I strongly believe is okay:
-Netgate's documenation and troubleshooting guides (https://docs.netgate.com/)
-pfSense 2.x cookbook Second edition(kindle) by Zientata, David
-Mastering OpenVPN (kindle) by Crist,Eric F,KEIJSER, JAN JUST
I look forward to your wise counsel and I will provide updates :)
First things first. You need to be using your ISP’s modem/router as a Gateway only. Is it possible to “Bridge” that modem or make it “pass thru”? Meaning the ISP modem just acts as a Gateway and your pfSense WAN connection pulls a public IP from your ISP. Right now, your modem is pulling the Public IP and then gives pfSense a local IP of 192.168.100.X. You can make your setup work by port forwarding requests or placing the WAN IP of your pfSense box in the modem’s DMZ. HOWEVER, having the ISP's modem be a Gateway and nothing else will really simplify where your problem is.
In OpenVPN’s connection where it states “TLS key negotiation failed to occur within 60 seconds (check your network connectivity)” This would mean check your network connectivity of the OpenVPN Server. Your issue is that the OpenVPN client isn’t getting through to the OpenVPN Server. This is probably because your ISP’s modem isn’t passing it to pfSense. This is why your ISP modem needs to be “Bridged”. I’d bet if you looked in the pfSense Firewall logs when the OpenVPN Client is trying to connect, you won’t see anything about a failed connection. This is because the OpenVPN client request never gets past your ISP’s modem.
Lastly, I just sent you a PM. Please check it.
@thatguy thank you for detailed response and helping me to understand more about this beautiful software. I'm happy to report that I found the error to be that of my ISP's Fibre Modem/Router device. They confimed that here was a faulty firmware update that caused bugs. The bug I was experiencing in particular was that for some reason that I do not know, certain addresses were not resolving DNS as I mentioned above, 116 being one of them which was the 'WAN' address of my pfsense. Even when I changed that IP address, and altered that port mapping rule, the firewall logging of my Fibre modem/router is showing attempts to pass that VPN client connection to that 116 address still. I was able to speak to the ISP about the issue and I was able to reset their device. From there, I was able
to Renter the port mapping rule (I actually use Port Control Protocol (PCP) instead and this resolved the issue. Thank you for your response and for furthering my knowledge.