IPSEC DNS Traffic issue
I have a site to site tunnel up between two locations. One site has two Windows servers running. The other location has workstations that authenticate to one of the servers and Remote Desktop to the other server. I can ping and Remote Desktop to the servers with no issues. The firewall rules on Netgates allow all traffic across the tunnels. The workstations at the remote site have their DNS settings hard set to the server at the other site. However I can not browse the web using the workstation browsers as they cannot reach the DNS service on the remote server. If I change the DNS servers to the Google DNS servers they can browse just fine. There is nothing wrong with the servers. If I put the old Netgate Monowall firewalls back in place all runs fine. It's just the DNS traffic which is not transversing the tunnel. Anybody have any idea as to why this is happening. I am dumbfounded. Thanks in advance for any help on this matter.
My suggestion would be to not debug DNS issues with a web browser. Use DNS tools to troubleshoot DNS issues then your web browsing will work.
Not sure what you have on Windows other than the wonderful
nslookupbut I would use dig something like this on the workstations:
dig @ip_address_of_your_DNS_Server www.google.com
There is probably an equivalent using nslookup. You do
server ip_address_of_your_DNS_Serveror something.
If that does not work, figure out why and you fix your network.
Again, there is nothing wrong with my network. All works as it should across a tunnel using two older Netgate firewalls running Monowall. It's not until I add the new pfSense firewalls does the issue present itself. All traffic traverses the tunnel with no issue (Remote Desktop, Web browser traffic to devices using fixed IP address, file sharing, etc..) it's just the DNS traffic which does not work so it's not my network. Also the firewalls are using identical rules as the Monowall firewalls.
If DNS isn't working then there is certainly something wrong with your network.
If there wasn't it would be working.
@Derelict I don't want to start an argument over this, but pretell how if the only thing changing is the firewall is it an issue with my network. The only thing changing is the firewall. Therefore the issue lies in a possible configuration issue with pfSense IPSEC or the rule set, which is why I'm asking for help trying to figure out.
Right, but the beginning issue is to manually do a DNS query, see the result, then walk from the client toward the server hop by hop until you find the cause. Until you do that everything is just a guess.
@Derelict I'm trying to get an idea of what's going before having to hook these things up at two different locations. I can tell you where the drop is going to happen, the tunnel. There are no other hops in between. Everything works perfectly with no issues. The only thing that is changing is the firewalls. So it's either the tunnel or the rule set. Since all the other traffic is crossing fine it makes no sense that DNS packets are not. This truly makes no sense as the rules are identical from the old firewalls running Monowall to the rules in the new firewalls running pfSense. As far as the rules on both sides LAN and IPSEC have only one rule, allow all traffic to pass.
If that is the rule then it would be working. So it must be something else.
Not sure what to tell you without seeing more information.
Not sure I understand your reluctance to do what amounts to standard network troubleshooting that is done when a problem arises to effectively isolate the problem area so it can be looked at more closely.
@Derelict I'll do it but it will be a wasted trip. This has to be done at 4:30am so there will be no chance of discussing anything with the units in place. They are also miles apart. So basically whats going to happen is I plug them in run the test. The breakdown will be at the tunnel. Which doesn't do anything but knowing the problem lies within the firewalls. Then we look at the firewall rules that are set to allow all traffic to pass through on both sides, which they do with the exception of DNS. Now do you see my reluctance? I have no problem doing what you ask but it will give no insight into the problem. Also this is not my first rodeo with firewalls. I've setup numerous Netgate units in this type of configuration with no issues. I also work with Sonicwalls and have tunnels that span a number of countries. What I am seeing in this one instance absolutely makes no sense and was hoping someone might have come across an issue like this or knows of a known bug. BTW, both units are updated to the latest version.
It is not a known bug. Thousands and thousands and thousands of people are running DNS over IPsec. Thousands. Maybe hundreds of thousands. Maybe even millions. You have almost certainly made a mistake somewhere.
I have a site to site tunnel up between two locations. One site has two Windows servers running. The other location has workstations that authenticate to one of the servers and Remote Desktop to the other server. I can ping and Remote Desktop to the servers with no issues.
So this is not currently set up?
@Derelict I had to remove the new firewalls once DNS was not working. I put the old firewalls back in place and all is fine. No mistakes made, everything is going across the tunnels just fine with just one rule set on the LAN and IPSEC networks which allows all traffic. That's why I am dumbfounded as long as the tunnels are up and the rules allow all traffic there should be no reason for DNS traffic not to go across as well. Like I said the rules are identical to the old Netgate Monowall firewalls. When I put them back online briefly on Wednesday at 4am I will get the info from the firewalls showing the tunnels up and connected and the rules for both firewalls. I was hoping someone would have an idea why the pfSense software would be doing this. First time I've encountered anything like this and it makes absolutely no sense whatsoever.
Add some evaluations of the states, packet captures, etc to see if you can determine exactly where the problem lies.
@Derelict Will do and will let you know what I find.
@Derelict It's not just DNS that's an issue it's also printing. We can send a request to the server at the main office to print to the remote printer at the remote location but the main server cannot communicate with the remote printer. I put the old firewalls back in place and the print job commences. Here are screenshots from both firewalls. PDF's labeled 101 are the main office and 102 is the remote location.
No idea what's in that zip file. Doesn't look like pdfs to me.
I suggest attaching images to posts like everyone else does.
@Derelict I sent the wrong zip file, I just reattached an external link to Droplr as it would not let me attach the zip file created by my Mac.
None of that is really any help.
Routes mean nothing to policy-based IPsec. The traffic selectors control what goes where, not the routing table.
@Derelict What do you want me to acquire from the firewalls. This is a plain Jane install. Only thing configured was Dynamic DNS, IPSEC VPN and the firewall rules to allow everything. Nothing else was touched. Do I just restore the firewalls and start from scratch? This just does not make sense.
Draw a network diagram specifically containing the pieces that are not working.
Problems like this are almost always caused by firewalls on the target servers themselves. As in the target DNS server is not allowing DNS queries sourced from the remote network.
Hence my original request that you troubleshoot this using DNS resolution tools, not RDP, web, etc.
@Derelict There is nothing special going on here whatsoever. They are using the same network addresses as old the firewalls. It has nothing to do with the servers. The only thing changing is the firewalls both of which are configured identically. The old Netgate firewalls are running MonoWall. The new firewalls are preconfigured with pfSense. They have identical rules, IPSEC configuration and using the same dynamic DNS. There is nothing on the servers that would limit anything. When the old firewalls are connected everything operates properly. It's not until I bring up the new firewalls do these issues present themselves. Nothing is being touched on the servers. I can Remote Desktop into both servers from the remote site with no issues. However I cannot connect to the DNS service at the main office from the remote site. Also the main site server cannot connect to the remote printer. The minute the old firewalls are reattached everything works flawlessly. There is something going on in the new firewalls but all of the settings are identical to the old firewalls. IPSEC is fine, the tunnels come up with no issue. The firewall rules are identical to the old firewalls, all traffic is allowed through. There is nothing being blocked. However DNS packets are not going across nor printing services. I could understand if I had rules set for specific services and something wasn't entered right but when the only rule is to allow all traffic there is nothing to screw up there.
None of that gives us anything to go on either.
Please understand that if everything was configured correctly, it would be working.
What is not configured correctly cannot be gleaned from you saying everything is configured correctly but not showing your work.
You have not shown us anything since you opened this thread over three weeks ago.
I asked what you wanted a couple of posts ago. Rather than telling me exactly what you need to see, you'd rather blame the server configuration. What network or firewall setting is going to change on its own dependent of which firewall is running? There is something off with the configuration of the firewall whether it be a configuration issue or corruption in the firewall itself. I'm just looking into guidance on where the problem could lie in the firewall and that's it.
This is simple network troubleshooting. You check DNS. Does it work? No? fix it. Then go hop-by-hop until you find the problem.
All you have said is "There is nothing wrong with my network. Tell me what's wrong with my network."
@Derelict Why do you keep harping on the network? Why can't you wrap your head around the fact that the only thing changing is the firewall. When the old firewall is in place everything functions properly. It's not until the new firewall is in place does this issue come up. So it's either a configuration issue on the new firewall or some sort of corruption. I'm trying to get help trying to track down where the issue lies inside the firewall. I have asked what you need to see from the firewall to try to figure out what's going on, but you continue to go after a working network that has had 0 issues until the new firewalls where introduced. As soon as the old firewalls are brought back up everything works fine again. As far as the link you sent:
If the WAN settings were off the tunnel would never be established and it is.
Is set properly as I can Remote Desktop to both remote servers.
Only one rule is set for LAN and also for IPSEC - Allow All
I can ping everything on both sides of the network. Remote -> Main & Main -> Remote
The main fact that I have allow all rule and Remote Desktop works and DNS and printing does not with the new firewalls, shows that it's something within the firewall not the network. I can state this with 100% confidence as ONCE AGAIN, the old firewalls which were purchased from Netgate work perfectly. It's not until I put the new firewalls up that these issues come up.
So please for the love of God ask me for some diagnostic data from the firewalls themselves to try to track this down this issue rather than continuing the harp about the existing and 100% functioning network.
@TechUnplugged Also there are no hops other than the tunnel.
Main Office (Servers) -> Switch -> Netgate Firewall -> Internet <- Netgate Firewall <- Switch <-Workstations
Great Apply IP addresses and networks to all of that and show your configuration. Need to see all of the interfaces, all of the interface rules including IPsec tabs, all of the IPsec configuration, etc. Then explain exactly what is NOT working in a manner such that there is no guessing involved.