"Rogue" Static Route Keeps Being Recreated (Solved)
I'm having a strange issue where a static route to a single host keeps being added to my routing table (on the wrong interface, no less) that breaks communication with that host.
I have a permanent network with a Watchguard XTM Firewall / Router handling OSPF, Packet Filtering, NAT and DHCP and a VM running pfSense handling DNS resolution (using Unbound) and a few other things. The network looks like this:
All of this works totally fine and I'm happy with it. The problem comes in when I insert the server I use for work. It's a fully virtualized environment that I carry around with me and use for training and it uses pfSense for pretty much everything (Filtering, OSPF, NAT, DNS, RADIUS…). I created an OSPF transit network to bridge the internal networks together and added a domain override on each pfSense system so I could connect to my local network and manage everything on the mobile environment. It's designed so I can quickly pack up and pull the full environment with me without any configuration changes. Here's how it looks:
(Yes, I realize this setup is double-NATed. It's not an issue for what I need to use it for).
Anyways, I was having a terrible time trying to get the two pfSense servers to forward DNS queries between each other. After a ton of painstaking troubleshooting, I realized it's because of this:
For some insane reason, pfSense on the training server is adding a static route saying 192.168.1.2 (the IP of pfSense on the local network) is on the DMZ, even though it knows through OSPF that the subnet it belongs to is on the Transit network. There are no static routes defined in System / Routing and there are no interfaces that share that subnet or IP. When I get rid of that route manually with the route command, everything works fine, but it just comes back a few minutes later. :o
I even removed everything from the DMZ to make sure it's not finding one through ARP (and there isn't an ARP entry for that IP address). I have no idea where this route is coming from or how to stop it from magically appearing in the table, and Google isn't coming up with anything.
Here is my Quagga config info:
What is going on here? Am I going crazy?
A route with a MAC destination like that is usually a tell-tale sign that the interface it's on received that IP address as a DNS server from DHCP on that interface. Is that interface set to obtain an address via DHCP?
System > General, uncheck "Allow DNS server list to be overridden by DHCP/PPP on WAN", then save/apply that interface again. See if it comes back.
It can't be coming from Quagga or it would have a "1" flag listed for the route.
Ah, well that is probably what is happening. It needs to use DHCP to pull an IP/DNS server for the outside (because as I said, I take the server with me. I don't have control over the networks I'm connecting it to). But the part that's confusing me is I have my Watchguard (which hands DHCP to it) set to give the Google DNS servers to the DMZ, not the internal one.
Edit: Nope, you're right.
Am I right in thinking it's an issue with my Watchguard sending DNS servers its not supposed to then? Or is there a setting that is forcing that DNS server onto my WAN in pfSense?
Got it. Turns out Watchguard distributes its global DNS server addresses to all DHCP clients, even if you have others configured on that interface. I just left the global ones blank and configured them on a per-interface basis. Thank you so much for your help!