@johnpoz

You are right, this android device joined the IOT SSID. On pfsense I have majority of IOT devices static DHCP binding with DNS assigned. But for dynamic DHCP pool I did not specify DNS server. So it was using .30.1 (pfsense gateway) as DNS and I have DNS redirect configured for external DNS servers not pfsense itself. This is resolved.

Thank you very much for pointing out the issue.

Got it! I had to add "LAN" to the list of authorized outbound network interfaces for unbound. Without that, it wasn't able to send traffic from an IPv6-enabled interface (since my WAN interface has no IPv6 address).

A HUGE thank you for taking the time to point out the relevant parts in the documentation and explain them.

Bart

@mathomas3

Timings Name server Query time 127.0.0.1 No response 192.168.1.1 No response

That sure doesn't look normal..

@randombits I have not looked into what Pis do - and would guess depends on what OS your actually running on them. But its possible it could/should also send out a gratuitous arp.

This is basically the device just telling the network on its own, hey if your looking for IP address x.x.x.x - that is me, here is my mac.

This should update any cache..

But sure arp cache should be something you should be aware of when you swap in the other device, that if you have any issues talking to it - make sure to check the cache on the device trying to talk to it, that its cache is not pointing the first devices mac.

@techvic said in DynDNS not updating IP by itself but only with "force update":

but in that scenario the DynDNS is not updated even though I have the entry in the log that claims it updates the DynDNS.

That's why I was asking for what was shown after the line :

909cc90f-25d8-48ca-b1a4-941036a87525-image.png

When using verbose mode (you are) :

cb28d7c4-e023-4b19-b8c6-ed60f681d584-image.png

the answer coming from the dyndns service of that https request will get shown.
That answer also proofs that the dyndns servcie was contacted.
These are a bunch of "Response Header:" and "Response Data:" lines.

In general that's not a known issue. Pretty much everyone here at Netgate runs with a private domain entry for our company domain and things hum along as usual.

unbound can get cranky sometimes if it is trying to reach a specific upstream server and it doesn't respond. Keep an eye on Status > DNS Resolver entries when it works vs when it doesn't work. You can get the same output from the shell with:

: unbound-control -c /var/unbound/unbound.conf dump_infra

Odds are when it stops responding there is an entry in there for a server that has also stopped responding. Restarting unbound clears all that knowledge and forces it to try again. You could also try manually flushing things for that domain (or all domains) to see if that's sufficient to make it try again:

: unbound-control -c /var/unbound/unbound.conf flush_zone foo.com

There are some other similar commands to try listed in the docs:

https://docs.netgate.com/pfsense/en/latest/services/dns/resolver-cli.html

figured it out, there was an old dhcp reservation on one of the carp routers that were not synchrnoised.

@qbhatti said in Pihole as secondary dns:

blocks some google ads so it means I cant click on shopping items or sponsored items when searching.

Seems like a problem of the ad blocking list your are using in Pihole.
You could try a safe one like OISD Basic that has no false positives.

@johnpoz I am hallucinating indeed I see that I can use other dns that are not those of my provider. I don't know if it's their mistake or they have changed their policy and now they allow us to use others. If so, this has solved my problem. Ufff!
Thank you very much for your help.

@johnpoz
I just verified. In the Network preference of the mac, under "DNS", I had specified a bunch of IP addresses of entities other than my own ISP (like 1.1.1.1 or 9.9.9.9 ...), owned by Google, Clouflare and possibly others. I erased all entries in this field yesterday and just checked now : it contains one entry it added itself, which is 198.168.xxx.1 : the LAN address of my SG-1000 pfSense firewall-router, in the private vlAN of which this mac is client.

alright, so for those interested:

it appears that pfSense uses the standard ISC DHCP server its config is stored in /var/dhcpd/etc/dhcpd.conf When making dhcp scopes, I can see there there are some reverse declarations at the bottom of the zone file. The standard zone 12.10.10.in-addr.arpa. { primary 10.10.12.14; key "mykeyhere";

kinda stuff.
4. In looking at the logs from the DNS server (BIND) I can see registrations of the forwards, but not fo the reverseses...
5. After setting up my own DHCP server and creating a config file, I noticed that I needed some config the get the reverse zones to register. Something kinda like this? (config I scavanged from other forums)

ddns-update-style standard; ddns-rev-domainname "in-addr.arpa."; I dont see any options is the pfSense gui that controls for either of these options, even though it creates the reverse registration...

At this point, I think this topic is closed, but anyone else feel free to comment regarding DHCP and reverse registration.

Thanks again all!

@bob-dig Yes, what you say is true. Luckily my prefix(es) are not dynamic and I'd need to do quite a few changes to flip to what you suggest. I'll probably think about that as a longer-term project but for now this solution works for my specific use case.

@chris1284 when I get a chance I will try and duplicate this.

@br116 said in IP Addresses not showing up:

Is there a reason for this?

You're probably missing important info.
If you set up a device with static IP settings, like : a screen shot of one of my APs :

92141b30-4402-496b-a5c3-369802092bc6-image.png

then the DHCP client process on that device, my AP, and your device, isn't used.
So, my AP will never emit a DHCP request, and the pfSense DHCP server will never receive a request for a lease from this device.
Because : DHCP isn't used by this device.
So : no lease.
Because you chose not to use DHCP for your device.

@derelict this has come up multiple times ;)

Here is an old thread where went into much detail about when the dns server your redirecting on on a different vlan, etc. or when its on the same network and you get back the unexpected IP, etc..

https://forum.netgate.com/topic/139457/transparently-intercept-and-redirect-dns-traffic-to-an-internal-dns

Personally not a fan of redirection either, either use the dns I handed you - or not getting dns ;)

But I have done on one of my vlans just to shut up some iot shit that insists on trying to talk to 8.8.8.8, so just redirect it to my pihole - there you go buddy googledns answered you ;) If you would just use the freaking dns I handed you with dhcp you dumb crappy pos we wouldn't have to do such nonsense.

@gertjan Thank you.
I'm not skilled in programming that's why I asked where to submit a request.
If I do a pull-request and edit things which most likely is wrong, someone has to view this and I'll wasted his/her time, because.. I suck.

I'll gladly donate 10usd to whomever does it. Or donate to a cause by his/her choosing.

@steveits I have 22h2 on windows 10, and the dhcp server is clearly still listed there.

Ethernet adapter Local: Connection-specific DNS Suffix . : local.lan Description . . . . . . . . . . . : Killer E2600 Gigabit Ethernet Controller Physical Address. . . . . . . . . : B0-4F-13-0B-FD-16 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes IPv4 Address. . . . . . . . . . . : 192.168.9.100(Preferred) Subnet Mask . . . . . . . . . . . : 255.255.255.0 Lease Obtained. . . . . . . . . . : Friday, November 11, 2022 6:20:38 AM Lease Expires . . . . . . . . . . : Thursday, November 17, 2022 6:20:38 AM Default Gateway . . . . . . . . . : 192.168.9.253 DHCP Server . . . . . . . . . . . : 192.168.9.253 DNS Servers . . . . . . . . . . . : 192.168.3.10 NetBIOS over Tcpip. . . . . . . . : Enabled

@schirrms when you forward, the dns you forward to either does dnssec or it doesn't that setting can cause issues if your asking unbound to do dnssec, but its not actually resolving.

That setting when you click forward should in my opinion be auto unchecked, there is really no scenario where you would forward and also want unbound to try and do dnssec..

Where you forward either already does it or it doesn't - that setting is only going to cause extra queries and most likely issues.

If you want to use dnssec when you forward, pick a server to forward to that does it - which is most of them. If you don't want to do it - then pick one that doesn't. Many of the big providers have a different IP to use that doesn't do it.

For example quad9 has 9.9.9.10 that doesn't do dnssec

9.9.9.10 Provides: No security blocklist, no DNSSEC, No EDNS Client-Subnet sent. Please use the unsecured secondary address of 149.112.112.10

Having unbound do dnssec only makes sense when its resolving stuff on its own.

@wurst Great that my post helped you finish setting this up!

Even when I figure something out myself or through other research I like to come back and finish a thread so there is a working solution for others to find if they come looking with the same problem.

Just to clarify for others (I can't edit my post now) the route-gateway commands are added to Advanced configuration, custom options in the OpenVPN server and client configuration for the link.

Regarding my two domain configuration - I did some testing with dhcrelay at the command line and unfortunately it is not possible to run two instances with the same upstream link - the daemon binds to the upstream interface (ovpnc1) with a BPF (packet filter) binding, however the second instance refuses to start saying the ports are already in use. This seems to be a coding limitation of dhcrelay because you can actually have more than one application listening to the same UDP ports on the same interface if you use raw sockets. (I guess it doesn't use raw sockets, which is surprising for DHCP)

So the only way to run two instances would be to use two different VPN tunnels to the two different domains, and at least one of the instances would need to be run from the command line or a startup script... messy and not very maintainable in the future.

However for my purposes one instance is actually enough. I realised that you can add many DHCP servers in the dhcrelay config, in the GUI as well, so I simply added all 5 of them - two are primary DHCP servers for the two domains, two are failover DHCP servers for the two domains (usually inactive) and one is the PXE boot server.

This causes every client DHCP broadcast request to be sent to all 5 servers as unicast. Not very efficient, but it works, as only the server which owns the scope for that particular VLAN (and is not in failover standby) will reply, the others just ignore the requests. And in the case of PXE boot the PXE boot server is the one that replies.

This can be seen here, captured on the ovpnc1 interface, when a client on VLAN 1 sends a DHCP request where only the primary DHCP server for Domain 1 responds:

16:17:41.702734 IP 172.23.0.2.67 > 10.0.0.250.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 334 16:17:41.702758 IP 172.23.0.2.67 > 10.0.0.248.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 334 16:17:41.702774 IP 172.23.0.2.67 > 10.0.0.249.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 334 16:17:41.702787 IP 172.23.0.2.67 > 10.0.0.253.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 334 16:17:41.702801 IP 172.23.0.2.67 > 10.0.1.6.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 334 16:17:41.719679 IP 10.0.0.253.67 > 10.20.1.254.67: BOOTP/DHCP, Reply, length 330

And here, when a client on VLAN 2 sends a DHCP request where only the primary DHCP server for Domain 2 responds:

16:18:59.910858 IP 172.23.0.2.67 > 10.0.0.250.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 336 16:18:59.910880 IP 172.23.0.2.67 > 10.0.0.248.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 336 16:18:59.910894 IP 172.23.0.2.67 > 10.0.0.249.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 336 16:18:59.910908 IP 172.23.0.2.67 > 10.0.0.253.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 336 16:18:59.910921 IP 172.23.0.2.67 > 10.0.1.6.67: BOOTP/DHCP, Request from b8:03:05:c7:0e:6b, length 336 16:18:59.925943 IP 10.0.0.248.67 > 10.22.1.254.67: BOOTP/DHCP, Reply, length 333

As we only have a small number of devices at this remote site I'm not worried about the overheads of doing this. Even if I was able to separate the forwarding requests for the two domains I would still have 3 requests per domain anyway, primary and failover DHCP and (shared) PXE boot server.

People should keep in mind that this configuration of relaying to all DHCP servers can potentially cause information leakage - eg the DHCP servers in one domain see queries from devices in the other domain even if they aren't responding to them. So you would only want to do this when the two domains are "friendly" and owned by the same organisation.

If not separate dhcrelay clients on other devices or separate tunnels would be warranted to avoid DHCP servers seeing queries from outside their organisation.

I've gone ahead and followed this guide on my firewall to avoid having to configure preferences against a system behind the firewall.

And it blocks everything it needs to from LAN clients after I configured the firewall rules referenced in the first post.

However I have a Wifi mesh system linked to the firewall that seems to agree with some blocks, but connecting to some hosts from my smartphone isn't blocked when they should be. Is there something more that needs to be configured?

@jegr

Why exactly? Multiple domains won't make that "impossible". That you have to explain.
My impression was that you suggest to use one of those domain names also for internal use. Since I have multiple that seems impossible to me, apart from the fact that i do not like the idea.

Otherwise a Host Override will do better as it won't have traffic go to your firewall to be reflected
The traffic from my internal systems are passing directly to the also ^publicly accessible systems^. However in opposite to what you suggest, they do pass the firewall, since the public servers are in the redzone and for example the pc's are in the pc-zone.

However I do not understand the NAT reflection idea. I had a look on https://docs.netgate.com/pfsense/en/latest/nat/reflection.html And do not get the impressing that there is a better solution. But perhaps I am wrong.

Dynamic ones - no. Statically assigned ones are no problem AFAIK as those get imported when starting/restarting the resolver and won't be refreshed that often.
I did not ever released that, I do not like it, however since the things which should have a stable address have either a machine internal fixed address(es) or have one statically assigned by the dhcp server based on MAC and DUID that will probably not be a big problem.

DNS-resolver forwarding local domain to rootservers!??
That issue have been solved recently by changing DNS resolver type to static. Which should IMHO be the default.

To check: What do you mean by "default zone"?
Yep exactly there. That is my toplevel. All the clients are in vlans having <vlanshortname>.lan as subdomain.

Note that .lan and its subdomains are of course not visible from the outside.

Related to DNS options, I have the following options in that field
server:
log-queries: yes
#log-replies: yes
#log-tag-queryreply: yes
local-zone: "use-application-dns.net" always_nxdomain
Not so sure about the final line 😧

Related to blocking SSH from e.g. my guest vlan. I am not sure about the pfSense behavoir (I should test some time).
I have two issues here:

the gateway is inside each vlan (?), and from that you could expect that there is no rule which could block access to that gateway. Never the less I think that you can block the functions provided by the gateway. Second is that you have to add rules for that. Best way there is probably two rules in the floating rule set.
First defining what is allowed
Second refusing the rest

I would prefer an option where you could define the listening addresses for the pfSense GUI.

So perhaps you didn't know, but you can use the ports of SSH/Web and forward them to another host from your WAN without doing anything in pfSense
I do not understand this, however I have only one publicly reachable SSH-/SHTP-server and that one is reachable via port forwarding. pfSense itself is on port 2222 and the pfSense management is not reachable from the wan, not by any port/means.

I don't know your system or HW specs
Related to the hardware specs
Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz
Current: 2099 MHz, Max: 3500 MHz
4 CPUs: 1 package(s) x 4 core(s)
SSD 16 GB-ram

So reasonable powerfull, however I have 10gbit lan's internally, and transferring data from the NAS (greenzone) to the PC-zone is passing the FW.

I also assume that applications like pfBlocker are only checking the WAN, which of course has a much lower speed. But why should I add that complexity If I only want to block a minimal number of domains!?

Note that I not only use the firewall to block incoming traffic, but also try to block some outgoing traffic (using DNS). Not so easy 🎃

@321liftoff Yes they are but why would you need to use transparent ClientIP? That wasn't mentioned anywhere in your question, that's why I'm wondering. Does your service actually need the source IP of the client to work?

Cheers

@jegr the other "DHCP server" is an Android tablet with WiFi sharing.
And so it works without further ado.
I suspect that the client does not understand something. Of course, a DCHP server in the IP range 1.1.1.x does not exist.

I'm just wondering why the "dumb" DHCP server works on the Android tablet and the pfsense DHCP server doesn't.

@jegr Thanks for the suggestions. I dont use pfBlockerNG since I have my own Unbund DNS running on my network in Docker on Synology. This way I am able to stay most current with Unbound versions. Big plus is that I am able to use Unbound full API when it comes to RZP and dns trigger actions. pfBlockerNG is not near what pure Unbound can provide in functionality.

AS I mentioned I do have port forward enabled and multiple interfaces. I use firehol ipset level2 and 3 plus some additional ipsets. I block these aliases at the interface group level.
These are configured to block any internal traffic going OUT to any of the IP in these lists.

Running it about a week and I have few hits. Its a low count.
That proves to me that it is working and catching bad actors as it should.

@cjbujold That isn't much information to deal with.

are you running the forwarder or resolver? if it's the resolver, do you actually resolve or have the option for forwarding active? are you forwarding to a secure service (DoT)? why do you have that much DNS servers configured? what is the setting in System/General about DNS servers and fallback strategy? how is the forwarder/resolver configured? are multiple WANs in play? how about host/domain overrides and what is your configured domain in System/General? does your DHCP server advertise a domain and domain search list?

DNS resolution may be slow for some domains or all, that depends on a lot of things. If the resolver is in use and it is slow for some domains but not for others - that's the internet. The DNS resolver is completely resolving the way to the upstream DNS server of that domain e.g. blah.example.com would mean it needs to head over to the ROOTs for .com, then to a .com nameserver for example.com and then ask the nameserver for example.com whoTF is blah.example.com. That's the complete tree and how resolving is done. That can result in the first query as slow but the subsequent calls are cached and therefore very fast. The other point is if a DNS forwarder is down or (geo)blocking or censuring, you completely ignore that as unbound will resolve the domain via the ROOTs, the domain etc. and ask the authoritative nameserver of the domain itself after the IP. Not a big forwarder that can be altered.

If you do forwarding all calls are forwarded to those name servers you've listed in the screenshot. That may be a bit faster but you send all DNS traffic that way. If they are down - you are down (have no resolution). Multiple servers may help but often DNS calls aren't working in parallel (or not very good) so if the first or both first and second forwarders are down or VERY slow, you'll often be stranded.

The localhost entry may also be slow if MultiWAN is in play and you have problems switching between the uplinks or is flapping etc. etc.

So as said, there can be many thing afoot that can play into slow resolution. Even windows itself can play a stupid part in it. If you have multiple domains in your Windows search path, Windows tries those first adding it to every DNS call. So you ask "blah.example.com" and have a search path of "my.local.home" and Windows stupidly asks your DNS for "blah.example.com.my.local.home"! - getting nothing but creating a question for the upstream DNS, then even asks for "blah.example.com.local.home" (parent domain) before asking about the correct. All that useless questions add up and make DNS resolution slow. So one can add a few lines to unbounds configuration and add/change it from transparent mode to static so it won't relay those useless questions to upstream DNS servers but immediatly reply with "nope!" and not waste much time.

So perhaps that can already shed some light into internals until you can add a few more info tidbits :)

Cheers

@steveits said in Namecheap Dynamic DNS seems to be broken again:

@occamsrazor said in Namecheap Dynamic DNS seems to be broken again:

why do the existing Namecheap dyndns services I set up long ago still work fine, and it's only the new one I created recently that has the issue

If you look at your system log does it actually succeed? Or are you assuming it's working because it's green? :) I have a daily 1am entry with "Unknown Response." I suspect (without looking) that "Unknown Response" != "fail" in the code...? If the IP didn't change no one would know it was failing.

You could be right on that. My IP never actually changes (!) these DDNS services are really just in case it does. See my post here:
https://forum.netgate.com/topic/173646/namecheap-dynamic-dns-seems-to-be-broken-again/9?_=1667925480997

No, with the existing ones I still get "unknown response" but then I DO get a "Configuration Change" message. With the new ones I don't get that. I'm not sure what it all means.

@gertjan @johnpoz @viragomann @SteveITS I seem to have fixed the issue over the weekend. I am not sure what was wrong but re-installing with the newest version allowed me to use the DNS Resolver. I was using an older version. But installing with 2.6.0 fixed the issue for me. I did notice, however, that when rebooting with the older version, the DNS Resolver service was taking a while to start up. I never actually chekced the services running on the router so its possible that the service was just not able to start. I did "Restart" the service through the Web GUI a few times and it never gave me any indication that it didn't work. Its possible the service was not actually started or in a hung state. Thanks for all your help though, really appreciate all the responses!

Just a quick update on this, over this last weekend, I finally implemented pfSense with the forwarding of DNS/DHCP to one of my Linux systems where I configured the DHCP with static IP assignments, and to my initial shock, I got everything right and everything is "just working".

Thanks for the information and pointers on how to get this configured.

Now to figure out how to get the hostnames I have configured in the DHCP static assignments to populate instead of the hostname that the devices are reporting...

@brookbphx Hey this is the same exact nonsense that I have been dealing with for years now. When I read your post it sent shivers down my spine because I'm like 'OMG I'm not the only one!' HAHA. I'm running a Netgate FW appliance with Netgear WAX630E APs. Typically, I have to toggle WiFi on/off (iPhones/laptops/etc) + attempt to reconnect to WiFi over and over before I finally get an IP Address. I would say that you can't imagine how frustrating this is, but sadly you probably do! Have you figured out a sustainable solution? Is it better to just use some other system like Windows Domain Controller for DHCP instead?

@dominikhoffmann You're welcome.

@markn6262 Not defensive just stating facts.. If you setup dns, any dns be unbound, be bind, be powerdns, be dnsmasq.. And you clients ask it and it gives the answer you put in. But then something else gets some other answer - then is blatantly obvious this other thing isn't asking it..

Unless you on purpose setup a view, which if you had - you would of known you did this.

Maybe you are just unaware of the goal of these apps and browsers and even OSes to bypass what you tell it to use for dns, and use them, or their partners via doh..

But you set something to give X back as an answer when you ask for some fqdn, and you don't get that answer - and other things that do get the answer you put in. Not sure what sort of snake needs to bite you on the nose for you to see that you didn't ask the thing you setup..

the first answer to you question telling you this

many browsers and devices use DNS over HTTPS (DoH) and bypass local DNS servers by default, nowadays.

You then stated

One mobile device gets a reply to pings and routes from the Lan gateway and the other mobile device gets a reply from the Wan gateway

This clearly shows that your whatever is not asking unbound.. If other devices that ask it get what you put in.

The is no need to "troubleshoot" your setup, if clients get what you put in, then what you put in is working. You can look around, this is not a uncommon issue where users whatever was using doh for the reason they were not getting the expected answer they put in.

@gertjan Thanks a lot for the detailed feedback. I will try that, as soon as I can. I have now experienced the same problem on a total other network, so this points at MacOs network as the problem. Will test and get back.

Thanks again