Unable to illegal DNS record from pfsense (DNS-resolver corruption)

asadz

You are very helpful i for once saw a glimmer of hope. See the output plus config

does this mean its cache gone bad or some static entry somewhere which is returned every time

asadz

@johnpoz
unbound-control -c /var/unbound/unbound.conf flush sb.scorecardsearch.com. but it return back.

Also i'm unable to ping or tracert to 100.2.3.4, perhaps due to ttl 0?

johnpoz

@asadz well that shows unbound would ask the authoritative NS for that domain.

I don't see how you could be getting back a 100.x address from them?

What do you have setup for host or domain overrides? Do you have anything in the custom options box of unbound?

asadz

@johnpoz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

@asadz well that shows unbound would ask the authoritative NS for that domain.

I don't see how you could be getting back a 100.x address from them?

What do you have setup for host or domain overrides? Do you have anything in the custom options box of unbound?

No I checked there is nothing under custom rest of setting I showed as ss. One off thing that happened during pfblockerng force reload , which was performed right at the time of Cron job was seconds away from execution and it gave me some fatal error which was later was fixed by restart of pfsense.

You can Dec 14 log dns reply.log before there was no log entry for this. Also when I ping via up address for e.g

Abc.HQ.local 192.168.8.9

I get the correct address even when I do
Ping - a

But nslookup abc.HQ.local brings 100.2.3.4

Also when I do wireshark I see A query record returned by DC for 100.2.3.4 and not by pfsense. I flush cache at DC but no change, also rebooted DC as well

asadz

@johnpoz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

@asadz well that shows unbound would ask the authoritative NS for that domain.

I don't see how you could be getting back a 100.x address from them?

What do you have setup for host or domain overrides? Do you have anything in the custom options box of unbound?

Following your advice I check .local resolution as you suggested, I see strange entry the yellow is the domain that is owned by me, but I see .local appended queries going to all places e.g github,isc etc.

Does it look like poisoning, how can I remove these entries?

asadz

Also, i found
Services> SnortPass> ListEdit

passlist included this IP address, although the list was not attached to any snort interface.

johnpoz

@asadz again that is common practice for pretty much any client.. It will append the search suffix..

Again a single level domain (sld) like .local is not a good idea, especially with .local because of its special use. MS did change their stance on these quite some time ago.

https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/single-label-domains-support-policy

"Existing products may continue to function with SLDs, but SLDs aren't a recommended configuration for future deployments and may not work with some products or versions."

Its a been quite some time since I have had to deal with MS world and dns issues like this. But it is also best that if your a MS shop to have your clients get their dhcp and dns from your AD.. Clients should point directly to your AD for dns, and then MS dns can forward to pfsense to let it resolve stuff that is not in your namespace.

You can look to how to alter the suffix search, and or another problem that can come up is the devolution of the domain as well.

My advice would be to point your clients to AD DNS, and move away from the .local sld. Use something like yourname.home.arpa. This is the new tld that is recommended for local domain use.

asadz

@johnpoz Thank you for your education, I will take this mater related to .local sld internally with the IT team.

But, we are using DC as DNS server it has SOA records, along with forward and reverse lookup zones configured up.

I'm not still not sure, about following:-

why my internal hostnames are resolved to 100.2.3.4 address, where this DNS response comes from, why can't I change to point to correct IP address?
Why ttl is set as 0 for sb.scorecardsearch.com, I read if the value is artificially set (due to some error) then the upstream cache server would reject for new DNS TTL response.
Also the resolution for the address, has changed the DNS mapping for internal hosts

e.g abc.bank.local -> sb.scorecodesearch.com

dns.bank.local -> sb.scorecodesearch.com :100.2.3.4

even incorrect why this DNS A record response change internal mapping of host on network?

Thank you for help

johnpoz

@asadz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

But, we are using DC as DNS server it has SOA records

but your clients are pointing to unbound on pfsense, and then it forwards .local to your AD via a domain override?

You should point all clients directly to the AD for dns, and have it forward to pfsense for anything not in your namespace. You can then still leverage any filtering of dns this way that is outside your namespace.

sb.scorecodesearch.com :100.2.3.4

If you query your AD directly for sb.scorecodesearch.com what does it return? what does it return if you ask for sb.scorecodesearch.com.local

Did you mention you found this 100.2.3.4 address on some setting in pfsense? Is this the IP you return for when something is blocked?

I do not show sb.scorecodesearch.com as a valid domain.. But before you used sb.scorecardsearch.com

Do you have wildcards setup in your AD dns... This can be very problematic when a client appends your tld to its searches, also that 100.2.3.4 is a valid IP on the internet

;; QUESTION SECTION:
;4.3.2.100.in-addr.arpa.                IN      PTR

;; ANSWER SECTION:
4.3.2.100.in-addr.arpa. 86400   IN      PTR     pool-100-2-3-4.nycmny.fios.verizon.net.

Could be possible your isp is doing some sort of dns shenanigans and for whatever reason when it intercepts your dns it returns this IP?? Is verizon your ISP by chance?

Keep in mind this can get really convoluted when devolution of a fqdn starts happening on a client where they can take out multiple levels in a fqdn say host.sub1.sub2.tld when using single label..

You will want to look into how devolution works, I know it can be modified from its default of 2 and only 1 or completely disabled.

When a client starts messing with the fqdn that is actually queried you can run into all sorts of weirdness.. Especially if wildcard returns can come into play on your dns, etc.

DNS Devolution

https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/ee683928(v=ws.10)?redirectedfrom=MSDN

There might be newer info on this - but again I have been away from having to deal with MS dns weirdness for quite some time.. But yeah it can be problematic depending on the circumstances of what exactly is being asked for, what your using for you domain and what clients are doing with search suffix and or devolution of some fqdn.

asadz

@johnpoz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

but your clients are pointing to unbound on pfsense, and then it forwards .local to your AD via a domain override?

I check my DHCP server on pfsense and its set DNS servers in order of
DC1
DC2
and Firewall interface

Plus, I think you determination of resolution order comes from the output of outbound command you showed earlier. Is that right?

unbound-control -c /var/unbound/unbound.conf lookup sb.scorecardsearch.com

but in this case the authoritative ns for this domain cannot be .local, it outbound to find the authoritative NS?

pls see output of nslookup on DC, it shows 100.2.3.4

I do not show sb.scorecodesearch.com as a valid domain.. But before you used sb.scorecardsearch.com

yes it was an error /typo thank you for pointing it out.

On snort I see the incoming traffic from wan-infra being blocked and whats strange its coming for 100.2.3.4

Do you have wildcards setup in your AD dns... This can be very problematic when a client appends your tld to its searches, also that 100.2.3.4 is a valid IP on the internet

I cannot find using mmc /dns-server on DC.

Now, the ISP is not verizon

johnpoz

@asadz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

I check my DHCP server on pfsense and its set DNS servers in order of
DC1
DC2
and Firewall interface

Yeah I would remove pfsense from that equation. You really have no idea what NS will be asked by a client at some time in the future. It at some point will not be in that order.

1433 is default MS SQL port.. So sure if your client wanting to talk to your sql server tried to go to some public IP out on the internet ie this 100.x address then sure snort might warn you of that.

If your MS dns, that 3.6 address returns 100.2.3.4 then that is where clients would go when they try and use that fqdn.

asadz

@johnpoz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

@asadz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

I check my DHCP server on pfsense and its set DNS servers in order of
DC1
DC2
and Firewall interface

Yeah I would remove pfsense from that equation. You really have no idea what NS will be asked by a client at some time in the future. It at some point will not be in that order.

1433 is default MS SQL port.. So sure if your client wanting to talk to your sql server tried to go to some public IP out on the internet ie this 100.x address then sure snort might warn you of that.

If your MS dns, that 3.6 address returns 100.2.3.4 then that is where clients would go when they try and use that fqdn.

Ok but the PTR records for DC should provide the response there is no 100.* Network on infra so a response coming back can resolve to any of local machines as currently abc.doman.local or xyz Al point to 100.*

where I can reload or refresh this mapping ? Do I need to rebuild DNS resolver?

johnpoz

@asadz said in Unable to illegal DNS record from pfsense (DNS-resolver corruption):

Ok but the PTR records for DC should provide the response there is no 100.*

What does that have to do with a forward query.. And yes there is a PTR for that 100 address, be it on your AD dns or not.

Not sure why you think a client would validate that the PTR matches the fqdn for some IP given back in A query. Yes some software can do this - smtp is known for doing such a check but that is not something typical to happen.

I have not had to play with MS dns for quite some time. But yes if however it finds that 100 answer it would cache, maybe you have a configured device registering that in your AD dns?

You would have to look to your AD dns on where it might have that entered or cached. But clearing the cache would not stop it from caching it again if when asked for something it resolves that something back to the 100.x address.

So if a client asks your AD dns for something that is not in your namespace what does it do - does it resolve it, does it forward it to pfsense? Does it forward it to some external dns?

Where exactly did you find this 100 address - you mention something about an interface? What I can tell you is that does not resolve on the public internet for sb.domain.com be it either of the examples you have given. And anything.whatever.etc.local would not resolve anywhere on the public internet

If you are using say pfblocker on pfsense - and your AD forwards to pfsense, then its possible that IP is getting returned as your blocked IP?

asadz

Not sure why you think a client would validate that the PTR matches the fqdn for some IP given back in A query.

This is since for .mydomain.local both SOA tells the administrative / authoritative zones for abc.mydomain.local is domain controller, and I expect it to return the correct address.

I have not had to play with MS dns for quite some time. But yes if however it finds that 100 answer it would cache, maybe you have a configured device registering that in your AD DNS?

I check all records there is none for 100.* address.

asadz

So if a client asks your AD dns for something that is not in your namespace what does it do - does it resolve it, does it forward it to pfsense? Does it forward it to some external DNS?

according to DHCP settings it should goto pfsense for addresses where NS is no authoritative, proof is in the DNS-REPLY.LOG from pfblockerng below

https://www.reddit.com/r/pfBlockerNG/comments/zneaec/comment/j0hpsw1/?context=3
see the dns-reply.log, (I cannot put here as it get flagged as spam idk)

this is all outbound forward request made by DC the IP address is DC, and you can in response it brought back 100.*. This is the first entry of its own kind of entire /var/log. Under snort this IP address was present as PASSLIST but that passlist was not associated on snort interfaces.

Currently pfblockerng is disabled. Previously I couldn't ping this IP too, but now I can when I remove its alert entry in snort.

And anything.whatever.etc.local would not resolve anywhere on the public internet

I agree its not, but for all .local address nslookup it bring to 100.x which is no the correct PTR record for this zone.

asadz

based upon tracert results the query is going outside the network to find 100.2.3.4

johnpoz

@asadz well yeah if clients asks for something and gets back a IP that is not local - it would try to get their via its gateway.

The question is here is why/how is it getting this 100.2.3.4 address for what is being asked..

Where does AD get its answers when what is asked is not in your namespace? Does it forward, to where - does it resolve? Tell you right now .local would not resolve on the public internet.

You would get back a SOA for the roots because .local is not a valid TLD. There should be no way anything .local should be resolvable to anything outside your network.

Your 2 example sb.scorecard or sb.scorecodesearch.com do not resolve to 100.x on the public internet either. So somewhere you have it setup to return that 100.x address for something, be it a host entry in unbound or your AD, or some sort of wild card, etc. when there is a .local tld

100.x which is no the correct PTR record for this zone.

Not understanding your thought process on how that has anything to do with anything..

To get to the bottom of this - just take pfsense and your AD talking to each other out of the equation. Again what is your AD dns setup to do when you ask it for something it is not authoritative for - its either going to forward, or its going to resolve. If forwarding to pfsense change that so it forwards to say gooledns or cloudflare or something.. On pfsense remove any domain overrides for .local etc..

Now query each of them for your fqdn that is returning the bad 100.x address.

asadz

This post is deleted!

johnpoz

@asadz flush the entries where? Not sure why you care about connections and traceroutes - if stuff your client want to talk to are resolving to this IP address, then its a give they will attempt to make these connections.

Those all go away when you have correct resolution of what your actually trying to get to - so now there is a proxy as well - proxies unless transparent will resolve things for where the client asks them to go, resolution is done by the proxy not the client when proxy is explicitly set on the client.

asadz

@johnpoz yes using HAPROXY but its for reverse proxy , I don't think it will effect how for addresses that local e.g .local.