DNS Resolver - getting IPv6 results when there is no IPv6?

JKnott · Apr 3, 2020, 2:15 PM

@johnpoz said in DNS Resolver - getting IPv6 results when there is no IPv6?:

Devices, Oses, Applications sure can ask for AAAA even when the device has no IPv6 address..

Well, I just used Wireshark on a computer running Linux. I pinged google.com, yahoo.com and ipv6.google.com. I didn't see any AAAA requests, only A. When I tried an IPv6 only host name, I got an unknown host message. What happens if you set up an IPv4 only network and watch with Wireshark? Also, do apps actually request IPv4 & IPv6 addresses? Or do they just request an address and use whatever comes back? I know the OS can be configured to prefer one or the other.

johnpoz · Apr 3, 2020, 2:32 PM

Here look...

$ ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : i5-win
   Primary Dns Suffix  . . . . . . . : local.lan
   Node Type . . . . . . . . . . . . : Broadcast
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : local.lan

Ethernet adapter Local:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Realtek PCIe GbE Family Controller
   Physical Address. . . . . . . . . : 00-13-3B-2F-67-62
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 192.168.9.100(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.9.253
   DNS Servers . . . . . . . . . . . : 192.168.3.10
   NetBIOS over Tcpip. . . . . . . . : Enabled

No IPv6.... But when set debug on nslookup you see it asking for AAAA when all I asked for was www.google.com

Again - it will depend on the OS, the application, etc.. But yes is quite normal to see queries for AAAA even when your not running IPv6

I would assume linux is far better at not doing this then a windows or windows applications ;)

Here for example off one of linux boxes.

user@ombi:~$ ifconfig
ens3      Link encap:Ethernet  HWaddr 02:11:32:28:77:34  
          inet addr:192.168.2.22  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::11:32ff:fe28:7734/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1020393 errors:0 dropped:2 overruns:0 frame:0
          TX packets:463097 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1332305940 (1.3 GB)  TX bytes:106337407 (106.3 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:1972 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1972 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:187779 (187.7 KB)  TX bytes:187779 (187.7 KB)

user@ombi:~$ nslookup
> set debug
> www.google.com
Server:         127.0.0.1
Address:        127.0.0.1#53

------------
    QUESTIONS:
        www.google.com, type = A, class = IN
    ANSWERS:
    ->  www.google.com
        internet address = 172.217.4.196
        ttl = 2396
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
Name:   www.google.com
Address: 172.217.4.196
>

no query for AAAA

JKnott · Apr 3, 2020, 2:52 PM

@johnpoz said in DNS Resolver - getting IPv6 results when there is no IPv6?:

Again - it will depend on the OS, the application, etc.. But yes is quite normal to see queries for AAAA even when your not running IPv6

Did it return an AAAA record?

johnpoz · Apr 3, 2020, 2:59 PM

No it wouldn't because one wasn't asked for ;) You can see in the debug exactly what was asked for.

JKnott · Apr 3, 2020, 3:56 PM

@johnpoz

I just tried using the Linux "host" command for ipv6.google.com. It showed the IPv6 address, just as your nslookup example did. Regarless, that does not represent an attempt to get an AAAA record by an OS, when on an IPv4 only network.

johnpoz · Apr 3, 2020, 4:42 PM

what??? I have no idea what your going on about... You can see in my windows nslookup that I have no IPv6 address at all, doesn't even show link local and still queries for AAAA, even though I did not call out in nslookup A or AAAA - it on its own asked for A and then AAAA

linux even having link local, does not query for the AAAA record.

Lets state this again.. It would be up to the OS, or the application if it asks for AAAA or not.. That may or may not happen depending on your OS and or your applications..

But just because your IPv4 only network - that doesn't mean that AAAA might not be queried for.. So seeing AAAA queries is quite normal even in a IPv4 only network..

Its just another record, like TXT or CNAME or PTR or SRV, etc. It really has little to do with the actual protocol.. Other than that has been the RR designed to handle IPv6 addresses for dns. Like A records.

Here for example - my NAS is static, has no IPv6 configured.. I do not run any sort of slaac dhcpv6 on this network.. It does not have IPv6 configured... And yet all on its own going about its business it queries for AAAA

It is set ti IPv6 OFF

And yeah just on its own, no client doing anything, etc.. Its normal operation - it queries its configured IPV4 dns for AAAA..

JKnott · Apr 3, 2020, 5:33 PM

@johnpoz said in DNS Resolver - getting IPv6 results when there is no IPv6?:

what??? I have no idea what your going on about...

You used nslookup, I used host. Same function. Both are used to obtain IP addresses for information purposes, not for actually accessing a site. On the other hand, the OS will request A or A & AAAA records, according to what the computer can handle.

Bottom line, if a computer has an IPv6 address, beyond link local, it will ask for AAAA records, otherwise not. You can try as I did with a test network and Wireshark or Packet Capture to verify.

johnpoz · Apr 3, 2020, 5:49 PM

if a computer has an IPv6 address, beyond link local, it will ask for AAAA records, otherwise not.

No this not true... You have no freaking idea what the application might do... I have shown you direct examples of a box with ZERO ipv6 address - and it still asking for AAAA... Its just a freaking record, how the application is written determines what is might ask for. You could also write your application to ONLY query A, even if the box only had IPv6... AAAA is just a record..

Yup applications and OSes can do different things..

I don't understand why your so confused about this.. The DNS resolver has no control what its get asked... If its asked for AAAA then it will return those.. If they exist if not, then it will return SOA, etc.

If the client asks for AAAA and there no AAAA record, then it will return the SOA, etc. etc..

;; QUESTION SECTION:
;www.reddit.com.                        IN      AAAA

;; ANSWER SECTION:
www.reddit.com.         3600    IN      CNAME   reddit.map.fastly.net.

;; AUTHORITY SECTION:
fastly.net.             460     IN      SOA     ns1.fastly.net. hostmaster.fastly.com. 2017052201 3600 600 604800 30

If the OP got back AAAA for something - HE ASKED FOR IT!! Be it he was aware of it or not...

Now many servers are starting to REFUSE the any query... But if you query a NS for ANY, you will get back all records for that host.. .So if it has A and AAAA you would get back both, etc. etc..

JKnott · Apr 3, 2020, 5:59 PM

@johnpoz said in DNS Resolver - getting IPv6 results when there is no IPv6?:

No this not true... You have no freaking idea what the application might do... I have shown you direct examples of a box with ZERO ipv6 address - and it still asking for AAAA... Its just a freaking record, how the application is written determines what is might ask for. You could also write your application to ONLY query A, even if the box only had IPv6... AAAA is just a record..

Do applications specifically ask for IPv4 or IPv6 addresses? Or do they just ask for an address? Nslookup and host are applications that are used to look up the addresses for a site and so would request both. On the other hand an app connecting to a site just needs a working address.

Here are nslookup and host used to find addresses.

nslookup google.com
Server: xxxxxx.yyyy.net
Address: fd48:1a37:2160:0:216:17ff:fea7:f2d3

Non-authoritative answer:
Name: google.com
Addresses: 2607:f8b0:400b:801::200e
172.217.165.14

host google.com
google.com has address 172.217.165.14
google.com has IPv6 address 2607:f8b0:400b:801::200e
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.

As mentioned, the app's purpose is to list addresses.

Now, if you open a browser, it just needs an address, either IPv4 or IPv6. Does it actually request both? I could be wrong, but I don't think so.

Also, it makes no difference if the DNS servers are reached with an IPv4 or IPv6 address, as both return the same info.

JKnott · Apr 3, 2020, 6:16 PM

@johnpoz

Incidentally, both C and Python have a gethostbyname() function. I don't see any mention of choosing IPv4 or IPv6.

mmiller7 · Apr 4, 2020, 2:32 AM

@Gertjan Some of the devices (like my work-issued laptop) I can't change any settings because I'm not an administrator. Others aren't my computers (roomates, friends when we aren't in lockdown for a pandemic, smartphones that have IPv6 when they are on 4G) and probably shouldn't be changed.

I've already tried the block-IPv6 checkbox (I've had an ISP before with broken IPv6 implementation and had to use that before we moved) didn't seem to make a difference so I put it back to allowed.

My Linux Mint box (main machine) I have a "Scope:Link" IPv6 address when I look at ifconfig but no global IPv6 address...and the router has no IPv6 address for the WAN.

I'll try and find the right filters to capture only DNS traffic in Wireshark and see if I can make any sense of what apps are requesting (if I can reproduce it...)

EDIT: That was quicker than I expected...yeah seems command line 'ftp' is asking for AAAA records. I don't understand why though, or what to do about it yet..

I do notice, this time it listed both IPs on the command line...I think when I have trouble it only lists the IPv6. So maybe that's something with when my upstream DNS hickups on one of the queries somehow?

Resolving cddis.nasa.gov (cddis.nasa.gov)... 198.118.242.40, 2001:4d0:241a:442::52
Connecting to cddis.nasa.gov (cddis.nasa.gov)|198.118.242.40|:21... connected.
Logging in as anonymous ... Logged in!

EDIT2:
And while I didn't happen to have wireshark up at the try when it reproduced the error, sure enough I got a different result on the temrinal...

Resolving cddis.nasa.gov (cddis.nasa.gov)... 2001:4d0:241a:442::52
Connecting to cddis.nasa.gov (cddis.nasa.gov)|2001:4d0:241a:442::52|:21... 
failed: Connection timed out.

JKnott · Apr 4, 2020, 2:41 AM

@mmiller7 said in DNS Resolver - getting IPv6 results when there is no IPv6?:

And while I didn't happen to have wireshark up

You can always use Packet Capture and download the capture so you can read it with Wireshark.

mmiller7 · Apr 4, 2020, 3:22 AM

Finally reproduced during capture. I think I see why it's not using IPv4, but I don't know why it's having a server-failure code?

johnpoz · Apr 4, 2020, 9:42 AM

Well will ya look at that there -- client did a AAAA query - who would of thunk it there @JKnott...

Why would it it do that I wonder????? He doesn't have an IPv6 address... <rolleyes>

As to why your getting back SERVFAIL... your going to have to follow those breadcrumbs... resolves here just fine.

;cddis.nasa.gov.                        IN      A

;; ANSWER SECTION:
cddis.nasa.gov.         3599    IN      CNAME   ftp.cddis.eosdis.nasa.gov.
ftp.cddis.eosdis.nasa.gov. 14399 IN     A       198.118.242.40

Its a cname, so maybe you have problem with the cname... Part of the problem when you throw your dns over the fence to someone like 1.1.1.1 or 8.8.8.8 and it doesn't work.. Its a black box... Now resolving on the other hand.. That you just see why - does the authoritative ns not answer, walk the tree down to see where it has problem... Other then hey googledns whats IP for www.domain.tld??

In your sniff - your dns query took 2.25 seconds to get a response.. Dude all kinds of shit would fail.. why do you have such a long response time.. A common timeout for dns is 2 seconds, then would switch over to tls maybe.. Maybe that worked.. I would take a serious look to your dns if that is typical??

Your query should be ms, not whole seconds..

A query to say 1.0.0.1 was 11ms for that.

;; Query time: 11 msec
;; SERVER: 1.0.0.1#53(1.0.0.1)

mmiller7 · Apr 4, 2020, 1:44 PM

And probably 80% of the time it (and other sites) resolve just fine here. Then randomly it won't.

Is there somewhere on pfSense that I don't know about where it may be logging what happened or what failed to respond? Is there a way to make it log failures to get responses from upstream without logging the billions of good requests from everything all over my network to wade thru?

Since it's infrequent that it fails, I don't know how I can easily tell where it's failing without some kind of logs to analyze.. pfSense is the DNS server for everything on my LAN, then it sends it off to the upstream servers. I would have thought with 4 DNS servers from 2 providers it should be able to resolve reliably from one of them? Does it not try others if the first doesn't answer?

johnpoz · Apr 4, 2020, 1:59 PM

You got an answer... It was FAIL, and it took 2.25 seconds.. That is not correct..

How long does something that is not cached locally take to respond? typically?

johnpoz · Apr 4, 2020, 2:27 PM

@mmiller7 said in DNS Resolver - getting IPv6 results when there is no IPv6?:

Does it not try others if the first doesn't answer?

It does, after timeout... Which quite often the client has already given up... And your not asking 4, your using 4 different IPs for 2 different services that are both anycast... The odds that one of those would fail and the other pass is just not realistic..

Servfail is not timeout.. So why would it check the other one - why would NS x be able to resolve it, while others said its broken!

If you want to blast multiple ns and get the first response - then use dnsmasq as the forwarder.. It does that out of the box, but can be switched to sequential mode. Unbound would be sequential..

Off the top I don't know you can not just log servfail ;) highly unlikely. But your client can give you that right away... Just query with something that you can see the actual response with, other than just some application client.

First thing would of done when you were just getting back IPv6 in your ftp client would of did a dig to that fqdn to see what was coming back..

dig
host
nslookup

All would of shown you the servfail response.. You could put nslookup debug mode to get more info... But dig is really the tool of choice..

Also 80% of the time is HORRIBLE... 99.999% of the time is how dns should work and you shouldn't even have to think about it.. If your having so many issues that you just off the top of your head throw out it works 80% of the time, you got some serious problems...

mmiller7 · Apr 4, 2020, 2:44 PM

My understanding of dnsmasq is it is just a forwarder - doesn't have the same features I'm using (local DNS resolution, VPN/DHCP/static DNS, DNSSEC, SSL/TLS, Query Name Minimization, Prefetch caching, etc.

While my internet is fast, there is sometimes high latency (I've had hours of multi-second PING replies) and low speeds (like 40-100Mbps instead of 500Mbps as usual for my Gigabit))/low packet loss (around 0.1-0.2% for a short bit usually at least 1 day a week according to the pfSense graphs) so I try to utilize a number of the pfSense caching features because it can make a noticeable performance difference at peak times (evenings/weekends). ISP's answer is "its working right now" and "we have high demand right now" when I call. I've also ruled out my pfSense box for those issues going straight to the modem, so I have to live with that.

The more strange thing, these issues really do seem to have cropped up in just the past ~2 weeks. It started a few days before I realized 2.4.5 was released (I noticed when I logged in to check why things were acting up).

johnpoz · Apr 4, 2020, 2:57 PM

So your doing DOT - that would explain your 80% success rate and shit response time of 2.25 seconds vs couple of ms..

You understand when you forward..Where you forward does dnssec or doesn't.. If it doesn't you asking for it does squat!! So yeah if you forward to dns that is doing dnssec, you auto get dnssec... So that doesn't matter.. You want dnssec support in unbound because its a RESOLVER.. Your using it as just a forwarder - so pointless what features it supports or doesn't your just asking something upstream..

If your concern is OMG my isp might be sniffing that I want to go to ftp.nasa.gov - and you don't have a normal vpn you can hide that from them... Then ok that would necessitate utter shit for dns performance by forwarding everything to xyz over tls..

Seems to me all your pain is self inflicted because you feel your dns is spying on your dns queries and you trust both cloudflare and google so much more that your just going to hand them all your dns queries, all of them.. And why not make it slower to boot.. Because ya know what dns was a bit too fast before ;) So I want to complicate the shit out of it, and slow it down over encrypted tls..

And then just use a wide open protocol that is not encrypted to grab some stuff via ftp, that my isp can just see everything I grab anyway - but you know F them for being able to see dns query for it ;) hehehehe

mmiller7 · Apr 4, 2020, 2:57 PM

@johnpoz said in DNS Resolver - getting IPv6 results when there is no IPv6?:

You got an answer... It was FAIL, and it took 2.25 seconds.. That is not correct..

How long does something that is not cached locally take to respond? typically?

When it works correctly it seems more or less instant. Looking at the one Wireshark where it worked maybe 0.1 second or less between the request and reply that had an IP?

I notice now in the one that did work when I captured it got SERVFAIL for both A and AAAA followed by a success on both. I don't understand that either.

@johnpoz said in DNS Resolver - getting IPv6 results when there is no IPv6?:

Servfail is not timeout.. So why would it check the other one - why would NS x be able to resolve it, while others said its broken!

Seems like in my case, that is the case though. Sometimes it can't be resolved by at least one (still don't know how to tell WHICH upstream one was queried if it stops at the first "dunno" reply?)

I see this at work annoyingly often where one DNS server (they use MS AD servers) will be "fussy" and return failures when another works, for some reason it mainly seems to affect Linux clients not Windows there. Go figure.

Off the top I don't know you can not just log servfail ;) highly unlikely. But your client can give you that right away... Just query with something that you can see the actual response with, other than just some application client.

Just tried 'dig' from my computer and I don't see where it gives me much useful. Got a fail from pfSense 192.168.1.1, but why? Which upstream failed? I wish I had like a DNS Traceroute.

matthew@Inspiron13-7378 $ dig cddis.nasa.gov

; <<>> DiG 9.10.3-P4-Ubuntu <<>> cddis.nasa.gov
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 26682
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;cddis.nasa.gov.			IN	A

;; Query time: 2444 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sat Apr 04 10:55:46 EDT 2020
;; MSG SIZE  rcvd: 43

matthew@Inspiron13-7378 $ dig cddis.nasa.gov

; <<>> DiG 9.10.3-P4-Ubuntu <<>> cddis.nasa.gov
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22536
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;cddis.nasa.gov.			IN	A

;; ANSWER SECTION:
cddis.nasa.gov.		593	IN	CNAME	ftp.cddis.eosdis.nasa.gov.
ftp.cddis.eosdis.nasa.gov. 14393 IN	A	198.118.242.40

;; Query time: 263 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sat Apr 04 10:55:51 EDT 2020
;; MSG SIZE  rcvd: 90

Also 80% of the time is HORRIBLE... 99.999% of the time is how dns should work and you shouldn't even have to think about it.. If your having so many issues that you just off the top of your head throw out it works 80% of the time, you got some serious problems...

I can't disagree there! Used to be rock solid and I never did think about it.