Problems with pfSense IPV6 DNS function (does it exist!?)

louis2

@johnpoz

It is clear the DNS ACL's are the problem!

Below you see three tests:

using cloudfare DNS
using the pfSense GW without explit ACL
using the pfSense GW with an ACL added

So its a bug

Also note wonder

if these ACL's are implemented via unbound
or that it is in a normal FW-rule ... under the surface ......

louis2

@johnpoz

Hum, I was looking for the config files below the DNS ...

Found /etc/resolv.conf
nameserver 127.0.0.1
nameserver 8.8.8.8
nameserver 2001:4860:4860::8844
nameserver 1.0.0.1
nameserver 2606:4700:4700::1001
search lan

find / -name unbound.conf
/var/unbound/unbound.conf
/usr/local/share/strongswan/templates/config/plugins/unbound.conf
/usr/local/etc/strongswan.d/charon/unbound.conf
/usr/local/etc/unbound/unbound.conf

Where /var/unbound/unbound.conf turned out to be the used config ...

unbound.zip

The content of the /var/unbound/remotecontrol.conf
remote-control:
control-enable: yes
control-interface: 127.0.0.1
control-port: 953
server-key-file: "/var/unbound/unbound_server.key"
server-cert-file: "/var/unbound/unbound_server.pem"
control-key-file: "/var/unbound/unbound_control.key"
control-cert-file: "/var/unbound/unbound_control.pem"

I will create a bug report, referring to this thread

johnpoz

@louis2 said in Problems with pfSense IPV6 DNS function (does it exist!?):

So its a bug

Users always jumping on the its a bug wagon..

What version of pfsense are you even using? Are you using 2.7 or 23.01 beta? Or stable release?

louis2

@johnpoz

My actual version is 2.7 built on Fri Jan 06 06:05:17 UTC 2023

And even if the problem is only related to the 2.7 build, it is still a bug / something to be fixed.

johnpoz

@louis2 said in Problems with pfSense IPV6 DNS function (does it exist!?):

it is still a bug / something to be fixed.

What snap are you on - for all we know it was already taken care of.. Your whole thread should of been in the dev section if your using a snapshot/beta..

I agree it needs to be corrected if it an actual problem - but before going taking the time to report a "bug" I would bring up your issue in the dev section for 2.7, what snap are you using, etc.

edit: should prob move this whole thread to here

https://forum.netgate.com/category/88/ce-2-7-0-development-snapshots

And you can add what snapshot your on, and see if anyone else seeing the problem - then they can fix it, etc. For all we know it was a temp issue on your specific snap

edit: I moved this thread to 2.7 dev - could you post up your snapshot version your using, and update to the current snap and validate it still an issue with auto acls adding IPv6, etc.

louis2

@johnpoz

I do not know if the problem is only related to the 2.7 release. Perhaps I doubt.

I also did not know if I did something stupid or that it was a bug in first instance.

But If you think it should be in the def section two things:

be so kind to test first if it does work on a stable release
and if so be my guest move this thread to the dev section, that is something I can not do

johnpoz

@louis2 I have moved it.. If it was an issue with stable I would think would of been reported long ago.. But sure I can test it on my 2.6 vm install and my 22.05 main setup.

Gertjan

@louis2

Not related .... but :

These are not comment fields for you own usage.
If you fill them in, use the real host name.
To find out :

[22.05-RELEASE][root@pfSense.wth.net]/root: host 1.0.0.1
1.0.0.1.in-addr.arpa domain name pointer one.one.one.one.
[22.05-RELEASE][root@pfSense.wth.net]/root: host 8.8.8.8
8.8.8.8.in-addr.arpa domain name pointer dns.google.
etc.

cursixx

@louis2 I'm having the same problem with dev plus 23.01 and dev CE 1/06.

When using DNS Resolver (Enable Forwarding Mode on and off) to the LAN ipv6 address I get a query refused. ipv4 LAN address works fine

If I switch to DNS Forwarder both ipv4 and ipv6 return results as expected.

The symptoms I first noticed was, when loading sites in chrome I would get dns_probe_finished_nxdomain for 2-3 secs and then the site would load. ios and andriod devices would have problems too.

My setup is pretty much default.

johnpoz

@cursixx you get a refused returned or dns just fails? Refused is indication that acls are not allowing your query, fail or timeout would point to possible other issue. I do believe there has been some issues related to unbound and using IPv6 in general.

cursixx

@johnpoz yes unbound returns "query refused" only with ipv6 LAN address as the DNS. I'm not sure when this started but it has been going on for at least a few weeks for me. I update dev just about everyday but switch back to stable from time to time.

Anything you want me to test with this?

louis2

@gertjan

Thanx! I did enter that a while back not reading, just assuming what should be there

At this moment I doubt a bit about what should be there

1.0.0.1.in-addr.arpa or
one.one.one.one. (with or without ending dot !?)

For now I defined it as shown in the picture below.
nslookup one.one.one.one. with and without dot function both !??

Gertjan

@cursixx said in Problems with pfSense IPV6 DNS function (does it exist!?):

Anything you want me to test with this?

If you use unbound without pfBlockerng-devel, you could set Services > DNS Resolver > Advanced Settings log level to 4, 5 or even 6.
Be careful : don't leave your system in this sate forever. unbound now start to log a lot (like a huge load of a lot) log lines. These will get rotated very often, but still, see this as a test - and stay onto it.
Or, if you have unbound and pfBlockerng-devel, and you use the python mode, : go here : Firewall > pfBlockerNGLog > Browser and select the dns_reply.log.

As soon as you detect an issue, go check the log file and see if unbound received the DNS request (you have the domain name, so search for it) and see what unbound received the request, and what it did with it.

Check also your device's IPv6, which should match with what you find here : /var/unbound/access_lists.conf

Example :
I used nslookup to resolve :

nslookup www.google.es. 2001:470:1f13:beef:2::1

where 2001:470:1f13:beef:2::1 is the LAN IPv6 of my LAN.
In the dns_reply.log I saw :

DNS-reply,Jan 9 16:29:34,reply,A,A,300,www.google.es,2001:470:1f13:5c0:2::88,74.125.140.94,US
DNS-reply,Jan 9 16:29:35,reply,AAAA,AAAA,300,www.google.es,2001:470:1f13:5c0:2::88,2a00:1450:4007:81a::2003,IE

and that was correct, nslookup was asking for the A and the AAAA.

@louis2 said in Problems with pfSense IPV6 DNS function (does it exist!?):

nslookup one.one.one.one. with and without dot function both !??

Look in the dns_reply.log file and you have you answer ;)

When you do a :
nslookup somethingspecial.tld

You will see this (these) line(s) :

DNS-reply,Jan 9 16:36:43,reply,A,SOA,3600,somethingspecial.tld.my-pfsense-local-domain.net,192.168.1.2,SOA,unk

If you don't stop with a dot, unbound (nslookup probably) will add the device's local network domain.

That why this works :

C:\Users\gwkro>nslookup gauche2
Serveur :   pfSense.my-pfsense-local-domain.net
Address:  192.168.1.1

Nom :    gauche2.my-pfsense-local-domain.net
Addresses:  2001:470:1f13:beef:2::c7
          192.168.1.6

"gauche2" is a PC in my pfSense LAN.

I could (should !) also ask :
gauche2.my-pfsense-local-domain.net.
which is de fully qualified host name of that device.

marcosm

This sounds like a mis-configuration somewhere rather than a bug. For reference, the ACLs are stored in /var/unbound/access_lists.conf. It works correctly here:

#> nslookup google.com 2001:db8::1
Server:  gw.home.arpa
Address:  2001:db8::1

Non-authoritative answer:
Name:    google.com
Addresses:  2607:f8b0:4023:1006::64
          2607:f8b0:4023:1006::65
          2607:f8b0:4023:1006::8a
          2607:f8b0:4023:1006::8b
          142.251.32.238

louis2

@marcosm

Yep thanks explaining where the config file is. However, yep there are no IPV6-entrys appart of the test one I entered manually.

[2.7.0-DEVELOPMENT][root@pfSense.lan]/root: cat /var/unbound/access_lists.conf
access-control: 127.0.0.1/32 allow_snoop
access-control: ::1 allow_snoop
access-control: 10.128.244.0/22 allow
access-control: 127.0.0.0/8 allow
access-control: 192.168.1.0/24 allow
access-control: 192.168.2.0/24 allow
access-control: 192.168.2.1/24 allow
access-control: 192.168.5.0/24 allow
access-control: 192.168.9.0/24 allow
access-control: 192.168.10.0/24 allow
access-control: 192.168.11.0/24 allow
access-control: 192.168.13.0/24 allow
access-control: 192.168.14.0/24 allow
access-control: 192.168.17.0/24 allow
access-control: 192.168.18.0/24 allow
access-control: 192.168.88.0/24 allow
access-control: 192.168.100.0/24 allow
access-control: 192.168.110.0/24 allow
access-control: 192.168.111.0/24 allow
access-control: 192.168.120.0/24 allow
access-control: 192.168.130.0/24 allow
access-control: ::1/128 allow
#IPV6-test
access-control: 2a02:xxxx:yyyy:11::/64 allow

louis2

@gertjan

I am still confused. I did not have a look at the logging yet,

however:

nslookup example.com dns.google AND
nslookup example.com dns.google. AND
nslookup example.com 2001:4860:4860::8844 AND
nslookup example.com 8.8.8.8

Are all providing the same correct answer. And all provide the correct server name dns.google (without an domain added).
So that does not match your example .. / screen captures ....

The second column in the DNS entry screen say
Hostname
Enter the DNS Server Hostname for TLS Verification in the DNS Resolver (optional)

So that suggests a name not an address ....
Is 8.8.8.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.6.8.4.0.6.8.4.1.0.0.2.ip6.arpa. a name or an address !? I would say an address

The documentations says
The FQDN of the DNS server, used to validate DNS server certificates when using DNS over TLS (DNS Resolver Configuration).

Further on:
https://www.nslookup.io/domains/dns.google/dns-records/ptr/

2001:4860:4860::8888 => 8.8.8.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.6.8.4.0.6.8.4.1.0.0.2.ip6.arpa. 86400 PTR dns.google
dns.google does not have any PTR records.
dns.google. does not have any PTR records.
dns.google.com does not have any PTR records.

So I will do some debugging, but to say that the page or documentation is helpful ..... hardly. One or more examples would help.

Then we have this setting

Showing IPV4 behavoir .... I assume that behavoir is also used for IPV6 ....

cursixx

@gertjan Here is the debug log. Let me know if you think I missed something (sorry for the picture. spam filter would not let me post it as code)

johnpoz

@louis2 said in Problems with pfSense IPV6 DNS function (does it exist!?):

used to validate DNS server certificates

this is big take away.. that fqdn is used on the cert to validate your talking to the right dot server..

So for example if I do a dot query to 1.1.1.1 and tell it to validate everything then using something other than the CN or what is in the SAN of the cert would fail.. A normal dot query should be validating this..

You can get all the info of CN and SANs that are in the cert with some use of openssl, notice the output below for the CN and SANs in that cert being used at 1.1.1.1 on port 853.

But from a kdig asking for validation you can see it fails if I use a fqdn that is not CN or SAN.

root@NewUC:/home/user# kdig -d @1.1.1.1 www.google.com +tls-ca
;; DEBUG: Querying for owner(www.google.com.), class(1), type(1), server(1.1.1.1), port(853), protocol(TCP)
;; DEBUG: TLS, imported 124 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, C=US,ST=California,L=San Francisco,O=Cloudflare\, Inc.,CN=cloudflare-dns.com
;; DEBUG:      SHA-256 PIN: MnLdGiqUGYhtyinlrGTC4FZdDyDXv4NOWFGnXW3ur14=
;; DEBUG:  #2, C=US,O=DigiCert Inc,CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1
;; DEBUG:      SHA-256 PIN: e0IRz5Tio3GA1Xs4fUVWmH1xHDiH2dMbVtCBSkOIdqM=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.3)-(ECDHE-X25519)-(ECDSA-SECP256R1-SHA256)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 30091
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 1232 B; ext-rcode: NOERROR
;; PADDING: 405 B

;; QUESTION SECTION:
;; www.google.com.              IN      A

;; ANSWER SECTION:
www.google.com.         128     IN      A       142.250.191.132

;; Received 468 B
;; Time 2023-01-09 13:12:01 CST
;; From 1.1.1.1@853(TCP) in 61.0 ms
root@NewUC:/home/user#

But if I tell it wrong host name..

;; From 1.1.1.1@853(TCP) in 61.0 ms
root@NewUC:/home/user# kdig -d @1.1.1.1 www.google.com +tls-ca +tls-host=badname
;; DEBUG: Querying for owner(www.google.com.), class(1), type(1), server(1.1.1.1), port(853), protocol(TCP)
;; DEBUG: TLS, imported 124 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, C=US,ST=California,L=San Francisco,O=Cloudflare\, Inc.,CN=cloudflare-dns.com
;; DEBUG:      SHA-256 PIN: MnLdGiqUGYhtyinlrGTC4FZdDyDXv4NOWFGnXW3ur14=
;; DEBUG:  #2, C=US,O=DigiCert Inc,CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1
;; DEBUG:      SHA-256 PIN: e0IRz5Tio3GA1Xs4fUVWmH1xHDiH2dMbVtCBSkOIdqM=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is NOT trusted. The name in the certificate does not match the expected. 
;; WARNING: TLS, handshake failed (Error in the certificate.)
;; ERROR: failed to query server 1.1.1.1@853(TCP)
root@NewUC:/home/user#

if I use correct hostname, see the below openssl output to what all the valid SANs are in that cert

root@NewUC:/home/user# kdig -d @1.1.1.1 www.google.com +tls-ca +tls-host=one.one.one.one
;; DEBUG: Querying for owner(www.google.com.), class(1), type(1), server(1.1.1.1), port(853), protocol(TCP)
;; DEBUG: TLS, imported 124 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, C=US,ST=California,L=San Francisco,O=Cloudflare\, Inc.,CN=cloudflare-dns.com
;; DEBUG:      SHA-256 PIN: MnLdGiqUGYhtyinlrGTC4FZdDyDXv4NOWFGnXW3ur14=
;; DEBUG:  #2, C=US,O=DigiCert Inc,CN=DigiCert TLS Hybrid ECC SHA384 2020 CA1
;; DEBUG:      SHA-256 PIN: e0IRz5Tio3GA1Xs4fUVWmH1xHDiH2dMbVtCBSkOIdqM=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted. 
;; TLS session (TLS1.3)-(ECDHE-X25519)-(ECDSA-SECP256R1-SHA256)-(AES-256-GCM)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 62622
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 1232 B; ext-rcode: NOERROR
;; PADDING: 405 B

;; QUESTION SECTION:
;; www.google.com.              IN      A

;; ANSWER SECTION:
www.google.com.         289     IN      A       142.250.190.4

;; Received 468 B
;; Time 2023-01-09 13:14:10 CST
;; From 1.1.1.1@853(TCP) in 63.2 ms
root@NewUC:/home/user#

As to your log - seems it pretty clear to why its refusing you, that "::/128 refused" where that is coming from not sure.. But that would explain why its refusing you.

root@NewUC:/home/user# openssl s_client -connect 1.1.1.1:853 </dev/null 2>/dev/null | openssl x509 -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0d:1c:7a:f2:8e:5f:27:17:db:b2:7f:41:08:20:bd:f7
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
        Validity
            Not Before: Sep 13 00:00:00 2022 GMT
            Not After : Sep 13 23:59:59 2023 GMT
        Subject: C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = cloudflare-dns.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:fc:3e:51:ef:74:1d:c6:da:78:ba:ae:a5:8a:4a:
                    dd:d9:0b:e6:e2:5b:57:31:57:de:d3:bf:b6:d9:8a:
                    3b:4f:d2:54:54:88:cf:bd:2e:65:e7:66:eb:c5:df:
                    d0:31:54:52:a7:2c:ee:12:86:a3:9a:66:c1:ea:06:
                    79:03:ba:1b:f0
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                0A:BC:08:29:17:8C:A5:39:6D:7A:0E:CE:33:C7:2E:B3:ED:FB:C3:7A
            X509v3 Subject Key Identifier: 
                D2:63:BA:94:D6:54:7F:4C:85:14:08:3A:1C:85:56:29:EF:59:8F:CC
            X509v3 Subject Alternative Name: 
                DNS:cloudflare-dns.com, DNS:*.cloudflare-dns.com, DNS:one.one.one.one, IP Address:1.0.0.1, IP Address:1.1.1.1, IP Address:162.159.36.1, IP Address:162.159.46.1, IP Address:2606:4700:4700:0:0:0:0:1001, IP Address:2606:4700:4700:0:0:0:0:1111, IP Address:2606:4700:4700:0:0:0:0:64, IP Address:2606:4700:4700:0:0:0:0:6400
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 CRL Distribution Points: 
                Full Name:
                  URI:http://crl3.digicert.com/DigiCertTLSHybridECCSHA3842020CA1-1.crl
                Full Name:
                  URI:http://crl4.digicert.com/DigiCertTLSHybridECCSHA3842020CA1-1.crl
            X509v3 Certificate Policies: 
                Policy: 2.23.140.1.2.2
                  CPS: http://www.digicert.com/CPS
            Authority Information Access: 
                OCSP - URI:http://ocsp.digicert.com
                CA Issuers - URI:http://cacerts.digicert.com/DigiCertTLSHybridECCSHA3842020CA1-1.crt
            X509v3 Basic Constraints: 
                CA:FALSE
            CT Precertificate SCTs: 
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : E8:3E:D0:DA:3E:F5:06:35:32:E7:57:28:BC:89:6B:C9:
                                03:D3:CB:D1:11:6B:EC:EB:69:E1:77:7D:6D:06:BD:6E
                    Timestamp : Sep 13 22:48:42.079 2022 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:44:02:20:13:E1:7D:A3:DD:99:5A:FF:0B:9F:41:A7:
                                37:EA:F3:4D:82:23:03:09:43:DD:AC:2D:C8:9C:C8:4E:
                                3C:34:32:F7:02:20:35:1A:CE:22:F9:06:48:3D:8D:0D:
                                C9:EB:9D:6C:92:C2:03:BA:59:D6:94:EE:8F:71:38:E8:
                                26:00:12:76:48:0F
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 35:CF:19:1B:BF:B1:6C:57:BF:0F:AD:4C:6D:42:CB:BB:
                                B6:27:20:26:51:EA:3F:E1:2A:EF:A8:03:C3:3B:D6:4C
                    Timestamp : Sep 13 22:48:42.151 2022 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:A2:B6:45:5D:DE:32:F8:03:3C:9C:BD:
                                5F:49:1B:8B:79:1B:FB:39:68:73:53:43:83:4E:77:8A:
                                37:D2:56:76:E2:02:21:00:84:92:F2:CA:5E:AB:16:F4:
                                50:EC:8D:95:F0:84:36:B0:A8:99:B8:75:05:9E:A0:4A:
                                F7:C2:16:F6:8E:23:1C:4F
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : B3:73:77:07:E1:84:50:F8:63:86:D6:05:A9:DC:11:09:
                                4A:79:2D:B1:67:0C:0B:87:DC:F0:03:0E:79:36:A5:9A
                    Timestamp : Sep 13 22:48:42.231 2022 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:21:00:9A:01:8C:AF:2F:41:72:8B:57:0B:1C:
                                1C:07:90:22:B1:C3:D7:FE:80:BB:FB:D2:B5:AA:CE:60:
                                5F:0F:62:0E:DF:02:20:0A:8C:4B:CE:DB:26:DF:B8:5E:
                                67:D6:9E:68:7C:CE:C4:D4:15:9A:53:3B:80:1F:07:F6:
                                6B:80:6B:07:18:86:A0
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:65:02:30:19:d7:9a:79:6e:69:cc:19:6f:dd:c3:d0:36:c8:
        54:d0:4e:f4:6f:1d:46:c0:ca:2f:fb:22:19:95:f8:70:75:0b:
        59:d9:ad:16:31:6e:1a:26:1b:16:df:71:0d:55:cd:4a:02:31:
        00:ea:dc:2e:09:98:fe:b1:fb:07:f5:31:5f:b6:7c:1a:be:38:
        d9:c5:39:9a:6a:0b:9d:8f:54:67:13:4b:c1:12:86:26:2a:11:
        6d:98:d3:91:1d:02:eb:e1:77:76:71:37:8e
root@NewUC:/home/user#

If you don't want to look through all that, the important info from the cert is this

X509v3 Subject Alternative Name:
DNS:cloudflare-dns.com, DNS:*.cloudflare-dns.com, DNS:one.one.one.one, IP Address:1.0.0.1, IP Address:1.1.1.1, IP Address:162.159.36.1, IP Address:162.159.46.1, IP Address:2606:4700:4700:0:0:0:0:1001, IP Address:2606:4700:4700:0:0:0:0:1111, IP Address:2606:4700:4700:0:0:0:0:64, IP Address:2606:4700:4700:0:0:0:0:6400

big fan of dig, and use it pretty much everyday - but kdig is much better with working with dot and or doh. And with anything cert related use of openssl is your go to tool.

cursixx

I did a factory reset of my config and set my WAN and LAN interfaces (dev 2.7 1/9). Never even logged in and I'm still getting query refused with ipv6. I just wanted to rule out any of my config changes as the issue.

Then I loaded a clean install of stable 2.6 and it seems to be working as expected

Bob.Dig

@marcosm I can confirm this bug for 23.01.r.20230202.1645 too. At least for ULAs.