pfsense blocking certain/some sites

bingo600

@johnpoz
Nice detective work

Now i have to read up on the various dig features

/Bingo

johnpoz

@bingo600 why he would be trying tcp and not udp, not sure. But not answering tcp can cause issues for sure..

Sometimes it is just crazy to me the lack of any understanding of dns for companies that provide servers like websites.. And control their own dns.

Notice the ns are ns11 and ns12 bsnl.in its not like they are having their dns hosted by someone - they have registered their own name servers..

If your going to to do - please have some clue to how to setup and manage dns ;) And if your going to use dnssec - read the rfcs for gosh sake ;) Notice the 11 warnings where they are using the the wrong algo.

Notice this sort of error, when set the limit to 512, which was old limit without edns etc. that would for move to tcp from udp.. You can for sure exceed 512 when doing dnssec

$ dig @ns11.bsnl.in portal2.bsnl.in +dnssec +bufsize=512
;; Truncated, retrying in TCP mode.
;; Connection to 218.248.240.178#53(218.248.240.178) for portal2.bsnl.in failed: connection refused.

see msg size when I ask for dnssec info

;; Query time: 309 msec
;; SERVER: 218.248.240.178#53(218.248.240.178)
;; WHEN: Mon Oct 03 23:08:30 Central Daylight Time 2022
;; MSG SIZE  rcvd: 798

If I set bufsize to 798 it works, if set to 797 is fails because its trying to fall back to tcp, which isn't working.. So yeah I could see lots of problems with this domain for sure. If for whatever reason the connection to the NSers are being limited to the udp msg size. below the size of what that server is sending back.

As a hack/workaround to getting this to work, you could forward to say one of the big boys, google, quad9, 1111 etc.. You could setup a domain override for this specific domain, so that when you ask unbound for this specific domain, it forwards to say 8.8.8.8

The correct thing to do would be to figure out why this site isn't answering on tcp and have them fix that, or figure out exactly why there is a limit in the udp size going on..

Maybe his isp is limiting it? And works on other sites because they answer on the tcp fallback, but this site fails because its its not answering on tcp..

edit: see here when I set udp 512 limit, in my query.. It sends back hey this is Truncated the answer will not fit in that size. So the client asks via tcp, but the server not answering on tcp.

I don't know why pfsense would be limiting its size in its query, not exactly sure why client would be doing that either, so the limit has to be happening somewhere between I would think.

Would be interesting to see the query that is leaving pfsense.. Sniff on the wan while your client asks for the portal.bsnl.in would be good info. See in my sniff, I can tell if a payload size is limited by pfsense. Notice when I just ask, does it ask if a directed query from the client asks ns11 or ns12 directly?

For pfsense to ask, you might have to flush cache of pfsense between queries for it, etc.

Normally should look like this.

edit2:

Has the advanced edns options in unbound been messed with?

bingo600

@johnpoz said in pfsense blocking certain/some sites:

Has the advanced edns options in unbound been messed with?

I doubt it had been messed with by OP

Nice debugging

I'm forwarding pfSense (unbound) DNS queries to my local bind9 server(s).
Primarily because i already had that in place, with working DDNS registration for my home domain.

And i made dual secondary DNS servers in the summerhouse (blush), that my summerhouse pfS is using as forwarders. Summerhouse DNS'es are also registering DDNS on the primary DNS, making all DDNS to replicate via primary to secondaries , and available on both sites.

So any queries from me would be done by bind9.

Btw:
That https://dnsviz.net/ is quite revealing.
Now i have to look into the possibility of RR sigs on my "public domain" (hosted) - Thanx or ..... Now i can't get that out of my head

/Bingo

Gurveer

@johnpoz how to forward this portal2.bsnl.in or portal.bsnl.in to 8.8.8.8 or 1.1.1.1? Please help

Gertjan

@gurveer

First :

Then :

Save and then Apply.

Gurveer

@johnpoz @bingo600 @Gertjan @rcoleman-netgate @stephenw10 @viragomann thanks mine was set to automatic value based interface mtu changed to unbound default sites started working flawlessly after which i enabled dns forwarding too. Huge thanks you guys

bingo600

@gurveer
Why do you use forward, and not just let pfSense resolve ???
Any specific reason or ???

Another +1 for @johnpoz

Gertjan

@bingo600 said in pfsense blocking certain/some sites:

Any specific reason or ???

Something like : The name servers involved being unable to communicate over TCP (so these names servers 'break' DNS) + a total fxckxd up DNSSEC = his unbound resolver refuses to work.

All this said, something amazing is happening : I can look at the site :

and I'm not forwarding, I'm using the pfSense resolver unbound with default settings + dnssec hardening.
So, I tend to think 'something' is not good on @Gurveer's side.
If my pfSense + unbound resolvers works
It should also work for @Gurveer.

stephenw10

I think Johnpoz meant to add a domain override for bsnl.in to 8.8.8.8 or similar so everything else is still resolved directly rather than use forwarding mode.

johnpoz

@stephenw10 exactly..

But did he mess with his udp size? If he did he prob doesn't need to forward and this domain would work.

Gurveer

@bingo600 nothing just did on whim it is fine to run forwarder ?

johnpoz

@bingo600 said in pfsense blocking certain/some sites:

I doubt it had been messed with by OP

You would think that huh - but I find that quite often users click on shit all the time.. Until actually see screenshot or a sniff showing it using 4096, etc. You have no idea what is actually happening other than that ns isn't answering on tcp, and it sends back large info that could exceed 512, etc.

stephenw10

Well I have two machines here hitting this and I'm pretty certain I never changed the buffer size. Both are set to the default (auto based on MTU). Both have 1500B WANs.

Just digging to see what the actual buffer size is but I don't know where that is...yet

stephenw10

Looks to be 512 in unbound.conf...

stephenw10

Mmm, and on working devices:

edns-buffer-size: 1432

That's in 22.11 so newer Unbound version

johnpoz

@stephenw10 that was changed from default, see my screenshot shows 4096 as default.. on 22.05

I had not read that unbound was changing their default settings. But sure pfsense could, default for unbound I think is 1232, I think that came out of a flag day for dns back a couple of years ago.

But that can be overridden with that actual bufsize setting, etc. But the max-udp-size: I believe defaults to 4096.

But if your having issues with a NS that is not answering on tcp, it would behoove you to validate what your settings are, etc. To make sure they are appropriate for your network connection.

stephenw10

That isn't the pfSense default config though. At least not any longer. It should be Auto based on MTU there since 2.5 https://redmine.pfsense.org/issues/10293
If that's a machine that was upgraded from before that it might still be 'Unbound default'. Several of mine are which is why they weren't hitting it.
Still can;t see why these two machines I have ended up at 512 when all their interfaces are 1500B MTU. However I'm not seeing it on any 22.11 test box so it might be moot.

Steve

johnpoz

@stephenw10 I agree with a setting based on mtu is prob the best scenario..

IMHO the problem here is that the NS for this domain doesn't answer on tcp - which is going to cause problems for sure..

I see a few options here, make sure unbound is using an appropriate size for your network and its connection. If settings are correct, and still having issue then the hack/workaround for such a domain would be to forward for that "specific" domain to remove your connections issue with large udp.

The nuclear option sure is to forward all your queries and let them worry about it ;) But to me that is burning down the house to kill a spider..

stephenw10

Yes, absolutely the NS should respond to TCP and the fact it doesn't is broken behaviour.

It's interesting that it's shown this edns buffer issue though. It looks like there probably is a bug there because it shouldn't be 512 in a default config where all the NICs are 1500B MTU.

It's also interesting that it still failed for me with DNSSec disabled where queries are much smaller.

Steve

Gertjan

These are the NS :

[22.05-RELEASE][root@pfSense.xxxxx.net]/root: dig bsnl.in NS +short
ns11.bsnl.in.
ns12.bsnl.in.

Not good :

[22.05-RELEASE][root@pfSense.xxxxx.net]/root: dig portal2.bsnl.in @ns11.bsnl.in +tcp
;; Connection to 218.248.240.178#53(218.248.240.178) for portal2.bsnl.in failed: connection refused.

Who is 218.248.240.178 :

[22.05-RELEASE][root@pfSense.xxxxx.net]/root: host 218.248.240.178
178.240.248.218.in-addr.arpa domain name pointer ns11.bsnl.in.gstkarnataka.gov.in.
178.240.248.218.in-addr.arpa domain name pointer ns11.bsnl.in.
178.240.248.218.in-addr.arpa domain name pointer ns11.bsnl.in.gvmc.gov.in.
178.240.248.218.in-addr.arpa domain name pointer ns11.bsnl.in.eofficeharyana.gov.in.

Multiple reverses .... hummm.
It's "ns11.bsnl.in." who was refusing to answer.

Good :

[22.05-RELEASE][root@pfSense.xxxxx.net]/root: dig www.test-domaine.fr @ns1.test-domaine.fr +tcp +short
5.196.43.182

@Gurveer : go have a talk with those who manage your ns11.bsnl.in. and ns12.bsnl.in.