DNS suddenly broken [on some VLANs]

johnpoz

@TheNarc here is the thing should be is nice but verify verify verify is networking matra you should live by

And not using explains what your seeing

TheNarc

@johnpoz Fair enough. So I re-ran some testing, and did verify that the requests were being forwarded to 1.2.3.4 via my Nord interfaces (by examining states). However, they were being forwarded on port 853 because I had unbound's "Use SSL/TLS for outgoing DNS Queries to Forwarding Servers" option enabled. As soon as I disabled that, I began resolving successfully again even though I've got 1.2.3.4 set as my only DNS server.

Note that I did not need to disable DNS over TLS when I had valid DNS servers configured in System -> General Setup. But I recalled that when I set Nord's DNS servers there, I was also failing to resolve anything. So next I tried setting my DNS servers back to Nord's, but leaving DNS over TLS for forwarding servers disabled . . . and that worked.

So, would I be off-base in interpreting these test results as:

Nord's DNS servers do not support DNS over TLS
Nord is redirecting DNS, but apparently only when you attempt to use a "bogus" DNS server (e.g. 1.2.3.4). Because otherwise, given my first point, I should fail to resolve anything when I set my DNS server as, say, 8.8.8.8 and enable DNS over TLS for forwarding. But that works.

I guess I could imagine their DNS servers not bothering to support TLS, because they only allow connections to them from clients already connected to their VPN. But it's not clear to me what their motivation would be for this (apparent) selective redirection. Though like I said, maybe I'm way off in my interpretation of what they're doing now . . .

Quick update: running nmap -p 853 on Nord's two DNS servers (103.86.96.100 and 103.86.99.100) showed the port as open, so I attempted to resolve using them with DNS over TLS (+tls arg to dig) going straight out my WAN (verified by checking states) and that worked. So it seems that Nord's servers do support DNS over TLS and do not restrict queries to only clients connected to their VPN. That kind of blows away the theory I put forth for why the Nord DNS servers were not working when I was forwarding to them with DNS over TLS enabled, and I don't yet have a new theory.

DaveP 0

I've the same issue with Nord VPN.

I 'talked' to one of their support staff earlier and they acknowledge the issue and say it is being worked on. I've to keep monitoring and get back if it persists. As this has been ongoing for a week or so I suspect I will be in touch with them again on Monday.

Pity really as up to now I've not had a issue with them. But if it isn't fixed then I'll be moving on. I see AirVPN which I've never heard of is recommended above. Can anyone confirm that I can set that up on PfSense as I have Nord. I don't want to make any assumptions

johnpoz

@DaveP-0 there should be zero reason why you couldn't setup any vpn service with pfsense. If they are using openvpn - your using the openvpn client - why wouldn't you be able to connect?

DaveP 0

@johnpoz Sorry missed your reply as was busy at home. There isn't a problem with OpenVPN it works fine as per the NordVPN setup.

The problem is that I can forward to a DNS provider such as my ISP, QUad 9 or something and it works fine. If I try and bypass these and go to the root servers to build up my own DNS values it fails and DNS will not return any values. Above it says that this is an issue with Nord, which I am using, I've chased them about this but so far have no response on it.

Regards,

TheNarc

@DaveP-0 I think @johnpoz was referring to your question about AirVPN, just saying that so long as AirVPN uses openvpn (which pretty much all VPN providers offer) then you should have no problem setting up a client connection in pfSense.

As to Nord, I have seen more references popping up confirming this issue, but still no real details or information about a possible fix, so I'll just be using forwarding mode until/if it's resolved (pun intended).

johnpoz

@TheNarc yeah there for sure clearly something going on with nord, they are clearly intercepting dns traffic... Smoking gun in another thread when did a directed query to 1.2.3.4 and got an answer.. 1.2.3.4 doesn't do dns that is for damn sure!

DaveP 0

@TheNarc Duh! You are correct I didn't read it correctly. I've turned off the autorenew with Nord and it is up for renewal in a few months. That gives me to to find a new provider if Nord is causing this which it appears. I've written to them now via email and lets see what they say to that email.

DaveP 0

I went reading their blurb about DNS servers and found this sentence

NordVPN offers private DNS in its apps, which ensures tighter security and privacy for your traffic. The NordVPN native applications automatically use NordVPN's DNS servers when connected to VPN. This prevents DNS leaks during your VPN connection, ensuring that your NordVPN private DNS requests are safe.

Maybe what they mean by 'The NordVPN native applications automatically use NordVPN's DNS servers when connected to VPN' is 'We intercept your DNS queries and securely process them via our servers when you are connected via our VPN'. That could be why the automagically send them through normally.

So they are intercepting them and turning off forwarding in PfSense and making it request the root severs can't give an IP as it doesn't know the full domain you have requested just the first character for the root server so it is screwed and returns nothing of use. In which case you would think that their support staff would advise that this is unlikely to get fixed and they see it as a feature. In which case I need to change VPN provider if i want that functionality. OR I reroute DNS queries so they don't go into the VPN tunnel but go directly out via my ISP where accessing the root servers works.

So lets see what Nord say. I've a while before I am due to renew my Nord subscription but I have stopped auto renew in the system.

johnpoz

@DaveP-0 unless you were using qname minimization, and had strict set it should fall back.. If your not doing that the roots do get asked for the fqdn your asking for, but they just respond with the NS for the TLD..

With qname min it would only send them the .tld, but without stict it will fall back to fqdn, etc..

so you would send .com to roots, for example and then domain.com to the qtld servers root handed you, and then only when talking to NS for the domain.tld would you send the fqdn like www.domain.tld

I have this set, but not strict - because that is known to break some major players that have crazy daisy chained names setup, etc.

but more likely that dnssec is just failing if they are intercepting.. You could try turning that off and stil resolving - and try turning off query name min settings if you have those set in unbound advanced settings.

But I don't want any service intercepting anything of mine!! If you want to provide some so call "private" dns for me to use great - I will set my stuff to use those if I want to.. But don't go calling dns interception something your doing to help me!! Cuz you know that is not the reason your doing it!!! Your app, sure ok - but if I just create a connection with my own device via a openvpn settings - you better not be intercepting my dns traffic..

DaveP 0

@johnpoz Sorry I only speak English. No speaky networky things. Just joking but only a little bit.

I'm not a networking person but I have a technical background where networking is the weakest bit. I know enough to set things up but not enough to really debug more than the basic things. I didn't sacrifice enough small furry animals to dark gods when I was learning networking. Plus it was easier back then with coaxial cable and 10-Base-T connections for business use and I had a dial up modem.

I have turned off dnssec because it was recommended to remove something that is known to create issues. query name min settings was turned off in my setup already.

The system worked fine without forward and performing its own lookups with DNSSEC on and query minimum name settings off until a few weeks ago. Of course because I play with things I always assume its something I have done and I'd just installed PfBlockerNG which is a DNS filter so I didn't have to think hard about whose fault it was. I then take everything back to basics to only find it was still going wrong. What a pain that was.

I also don't want people monitoring or intercepting me which is why I set this complex system up in the first place. It is frustrating that this is going on with the provider i use for a security product.

johnpoz

@DaveP-0 said in DNS suddenly broken [on some VLANs]:

provider i use for a security product.

Not sure where you got the idea that vpn services were a security product? They sell you the idea that we won't sell your info to someone else.. Do you really believe that????? really???

I never understood the logic in hey my isp knows where I go, so let me hide that from that that from them and give it to someone else and also pay them money to hide it from the first guy??

But hey if you want to pay company X to connect you to the internet, but then hey they might know where I go, so lets pay company Y to hide that info, because you know company Y is way more trust worthy, and I pay them 3 bucks a month to slow down my internet and and hide where I go from company X.

And then let me complain when this service that is only out for my best interest isn't...
.

Gertjan

@johnpoz

Not (only) hiding or security.
It's also the darker part that never gets mentioned during the in-video youtube adds : the local ISP can be checked upon by government, in France and most Europe countries they do, and when you download and share that "Disney" movie, you will get a warning, then a letter against signature, and then they will cancel your connection.
VPNs are less subjected to this as you chose their end-point, your new WAN, outside of your country.

Another related example : I've a captive portal I'm using to offer a 'internet connection' to my hotel clients.
I don't know what they are doing with their connection, and I don't want to know.
But I can, in case of a doubt, route that interface over to a VPN to, for example, a VPN in Paris (I'm based in France). Just so that my quest do not pollute my ISP WAN IP, which is a quasi static IP. Not that I'm mailing from this IP, or that I host something at this IP.
The main subscriber of the ISP stays responsible for the connection. The (my) ISP contract says clearly : do not share your connection with people that you do not trust.

DaveP 0

@johnpoz I understand your viewpoint but for me it is as @Gertjan says. It is a single layer, and the only one I can get, that stops the stasi from just contacting my ISP and taking my information. If Nord do collect and pass the stats on then I'm no worse off that I was without them. the DNS part is because I don't fully trust anyone and want to make sure I'm getting to where I'm supposed to. Having all my eggs in a different basket isn't what I want either.

Gertjan

@DaveP-0 said in DNS suddenly broken [on some VLANs]:

and want to make sure I'm getting to where I'm supposed to

That's DNSSEC. Works out of the box for those who resolve - do not forward.

DaveP 0

Got a reply from Nord.

Thank you for your reply.

It seems we are experiencing some issues with custom DNS addresses, our developers are aware of the issue and are working to resolve it.

We would rather not give you an estimation when we do not actually have one for certain.

Telling you that it will take 5 days and if in reality, it ends up taking 2 months - would be bad for both sides involved.

We understand that waiting is rather frustrating, and we can only apologize in the name of the company.

In the meantime, let us know if you have any other issues or questions.

Which doesn't actually say anything really. I'll not hear anything more I suspect but I'll let you know if I do or it changes.

Gertjan

@DaveP-0 said in DNS suddenly broken [on some VLANs]:

It seems we are experiencing some issues with custom DNS addresses ...

I'm reading - and understand this :

On the VPN server side, like user authentication etc, all is fine.
Their DNS are fine also.
They've added something between those two, like a policy routing, or a DNS redirect "NAT" firewall rule.

Maybe this is futuristic new their way to "protect" their clients - and this part will be soon shown on their advertisement video's "We protect your DNS requests without any setup issues" - and on the other side, they collect all your DNS so they ...... and know I will be polite, as I can't be sure, but your DNS data is worth $$$ so everything comes nicely together : the making more money show must go on.

I presume they are experimenting with some server nodes, just to check if they can pull it of.

And I get it. Its the same song again : I was working for them ?! Nice move, this is what I would do ^^

johnpoz

@DaveP-0 said in DNS suddenly broken [on some VLANs]:

NordVPN offers private DNS in its apps, which ensures tighter security and privacy for your traffic.

So seems like they aren't really doing that.. But just redirecting all the dns traffic to their own servers.. Vs the app say pointing to their server on 4.5.6.7.. Or even some rfc1918 IP on their network.

Why should such a thing take 5 day or 2 months to fix? Clearly you made a change that users noticed and breaks stuff, roll the change back and back to the drawing board.. Should take all of 2 hours at most to fix ;)

They are like hey only a small % of our users are complaining.. Screw them is how I read this response..

Or how about hey here is a new feature, click the use our dns checkbox.. If you don't want to use ours, uncheck it and we won't mess with your dns.. Shouldn't all of this been worked about before they rolled it out?

DaveP 0

That was my thoughts. I can't imagine any issue like this taking weeks never mind months to fix either. So I came to the same conclusion.

I would have thought it was an easy setting to make as well. Chose our DNS servers, use your own or build your own using the root servers.

Everyone is looking for ways to increase their income. It would be stupid of them to sell this data as it would have a big knock on effect.

DaveP 0

I wrote the following to Nord. Lets see how it goes.

I don't understand how this issue has taken as long as it has never mind a further 2 months never mind 5 days more. Networking may be a black art but it follows very simple rules. Can you clarify what the issue you are working on is.

From my understanding the issue I am having can only be caused by Nord intercepting my DNS queries as you say on your website. 'NordVPN native applications automatically use NordVPN's DNS servers when connected to VPN.' I read this as you intercept my DNS and direct it to your DNS servers. Can you confirm if this is true or not.

This makes me concerned that Nord are collecting this data as it is valuable to marketing people. Can you confirm that you are not collecting any DNS data.

I look forward to hearing from you.