ATT Internet AIr

[ahole4sure]

@stephenw10
Got back on site and restarted the WAN
It is back up and going on 192.168.2.2
If I do the test gateway group , I am still losing DNS/Internet with WAN as Tier 1 and primary internet

But I did get differnt result with the command line
see below

[stephenw10]

Hmm, well that looks OK. Try setting Unbound (DNS Resolver) to forwarding mode so it uses the configured remote DNS server directly. You should also disable DNSSec whilst testing that.

[tman222]

Just wanted to chime in here and mention that I have been using pfSense with AT&T Internet Air in failover configuration for about a year now without issues (primary WAN is fiber). During that time I have not observed any evidence that DNS might be intercepted somehow on the Internet Air side - DNS is working fine for the failover gateway group I've got setup with Unbound in resolver mode. So I think in this case it's still a misconfiguration somewhere in the pfSense settings.

What do you have set for DNS Server Override and DNS Resolution Behavior under General Setup?

Also, this section in the pfsense docs will be good to review if you're switching over from resolver to forwarder and using multiple wan connections:

https://docs.netgate.com/pfsense/en/latest/multiwan/interfaces-and-dns.html#dns-configuration

Hope this helps.

[ahole4sure]

@stephenw10 @tman222
Ok so I'm not completely sure which setting fixed things BUT by setting DNS Resolver to forwarding and disabling DNSSec - the WAN (ATT Internet Ait ) Internet worked when enabling the Test Gateway group1

@tman222
I attached screnshots of my settings - how does that compare to yours? Do you have the Internet Air setup on a seperate "local" subnet like I do? Shame it doesn't have a full "bridge mode". Also I could not seemingly get DMZ to work in that with it enabled my pfsense interface still shows the local IP address - it doesn't pass through the public IP

The screeshots show the active test gateway. The command line results AND the succesful "speedtest" (shich was never accesible before) with the test gateway and then the regular gateway. AND my general setup settings page for @tman222 question

How should I permanenetly set things ? Any explanation or thoughts on what is happening?

[ahole4sure]

@tman222 @stephenw10

[ahole4sure]

@tman222 @stephenw10

[tman222]

@ahole4sure - glad you got things up and running.

My settings are similar except that I'm using Unbound in resolver mode so I have no additional DNS Servers defined under General Setup > DNS Server Settings.

Which gateway did you end up getting from AT&T for the Internet Air service? The one I received does allow for IP passthrough so I can get a public IP on the pfSense side.

Hope this helps.

[stephenw10]

Mmm, interesting. Yeah it should work is resolving mode behind a failover gateway group as @tman222 shows. Curious that it's failing for you.

You should not have a gateway set against 8.8.8.8 so it's available for use on either WAN. Especially since you have at least one device hard coded to use that directly.

[ahole4sure]

@stephenw10
I guess I’m confused a little
Maybe even with terminology

I have DNS Resolver enabled. I never disabled it. I just enabled “forwarding mode” in the DNS Resolver settings.
And another thing I don’t understand is that in the General Setup page I have WAN2 supposedly hardcoded to use 8.8.8.8, and WAN to use 8.8.4.4
But based on the command line results 192.168.2.2 used 8.8.8.8 to resolve the address

So I’m confused as to what is happening
AND how I should have things setup??

[ahole4sure]

@tman222
Yeah I am a bit confused because I have DNS Reolver enabled - but under those settings I have check to use DNS Forwarding mode -- so I guess I don't understand what that means??

Attached is the model Internet Air that I am using.
In the settings I do have an option to set DMZ to a particular address and to "allow IP passthrough" but it doesn't seem to be actually doing the passthrough??
Do you have actual settings like a "bridge mode" - could you possibly send a screenshot of your Internet Air settings page where you are able to accomplish "passthrough"?

[stephenw10]

Unbound in it's default config acts as a DNS resolver. That means it queries the DNS root servers directly. If you run a DNS leak test whilst using that it will show your WAN IP address.

When you set it in forwarding mode (or use the separate DNS forwardibg service, dnsmasq) queries from LAN clients are forwarded to external DNS servers. Either those configured in general setup or passed by your ISP via DHCP. There they may be forwarded again If you run a DNS leak test when running that it will show the public IP of the final server that resoles queries.

Both modes should work behind failover WANs.

[ahole4sure]

@stephenw10
I guess I'm still confused since "forwarding" is still a sub-section under the main page of DNS Resolver and is a sub-setting under enabling DNS Resolver
From the explanantion as I understand it - it's like the setting page allows you to enable unbound or DNS Resolver and then bypass it (or disable it) all on the same page ??

[stephenw10]

There are two DNS services that can be enabled in pSense.

The 'DNS Resolver' is Unbound and that resolves directly but can be set to forward also.
The 'DNS Forwarder' is dnsmasq. It's what pfSense used by default in much older versions and is included for compatibility. It can only run as forwarder.

[ahole4sure]

@stephenw10
So whats the differnece beteween DNS Forwarder (which I do NOT have enabled ) and the DNS Query Forwarding - enable forwarding mode , which I do have enabled ??

[stephenw10]

Both will forward queries. dnsmasq is included for backwards compatibility with much older configs.

Unbound is a lot more flexible and powerful, even in forwarding mode. If you are using dns based filtering via pfBlocker-NG that can only use Unbound for example.

[ahole4sure]

@tman222
Hey sorry to bug you again - I was hoping you could answer some questions about your setup. I have tried diligently to get info about my “internet Air for Business” device and today the tech support rep that I finally was able to speak to state and read to me the disclaimer - “this device will support IP passthrough”.

I think because of this I am not able to obtain a public IP address when this device is connected to my pFsense router - thus the 192.168.2.2 address that I am getting with DHCP despite enabling the DMZ.

I run a few services behind my firewall that for failover to truly be functional I not only need the outgoing internet access but the incoming connection to those services is almost as essential!

Probably related - I was testing today and when I failed the primary internet WAN2 my DDNS service came up with an entirely differnt IP address than the one I would obtain my going to whatismyIP But even if went to cloudflare and manually set the IP address for my services domain names I was still unable to access them from outside.

Maybe you could share which “Internet Air” device you have had success with and the settings you have used for that device to get IP passthrough. Maybe I could trade mine for the type youare using

Thanks

[stephenw10]

If the DMZ works correctly then all traffic will be passed to pfSense and can be passed to something behind it. Since those services are already working behind pfSense in a private subnet they should work fine in that situation.

[ahole4sure]

@stephenw10 I think the big term here is IF!
How do I determine if DMZ is working correctly? The pfsense box only reports the local private IP for the ATT gateway - no public address.
Also if I fail the main internet , WAN2, then when I go to the DDNS page the IP address that is reported to Cloudflare is different than the IP address I get when I go to whatismyIP on a LAN computer. And different still than the IP report on the "status" page for the ATT gateway.

But most importantly when I turn off the DDNS function and manually change the Cloudflare DNS IP address for any of my services behind my HAProxy I am not able to reach them from outside. And I even have one service that is not behind Haproxy and just has a port forward (set in both the WAN and WAN2 rules) and I am not able to reach it either.

[stephenw10]

Send some traffic to the public IP and see if it arrives at pfSense.

Either send traffic directly from some other connection (cell phone etc) or use a port checker site. Then check the firewall logs in pfSense. Or run a pcap for that directly on the WAN.

[ahole4sure]

@stephenw10
Thanks again for help - I guess I should be more patient , failovbut I have only 2 hairs left lol

I have failed the main WAN2. The WAN is active and I get a differnt
is not IP address when I check whatsmyip on a LAN PC. The same IP (this time) appears in the DDNS. I try with a port checker on my phone port 443, and port 32400 which should be open AND I see no denials.

I think my ATT gateway is not forwarding into pfsense