ATT Internet AIr

[ahole4sure]

@stephenw10

and https://mysyno.mydomain.com resolves to the internal IP 192.168.144 but to port 443
and haproxy is supposed to send it to 192.168.1.144:5001

[ahole4sure]

@stephenw10 and I mean internall

It works correctly externally

[ahole4sure]

@stephenw10

So I activated NAT Reflection > Pure NAT

I now can access several of my services internally and externally — all of which are https through harrows

The only remaining issue, even after enabling the NAT reflection, is the firewall interface which is reachable external via https://pf.mydomain.com
But internally only available by adding the port :8000

So NAT reflection fixed everything except firewall management page!

[stephenw10]

@ahole4sure said in ATT Internet AIr:

So I activated NAT Reflection > Pure NAT

I now can access several of my services internally and externally

Then it's not using HAProxy since that would redirect traffic past HAProxy entirely. NAT reflection only works with port forwards and you shouldn't have any of those if you're using HAProxy. It sounbds like you must have them though.

You have the pfSense webgui set to use port 8000 directly?
Do you still have the port redirect enabled in Sys > Adv > Admin Access: WebGUI redirect ?
That should redirect http requests to https on port 8000.

[ahole4sure]

@stephenw10 I have NO port forwards
see pic

I have been trying to get something else to work. I saw a post where someone did an haproxy frontend on to listen on the LAN and then forward to the correct port
see pic

I was thinking it was working last noght but then when I got up this AM it didn't seem to be working , BUT after i reached the firewall by 192.168.1.1:8000 the forst time , it then began working

Also wondering if I should have my wildcard domain cert here or should I leave as the default webui config?
ssee pics

[stephenw10]

Well you have disabled the webgui redirect which is why you have to enter the port directly.
Did you do that to prevent it interfering with some other service?

I'm not seeing a screenshot of your port forwards page. Unless it was much earlier in the thread?

[ahole4sure]

@stephenw10

Back again lol
Trying to get ready for the holidays so my side line IT job has suffered

During the last week- both of the ATT moderns gateways have remained online ( one with the direct interface connection and the other with the VLAN connection ) - that is with the static IP configuration.

Got around to a few fail over tests. Latest one tonight.
Upon unplugging the WAN cable from the fiber modem fail over is successful but 2 issues remain

1 ) the ATT modem just doesn’t seem to behave as far as DNS resolution (will ping IP addresses but can’t resolve DNS) - unless I check Enable DNS Forward Mode in the DNS Resolver settings. We explored this before but never figured out why

2. possibly bigger issues - when I plug the fiber modem cable back in the Frontier fiber internet cable just won’t come back online. Even if I “reset all states” it won’t come back online. The only way I can get the fiber modem Tier 1 gateway back online is to full reboot pfsense.

Ideas?

Thank you

[Gblenn]

@ahole4sure Hmm, how have you set up your resolver? Which interfaces have you selected in the list?

[stephenw10]

Hmm, odd. Unbound should be able to resolve directly via any outbound interface as long as it's selected. By default it tries to use all interfaces.
Do you see any blocked traffic? Do you see states on port 53 when failed over to the AT&T modem?

The Frontier gateway shows as off line? Or the NIC never links?

[ahole4sure]

@stephenw10 @Gblenn

I did not check for "blocked traffic". If I test again this PM would I just fail the WAN and then check the logs for the firewall? And where / how is best way to monitor or look for "states" on port 53?

As far as the Frontier WAN interface -- the gateway on the status page shows red or "offliine" for much longer than expected (like a couple minutes), then it flashes to blue "unknown", then back to offline, then back to blue unknown -- ultimately after 4 or 5 minutes it settles in on blue unknown and just never links back up. I then tried reset states --- Ulitmately the only thing that lets it reconnect is just go to Diagnostics and choose "normal reboot". Back online , like normal, after the reboot.

[Gblenn]

@ahole4sure I guess the two problems are separate and not necessarily related. So on the first one about the DNS issue, which I responded to, have you checked the settings for the resolver? Which interfaces have you selected for it to use?
In the forwarder, there is no such setting and I assume it's automatic as @stephenw10 suggests.

[stephenw10]

Go to Diag > States and filter by :53

What is logged in the System and Gateway logs when you reconnect the Frontier modem?
How are you running the failover test? Physically unplugging something?

[ahole4sure]

@stephenw10 I'll have to retest and check the logs

And yes to fail - I just unplug the ethernet cord from the back of the frontier modem

[ahole4sure]

@stephenw10
Here are the screen shots of the info
Apologies and thanks for help!!

They start with states during failover
Show the DNS error on web browser
(the is with Forwarder Mode off under DNS Resolver settings)

Then shows what happens after waiting for Frontier WAN to come back up

Ultimately would not fully connect and come back up untill full reboot of pfsense

[ahole4sure]

@stephenw10

[ahole4sure]

@Gblenn
I have checked ALL for both

Is that correct?

[stephenw10]

Hmm, no DNS states shown there on states with ports starting with 53. You might need to filter differently to see that in the states.

I don't see the link state change in the logs there. Does it actually show the that when you reconnect the modem?

[Gblenn]

@ahole4sure said in ATT Internet AIr:

@Gblenn
I have checked ALL for both

Is that correct?

Yes that is correct, and I'm starting to think that ATT is actually blocking access to authoritative servers... If this is true you either have to stick to Forwarder mode, or ask ATT to stop blocking. I wonder if there is some setting you can change for your account perhaps?

On the fiber issue, there are a few things that stand out to me.

1. Gateway shows an IP but status is pending. So that IP is probably cached data in pfsense, the link is up (eth to frontier box) but pfsense has not verified the DHCP lease?

2. I notice you have IPv6 enabled on the fiber WAN. Could that cause issues? Try disabling it and see if that makes any difference when testing?

3. Have you tried pulling the fiber instead of the ethernet cable to pfsense? If that is possible to do?
   Keeping the physical link up on pfsense interface thoughout the test... If there is something on that side that isn't really working as it should?
   If you look at the dashboard instead of Status / Gateways, you will also see the negotiated settings for the interface. Does that seem ok, and is it the same before and after pulling and reconnecting the cable?

[ahole4sure]

@stephenw10
Yes it does show this when I reconnect the Frontier fiber modem -- I will have to try it again this PM to try to get the states again
As far as ATT - wonder if @Gblenn is correct - I have been reading that ATT has some sort of DNS assist that could be hijacking my DNS queries -- not sure how to determine that.

https://gist.github.com/CollinChaffin/24f6c9652efb3d6d5ef2f5502720ef00

@Gblenn
I will retest with IPv6 disableld tonight ! I see (after some ressearch) that ATT could be hijacking the DNS. I will look into this. Thanks

[stephenw10]

Yes if they're redirecting DNS that would break the resolver.