ATT Internet AIr
-
Hmm, nope you absolutely shouldn't need that rule on a WAN. That passes traffic from the modem side into the firewall which should not be needed.
-
@ahole4sure A rule like that will be needed for your NAS- or Guest-VLANs only.
But not for the VLAN you have for the ATT modem (rosegate...). Not sure anymore which VLAN is used for what though...
-
The saga continues -- it appears that the second (in my discussions) of my two ATT modems may be bad. The back end ATT people swear that it is provisioned correctly. They are overnighting a replacement device with new SIM tomorrow.
On another note - I did as @Gblenn suggested and set up an additional test scenario and I was able to get Modem #1 to work through the TP- Link switch
So her is the current problem -- I have simulated power failures and reboots of the pfsense box. The modem and switch boot quicker on power failure AND if I just do a reboot of the pfsense box without booting the modem - I am unable to reegain connection. The connection is restored after modem manual reboot. During the time of trying to regain connection the modem just cycles through connection and disconnection to the pfsense box. (screenshots are 5 sec apart)
I assume it is just not renewing the lease - but can I force it???
Have you ever seen this behavior before? Any fix or workaround? I am trying to make this as self fixable as possible since I will eventiually deploy 5 physical hours away from me with no tech savvy on site employees. -
What do the pfSense logs show when that's happening? Check the system and dhcp logs.
-
@stephenw10
Soi strangely enough , while testinng the TP-Link switch, and this time without power failure or reboot - the gateway just went down (not sure exactly when) but has remaained down for several hoursWhen I checked the modem it was doing that cycling connecting , disconnecting thing
The only relevant entries in the log (as far as current time-wise) were int he DHCP log
see attached🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view
-
@stephenw10
I rebooted the modem and connected to the Linksys switchThe modem shows connected to the pfsense igb3 mac address , but the interface never showed the IP address this time, and the gateway never showed coming online
BUT the cmd ping lets me ping google.com from the OPT6VLAN10 interface that doesn't show up as online ???Also at the end -- do you have any idea waht those numerous "default deny" things are in my firewall logs -- for both my WAN2 and my LAN. ?? There are just so many !!
I didn't even know I had a "default deny" rule
🔒 Log in to view 🔒 Log in to view 🔒 Log in to view🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view
-
I don't think that ping is real. It doesn't show a source address in the output. That should appear like:
🔒 Log in to viewBut since it doesn't it implies OPT6VLAN10 doesn't have a valid address.
The DHCP logs there simply show no servers responding.
-
@stephenw10
So does that mean that the ATT servers are "to blame" in this case?I need to make a decision soon -- I have enjoyed learning and pushing through the process but sooner or later I gotta decide --
-
failover internet at my second location is not an option
-
I need a different gateway than the Nighthawk (the odd think here is that if I stay away from VLAN connection the Nighthawk seems to be stable (and survive reboots and simulated power failures)
So on the one hand it seems like the Nighthawk>VLAN>pfSense scenario is to blame , while on the other hand is it just the Nighthawk to blame??
Any thoughts on how I might should proceed to getting to the source of the issue?
Running another ethernet cable to my proposed modem location is just not an option - it has about a 10ft run UNDER concrete floor to get to the outer wall and that run is what is feeding the cameras -
-
@ahole4sure That blocked device that you have showing in the picture from the ATT modem is your TPLink switch, right? I wonder if that may play a part in this? The ATT modem is connected to the only device it's trying to block?!
I think you should set the IP manually and try removing that entry in the ATT modem. If you haven't done it already, it's under System - IP Setting and there you set DHCP to disable and enter the IP you want when accessing it.
-
@Gblenn Are you suggesting that I go back to trying to manually set the IP address for the VLAN interface to the static address I have form ATT? I hasn't worked in the past but I'm up for anything -- I had hoped that I could get DHCP to work and it DOES when connected directly to the pfsense (but the issuess start when I thow the VLAN into the mix)
-
@ahole4sure No, I meant the management IP for the TPLink switch. I believe you set that block in the ATT modem so it wouldn't pick up that MAC instead of pfsense.
So keep everything as it is, set the correct MAC (for pfsense) in the ATT modem, and remove the blocking. AND, set the IP of the TPLink switch to whatever it is that you want it to be. I suppose you have already set it as static in pfsense DHCP, but still. Just to make sure it doesn't try to get an IP from the ATT modem. -
Like this
-
@Gblenn
So in reesonse to your initial reply I switch my interface to static - (so far it is staying pretty stable) I actually think that I had not gone back to that AFTER I found out one of my modems was "bad"
I would like to be able to keep using DHCP ( like appears to have been working well with dorect connection to the pfssense interface as compared to the VLAN connecting through the switch
But at this point , just getting it working is all I care about!
And it appears that ATT has no problem with providing my static IPI had already set both the Linksys and the TP-Link to static as per your pic -- that didn't really change anything
I just don't know why connecting through the VLAN screws up the DHCP delivery and stable connection ??That said - I can connect to my ATT modem after I added the virtual IP address in that subnet to the VLAN interface (that address is 192.168.2.1)
For some reason - with a ethernet cable connected as a trunk to port one of the switch and the ATT modem connecteed to port 2 of the switch I can't connect to the management interface of the switch (192.168.3.100) -- any suggestions for that ?THANKS again🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view 🔒 Log in to view
-
Those IPAlias VIPs are all conflicting. You can't have the same subnet defined on different interfaces.
-
@stephenw10
Oh crap , my bad
I thought you had said I couldSo I can have multiple subnets on one interface, but not the same subnet on two different interfaces??
-
Exactly. Otherwise the system doesn't know where to route traffic. The interface must be unique for each subnet in the table.
-
@stephenw10
So I guess a VLAN general question
If I try to go through the igb3 interface connection the connection to the switch 192.168.3.100 would not connect at all
The trunk (port 1 of the switch) is connected to the igb3 port of pfsense
The VIP configured in way # 1 doesn’t allow connection at all
The VIP configured in way #2 works it connects. But the connection is so slow it’s almost unusable![alt text] ![IMG_0336.jpeg]🔒 Log in to view 🔒 Log in to view (/assets/uploads/files/1733452318123-img_0336.jpeg)🔒 Log in to view -
@ahole4sure I did not mean for you to set the pfsense interface to static, I meant only the TPLink switch.
The problem you were having before was that the TPLink was "stealing" the IP that was handed out by ATT meant for pfsense. It seems you already had it set as static now.
The other way to secure that pfsense would get the IP, was to enter the pfsense MAC into the ATT interface and set the Passthrough mode to static there. In this context in the ATT modem, static means the IP should only go to one single device (the one defined with the MAC). So even though it seemed like that setting didn't really survive a reboot, it should secure that you get your Public IP via DHCP as desired.So basically the only thing I was thinking you should be testing was to remove this Blocked device, from the ATT settings you did some time earlier (I suppose as a way to make sure the IP was handed out to the right interface).
My thinking was that this strange and continous disconnect and reconnect that the ATT modem is doing, is because you have the MAC of the switch set as Blocked. So the ATT modem blocks it, and then there is no connection and it also no longer see's the MAC so it tries to connect and discovers that MAC again... and then it cycles again....
The way to reach the ATT interface is going to be via the WAN port of pfsense which is back to that discussion above. One way is to set a static route in System > Routing and just have the 192.168.3.100 IP set as going out that gateway.
-
Yup you'd need that VIP on the VLAN not the parent NIC because I think you have removed VLAN1 from the trunk port in the switch? Otherwise it could be on igb3 directly.
So really it depends what VLANs the switch gui is configured to listen on.
-
@Gblenn
I can't thank. oyu enough for continuing to try to help me resolve this issue -- and yes this strange connecting and disconnecting has to have a sourceSome addn't info that I may not have made clear -- I HAVE had the MAC address in the ATT device sice it was discussed before. Of note, if you connect to a different device the MAC address changes so I have had to enter the MAC address (if the device was out of the needed environment) and the apply the change and then disconnect the device. Currently the MAC address appears to be "sticking" and having it there does not seem to help the weird connect/disconnect issue when trying to connect through VLAN and still use DHCP at the pfsense level. (the ATT dhcp has been off as well).
The other reason that kinda made me think the "block device" was not a source of the problem is that it happened on the Linksys switch as well (and that was not the blocked MAC address - the blocked MAC address was form the TP-Link).I can defintely try to unblock the "block" but I don't think it is involved🔒 Log in to view 🔒 Log in to view
And @Gblenn and @stephenw10
At this point - having spent tens of hours on this -- I wonder if I should just quit trying to include DHCP into the mix since ATT has given me a ststic IP for now (I was originally trying to future prrof things in case I didn't always have a static and tryign to accept the challenge to "make it work") -
Mmm, if it works reliably with a static configuration I'd go with it. Maybe AT&T don't expect DHCP to work in that setup.
-
@stephenw10 said in ATT Internet AIr:
Mmm, if it works reliably with a static configuration I'd go with it. Maybe AT&T don't expect DHCP to work in that setup.
@ahole4sure Alternatively, change it so you only have VLANs on your LAN (igb1) interface and keep the WAN interfaces clean and directly attached to the ATT modem(s).
-
Hopefully one of you guys are around -- I was still trying to experiment and get an understanding of why things wouldn't work
(also noticed some pretty long reboots - weird behavior)NOW THE BOX IS UNACCESSIBLE
It's been so long since I set it up. I have it out of the rack setup and on the bench. Only one cable connected.to LAN. No access.
It will be a disaster if all my setup is gone (spent hours settting up Wireguard VPN, and even more hours setting up HaProxy stuff) - but I have no backup!! dumb I get it
Whats best way to troubleshoot and hopefully save something
-
@ahole4sure Hmm, try connecting a WAN cable to see if that helps. I'm not sure but I think I have seen that happen when it's disconnected. Also, check your IP settings in the PC you are using. Does it have an IP in the correct subnet?
BTW, I think the long boot time may be due to it waiting to see if WAN is there, so there is a pause in the boot process...
-
-
@Gblenn Was able to get an HDMI connected and here is the error -- looks like a disater
I ordeered a new box on Amazon at 330 amBut if I have lost all my config the I am SUNK
-
@ahole4sure Check if there is something in this thread that can help you?
https://forum.netgate.com/topic/185312/config-xml-empty-subsequent-inability-to-restart-properly-and-what-i-did/2
In particular that he was able to log in via console (SSH?) and replace the empty file with another one. I guess though that if you don't have any backup at all, it will be really tough if it is empty. It is /cf/conf/config.xml that you are looking for.
I "consoled" in and did some snooping.
The error I saw was a PHP stack trace that ultimately pointed at config.xml being empty (pasted after my closing)
I looked and saw that...it was an empty file (zero length)
I found a recent backup that was non-empty (the latest was empty) and copied that into place.
I then rebooted and things returned to normal. -
@Gblenn I guess I don't know how I would console in since it looks like I have no access via the ethernet ports
-
But using a keyboard directly can you bring up the menu? Or just a prompt?
That looks like either filesystem damage or a failed upgrade. Was is hard powered off?
If it's running UFS try running FSCK frm single user mode:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-check.html#manual-filesystem-check -
@stephenw10
Did fsck 5 timesHere is the same error after reboot
And yes I am a dumb_ss - I think it might have been hard powered off -- I was distracted with comething else on another screen and I think I held the power button longer than normal🔒 Log in to view
-
What are my options??
Is there anyway to find a config file anywhere in the pfsense OS?
Or do I have to reinstall? Will a re-install possibly find a backup? -
Yes you should be able to access the config file in /cf/conf/config.xml and the previous 30 files in /cf/conf/backup if you can get a command prompt.
You can access it from single user mode but your options for copying it off are limited from there.
If you can get a prompt after allowing it to boot normally you can try to scp it to some other device.
If you have a serial console you can always just copy/paste it directly if required.
You can insert a fat formatted USB stick (or use the fat partition on the installer image) mount that and copy the config file to it.
But, yes, reinstalling should give you the option to recover the config file.
-
@stephenw10
So I have downloaded the pfsense latest version and I am flashing to USB nowRe-install shoul give an option to restore?
And do you agree that is basically my best option now? I don't think I still have the serial cable -- I am using HDMI and a keyboard
-
@ahole4sure And it is not responding if you try to SSH into it?
-
@Gblenn no I cannot get any port to respond to ssh
Am i doing nything wrong -- just connect ethernet to prior LAN port?
It is not allowing any communication that way -
If you're using the Net Installer it shows you the config it has recovered and will use so you can know it worked.
-
@ahole4sure said in ATT Internet AIr:
Am i doing nything wrong -- just connect ethernet to prior LAN port?
Probably not, it should respond if it gets that far during boot. But I suppose you should also see the menu on the screen (via HDMI). So I think you have to rely on the installer that stephen10 mentions.
-
You didn't have Auto Config Backup enabled?
-
@stephenw10 not sure
-
@stephenw10
So for thee reinstall it does ask for rescue from a config.xml file. Then asks me where - I click on ada0 disk. and click ok and it immediately goes baack to thei main install screenI first tried ZFS but the end it said I only had one disk, not enough disks for mirror
I went back to look at UFS install and it gets to a point that it asks if I want to partition the disk and it says it will wipe the disk ....
Wonn't that then erase my config file ??