pfSense dropping connection to WAN or something
-
This post is deleted! -
FFS I can't edit my post, so here we go again :(
I have started to experience the most bizarre connection problems. Some time during friday, internet connection started to intermittently and repeatedly drop for anything from 5 to 20 minutes. I didn't make any changes to network settings and I couldn't figure this out since it started.
My/Our setup is ISP's AP on the roof, managed switch (TP-Link T1700G-28TQ) and pfSense virtualized in ESXi.What exactly is happening is there is no internet connection but local network is still working perfectly fine. The bizarre thing about all this is when I connect my PC directly to the AP, everything is perfectly fine - as in internet working, pings working with expected latencies etc.
A guy from the IPS sent me screenshots from the AP's log where the LAN connection is randomly disconnecting, usually for a second or two, but it also lasted a few minutes. This doesn't mirror the length of internet disconnections as seen from down here however, which is why I doubt it's physical problem with the cable - I originally thought it could, but then there was several hours disconnection-free window during the night until 9 or so.I doubt it's the switch since local network works 24/7 without problems.
My completely uneducated guess would be pfSense, but I don't really know anything about troubleshooting this kind of stuff properly. I'm happy I managed to get it working back when I built the server. Anyway, I am not entirely sure, but it seemed like the disconnects were temporarily fixed after rebooting the VM running pfSense. Resetting states didn't make a difference and I had no idea what to look for in the logs. -
If the AP shows it's LAN losing link does the pfSense system log show it losing link on the WAN?
Losing physical link is usually a hardware issue, bad cable, bad NIC etc.
Steve
-
@stephenw10 Like I said, the AP is usually losing LAN connection only for seconds (rarely for a few mins). The main problem is pfSense completely losing access to outside of local network for up to 20 mins or so.
I have just almost cofirmed that rebooting pfSense instantly fixes the problem until the next time it happens.I spent most of the day reinstalling and manually reconfiguring the virtual machine and it's still happening. I have no idea what to do now.
-
But does pfSense show the WAN NIC losing link? If it does that triggers all sorts of other scripts.
Check the logs covering the period of the outage. What's actually shown?
-
@stephenw10 You will have to point me in specific direction, I have no idea what to look at/for.
-
@Octopuss The problem might have something to do with presumably faulty PoE injector, because since I replaced it with another one, there hasn't been any disconnection. That doesn't explain why I had no problems connecting directly to the roof though, because the AP up there was still powered by the same injector...
-
Look at the main system log in Status > System Logs. Check for 'link state changed' messages from the WAN interface and any surrounding messages. There will likely be a lot there but it might show what triggered the link to go down if it wasn't hardware.
-
@stephenw10 Ok but in what category should I look further? There is nothing related to interface as far as I can see.
-
It would be in the main System/General log if it's losing link.
If it's not losing link but the AP is then what's in between those things? A switch?
-
@stephenw10 Oh the logs aren't sorted by newest entry by default, that confused me.
What happens is I randomly stop having internet access, I can't ping anything outside of local network, but the ISP doesn't see this on the AP's interface, obviously, because it's just data. The LAN port on the AP sometimes dropping connection as seen on their end could be something completely different and irrelevant and doesn't mirror the internet "disconnections" at all.
The reason I don't believe it's the switch is that local network works just fine all the time, AND more importantly when I reboot the entire virtual machine running pfSense, the internet connection is instantly restored.The other weirdness is when I swapped the PoE injector powering the AP for different one, the disconnects or dropouts almost completely stopped.
The setup is AP on the roof connected to TP-Link T1700G-28TQ switch, and then a server running ESXi with pfSense as a vritual machine.
-
@Octopuss The new PoE injector might not have been related at all. It has just happened again. I thought it was the reason since it didn't happen since yesterday, but apparently not.
This is so annoying. It started out of the blue on thursday during the day. I wasn't even at home, so absolutely nothing on the server was changed, settings-wise.I plan to upgrade the server to ESXi 8 (currently running 6.7) one of these days, so maybe it will go away then...
-
Mmm, well that sure sounds like a hardware issue then. Does the switch log PoE issues?
-
@stephenw10 The switch doesn't have any PoE functionality. I have an injector.
Besides, it must be a problem with pfSense, because the AP on the roof is running all the time. -
If it's something in pfSense it will almost certainly be logging something when the problem starts. It could be a link state change. But it could be a route or IP address change. Or if you're running something dynamic like Snort or pfBlocker it might be related to a list update for example.
