Short disconnects multiple times per day
-
Gateway logs show a bunch of "send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 31.18.xxx.254 bind_addr 31.18.xxx.209 identifier "WAN_DHCP " ", but the times don't really add up with my recorded times.
Never looked at the monitoring graph before, but it look pretty flat, I'm guessing there would be some sort of spike in case of a connection loss ?
Symptoms are short losses of connection for just a few seconds, usually enough to trigger some sort of "socket closed" error message in online games. It's kinda similar to a new DHCP lease (using static IPs for my internal network though), but way too frequent for something like that. I had it yesterday 3x in about 3h.
Shouldn't be a DNS issue. Maybe I'm just not noticing it during browsing, but it interrupts established connections with servers. Also I tried a nslookup on my client PC today seconds after I lost connection again and both nslookup and ping worked without a problem.
DNS is configured as localhost (resolver)>OpenDNS>GoogleDNS on my pfSense and pfSense>OpenDNS in the network settings of my PC. -
@XX302 said in Short disconnects multiple times per day:
pfSense on the QOTOM hardware
Probably not related, but https://forum.netgate.com/topic/144744/pfsense-mini-computer-and-battery-powerbank-questions/2 : Netgate and Qotom aren't mutual fans.
@XX302 said in Short disconnects multiple times per day:
Shouldn't be a DNS issue
but then
@XX302 said in Short disconnects multiple times per day:
DNS is configured as localhost (resolver)>OpenDNS>GoogleDNS on my pfSense and pfSense>OpenDNS in the network settings of my PC.
What about putting the Resolver to "all default" : just you, pfSense and Internet's root servers (if they are down, there will no Internet anyway. ) - no intermediates.
-
I mean, I didn't change anything on the Resolve settings. But alright, so you want me to just deactivate the resolver ?
-
@XX302 said in Short disconnects multiple times per day:
just deactivate the resolver
No way.
Just put in in the state you found it when you installed pfSense. The so called default settings. It should works just fine with settings out of the box. -
Entries like this in the gateway log are just the gateway monitoring service starting up showing what settings it's using:
Apr 27 12:21:40 dpinger send_interval 2000ms loss_interval 8000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 4000ms latency_alarm 500ms loss_alarm 40% dest_addr 8.8.8.8 bind_addr 194.75.xxx.xxx identifier "WAN_PPPOE "
Entries like this are actual gateway events:
Apr 27 15:54:10 dpinger WAN_PPPOE 8.8.8.8: Alarm latency 6293us stddev 178us loss 42% Apr 27 15:55:35 dpinger WAN_PPPOE 8.8.8.8: Clear latency 6203us stddev 213us loss 0%
Packet loss goes over the 40% level I have set for that link triggering an alarm. Then the alarm clears.
You are monitoring the gateway IP which is the default setting but it won't show you any disruption upstream of that. I suggest editing the gateway and changing the monitoring IP to something external such as 8.8.8.8 as I'm using there.
even the netgate formus right as I type this
What symptoms were you seeing when usong the forum?
Steve
-
While I was typing my post I got a small message in the bottom right about "your connection to the netgate servers has been disrupted", fixed itself a few seconds later.
I've added 8.8.8.8 as the monitoring IP for my gateway, thank you for that suggestion.
Don't really know where I can set the % but thats not that important right now, I've had no errors like you posted them in over a month, so again I'm pretty sure my WAN connection is stable.It has to be something between pfSense and my PC.
Is there a good monitoring tool for windows ?
If it was a permanent thing I could maybe troubleshoot it, but right now it's just sort of happening for a few seconds and I can't do shit about it. And even if I somehow fixed the problem, I would have no way of actually knowing if it works.
I need some hard logs I can work with.Oh and my DNS resolver is more or less default. The only thing I changed is adding some host overrides for some devices on my local network. I removed the ovveride for my pfSense box just in case that somehow screwed up something.
Oh one more thing, I basically have 2 connections from my PC to my pfSense box. One over my normal LAN network, and then a seperate network on a different LAN cable between my PC and the pfSense box.
My LAN has
allow IPv4 UDP LAN * this firewall 53
and
allow IPv4 * LAN * !RFC1918 *My management network has
allow IPv4 UDP LAN * this firewall 53
and
allow IPv4 TCP Management net * this firewall 443+80+22 -
@XX302 said in Short disconnects multiple times per day:
I've added 8.8.8.8 as the monitoring IP for my gateway
In that case, this :
Apr 27 15:54:10 dpinger WAN_PPPOE 8.8.8.8: Alarm latency 6293us stddev 178us loss 42%
Apr 27 15:55:35 dpinger WAN_PPPOE 8.8.8.8: Clear latency 6203us stddev 213us loss 0%is normal.
8.8.8.8 is a DNS server with " quiet a bit of people hanging on to it (half the planet by now ?)". The IP is probably a shadowed server all over the planet, but probably not the best ICMP "Monitoring IP".edit :
@XX302 said in Short disconnects multiple times per day:allow IPv4 UDP LAN * this firewall 53
And TCP ??? Not letting through TCP port 53 is very bad - many DNS requests and answers are to big, and are send over TCP.
(this probably solves your local issue). -
Alright changing that lol
So, if that is indeed the source of all of my problems, is there any monitoring tool like dpinger for windows ?
-
It also a question of selecting a good IP to monitor.
Most often it's an IP upstream known to you as the upstream gateway.
Using Google's IP's makes you being sending ICMP to Stockholm, then Berlin, Paris, Amsterdam, New York and Berlin again ....edit : wait ....
This is one of my 'munin' pages : https://www.test-domaine.fr/munin/brit-hotel-fumel.net/pfsense.brit-hotel-fumel.net/ping_google_public_dns_a_google_com.html - I do actually use a Google domain to ping against ... for years now .... -
When you are monitoring the gateway IP directly you can only say the connection was good to the gateway, the first hop in the ISPs network. If the ISP has some upstream issue you won't see that in the gateway logs or quality graphs.
So you cannot assume the data you have until now shows a good connection. Setting an external monitoring IP will show you better data from now on.
I've personally never seen any issue using 8.8.8.8 as the monitoring IP but, yes, it was not intended for that purpose. It uses anycast to provide a pretty local IP to you where ever you are.Steve
-
Google will probably work for now.
But thats only pfSense -> everything upstream, I need to be able to monitor from my PC upwards.Edit:
Yeah, I understand what you mean Steve. But I called my ISP and they didn't mention anything, so evem though it would be nice to just blame my ISP, I couldn't do anything about it in that case. And since I'm merely a enthusiast there is a high possibility that I misconfigured something here in my local network. -
Yup, changing the DNS firewall rule and deleting the host override for the router did absolutely nothing, still getting disconnects.
Nothing in the gateway logs so the WAN side is absolutely fine.What else can I do to track down this problem ?
-
So you change to monitoring 8.8.8.8 for the WAN? What do the WAN quality graphs look like?
Steve
-
...where was the quality graph again ?
Monitoring is on 8.8.8., but at this point I'm pretty sure it is not a WAN problem.
It feels like pfSense is dropping the connection on my LAN every few hours, but I don't know why it would do that.
Are there logs for the LAN that could reveal anything, or like I said any monitoring tools for windows you guys would recommend ? -
The graphs are in Status > Monitoring then hit the wrench icon and chose Quality and the appropriate gateway.
I would also check the processor usage for any spikes at that time.
Also check the system logs. I have seen systems where reloading everything caused system loading sufficient to introduce delays in opening connections.
Steve
-
@XX302 said in Short disconnects multiple times per day:
monitoring tools for windows
What about setting up a 'pfSense' monitor to one of your LAN devices that is always on ?
Btw : when the LAN NIC goes down and up, this is logged. related LAN services like the DHCP server would also restart.
Maybe a bad cable / bad contact somewhere / bad switch (check the power of the switch).
Or a bad NIC. -
@stephenw10
https://imgur.com/a/n7wjETx
and
https://imgur.com/a/51yPiDJ
Last disconnect was 15:55.Logs are clear, nothing during that time in General, Gateways, Routing, Firewall...
And it's not just a delay in opening connections, I don't really care about that, it is more the fact that established connections get droppen.
@Gertjan Alright, so how would I do that ?
LAN NIC is on the board of that QOTOM Mini PC, but I guess stuff like that would show up in the system logs, right ?
There is absolutely nothing. DHCP logs say also nothing...
Switch power is ok, I could switch the cable itself (self-crimped Cat6 iirc), but I need to see these disconnects on actual logs. So far I am only experiencing the effects, but there is not the slightest sign in pfSense that something happened during that time. -
Well that big spike looks suspicious but that was at ~2am if the time is correct there. There was an event though just before 16.00. Nothing dramatic at that scale but you should try using 1h at 1m resolution graph to see more.
You should also disable the processes line on the system graph as that swamps the other data. I can't see any spikes there though.Steve
-
https://imgur.com/a/E5GnI2v
The big spike was 15:08, the smaller one exactly 16:00.
I do know however that the disconnect was at exactly 15:54, and was over after less than a minute, so both of those spikes are too late/early. And it is just a jump from 2 to 4/11 ms, package loss was at 0% all the time, so I doubt it had something to do with it.Back to what Gertjan mentioned, how can I set up logging like that for LAN devices ?
I do have some new infermation hinting towards a possible problem further upstream, but I have to confirm that the connection is stable on my end before I can start calling my ISP and yelling at people.
I'm getting more and more desperate here every day, I just want my working Internet back.Alright, enough with the crying, if you guys had a problem likes this on your network, what would you do to find the issue ?
-
I would setup MTR or Smoke ping from a LAN side client to some places out of the internet and let it run until I saw failures then check where in the route is failing. I believe there are Windows variants of those but I've never tried them.
Steve