WAN Issue

raidflex

I have been having an issue for the past 1-2 months where out of no where the internet will at least partially drop. It can be a couple day or even week until it happens again. The WAN connection still has an IP address. What I have noticed is that when it does drop for some reason I can still connect to the OpenVPN server I have setup on my PFsense box, but once on the network I cannot ping or connect to any outside address, only internal devices.

I have also had a tech out from charter to test the connection, he also did a physical check off the wiring and found no issues. He did replace the modem, but the issue still exists. In the past I also checked the modem logs and there was no indication of issues and power levels were within spec. I have no splitters, it is a direct line to the modem.

Rebooting the modem or restarting the firewall always fixes the issue.

I am trying to determine if this is an ISP issue or not.

When the drop happens I notice the below in the logs.

Mar 16 18:41:12	dpinger		WAN_DHCP 24.151.*.*: Alarm latency 7392us stddev 1344us loss 21%
Mar 16 18:41:11	dpinger		WAN_DHCP6 fe80::201:5cff:****:*****em0: Alarm latency 7492us stddev 1439us loss 21%
Mar 16 18:14:05	dpinger		WAN_DHCP6 fe80::201:5cff:****:*****em0: Clear latency 8260us stddev 3008us loss 13%
Mar 16 18:14:02	dpinger		WAN_DHCP 24.151.*.*:: Clear latency 8977us stddev 4112us loss 15%
Mar 16 18:12:46	dpinger		WAN_DHCP 24.151.*.*:: Alarm latency 10100us stddev 4526us loss 21%
Mar 16 18:12:42	dpinger		WAN_DHCP6 fe80::201:5cff:****:*****em0: Alarm latency 10287us stddev 4562us loss 21%
Mar 14 11:17:38	dpinger		WAN_DHCP 24.151.*.*:: Clear latency 9593us stddev 8331us loss 6%
Mar 14 11:17:35	dpinger		WAN_DHCP6 fe80::201:5cff:****:*****em0: Clear latency 9537us stddev 4619us loss 7%
Mar 14 11:15:31	dpinger		WAN_DHCP 24.151.*.*:: Alarm latency 9830us stddev 4748us loss 22%
Mar 14 11:15:28	dpinger		WAN_DHCP6 fe80::201:5cff:****:*****em0: Alarm latency 10588us stddev 4125us loss 21%

When I combed through the logs I could clearly see these latency issues starting about 1-2 months ago, before this time there were no issues reported in the logs. When the connection is up I have no bandwidth or latency issues. There is also no particular time of day when the issue appears.

stephenw10

If you have only one WAN connection you can edit the gateway in System > Routing and set Disable Gateway Monitoring Action. That will prevent the firewall taking action when you hit a latency or packet loss problem. It will log stats for the gateway quality.
If you are using the gateway IP as the monitoring target you might try changing it to something external like 8.8.8.8 to get a better idea of the real connection quality.

Steve

raidflex

@stephenw10 said in WAN Issue:

If you have only one WAN connection you can edit the gateway in System > Routing and set Disable Gateway Monitoring Action. That will prevent the firewall taking action when you hit a latency or packet loss problem. It will log stats for the gateway quality.
If you are using the gateway IP as the monitoring target you might try changing it to something external like 8.8.8.8 to get a better idea of the real connection quality.
Steve

Thanks I will try this out and also change the monitoring target which will hopefully help to zero in on the issue.

raidflex

So its been about a month with no issues and then yesterday the connection dropped. Sure enough rebooting Pfsense fixed the issue. As you can see I changed the target to Google's DNS instead of the Gateway. Previously the connection was dropping about once a week.

Apr 14 18:21:05	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16364us stddev 3253us loss 6%
Apr 14 18:15:38	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 17923us stddev 5861us loss 21%
Apr 14 18:02:43	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16570us stddev 3711us loss 5%
Apr 14 17:57:13	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 17237us stddev 8385us loss 21%
Apr 14 17:56:43	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16511us stddev 2921us loss 5%
Apr 14 17:54:58	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 16985us stddev 3852us loss 21%
Apr 14 16:37:20	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16675us stddev 3238us loss 5%
Apr 14 16:32:09	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 16316us stddev 3158us loss 22%
Apr 14 16:26:53	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16193us stddev 2634us loss 5%
Apr 14 16:25:00	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 16416us stddev 3339us loss 21%
Apr 14 16:22:14	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16040us stddev 2329us loss 5%
Apr 14 16:20:14	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 15950us stddev 2729us loss 21%
Apr 14 16:18:46	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16854us stddev 3063us loss 5%
Apr 14 16:15:35	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 18039us stddev 5263us loss 21%
Apr 14 16:10:12	dpinger		WAN_DHCP 8.8.8.8: Clear latency 16450us stddev 3274us loss 9%
Apr 14 16:08:55	dpinger		WAN_DHCP 8.8.8.8: Alarm latency 17014us stddev 3972us loss 21%

This is quality over last two days, you can see the packet loss when it drops, otherwise there is virtually no loss.

stephenw10

Unless you have reason to think otherwise that looks like actual loss.

I would try physically disconnecting the WAN and reconnecting and also forcing the interface DOWN then UP in software to see if either of those restore the connection without rebooting.

Steve

raidflex

@stephenw10 said in WAN Issue:

Unless you have reason to think otherwise that looks like actual loss.
I would try physically disconnecting the WAN and reconnecting and also forcing the interface DOWN then UP in software to see if either of those restore the connection without rebooting.
Steve

So it just happened again, so this time I physically disconnected the WAN and reconnected it and the interface came back online. It also seems to be staying online.

y2raza

@raidflex Hi, so did this permanently fix the problem?

stephenw10

Anything that was happening in 2019 is unlikely to apply to current pfSense. A lot has changed.