WAN interface stops working every few days.
-
What would I want to focus on for the packet capture, WAN?
also, it seems like Unplugging the WAN cable for a minute or so and plugging it back in also resolves the high latency and drops.
-
I've found this in the gateway logs if it helps any. Also Guess I'll set a syslog server back up later today.
Aug 8 11:22:02 dpinger send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr [Redacted] bind_addr [Redacted] bidentifier "WAN01_CENTURYLINK_PPPOE " Aug 8 13:44:52 dpinger WAN01_CENTURYLINK_PPPOE [Redacted]: Alarm latency 544019us stddev 1364744us loss 10% -
Well, you could start with what happens when you try disconnecting/reconnection the WAN cable or disabling/re-enabling the interface. I also see it's OK after waiting a minute before plugging the cable in. You could compare the differences with not waiting. That sort of thing. About 1.5 years ago, I had a problem with IPv6 on my ISP. By using Wireshark, when pfSense booted up, I was able to identify the failing equipment, by name, at my ISPs local office. After I got that resolved, I saved a normal DHCP & DHCPv6-PD sequence, so that I'd always have something to refer to, should a problem happen again. Since I was rebooting pfsense, I couldn't use Packet Capture, so I used a managed switch, configured as a data tap,with my notebook running Wireshark.
-
This post is deleted! -
@bcruze
These issues began after I swapped my pfsense box from an optiplex 7010 to a Dell Poweredge R210 II. it was working without any issue prior.As of 2 days ago this behaviour is now occuring every 12 hours or so.
As of yesterday, I've already tried tweaking the system tunables per some suggestions for PPOE interfaces as well as BCE adapters.
I'm going to try buying an intel gigabit nic and see if it's an issue with the broadcom onboard adapter.
It also seems that manually setting my PPOE connection as offline, applying settings then going back and re-enabling it temporarily resolves the issue when it occurs whereas the monitoring service isn't managing to re-establish the connection.
-
What NICs were in the older box?
I assume this is a PPPoE connection from your logs, unless that's another gateway?
You could certainly try setting the gateway monitoring target to something different. Be sure it's actually the WAN and not just the target.
Carrier grade NAT should be all at the ISP if they are using that. I'm not sure which issue you're referring to @bcruze.
Steve
-
I'm not certain which was wan/LAN but I had two NICs, an onboard intel gigabit and a broadcom 5722.
I still have the 5722 floating around so I'm going to see if I can adapt a full length PCIE slot bracket onto it
I've already tried setting the gateway monitoring target to 8.8.8.8
https://techtilt.com/fix-for-pfsense-keeps-dropping-wan-intermittently-randomI also tried adjusting a number of settings in tunables.
https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html
net.isr.dispatch=deferred
kern.ipc.nmbclusters="131072"
hw.bce.tso_enable=0
hw.pci.enable_msix=0P.S. It doesn't look like syslogging is immediately available because i let my Splunk trial license lapse... I just applied the 500MB free license but it'll take a month or a reinstall before I can access the data again.
-
Installed the 5722 going to see how it handles the PPPOe... Diddnt have a correct full length bracket so I got creative to prevent it from getting knocked out of the slot and causing a short. https://imgur.com/gallery/dbfYaLi
-
Still occurring unfortunately, would I need to run a packet capture 24/7 until an issue occurs or would there be another route?
-
That may not tell you much anyway.
It's curious that rebooting the ONT corrects the issue but unplugging the cable does not. Those should be similar from pfSense's view. Obviously one resets the upstream connection too but if that were an issue then rebooting pfSense alone would not correct it.
A short pcap made whilst the connection is bad might show something. Bad packets etc.
It might need to be on the PPPoE parent interface though.
Steve
-
@stephenw10 said in WAN interface stops working every few days.:
That may not tell you much anyway.
It's curious that rebooting the ONT corrects the issue but unplugging the cable does not. Those should be similar from pfSense's view. Obviously one resets the upstream connection too but if that were an issue then rebooting pfSense alone would not correct it.
A short pcap made whilst the connection is bad might show something. Bad packets etc.
It might need to be on the PPPoE parent interface though.
Steve
Three things have been observed to correct the issue so far.
Rebooting pfsense
Disconnecting the WAN if's ethernet cable for ~15 seconds then plugging it back in.
Power Cycling the ONTpacket loss and latency skyrockets during these events.
I'm going to do a packet capture as well as take a close look at the PPPOE traffic the next time this happens. I'm curious to see if my WAN IP changes as well as what disabling and re-enabling PPPOE does.This issue began approximately 1 week after I had replaced my optiplex 7010SFF PFsense instance for the R210 II.
There are two other things in the same timeframe which "may may potentially attribute but I'd be surprised if they were the issue"- Minor heat wave where temps were in the upper 90's for a few days.\
- Unmounted ONT to physically inspect what type of optical cable it uses, It may be remotely possible that I somehow pinched the cable when returning the ONT back into it's cradle? I'm not certain whether that would manifest with these symptoms though. aside from the every 12-36 hour events pings, latency and packet loss are on par for gigabit.
-
I changed out the Broadcom NIC for an Intel one and I really, really hope this issues goes away with it.
I did a packet capture on the WAN interface and see a lot of ttl timeouts, TCP resets or unacknowledged acks. Existing socket connections continue to work without issue but any new connection attempts have an extremely high latency and packet loss. I'm not comfortable sharing this packet capture though because it could potentially contain some authentication info which could be reverse engineered..
I'll paste a snippet of it below.
I'm going to see if my ISP will send me a replacement ONT so that I can cover my bases there.
1 2020/231 17:46:10.570391 0.000000000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=123 2 2020/231 17:46:10.570398 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=252 3 2020/231 17:46:10.581041 0.010643000 71.36.120.123 205.251.197.145 DNS Standard query 0x5e31 A a.teads.tv OPT 4 2020/231 17:46:10.604454 0.023413000 71.36.120.123 182.161.72.6 DNS Standard query 0x8f03 AAAA ns28.criteo.com OPT 5 2020/231 17:46:10.604504 0.000050000 71.36.120.123 74.119.118.255 DNS Standard query 0x7900 AAAA ns22.criteo.com OPT 6 2020/231 17:46:10.604535 0.000031000 71.36.120.123 74.119.118.255 DNS Standard query 0xa752 AAAA ns27.criteo.com OPT 7 2020/231 17:46:10.604866 0.000331000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=128 8 2020/231 17:46:10.610319 0.005453000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=70 9 2020/231 17:46:10.612201 0.001882000 71.36.120.123 108.162.193.135 DNS Standard query 0xcf67 A ns.wpopt.net OPT 10 2020/231 17:46:10.612759 0.000558000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=42 11 2020/231 17:46:10.616211 0.003452000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=128 12 2020/231 17:46:10.623546 0.007335000 71.36.120.123 192.112.36.4 DNS Standard query 0xbbf4 A wpad.britannia.local OPT 13 2020/231 17:46:10.644779 0.021233000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=126 14 2020/231 17:46:10.644786 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=73 15 2020/231 17:46:10.644935 0.000149000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=627 16 2020/231 17:46:10.644941 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=627 17 2020/231 17:46:10.656326 0.011385000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1139 18 2020/231 17:46:10.662098 0.005772000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=125 19 2020/231 17:46:10.662255 0.000157000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1139 20 2020/231 17:46:10.663498 0.001243000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 21 2020/231 17:46:10.667730 0.004232000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1139 22 2020/231 17:46:10.671272 0.003542000 71.36.120.123 192.112.36.4 DNS Standard query 0x06ce A local OPT 23 2020/231 17:46:10.673319 0.002047000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1139 24 2020/231 17:46:10.673900 0.000581000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 25 2020/231 17:46:10.678912 0.005012000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=119 26 2020/231 17:46:10.683979 0.005067000 71.36.120.123 192.112.36.4 DNS Standard query 0x5d0d AAAA ns-1881.awsdns-43.co.uk OPT 27 2020/231 17:46:10.684778 0.000799000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1139 28 2020/231 17:46:10.684785 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 29 2020/231 17:46:10.690549 0.005764000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 30 2020/231 17:46:10.695846 0.005297000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 31 2020/231 17:46:10.696177 0.000331000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 32 2020/231 17:46:10.701793 0.005616000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=114 33 2020/231 17:46:10.701950 0.000157000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=243 34 2020/231 17:46:10.701956 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 35 2020/231 17:46:10.707201 0.005245000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 36 2020/231 17:46:10.712448 0.005247000 71.36.120.123 192.112.36.4 DNS Standard query 0xe812 A ns-1881.awsdns-43.co.uk OPT 37 2020/231 17:46:10.713558 0.001110000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 38 2020/231 17:46:10.715311 0.001753000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=50 39 2020/231 17:46:10.718780 0.003469000 71.36.120.123 204.13.251.136 DNS Standard query 0xb24a A ns3.p29.dynect.net OPT 40 2020/231 17:46:10.718817 0.000037000 71.36.120.123 156.154.65.210 DNS Standard query 0xd119 A elb-ore-amz.nimbus.bitdefender.net OPT 41 2020/231 17:46:10.719147 0.000330000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=114 42 2020/231 17:46:10.719154 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 43 2020/231 17:46:10.724919 0.005765000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 44 2020/231 17:46:10.730536 0.005617000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 45 2020/231 17:46:10.736308 0.005772000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=114 46 2020/231 17:46:10.736315 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 47 2020/231 17:46:10.741924 0.005609000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=46 48 2020/231 17:46:10.741931 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1140 49 2020/231 17:46:10.742357 0.000426000 71.36.120.123 74.125.250.87 STUN Binding Request user: QUk4jW0q5FYFBAXl:R6ng 50 2020/231 17:46:10.755482 0.013125000 74.125.250.87 71.36.120.123 STUN Binding Success Response user: QUk4jW0q5FYFBAXl:R6ng XOR-MAPPED-ADDRESS: 71.36.120.123:7162 51 2020/231 17:46:10.759071 0.003589000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=98 52 2020/231 17:46:10.759078 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=39 53 2020/231 17:46:10.766467 0.007389000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 54 2020/231 17:46:10.770543 0.004076000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=107 55 2020/231 17:46:10.770549 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=702 56 2020/231 17:46:10.771652 0.001103000 71.36.120.123 162.88.61.21 DNS Standard query 0x78ad A ns2.p29.dynect.net OPT 57 2020/231 17:46:10.776351 0.004699000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=77 58 2020/231 17:46:10.783203 0.006852000 71.36.120.123 205.251.193.209 DNS Standard query 0x4fc2 A ns-645.awsdns-16.net OPT 59 2020/231 17:46:10.783285 0.000082000 71.36.120.123 162.88.60.21 DNS Standard query 0xb882 A ns1.p29.dynect.net OPT 60 2020/231 17:46:10.799214 0.015929000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=75 61 2020/231 17:46:10.803738 0.004524000 216.239.38.10 71.36.120.123 DNS Standard query response 0x021f A mobile-gtalk.l.google.com A 74.125.195.188 OPT 62 2020/231 17:46:10.814661 0.010923000 192.35.51.30 71.36.120.123 DNS Standard query response 0x43c6 A dynamicnetworkservices.net NS ns1.dynamicnetworkservices.net NS ns2.dynamicnetworkservices.net NS ns3.dynamicnetworkservices.net NS ns4.dynamicnetworkservices.net NS ns5.dynamicnetworkservices.net NS ns6.dynamicnetworkservices.net NS ns7.dynamicnetworkservices.net NSEC3 RRSIG AAAA 2001:500:90:1::136 A 208.78.70.136 OPT 63 2020/231 17:46:10.816580 0.001919000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=69 64 2020/231 17:46:10.817005 0.000425000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=42 65 2020/231 17:46:10.822015 0.005010000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=76 66 2020/231 17:46:10.825603 0.003588000 199.19.54.1 71.36.120.123 DNS Standard query response 0x29cb A ultradns.org OPT 67 2020/231 17:46:10.825684 0.000081000 71.36.120.123 199.19.54.1 TCP 0 0 1 0 65228 44963 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=3298201888 TSecr=0 68 2020/231 17:46:10.827733 0.002049000 199.249.120.1 71.36.120.123 DNS Standard query response 0xd56c AAAA ns3-06.azure-dns.org OPT 69 2020/231 17:46:10.827740 0.000007000 65.22.162.17 71.36.120.123 TCP 0 0 1 1 65535 53 → 44907 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1452 WS=64 SACK_PERM=1 TSval=734570156 TSecr=474470959 70 2020/231 17:46:10.827793 0.000053000 71.36.120.123 199.249.120.1 TCP 0 0 1 0 65228 44964 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=1310942176 TSecr=0 71 2020/231 17:46:10.827892 0.000099000 199.19.53.1 71.36.120.123 TCP 0 1 1 0 0 53 → 44896 [RST] Seq=1 Win=0 Len=0 72 2020/231 17:46:10.827899 0.000007000 192.48.79.30 71.36.120.123 TCP 0 1 1 0 0 53 → 44909 [RST] Seq=1 Win=0 Len=0 73 2020/231 17:46:10.829917 0.002018000 198.51.45.66 71.36.120.123 DNS Standard query response 0x8e05 A tlx.3lift.com CNAME us-west-tlx.3lift.com CNAME dualstack.exchange-prod-582331669.us-west-1.elb.amazonaws.com OPT 74 2020/231 17:46:10.831699 0.001782000 71.36.120.123 64.4.48.3 DNS Standard query 0x1f8a A ns2-34.azure-dns.net OPT 75 2020/231 17:46:10.831732 0.000033000 71.36.120.123 205.251.199.144 DNS Standard query 0xf8f8 A ns-645.awsdns-16.net OPT 76 2020/231 17:46:10.837963 0.006231000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=35 77 2020/231 17:46:10.839231 0.001268000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=84 78 2020/231 17:46:10.841102 0.001871000 199.19.53.1 71.36.120.123 TCP 0 0 1 1 65535 53 → 44916 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1452 WS=64 SACK_PERM=1 TSval=3678409839 TSecr=721965227 79 2020/231 17:46:10.841109 0.000007000 192.5.6.30 71.36.120.123 TCP 0 1 1 0 0 53 → 44858 [RST] Seq=1 Win=0 Len=0 80 2020/231 17:46:10.843287 0.002178000 192.36.148.17 71.36.120.123 DNS Standard query response 0x801c No such name A bidder.criteo.com.britannia.local OPT 81 2020/231 17:46:10.845318 0.002031000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=246 82 2020/231 17:46:10.845324 0.000006000 2.22.230.67 71.36.120.123 DNS Standard query response 0x7135 A a16-65.akam.net A 23.211.132.65 OPT 83 2020/231 17:46:10.845601 0.000277000 71.36.120.123 96.7.49.67 DNS Standard query 0x8b78 AAAA a16-65.akam.net OPT 84 2020/231 17:46:10.847337 0.001736000 199.7.91.13 71.36.120.123 DNS Standard query response 0x2c86 No such name A local NSEC locker RRSIG OPT 85 2020/231 17:46:10.849520 0.002183000 192.48.79.30 71.36.120.123 DNS Standard query response 0x5869 A nsone.net NS dns1.p01.nsone.net NS dns2.p01.nsone.net NS dns3.p01.nsone.net NS dns4.p01.nsone.net NSEC3 RRSIG NSEC3 A 198.51.44.1 A 198.51.45.1 OPT 86 2020/231 17:46:10.849580 0.000060000 71.36.120.123 192.48.79.30 TCP 0 0 1 0 65228 44965 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=144028582 TSecr=0 87 2020/231 17:46:10.850848 0.001268000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 88 2020/231 17:46:10.851470 0.000622000 2.22.230.67 71.36.120.123 DNS Standard query response 0x31af A a5-64.akam.net A 95.100.168.64 OPT 89 2020/231 17:46:10.851627 0.000157000 192.48.79.30 71.36.120.123 TCP 0 1 1 0 0 53 → 44909 [RST] Seq=1 Win=0 Len=0 90 2020/231 17:46:10.851722 0.000095000 71.36.120.123 96.7.49.67 DNS Standard query 0x9ef7 A a5-64.akam.net OPT 91 2020/231 17:46:10.853770 0.002048000 213.248.216.1 71.36.120.123 DNS Standard query response 0xb526 A ns-1881.awsdns-43.co.uk NS g-ns-363.awsdns-43.co.uk NS g-ns-939.awsdns-43.co.uk NS g-ns-1518.awsdns-43.co.uk NS g-ns-1839.awsdns-43.co.uk NSEC3 RRSIG OPT 92 2020/231 17:46:10.856579 0.002809000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 93 2020/231 17:46:10.862037 0.005458000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=87 94 2020/231 17:46:10.862193 0.000156000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 95 2020/231 17:46:10.865180 0.002987000 71.36.120.123 216.252.166.11 DNS Standard query 0x4a91 A ib.adnxs.com OPT 96 2020/231 17:46:10.866755 0.001575000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 97 2020/231 17:46:10.867868 0.001113000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 98 2020/231 17:46:10.871459 0.003591000 71.36.120.123 156.154.65.210 DNS Standard query 0x451e A elb-ore-amz.nimbus.bitdefender.net OPT 99 2020/231 17:46:10.879433 0.007974000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=90 100 2020/231 17:46:10.879440 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 101 2020/231 17:46:10.879589 0.000149000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 102 2020/231 17:46:10.890823 0.011234000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 103 2020/231 17:46:10.896595 0.005772000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=94 104 2020/231 17:46:10.902368 0.005773000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=59 105 2020/231 17:46:10.902374 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1173 106 2020/231 17:46:10.902380 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=1174 107 2020/231 17:46:10.908063 0.005683000 71.36.120.123 200.7.86.53 DNS Standard query 0x760f PTR 4.d.3.2.0.4.f.2.b.0.d.3.0.0.7.2.e.c.5.0.9.1.6.f.0.2.c.f.7.0.6.2.ip6.arpa OPT 108 2020/231 17:46:10.913699 0.005636000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=73 109 2020/231 17:46:10.917560 0.003861000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 110 2020/231 17:46:10.919297 0.001737000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=96 111 2020/231 17:46:10.919452 0.000155000 96.7.49.67 71.36.120.123 DNS Standard query response 0x351a A a16-65.akam.net A 23.211.132.65 OPT 112 2020/231 17:46:10.919458 0.000006000 81.17.242.98 71.36.120.123 ICMP Time-to-live exceeded (Time to live exceeded in transit) 113 2020/231 17:46:10.919465 0.000007000 81.17.242.98 71.36.120.123 ICMP Time-to-live exceeded (Time to live exceeded in transit) 114 2020/231 17:46:10.919717 0.000252000 71.36.120.123 23.211.133.67 DNS Standard query 0xb7b3 A a16-65.akam.net OPT 115 2020/231 17:46:10.921607 0.001890000 96.7.49.67 71.36.120.123 DNS Standard query response 0x8d3e AAAA use2.akam.net SOA internal.akam.net OPT 116 2020/231 17:46:10.921867 0.000260000 71.36.120.123 23.211.133.67 DNS Standard query 0x2bb5 AAAA use2.akam.net OPT 117 2020/231 17:46:10.923758 0.001891000 199.253.182.182 71.36.120.123 DNS Standard query response 0xfdd1 PTR 4.d.3.2.0.4.f.2.b.0.d.3.0.0.7.2.e.c.5.0.9.1.6.f.0.2.c.f.7.0.6.2.ip6.arpa NS r.arin.net NS u.arin.net NS x.arin.net NS y.arin.net NS z.arin.net NS arin.authdns.ripe.net DS RRSIG OPT 118 2020/231 17:46:10.925005 0.001247000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=697 119 2020/231 17:46:10.925786 0.000781000 199.19.56.1 71.36.120.123 DNS Standard query response 0x3f87 A ultradns.org OPT 120 2020/231 17:46:10.925846 0.000060000 71.36.120.123 199.19.56.1 TCP 0 0 1 0 65228 44966 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=923319125 TSecr=0 121 2020/231 17:46:10.927895 0.002049000 198.51.44.2 71.36.120.123 DNS Standard query response 0xb0f4 A prebid.appnexusgslb.net A 68.67.129.85 OPT 122 2020/231 17:46:10.929922 0.002027000 65.22.163.17 71.36.120.123 DNS Standard query response 0x4662 A dmx.districtm.io OPT 123 2020/231 17:46:10.930703 0.000781000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=39 124 2020/231 17:46:10.931952 0.001249000 23.211.133.67 71.36.120.123 DNS Standard query response 0x4d8f AAAA a28-67.akam.net SOA internal.akam.net OPT 125 2020/231 17:46:10.932073 0.000121000 71.36.120.123 184.85.248.67 DNS Standard query 0x2361 AAAA a28-67.akam.net OPT 126 2020/231 17:46:10.934120 0.002047000 192.36.148.17 71.36.120.123 DNS Standard query response 0x940b DNSKEY <Root> OPT 127 2020/231 17:46:10.936146 0.002026000 23.211.133.67 71.36.120.123 DNS Standard query response 0x0d65 A a5-64.akam.net A 95.100.168.64 OPT 128 2020/231 17:46:10.936393 0.000247000 71.36.120.123 95.101.36.67 DNS Standard query 0x51e0 AAAA a5-64.akam.net OPT 129 2020/231 17:46:10.942964 0.006571000 192.48.79.30 71.36.120.123 DNS Standard query response 0x4b33 A nsone.net NS dns1.p01.nsone.net NS dns2.p01.nsone.net NS dns3.p01.nsone.net NS dns4.p01.nsone.net NSEC3 RRSIG NSEC3 A 198.51.44.1 A 198.51.45.1 OPT 130 2020/231 17:46:10.943024 0.000060000 71.36.120.123 192.48.79.30 TCP 0 0 1 0 65228 44967 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=2408885028 TSecr=0 131 2020/231 17:46:10.945072 0.002048000 199.254.48.1 71.36.120.123 DNS Standard query response 0xd584 AAAA ns4-06.azure-dns.info OPT 132 2020/231 17:46:10.947099 0.002027000 199.254.48.1 71.36.120.123 DNS Standard query response 0xb8a3 A ns4-06.azure-dns.info OPT 133 2020/231 17:46:10.947881 0.000782000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=85 134 2020/231 17:46:10.949130 0.001249000 199.253.182.182 71.36.120.123 DNS Standard query response 0xf9cc PTR 4.d.3.2.0.4.f.2.b.0.d.3.0.0.7.2.e.c.5.0.9.1.6.f.0.2.c.f.7.0.6.2.ip6.arpa NS r.arin.net NS u.arin.net NS x.arin.net NS y.arin.net NS z.arin.net NS arin.authdns.ripe.net DS RRSIG OPT 135 2020/231 17:46:10.950288 0.001158000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=46 136 2020/231 17:46:10.951244 0.000956000 182.161.72.6 71.36.120.123 DNS Standard query response 0x1f80 AAAA ns26.criteo.com SOA ns23.criteo.com OPT 137 2020/231 17:46:10.951251 0.000007000 8.8.8.8 71.36.120.123 ICMP Echo (ping) reply id=0x0e8f, seq=389/34049, ttl=118 138 2020/231 17:46:10.953275 0.002024000 172.217.14.196 71.36.120.123 TCP 0 0 1 1 65535 80 → 25037 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1430 SACK_PERM=1 TSval=2310718172 TSecr=149079597 WS=256 139 2020/231 17:46:10.953430 0.000155000 192.48.79.30 71.36.120.123 TCP 0 1 1 0 0 53 → 44909 [RST] Seq=1 Win=0 Len=0 140 2020/231 17:46:10.955459 0.002029000 96.7.49.67 71.36.120.123 DNS Standard query response 0x6fcb A as-sec.casalemedia.com CNAME as-sec.casalemedia.com.edgekey.net OPT 141 2020/231 17:46:10.959517 0.004058000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=84 142 2020/231 17:46:10.959829 0.000312000 192.48.79.30 71.36.120.123 DNS Standard query response 0x4c0b A nsone.net NS dns1.p01.nsone.net NS dns2.p01.nsone.net NS dns3.p01.nsone.net NS dns4.p01.nsone.net NSEC3 RRSIG NSEC3 A 198.51.44.1 A 198.51.45.1 OPT 143 2020/231 17:46:10.959895 0.000066000 71.36.120.123 192.48.79.30 TCP 0 0 1 0 65228 44968 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=3466359485 TSecr=0 144 2020/231 17:46:10.961788 0.001893000 192.112.36.4 71.36.120.123 DNS Standard query response 0xcbc7 No such name A wpad.britannia.local OPT 145 2020/231 17:46:10.962221 0.000433000 71.36.120.123 192.12.94.30 DNS Standard query 0x61b2 A appnexusgslb.com OPT 146 2020/231 17:46:10.963956 0.001735000 96.7.49.67 71.36.120.123 DNS Standard query response 0x1a52 A a16-65.akam.net A 23.211.132.65 OPT 147 2020/231 17:46:10.964207 0.000251000 71.36.120.123 95.101.36.67 DNS Standard query 0xf4ef A a16-65.akam.net OPT 148 2020/231 17:46:10.967801 0.003594000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=42 149 2020/231 17:46:10.982330 0.014529000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=79 150 2020/231 17:46:10.982337 0.000007000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=327 151 2020/231 17:46:10.985450 0.003113000 172.217.14.196 71.36.120.123 TCP 0 0 1 1 65535 443 → 39665 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1430 SACK_PERM=1 TSval=2778687698 TSecr=149079006 WS=256 152 2020/231 17:46:10.985456 0.000006000 172.217.14.195 71.36.120.123 TCP 0 0 1 1 65535 80 → 6268 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1430 SACK_PERM=1 TSval=1227369459 TSecr=149079007 WS=256 153 2020/231 17:46:10.987478 0.002022000 156.154.101.3 71.36.120.123 DNS Standard query response 0x1d19 AAAA nsc.nic.uk SOA dns1.nic.uk OPT 154 2020/231 17:46:10.989118 0.001640000 71.36.120.123 173.245.59.135 DNS Standard query 0x4482 A ns.wpopt.net OPT 155 2020/231 17:46:10.989148 0.000030000 71.36.120.123 172.64.33.135 DNS Standard query 0x5a54 A ns.wpopt.net OPT 156 2020/231 17:46:10.989633 0.000485000 96.7.49.67 71.36.120.123 DNS Standard query response 0xbb37 AAAA a9-67.akam.net AAAA 2a02:26f0:117::43 OPT 157 2020/231 17:46:10.989875 0.000242000 71.36.120.123 184.85.248.67 DNS Standard query 0xbdfc AAAA a9-67.akam.net OPT 158 2020/231 17:46:10.990877 0.001002000 71.36.120.123 172.217.14.196 TCP 0 1 1 1 343 25037 → 80 [ACK] Seq=1 Ack=1 Win=343 Len=0 TSval=149080659 TSecr=2310718172 159 2020/231 17:46:10.990892 0.000015000 71.36.120.123 172.217.14.196 TCP 0 882 882 1076 354 [TCP ACKed unseen segment] 39665 → 443 [ACK] Seq=882 Ack=1076 Win=354 Len=0 TSval=149080659 TSecr=2778687985 SLE=0 SRE=1 160 2020/231 17:46:10.991098 0.000206000 71.36.120.123 172.217.14.195 TCP 0 229 229 103 343 [TCP ACKed unseen segment] 6268 → 80 [ACK] Seq=229 Ack=103 Win=343 Len=0 TSval=149080659 TSecr=1227369723 SLE=0 SRE=1 161 2020/231 17:46:10.991362 0.000264000 71.36.120.123 172.217.14.196 HTTP 207 1 208 1 343 207 GET /gen_204 HTTP/1.1 162 2020/231 17:46:10.993567 0.002205000 176.32.99.148 71.36.120.123 TLSv1.2 46 1 47 1 2188 46 Application Data 163 2020/231 17:46:10.996531 0.002964000 172.217.14.196 71.36.120.123 TCP 0 1 1 208 66816 80 → 25037 [ACK] Seq=1 Ack=208 Win=66816 Len=0 TSval=2310719898 TSecr=149080659 164 2020/231 17:46:10.999340 0.002809000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=79 165 2020/231 17:46:11.000038 0.000698000 71.36.120.123 176.32.99.148 TLSv1.2 46 1 47 47 8209 46 Application Data 166 2020/231 17:46:11.002999 0.002961000 71.36.120.123 198.51.45.2 DNS Standard query 0x3b38 A tlx.3lift.com OPT 167 2020/231 17:46:11.005044 0.002045000 192.48.79.30 71.36.120.123 TCP 0 1 1 0 0 53 → 44909 [RST] Seq=1 Win=0 Len=0 168 2020/231 17:46:11.007073 0.002029000 192.112.36.4 71.36.120.123 DNS Standard query response 0xd94a No such name A local OPT 169 2020/231 17:46:11.008634 0.001561000 172.217.14.196 71.36.120.123 HTTP 314 1 315 208 66816 314 HTTP/1.1 204 No Content 170 2020/231 17:46:11.009259 0.000625000 192.112.36.4 71.36.120.123 DNS Standard query response 0xcd80 AAAA ns-1881.awsdns-43.co.uk OPT 171 2020/231 17:46:11.010640 0.001381000 71.36.120.123 172.217.14.196 TCP 0 208 208 315 347 25037 → 80 [ACK] Seq=208 Ack=315 Win=347 Len=0 TSval=149080664 TSecr=2310719910 172 2020/231 17:46:11.011283 0.000643000 23.211.133.67 71.36.120.123 DNS Standard query response 0x8337 A a16-65.akam.net A 23.211.132.65 OPT 173 2020/231 17:46:11.011532 0.000249000 71.36.120.123 95.100.173.67 DNS Standard query 0x5f5c AAAA a16-65.akam.net OPT 174 2020/231 17:46:11.013225 0.001693000 71.36.120.123 172.217.14.196 TCP 0 208 209 315 347 25037 → 80 [FIN, ACK] Seq=208 Ack=315 Win=347 Len=0 TSval=149080665 TSecr=2310719910 175 2020/231 17:46:11.013400 0.000175000 2.22.230.67 71.36.120.123 DNS Standard query response 0x0068 AAAA a22-67.akam.net SOA internal.akam.net OPT 176 2020/231 17:46:11.013406 0.000006000 81.17.242.98 71.36.120.123 ICMP Time-to-live exceeded (Time to live exceeded in transit) 177 2020/231 17:46:11.013535 0.000129000 71.36.120.123 95.100.173.67 DNS Standard query 0x2cbf AAAA a22-67.akam.net OPT 178 2020/231 17:46:11.015427 0.001892000 192.112.36.4 71.36.120.123 DNS Standard query response 0x5271 A ns-1881.awsdns-43.co.uk OPT 179 2020/231 17:46:11.018265 0.002838000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=42 180 2020/231 17:46:11.019300 0.001035000 71.36.120.123 8.8.8.8 ICMP Echo (ping) request id=0x0e8f, seq=396/35841, ttl=64 (no response found!) 181 2020/231 17:46:11.019344 0.000044000 71.36.120.123 192.12.94.30 DNS Standard query 0xf06c A ns27.domaincontrol.com OPT 182 2020/231 17:46:11.019378 0.000034000 71.36.120.123 192.12.94.30 DNS Standard query 0x0b0c AAAA ns27.domaincontrol.com OPT 183 2020/231 17:46:11.019410 0.000032000 71.36.120.123 192.12.94.30 DNS Standard query 0xaaef AAAA ns28.domaincontrol.com OPT 184 2020/231 17:46:11.022237 0.002827000 156.154.65.210 71.36.120.123 DNS Standard query response 0x8827 A elb-ore-amz.nimbus.bitdefender.net CNAME kube-nimbus-471965604.us-west-2.elb.amazonaws.com OPT 185 2020/231 17:46:11.022391 0.000154000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=80 186 2020/231 17:46:11.024265 0.001874000 204.13.251.136 71.36.120.123 DNS Standard query response 0x4bf1 A ns3.p29.dynect.net A 208.78.71.29 NS ns3.dynamicnetworkservices.net NS ns2.dynamicnetworkservices.net NS ns1.dynamicnetworkservices.net NS ns7.dynamicnetworkservices.net NS ns5.dynamicnetworkservices.net NS ns6.dynamicnetworkservices.net NS ns4.dynamicnetworkservices.net OPT 187 2020/231 17:46:11.026294 0.002029000 162.88.61.21 71.36.120.123 DNS Standard query response 0x4046 A ns2.p29.dynect.net A 204.13.250.29 OPT 188 2020/231 17:46:11.026762 0.000468000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=66 189 2020/231 17:46:11.028484 0.001722000 192.58.128.30 71.36.120.123 DNS Standard query response 0x47b3 A biz NS k.gtld.biz NS f.gtld.biz NS a.gtld.biz NS b.gtld.biz NS c.gtld.biz NS e.gtld.biz DS DS RRSIG OPT 190 2020/231 17:46:11.030513 0.002029000 192.5.5.241 71.36.120.123 DNS Standard query response 0x724c A biz NS a.gtld.biz NS b.gtld.biz NS c.gtld.biz NS e.gtld.biz NS f.gtld.biz NS k.gtld.biz DS DS RRSIG OPT 191 2020/231 17:46:11.030578 0.000065000 71.36.120.123 192.5.5.241 TCP 0 0 1 0 65228 44969 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=2411125480 TSecr=0 192 2020/231 17:46:11.032627 0.002049000 192.5.5.241 71.36.120.123 DNS Standard query response 0x847d A e.gtld.biz NS a.gtld.biz NS b.gtld.biz NS c.gtld.biz NS e.gtld.biz NS f.gtld.biz NS k.gtld.biz DS DS RRSIG OPT 193 2020/231 17:46:11.032683 0.000056000 71.36.120.123 192.5.5.241 TCP 0 0 1 0 65228 44970 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=259106889 TSecr=0 194 2020/231 17:46:11.036916 0.004233000 204.13.250.136 71.36.120.123 DNS Standard query response 0xfdbf A ns2.p29.dynect.net A 204.13.250.29 NS ns3.dynamicnetworkservices.net NS ns2.dynamicnetworkservices.net NS ns7.dynamicnetworkservices.net NS ns1.dynamicnetworkservices.net NS ns6.dynamicnetworkservices.net NS ns4.dynamicnetworkservices.net NS ns5.dynamicnetworkservices.net OPT 195 2020/231 17:46:11.037107 0.000191000 71.36.120.123 208.78.71.136 DNS Standard query 0x9ddf A ns2.p29.dynect.net OPT 196 2020/231 17:46:11.038999 0.001892000 205.251.195.18 71.36.120.123 DNS Standard query response 0x4d9e A ns-645.awsdns-16.net A 205.251.194.133 NS g-ns-1360.awsdns-16.net NS g-ns-1936.awsdns-16.net NS g-ns-465.awsdns-16.net NS g-ns-786.awsdns-16.net A 205.251.197.80 AAAA 2600:9000:5305:5000::1 A 205.251.199.144 AAAA 2600:9000:5307:9000::1 A 205.251.193.209 AAAA 2600:9000:5301:d100::1 A 205.251.195.18 AAAA 2600:9000:5303:1200::1 OPT 197 2020/231 17:46:11.039007 0.000008000 192.35.51.30 71.36.120.123 TCP 0 1 1 0 0 53 → 44915 [RST] Seq=1 Win=0 Len=0 198 2020/231 17:46:11.039467 0.000460000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=83 199 2020/231 17:46:11.039473 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=620 200 2020/231 17:46:11.041027 0.001554000 192.42.93.30 71.36.120.123 DNS Standard query response 0x4e36 A amplitude.com NS ns-579.awsdns-08.net NS ns-260.awsdns-32.com NS ns-1262.awsdns-29.org NS ns-1942.awsdns-50.co.uk NSEC3 RRSIG A 205.251.193.4 OPT 201 2020/231 17:46:11.043056 0.002029000 198.97.190.53 71.36.120.123 DNS Standard query response 0x47e6 A biz NS a.gtld.biz NS b.gtld.biz NS c.gtld.biz NS e.gtld.biz NS f.gtld.biz NS k.gtld.biz DS DS RRSIG OPT 202 2020/231 17:46:11.043114 0.000058000 71.36.120.123 198.97.190.53 TCP 0 0 1 0 65228 44971 → 53 [SYN] Seq=0 Win=65228 Len=0 MSS=1452 WS=128 SACK_PERM=1 TSval=2608320456 TSecr=0 203 2020/231 17:46:11.045161 0.002047000 162.88.60.21 71.36.120.123 DNS Standard query response 0x346c A ns1.p29.dynect.net A 208.78.70.29 OPT 204 2020/231 17:46:11.047467 0.002306000 71.36.120.123 216.239.34.10 DNS Standard query 0xa45d A mobile-gtalk.l.google.com OPT 205 2020/231 17:46:11.050920 0.003453000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=79 206 2020/231 17:46:11.050926 0.000006000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=107 207 2020/231 17:46:11.053726 0.002800000 205.251.193.209 71.36.120.123 DNS Standard query response 0x79d3 A ns-645.awsdns-16.net A 205.251.194.133 NS g-ns-1360.awsdns-16.net NS g-ns-1936.awsdns-16.net NS g-ns-465.awsdns-16.net NS g-ns-786.awsdns-16.net A 205.251.197.80 AAAA 2600:9000:5305:5000::1 A 205.251.199.144 AAAA 2600:9000:5307:9000::1 A 205.251.193.209 AAAA 2600:9000:5301:d100::1 A 205.251.195.18 AAAA 2600:9000:5303:1200::1 OPT 208 2020/231 17:46:11.055755 0.002029000 156.154.65.210 71.36.120.123 DNS Standard query response 0x0f00 A elb-ore-amz.nimbus.bitdefender.net CNAME kube-nimbus-471965604.us-west-2.elb.amazonaws.com OPT 209 2020/231 17:46:11.057944 0.002189000 64.4.48.1 71.36.120.123 DNS Standard query response 0x3e3f A ns2-34.azure-dns.net A 150.171.16.34 OPT 210 2020/231 17:46:11.059971 0.002027000 205.251.194.68 71.36.120.123 DNS Standard query response 0x3039 AAAA ns-38.awsdns-04.com AAAA 2600:9000:5300:2600::1 NS g-ns-1156.awsdns-04.com NS g-ns-1732.awsdns-04.com NS g-ns-5.awsdns-04.com NS g-ns-580.awsdns-04.com A 205.251.196.132 AAAA 2600:9000:5304:8400::1 A 205.251.198.196 AAAA 2600:9000:5306:c400::1 A 205.251.192.5 AAAA 2600:9000:5300:500::1 A 205.251.194.68 AAAA 2600:9000:5302:4400::1 OPT 211 2020/231 17:46:11.062155 0.002184000 2.22.230.67 71.36.120.123 DNS Standard query response 0x6d3c A a9-67.akam.net A 184.85.248.67 OPT 212 2020/231 17:46:11.062411 0.000256000 71.36.120.123 95.100.173.67 DNS Standard query 0xd1e4 AAAA a9-67.akam.net OPT 213 2020/231 17:46:11.064145 0.001734000 43.230.48.1 71.36.120.123 DNS Standard query response 0xa2b0 AAAA nsd.nic.uk SOA dns1.nic.uk OPT 214 2020/231 17:46:11.066017 0.001872000 74.125.250.87 71.36.120.123 UDP 19305 → 7162 Len=66 215 2020/231 17:46:11.066176 0.000159000 198.97.190.53 71.36.120.123 DNS Standard query response 0x09cf A a.gtld.biz NS a.gtld.biz NS b.gtld.biz NS c.gtld.biz NS e.gtld.biz NS f.gtld.biz NS k.gtld.biz DS DS RRSIG OPT 216 2020/231 17:46:11.067344 0.001168000 71.36.120.123 74.125.250.87 UDP 7162 → 19305 Len=42 217 2020/231 17:46:11.068301 0.000957000 216.252.166.10 71.36.120.123 DNS Standard query response 0xd535 A ib.adnxs.com CNAME g.geogslb.com NS ns1.gslb.com NS ns2.gslb.com 218 2020/231 17:46:11.068307 0.000006000 81.17.242.98 71.36.120.123 ICMP Time-to-live exceeded (Time to live exceeded in transit) -
Hmm, that sure starts to looks like an upstream routing issue.
What is sending that ICMP TTL exceeded response? What was the target?
Steve
-
@stephenw10 target was the google DNS server, 8.8.8.8
So far this week, the issue has manifested like clockwork almost every day between 10:50 and 11:15 AM. with one occasion where it also reoccurred near noon as well.
I've contacted my ISP and they beleive they saw some up line issues and have a tech coming out next week...
I'm getting very tiered of this issue very fast.
-
ISP replaced the ONT and I had been problem free until today when the behaviour appeared again..
I tried to do a tracert and every hop diddnt response and the last 8.8.8.8 had a response time of 1248ms
I was able to restore my connection by going to status>interfaces and then disconnecting and recconecting the WAN PPOE.
Could use some guidance on troubleshooting PPOE issues as well as reccomendations on a scripted workaround to automatically restart it if non responsive after a period of time.
-
You never said what in the route is sending TTL exceeded replies annd what the acrual message is. That's usually a sign there's a routing loop.
It looks like 81.17.242.98 and sending the replies back to 71.36.120.123 which I assume was your WAN IP at that time. What is 81.17.242.98 though? Something at your ISP?You can configure a PPPoE connection to reset at, say, 6am everyday. That will likely prevent this if it doesn't fail more often than that. Though it should not be required.
Steve
-
@stephenw10 said in WAN interface stops working every few days.:
You never said what in the route is sending TTL exceeded replies annd what the acrual message is. That's usually a sign there's a routing loop.
It looks like 81.17.242.98 and sending the replies back to 71.36.120.123 which I assume was your WAN IP at that time. What is 81.17.242.98 though? Something at your ISP?You can configure a PPPoE connection to reset at, say, 6am everyday. That will likely prevent this if it doesn't fail more often than that. Though it should not be required.
Steve
I'll have to grab that info the next time this behavior occurs, which specific info would I want to grab in this case?
Not sure on that specific AP, it was likely picking up traffic from some random device on my network.
Here's my config related to my PPOE wan if that helps any.
<wan> <if>pppoe0</if> <blockbogons></blockbogons> <descr><![CDATA[WAN01_CenturyLink]]></descr> <alias-address></alias-address> <alias-subnet>32</alias-subnet> <spoofmac></spoofmac> <blockpriv></blockpriv> <enable></enable> <ipaddr>pppoe</ipaddr> </wan> <vlan> <if>igb0</if> <tag>201</tag> <pcp></pcp> <descr><![CDATA[WAN_01_VLAN201]]></descr> <vlanif>igb0.201</vlanif> </vlan> <ppps> <ppp> <ptpid>0</ptpid> <type>pppoe</type> <if>pppoe0</if> <ports>igb0.201</ports> <username><![CDATA[REDACTED@centurylink.net]]></username> <password><![CDATA[REDACTED]]></password> <bandwidth></bandwidth> <mtu></mtu> <mru></mru> <mrru></mrru> </ppp> </ppps> <gateways> <gateway_item> <interface>wan</interface> <gateway>dynamic</gateway> <name>WAN01_CENTURYLINK_PPPOE</name> <weight>1</weight> <ipprotocol>inet</ipprotocol> <descr><![CDATA[Interface WAN01_CENTURYLINK_PPPOE Gateway]]></descr> <monitor>8.8.8.8</monitor> </gateway_item> <defaultgw4>WAN01_CENTURYLINK_PPPOE</defaultgw4> <defaultgw6>-</defaultgw6> </gateways> -
Nothing unusual there.
You can set a periodic reset as I said. You might try that to see if it does prevent the issue happening during the day.
Steve
-
@stephenw10 said in WAN interface stops working every few days.:
Nothing unusual there.
You can set a periodic reset as I said. You might try that to see if it does prevent the issue happening during the day.
Steve
It's unfortunately sometimes occurs more frequently then that. Last event was yesterday around ~1pm and it reoccured a short bit ago around 9:20am today.
I was not able to get the connection back this time by disconnecting and reconnecting the PPOE cconnection, ended up restarting PFsense.
Next step will likely be for me to disable snort for atleast a week or until the issue returns to see if the behaviour reappears.
I'm kind of grasping at straws right now though.....
------------ System logs from time period ---------
Aug 31 09:10:20 snort 67712 [1:2403428:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 65 [Classification: Misc Attack] [Priority: 2] {TCP} 80.82.77.227:33798 -> 71.36.122.177:443 Aug 31 09:10:57 snort 67712 [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 183.131.3.210:58864 -> 71.36.122.177:1433 Aug 31 09:11:25 snort 67712 [1:2403368:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 35 [Classification: Misc Attack] [Priority: 2] {TCP} 51.161.12.231:32767 -> 71.36.122.177:8545 Aug 31 09:13:13 snort 67712 [1:2403448:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 75 [Classification: Misc Attack] [Priority: 2] {TCP} 89.248.168.157:37856 -> 71.36.122.177:41065 Aug 31 09:14:38 snort 67712 [1:2403458:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 80 [Classification: Misc Attack] [Priority: 2] {TCP} 92.63.197.55:40327 -> 71.36.122.177:3377 Aug 31 09:15:07 snort 67712 [1:2403460:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 81 [Classification: Misc Attack] [Priority: 2] {TCP} 94.102.56.238:55872 -> 71.36.122.177:5900 Aug 31 09:16:09 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.20:57576 -> 71.36.122.177:3345 Aug 31 09:16:14 rc.gateway_alarm 27046 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:506.622ms RTTsd:787.570ms Loss:0%) Aug 31 09:16:14 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:16:14 check_reload_status Restarting ipsec tunnels Aug 31 09:16:14 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:16:14 check_reload_status Reloading filter Aug 31 09:16:15 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:16:15 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:17:07 snort 67712 [1:2403452:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 77 [Classification: Misc Attack] [Priority: 2] {TCP} 91.229.112.3:55957 -> 71.36.122.177:3310 Aug 31 09:17:07 snort 67712 [1:2403460:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 81 [Classification: Misc Attack] [Priority: 2] {TCP} 94.102.51.17:51800 -> 71.36.122.177:7291 Aug 31 09:17:07 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 94.102.51.17:51800 -> 71.36.122.177:7291 Aug 31 09:17:22 rc.gateway_alarm 11126 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:4120.023ms RTTsd:1799.455ms Loss:22%) Aug 31 09:17:22 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:17:22 check_reload_status Restarting ipsec tunnels Aug 31 09:17:22 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:17:22 check_reload_status Reloading filter Aug 31 09:17:23 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:17:23 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:17:27 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.142:45646 -> 71.36.122.177:17852 Aug 31 09:17:27 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.142:45646 -> 71.36.122.177:17852 Aug 31 09:17:35 rc.gateway_alarm 61503 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:3703.111ms RTTsd:2201.113ms Loss:11%) Aug 31 09:17:35 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:17:35 check_reload_status Restarting ipsec tunnels Aug 31 09:17:35 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:17:35 check_reload_status Reloading filter Aug 31 09:17:36 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:17:36 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:17:38 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.102:47924 -> 71.36.122.177:26098 Aug 31 09:18:31 snort 67712 [1:2403424:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 63 [Classification: Misc Attack] [Priority: 2] {TCP} 78.108.177.54:26525 -> 71.36.122.177:8080 Aug 31 09:18:32 rc.gateway_alarm 50465 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:0 RTT:310.577ms RTTsd:435.870ms Loss:0%) Aug 31 09:18:32 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:18:32 check_reload_status Restarting ipsec tunnels Aug 31 09:18:32 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:18:32 check_reload_status Reloading filter Aug 31 09:18:33 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:18:34 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:18:57 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 195.54.167.91:45181 -> 71.36.122.177:33355 Aug 31 09:19:52 snort 67712 [1:2403454:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 78 [Classification: Misc Attack] [Priority: 2] {TCP} 91.240.118.113:42826 -> 71.36.122.177:3391 Aug 31 09:20:03 snort 67712 [1:2400005:2773] ET DROP Spamhaus DROP Listed Traffic Inbound group 6 [Classification: Misc Attack] [Priority: 2] {TCP} 103.215.80.70:6000 -> 71.36.122.177:6780 Aug 31 09:20:44 snort 67712 [1:2403344:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 23 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.47:50206 -> 71.36.122.177:15573 Aug 31 09:20:44 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.47:50206 -> 71.36.122.177:15573 Aug 31 09:22:03 snort 67712 [1:2011716:4] ET SCAN Sipvicious User-Agent Detected (friendly-scanner) [Classification: Attempted Information Leak] [Priority: 2] {UDP} 193.203.14.202:5311 -> 71.36.122.177:5060 Aug 31 09:22:03 snort 67712 [1:2008578:6] ET SCAN Sipvicious Scan [Classification: Attempted Information Leak] [Priority: 2] {UDP} 193.203.14.202:5311 -> 71.36.122.177:5060 Aug 31 09:22:27 snort 67712 [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 103.48.25.131:63333 -> 71.36.122.177:1433 Aug 31 09:22:29 snort 67712 [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 103.48.25.131:63333 -> 71.36.122.177:1433 Aug 31 09:24:01 snort 67712 [1:2403452:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 77 [Classification: Misc Attack] [Priority: 2] {TCP} 91.229.112.4:55935 -> 71.36.122.177:835 Aug 31 09:24:26 snort 67712 [1:2403452:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 77 [Classification: Misc Attack] [Priority: 2] {TCP} 91.229.112.8:55838 -> 71.36.122.177:4004 Aug 31 09:26:21 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.24:43406 -> 71.36.122.177:22124 Aug 31 09:26:21 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.24:43406 -> 71.36.122.177:22124 Aug 31 09:27:05 snort 67712 [1:2403406:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 54 [Classification: Misc Attack] [Priority: 2] {TCP} 62.171.161.187:43973 -> 71.36.122.177:81 Aug 31 09:28:11 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.151:51260 -> 71.36.122.177:37606 Aug 31 09:28:11 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.151:51260 -> 71.36.122.177:37606 Aug 31 09:28:47 snort 67712 [1:2403429:59789] ET CINS Active Threat Intelligence Poor Reputation IP UDP group 65 [Classification: Misc Attack] [Priority: 2] {UDP} 80.82.77.212:48824 -> 71.36.122.177:49154 Aug 31 09:28:52 rc.gateway_alarm 69361 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:502.168ms RTTsd:986.015ms Loss:0%) Aug 31 09:28:52 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:28:52 check_reload_status Restarting ipsec tunnels Aug 31 09:28:52 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:28:52 check_reload_status Reloading filter Aug 31 09:28:53 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:28:53 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:28:56 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.53:57620 -> 71.36.122.177:6357 Aug 31 09:29:02 snort 67712 [1:2403344:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 23 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.46:52212 -> 71.36.122.177:15139 Aug 31 09:29:02 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.46:52212 -> 71.36.122.177:15139 Aug 31 09:29:12 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.141:45527 -> 71.36.122.177:17856 Aug 31 09:29:12 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.141:45527 -> 71.36.122.177:17856 Aug 31 09:29:44 snort 67712 [1:2403419:59789] ET CINS Active Threat Intelligence Poor Reputation IP UDP group 60 [Classification: Misc Attack] [Priority: 2] {UDP} 71.6.158.166:32064 -> 71.36.122.177:389 Aug 31 09:30:04 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.148:44932 -> 71.36.122.177:17867 Aug 31 09:30:04 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.148:44932 -> 71.36.122.177:17867 Aug 31 09:30:14 snort 67712 [1:2011716:4] ET SCAN Sipvicious User-Agent Detected (friendly-scanner) [Classification: Attempted Information Leak] [Priority: 2] {UDP} 51.89.217.179:5072 -> 71.36.122.177:5060 Aug 31 09:30:14 snort 67712 [1:2008578:6] ET SCAN Sipvicious Scan [Classification: Attempted Information Leak] [Priority: 2] {UDP} 51.89.217.179:5072 -> 71.36.122.177:5060 Aug 31 09:30:26 snort 67712 [1:2403452:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 77 [Classification: Misc Attack] [Priority: 2] {TCP} 91.229.112.11:48084 -> 71.36.122.177:10552 Aug 31 09:31:13 rc.gateway_alarm 93277 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:4050.647ms RTTsd:1954.397ms Loss:21%) Aug 31 09:31:13 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:31:13 check_reload_status Restarting ipsec tunnels Aug 31 09:31:13 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:31:13 check_reload_status Reloading filter Aug 31 09:31:14 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:31:14 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:31:23 rc.gateway_alarm 78618 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:4322.346ms RTTsd:1981.268ms Loss:14%) Aug 31 09:31:23 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:31:23 check_reload_status Restarting ipsec tunnels Aug 31 09:31:23 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:31:23 check_reload_status Reloading filter Aug 31 09:31:24 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:31:24 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:32:09 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 195.54.167.174:44528 -> 71.36.122.177:33339 Aug 31 09:32:41 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.143:44684 -> 71.36.122.177:17872 Aug 31 09:32:41 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.143:44684 -> 71.36.122.177:17872 Aug 31 09:32:58 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.12:41414 -> 71.36.122.177:62015 Aug 31 09:32:58 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.12:41414 -> 71.36.122.177:62015 Aug 31 09:33:17 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 195.54.167.94:45253 -> 71.36.122.177:33384 Aug 31 09:33:56 snort 67712 [1:2403431:59789] ET CINS Active Threat Intelligence Poor Reputation IP UDP group 66 [Classification: Misc Attack] [Priority: 2] {UDP} 80.82.77.245:44258 -> 71.36.122.177:120 Aug 31 09:34:18 snort 67712 [1:2403436:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 69 [Classification: Misc Attack] [Priority: 2] {TCP} 83.97.20.35:48991 -> 71.36.122.177:6664 Aug 31 09:34:28 snort 67712 [1:2403344:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 23 [Classification: Misc Attack] [Priority: 2] {TCP} 45.145.66.21:56468 -> 71.36.122.177:22979 Aug 31 09:35:11 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.230:40882 -> 71.36.122.177:3997 Aug 31 09:35:15 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 185.176.27.14:49426 -> 71.36.122.177:26187 Aug 31 09:35:25 snort 67712 [1:2403454:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 78 [Classification: Misc Attack] [Priority: 2] {TCP} 91.240.118.60:53196 -> 71.36.122.177:4184 Aug 31 09:35:38 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.10:57057 -> 71.36.122.177:27139 Aug 31 09:35:38 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.10:57057 -> 71.36.122.177:27139 Aug 31 09:36:18 snort 67712 [1:2010935:3] ET SCAN Suspicious inbound to MSSQL port 1433 [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 124.114.177.237:10566 -> 71.36.122.177:1433 Aug 31 09:36:35 snort 67712 [1:2403492:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 97 [Classification: Misc Attack] [Priority: 2] {TCP} 106.13.48.122:57394 -> 71.36.122.177:774 Aug 31 09:36:39 snort 67712 [1:2403344:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 23 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.5:42685 -> 71.36.122.177:5548 Aug 31 09:36:39 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.5:42685 -> 71.36.122.177:5548 Aug 31 09:36:59 snort 67712 [1:2403428:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 65 [Classification: Misc Attack] [Priority: 2] {TCP} 80.82.65.74:58855 -> 71.36.122.177:6000 Aug 31 09:37:09 snort 67712 [1:2403344:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 23 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.45:50080 -> 71.36.122.177:14956 Aug 31 09:37:09 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.45:50080 -> 71.36.122.177:14956 Aug 31 09:37:11 snort 67712 [1:2403344:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 23 [Classification: Misc Attack] [Priority: 2] {TCP} 45.145.66.22:56634 -> 71.36.122.177:33046 Aug 31 09:37:31 snort 67712 [1:2403342:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 22 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.15:56776 -> 71.36.122.177:3547 Aug 31 09:37:31 snort 67712 [1:2402000:5651] ET DROP Dshield Block Listed Source group 1 [Classification: Misc Attack] [Priority: 2] {TCP} 45.129.33.15:56776 -> 71.36.122.177:3547 Aug 31 09:37:33 rc.gateway_alarm 53811 >>> Gateway alarm: WAN01_CENTURYLINK_PPPOE (Addr:8.8.4.4 Alarm:1 RTT:4054.569ms RTTsd:2049.170ms Loss:21%) Aug 31 09:37:33 check_reload_status updating dyndns WAN01_CENTURYLINK_PPPOE Aug 31 09:37:33 check_reload_status Restarting ipsec tunnels Aug 31 09:37:33 check_reload_status Restarting OpenVPN tunnels/interfaces Aug 31 09:37:33 check_reload_status Reloading filter Aug 31 09:37:34 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN01_CENTURYLINK_PPPOE. Aug 31 09:37:34 php-fpm /rc.dyndns.update: phpDynDNS (): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry. Aug 31 09:37:48 snort 67712 [1:2403372:59789] ET CINS Active Threat Intelligence Poor Reputation IP TCP group 37 [Classification: Misc Attack] [Priority: 2] {TCP} 54.36.109.237:50023 -> 71.36.122.177:8443---------- Gateway logs from time period ------------------
Aug 30 13:32:43 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Clear latency 290791us stddev 369179us loss 0% Aug 31 09:16:14 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 506622us stddev 787570us loss 0% Aug 31 09:17:22 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 4120023us stddev 1799455us loss 22% Aug 31 09:17:35 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 3703111us stddev 2201113us loss 11% Aug 31 09:18:32 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Clear latency 310577us stddev 435870us loss 0% Aug 31 09:28:52 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 502168us stddev 986015us loss 0% Aug 31 09:31:13 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 4050647us stddev 1954397us loss 21% Aug 31 09:31:23 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 4322346us stddev 1981268us loss 14% Aug 31 09:37:33 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 4054569us stddev 2049170us loss 21% Aug 31 09:40:13 dpinger send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 8.8.4.4 bind_addr 97.120.6.183 identifier "WAN01_CENTURYLINK_PPPOE " Aug 31 09:40:30 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 507360us stddev 451625us loss 0% Aug 31 09:40:36 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 599186us stddev 671081us loss 22% Aug 31 09:40:46 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 1544978us stddev 1669473us loss 11% Aug 31 09:41:13 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 1609645us stddev 1562133us loss 21% Aug 31 09:41:18 dpinger send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 8.8.4.4 bind_addr 75.164.130.187 identifier "WAN01_CENTURYLINK_PPPOE " Aug 31 09:41:30 dpinger WAN01_CENTURYLINK_PPPOE 8.8.4.4: Alarm latency 589734us stddev 844410us loss 14%--- End logs----
I'll need to look closer at the PPP logs the next time this occurs, They were unfortunately flooded out when I restarted pfsense.
I've also been collecting data into Splunk, I'll need to go through that and set up filters when I have time today. -
Yeah the gateway logs look terrible. It's not failing on each of those events? Just very bad latency and/or packet loss?
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.