PPPoE - Connection drops - needs full reboot to bring it back up

breakaway

I've got a box running pfSense 2.4.4-release.

This box has been running trouble free for about 4 years now until about last week when it randomly stopped working from time to time. It seems that PPPoE just stops. The internet connection is delivered over fiber with a media converter from the ISP to a copper RJ45, then we plug our pfsense directly into it and setup a sub-interface on VLAN10 and do PPPoE over this interface.

So far I have done the following

Replace pfSense hardware thinking its failing
Replace all power calbes
Replace all patch leads with new ones

Remote end (ISP side) just shows PPP drop, nothing worthwhile. Here's system log. The outage happened roughly 08:34 local time. By 08:39 I talked someone through rebooting the gear on-site and it's back up. Note that log entires are newest-on-top.

Any ideas on whats happening here?

https://pastebin.com/YrDZ5vi9

NinoM4ster

Hello, Breakaway!

Something very similar happened with me a few months ago. The PPPoE just wouldn't connect. I checked my cables and everything, but no luck.

I called my ISP and they asked me to get a notebook and connect it directly to the Fiber Modem/Media Converter and try authenticating through Windows, and it also wasn't connecting (by this time I still didn't have this pfSense box, I was using a Mikrotik RouterBoard), so they determined that the issue was probably on the Fiber between the light pole and my house, so they sent two technicians.

And sure enough, after they replaced both ends (one of them even twice), it worked like a charm, and is working until now. (BTW, the connection was doing OK for almost a year, then this happened. a bad connector. and no, I haven't touched it.)

So, try giving them a call and ask them to test the signal strength on your end, and see if it helps. Also, before that, you can try using a notebook or a computer to authenticate directly on the Media Converter, as I did (you'll need the PPPoE credentials).

I wish you a good luck!

breakaway

Hi henrique, I have logged a job with the upstream provider now. I also suspect the fault is with either the fiber or their GPON converter (they don't let us plug directly into their network, they actually have a GPON modem/converter device that the fiber plugs into then our pfSense plugs into that.

My issue isn't pfSense not connecting but rather randomly dropping. At first I thought it happens under load but the connection was fine all night (night time is way more load because of off-site backups) but then dropped at about 8 am as the first people were trickling into our office.

Let's see what they come back with.

NinoM4ster

Honestly, I don't know if the bad connector was the cause, but for quite some time before it completely broke, I also had a lot of (PPPoE) connection drops.. sometimes even twice a day, and taking up to 30 minutes to come back, but some of it could just have been an unexpected maintenance on their side, or bad configuration at mine (since I was still learning how to properly set up the Mikrotik), I don't know.

So, yeah, that may be it. =)

Keep me updated!

breakaway

On Friday, I installed the Realtek v1.95 driver referenced in this thread. https://forum.netgate.com/topic/135850/official-realtek-driver-binary-1-95-for-2-4-4-release/13

The connection has been up for almost 48 hours now. I’m guessing that this issue was that the Realtek drivers built into pfSense just don’t work properly.

stephenw10

It will be interesting to see if that allows it to come up when it does go down for whatever reason.
Though you would hope that has removed the previous cause of it going down in the first place.

Steve

breakaway

The ISP claimed the RX & TX power is looking good from their end. It may be that there is nothing wrong with their equipment, it could just be idiosyncrasies in the realtek driver.

This only appears to impact a specific use case, when doing PPPoE interface on top of a VLAN interface. I have other connections that are DHCP or Static IP - those are all fine.

ha11oga11o

Sorry to bumping this topic up guys, but i have same issue since i changed ISP.
Connection randomly dissapears, dpinger service stops (cant be restarted), WAN traffic graph dissapears from main GUI page.

Anly way to reconnect is to reboot router.

pfSense is actual virtual machine on Proxmox.

BIOS Vendor: SeaBIOS
Version: rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org
Release Date: Tue Apr 1 2014

Version 2.4.5-RELEASE-p1 (amd64)
built on Tue Jun 02 17:51:17 EDT 2020
FreeBSD 11.3-STABLE

I see quite few similar issues across forum due long period of time. All of them is reboot router is cure... Im wondering, since i haven't found solution, can someone point me to right direction?

Many thnx in advance.

Gertjan

@ha11oga11o said in PPPoE - Connection drops - needs full reboot to bring it back up:

can someone point me to right direction?

No one has access to the logs that shows what and what happens.
The easy solution : go back to the old ISP.
Or : adapt your pppoe settings according the new ISP.
Btw : what ISP ? Mentioning the name would really help. If not, I can detail you how we manage our ISP here on the Nord pole.

ha11oga11o

Heres what log says,... its flooded with it:

https://pastebin.com/NeUvw6G6

I had to move so i had do use other ISP, old one was cable provider, this one is cr***ap. No acces to anything else,... sadly.

Thanks.

Gertjan

@ha11oga11o said in PPPoE - Connection drops - needs full reboot to bring it back up:

Heres what log says,... its flooded with it:
https://pastebin.com/NeUvw6G6

The logs says : no answer what so ever from the other side.

Is this true :

needs full reboot to bring it back up

?

Does your connection work after a reboot ?
Show also a log during a successful connect.

stephenw10

Yeah, nothing useful in that log. We need to see what happens when it disconnects. Does the remote side just stop responding it does it send a disconnect for example?

From those logs it looks like nothing is coming back at all from the remote side, as though the WAN isn't even connected. What is the modem status when this happens?

Steve

ha11oga11o

Thanks guys for kind interest into this. It just hapens random. I have LAN, VPN and WAN graph on dashboard... when that happens WAN graph disapear, but at config i still see it.

Gateways says WAN unknown after i manually reboot apinger servise. Ill try to catch log before reboot as bigger i can, and after reboot too. IP changes every 24 hrs so i dont matter if it was there too.

And yes,... after reboot all works fine. Graph come back, ping goes to 8.8.8.8 etc. Its like nothing happened.

stephenw10

Reboot pfSense or the modem? Or both?

ha11oga11o

Just pfSense reboot. I tried with modem, but got no results.

Heres my log from cpl last days. At about line 467 is reboot sequence which i did from gui.

https://pastebin.com/b9CuR43u

Thanks.

BTW, ISP is one of worst in mu country its called HT Eronet. From Bosna i Hercegovina. Im not sure it means anything since i doubt they have any useful public tech data. Tech support is synonym for "dont know", "cant be done", "no at all", etc etc. They needed 3 days and 5 tech guys on site just to bridge router, since they dont allow me to have admin access to it.You got the point.

Thanks again.

stephenw10

Yeah you can see the remote end just stops responding:

Feb 15 07:50:15 	ppp 		[wan_link0] LCP: state change Opened --> Stopping
Feb 15 07:50:15 	ppp 		[wan_link0] LCP: peer not responding to echo requests
Feb 15 07:50:15 	ppp 		[wan_link0] LCP: no reply to 5 echo request(s)
Feb 15 07:50:05 	ppp 		[wan_link0] LCP: no reply to 4 echo request(s)
Feb 15 07:49:55 	ppp 		[wan_link0] LCP: no reply to 3 echo request(s)
Feb 15 07:49:45 	ppp 		[wan_link0] LCP: no reply to 2 echo request(s)
Feb 15 07:49:35 	ppp 		[wan_link0] LCP: no reply to 1 echo request(s)

I would probably run a packet capture to be sure it's actually sending that traffic over the expected interface when that happens.

Steve

ha11oga11o

@stephenw10

I think actual link from ISP stop at some point. Since outages are known issue. But it looks pfSense does not reconnect back unless rebooted.

Last month i had 5 same issues. ISP sending also IPTV via same link. So, TV works, just PPPoE connection dont. So, ISP link is actually UP.

Every single time, pfSense reboot establish connection.

Ill try to explain topology here:

ISP cable come to their Modem/router which is in bridge so i can dial PPPoE. But their IPTV goes directly from their Modem/router via UTP cable to tvbox.

And that works. When internet really go out (had that isse twice so far), IPTV also stop working. And when they fix it everything works fine, including pfSense.

Gertjan

@ha11oga11o

Next time when pppoe doesn't reconnect, try to force an link down and up by taking out the ethernet 'WAN' cable, and re connect it, instead of rebooting pfSense.

ha11oga11o

Hello all,

first to thank you for kind assistance.

Second,... it looks all this mess was my fault. In some reason i had additional PPP interface configured but without account data in it. It looks when my ISP force IP change it tries to reconnect at pppoe1 instead pppoe0 - pppoe1 was ghosted so no reconnect it seems.

When i deleted it, it looks my problems stop there.

Many thnx for all, and please forgive me for my lack of knowledge.

stephenw10

Hmm, I would not expect that to happen but I'm glad you found it and were able to resolve the problem.

Steve