PPPoE WAN fails to reconnect after link loss



  • Hi,

    I'm using pfSense 2.4.3 with em Intel network interfaces and my WAN is PPPoE based. When the connection drops for any reason, pfSense does not reconnect until a full system reboot is done.

    The same WAN in a domestic router (e.g. TP-Link) reconnects just fine. I found a similar issue on Redmine but it is from six years ago.

    Observing the logs, I could not understand why this is happening.

    Here's two logs:
    Log #1: PPPoE connects OK in the system startup
    Log #2: PPPoE fails to reconnect indefinitely
    (I've changed the IPs to $_WAN_PUBLIC_IP_$ and $_ISP_GATEWAY_$ on both logs)

    Could anyone help me on this issue?

    Thanks!


  • Netgate Administrator

    I assume you have redacted the IPs from those logs?
    That does mean we can't assess whether or not they are reasonable.

    This line appears to be key [wan] IFACE: Adding IPv4 address to pppoe0 failed

    Does the IP it's trying to add conflict? The previous connection still exist somehow maybe?

    Do you see any other errors in the system log at that point?

    Steve



  • @stephenw10

    Hi Steve, thanks for your time.

    Yes, I've redacted the IPs replacing them with $_WAN_PUBLIC_IP_$ where my public IP was logged and $_ISP_GATEWAY_$ for my ISP's gateway. Please note that those two IPs are the only ones logged and they stay the same on both logs (#1 and #2).

    About the conflict, I think that is possible. Since the ISP is giving the same public IP to this PPPoE every time. Here's something I found in the system log:

    Jun 28 15:28:17     ppp         [wan] IPCP: rec'd Configure Request #2 (Ack-Sent)
    Jun 28 15:28:17     ppp         [wan] IPADDR $_ISP_GATEWAY_$
    Jun 28 15:28:17     ppp         [wan] $_ISP_GATEWAY_$ is OK
    Jun 28 15:28:17     ppp         [wan] IPCP: SendConfigAck #2
    Jun 28 15:28:17     ppp         [wan] IPADDR $_ISP_GATEWAY_$
    Jun 28 15:28:17     ppp         [wan] IPCP: rec'd Configure Ack #7 (Ack-Sent)
    Jun 28 15:28:17     ppp         [wan] IPADDR $_WAN_PUBLIC_IP_$
    Jun 28 15:28:17     ppp         [wan] PRIDNS 8.8.8.8
    Jun 28 15:28:17     ppp         [wan] SECDNS 8.8.4.4
    Jun 28 15:28:17     ppp         [wan] IPCP: state change Ack-Sent --> Opened
    Jun 28 15:28:17     ppp         [wan] IPCP: LayerUp
    Jun 28 15:28:17     ppp         [wan] $_WAN_PUBLIC_IP_$ -> $_ISP_GATEWAY_$
    Jun 28 15:28:17     ppp         [wan] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists
    Jun 28 15:28:17     check_reload_status         Rewriting resolv.conf
    Jun 28 15:28:18     php-fpm     61165   /rc.newwanipv6: rc.newwanipv6: Info: starting on pppoe0.
    Jun 28 15:28:18     php-fpm     61165   /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
    Jun 28 15:28:18     check_reload_status         rc.newwanip starting pppoe0
    Jun 28 15:28:19     php-fpm     3065    /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
    Jun 28 15:28:19     php-fpm     3065    /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: pppoe0).
    Jun 28 15:28:19     php-fpm     3065    /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting...
    Jun 28 15:28:19     check_reload_status         Configuring interface wan
    Jun 28 15:28:20     ppp         Multi-link PPP daemon for FreeBSD
    Jun 28 15:28:20     ppp         process 95029 started, version 5.8 (nobody@pfSense_v2_4_3_amd64-pfSense_v2_4_3-job-08 16:18 16-Mar-2018)
    Jun 28 15:28:20     ppp         waiting for process 13192 to die...
    Jun 28 15:28:20     ppp         caught fatal signal TERM
    Jun 28 15:28:20     ppp         [wan] IFACE: Close event
    Jun 28 15:28:20     ppp         [wan] IPCP: Close event
    Jun 28 15:28:20     ppp         [wan] IPCP: state change Opened --> Closing
    Jun 28 15:28:20     ppp         [wan] IPCP: SendTerminateReq #8
    Jun 28 15:28:20     ppp         [wan] IPCP: LayerDown
    Jun 28 15:28:20     check_reload_status         Rewriting resolv.conf
    Jun 28 15:28:20     ppp         [wan] IFACE: Removing IPv4 address from pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): Can't assign requested address
    

    Does this help at all? This keeps repeating until a full system reboot.

    Thanks a lot!


  • Netgate Administrator

    That just looks like the ppp output that's logged in both places.

    Hmm, this appears to be something specific to your ISP. I have PPPoE WANs and don't see this. Perhaps some timing issue.

    Is that intertelecom.ua ? Someone else might have hit it.

    Steve



  • @stephenw10 It's not intertelecom.ua. We're using a local FTTH ISP.

    The weird thing is that the issue only happens with pfSense. Routers with DD-WRT or OpenWRT can reconnect just fine.

    Is it possible to set a delay between reconnection attempts?


  • Netgate Administrator

    Are you spoofing the WAN MAC? What NIC driver is the WAN using?

    Steve



  • @stephenw10

    Are you spoofing the WAN MAC?

    Yes! Could it be the cause?

    What NIC driver is the WAN using?

    It's a Intel PRO/1000, using the em(4) drivers.

    Thanks!


  • Netgate Administrator

    I did see a similar report that spoofing the MAC broke PPPoE in some cases.
    I wouldn't expect that to be required for PPPoE, can you test it without spoofing?

    Also where are you spoofing it? On the PPPoE interface or on the parent interface?
    I could imagine an issue passing that through to the parent perhaps.

    Steve



  • @stephenw10

    I wouldn't expect that to be required for PPPoE, can you test it without spoofing?

    I can but it's not easy... Probably next weekend - needs to be in office off-hours and my ISP needs to authorize the correct MAC addr. I'll do and return with feedback.

    Also where are you spoofing it? On the PPPoE interface or on the parent interface?

    I'm spoofing directly on the physical interface (em0). I don't see the option to spoof on the pppoe0 interface.

    Thanks a lot!


  • Netgate Administrator

    Great. If that does work we can open a bug report to look into it.

    Interesting though, I see the MAC spoof option here on PPPoE interfaces in 2.4.3p1:

    0_1531062814856_Selection_438.png

    Steve



  • @stephenw10 Sorry for the delay. Unfortunately, the problem persists.

    Here's what I've done since then:

    • Updated to 2.4.3p1;
    • Fixed the authorized MAC address with my ISP. There's no spoofing anymore;
    • Recreated the PPPoE interface from scratch;
    • Tried another network port in the same machine;
    • Used PPPoE with the same ISP but different layer 1 topology. Reconnection works fine when using their wireless uplink. The problem only happens on the FTTH connection;

    Is there anything else I can do?

    Thanks!


  • Netgate Administrator

    Hmm, the logs still show the same thing? Can we see a current PPP log showing the failure?

    Steve



  • @stephenw10 Here's the updated logs. For safety, I've replaced the WAN IP with $_WAN_PUBLIC_IP_$ and my ISP's gateway with $_ISP_GATEWAY_$.

    Log #1 - Connection works after system restart

    Log #2 - Fails to reconnect if the connection dropped for any reason

    This line keeps appearing:

    IFACE: Adding IPv4 address to pppoe2 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists
    

    Thanks!



  • Hello, did you find out what was the problem ? Because I have the exact same problem with the exact same errors after upgrading to 2.4.4 .

    [opt1] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists 
    
    [opt1] IFACE: Removing IPv4 address from pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): Can't assign requested address 
    

    Tried everything, nothing helps. The connection works on VyOS and directly on the PC.



  • @joxxxx I have yet to find a solution. For me, the (re)connection works fine on other OSes and devices as well.

    Let me know you find out anything. Unfortunately I'm considering migrating to something else.



  • @vegbrasil After countless hours, I found out that the pppoe layer of pfsense 2.4.4 is bugged ! Something changed and because of that, the connection cannot be made with the above errors. Tried it also on a clean 2.4.4 install.

    Re-installed 2.4.3 and all works again without any problems ! The thing is that I cannot install any packages anymore because it wants me to update to 2.4.4...

    Btw, tried the dev 2.4.5 version and the bug is still the same.

    I hope this will be fixed because it mus affect a lot of people...


  • Netgate Administrator

    Hmm, there must be something different about your connection. I have two WANs here, both are PPPoE. The only issue I saw was when the ISP(s) had an outage. However the logs showed it as very broken, nothing like what you're seeing.
    If PPPoE was broken in general there would be hundreds of posts about it by now.

    Are you able to try a different NIC type?

    Steve



  • @stephenw10 I've been using pfSense with the same FTTH PPPoE connection and two different NICs models:

    • Realtek NICs (re): crazy bugs, completely unusable - expected, I guess;
    • Intel PRO/1000 NICs (em): Problem of the initial post: first connection is fine, re-connection never happens. If we forget this problem, everything is working great and I can max-out the connection (150 Mbps).

    Upgraded and tried clean install between 2.3.5 and all 2.4.x versions.

    Thanks!


  • Netgate Administrator

    Hmm, yes, this seems like it must be some other issue since you were seeing it in 2.4.3 also.

    Steve



  • I'm using a quad ethernet intel pro and never got any problems since v 2.0 or something like that on vmware and the vmx3 driver.

    After analysing a bit the problem, could it be that I have this problem because pfsense sees his local ip address as the FTTH gateway ?
    Maybe something changed in pf 2.4.4 in the pppoe connection that forbids the use of it ? (This could explain the two errors I get).



  • @joxxxx said in PPPoE WAN fails to reconnect after link loss:

    I'm using a quad ethernet intel pro

    I'm using the exactly same hardware and since we're having the same problem, we may have found the initial cause.



  • @stephenw10 On my side, 2.4.3 is working perfectly and reaching the 1Gbits of the fiber connection. The problem is 2.4.4 and above where it doesn't want to allocate the ip address it gets (I can see the received ip address in the logs, but the error prevents it to be used).



  • How did you get anywhere with this problem?

    I'm experiencing the same issue:

    Could it be the PPPoE is not receiving a gateway address and the IP address its been given is a /32 255.255.255.255 address

    This is the case with mine.

    I started on 2.4.3 upgraded to 2.4.4 (this causes the WAN to drop and it attempt another connection - 2.4.3 got this error and left the link up)

    Feb 4 12:22:20 ppp [wan] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): Destination address required
    Feb 4 12:22:20 ppp [wan] 118.xxx.xxx.xxx -> 0.0.0.0
    Feb 4 12:22:20 ppp [wan] IPCP: LayerUp
    Feb 4 12:22:20 ppp [wan] IPCP: state change Ack-Sent --> Opened
    Feb 4 12:22:20 ppp [wan] IPADDR 118.xxx.xxx.xxx



  • @timboau-0 I still have the issue, running the latest stable version of pfSense.

    This weekend I needed to visit a site that was offline due this reconnection problem. I'm thinking of making some sort of simple script to reboot the server if the WAN is offline for a few minutes...

    Thanks!



  • This post is deleted!


  • I'll add that im using APU intel nic no VM
    Tested clean install of 2.3.1 & 2.4.4

    2.3.1 has some additional information:
    Feb 4 02:35:58 ppp [wan] IPCP: LayerUp
    Feb 4 02:35:58 ppp [wan] 118.xxx.xxx.xxx -> 0.0.0.0
    Feb 4 02:35:58 ppp [wan] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): Destination address required
    Feb 4 02:35:58 check_reload_status Rewriting resolv.conf
    Feb 4 02:35:58 php-fpm 19680 /rc.interfaces_wan_configure: Accept router advertisements on interface igb0
    Feb 4 02:35:59 check_reload_status updating dyndns wan
    Feb 4 02:35:59 check_reload_status rc.newwanip starting pppoe0
    Feb 4 02:35:59 ppp [wan] IFACE: Up event
    Feb 4 02:35:59 ppp [wan] IFACE: Rename interface ng0 to pppoe0
    Feb 4 02:36:00 php-fpm 34057 /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
    Feb 4 02:36:00 php-fpm 34057 /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: pppoe0).
    Feb 4 02:36:00 php-fpm 34057 /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting...
    Feb 4 02:36:00 check_reload_status Configuring interface wan
    Feb 4 02:36:01 check_reload_status Reloading filter
    Feb 4 02:36:01 php-fpm 16228 /interfaces.php: Creating rrd update script
    Feb 4 02:36:02 xinetd 9811 Starting reconfiguration
    Feb 4 02:36:02 xinetd 9811 Swapping defaults
    Feb 4 02:36:02 xinetd 9811 readjusting service 6969-udp
    Feb 4 02:36:02 xinetd 9811 Reconfigured: new=0 old=1 dropped=0 (services)
    Feb 4 02:36:05 php-fpm 23933 /rc.interfaces_wan_configure: Accept router advertisements on interface igb0


  • Netgate Administrator

    /32 is valid for a PPP connection as is a gateway outside the subnet.

    Obviously no gateway at all isn't though.

    Steve



  • I got my config working on 2.4.3 (but I think it will break on 2.4.4) - I had a slightly different problem to the above.

    With 2.4.3 I can authenticate and the interface stays up
    2.4.4 I authenticate and the interface cant be configured as it doesnt have a gateway IP so it goes down and then re-connects. (repeat)

    I fixed my problem : (Not being supplied a gateway via PPPoE)
    Configuring the interface with PPPoE (as per usual)
    Then creating a NEW interface on the same port as STATIC (save)
    Then open the interface again and create a gateway using that same IP

    I can see that if I lost authentication now on 2.4.3 I would likely loose connectivity as the PPPoE wont try again (without a reboot or port cycle)
    I worry on 2.4.4 that it will fail the interface configuration due to the missing gateway then the 2nd interface wont work either..


  • Netgate Administrator

    Hmm, try it and see. Odd setup.

    Steve


Log in to reply