PPPoE WAN fails to reconnect after link loss



  • Hi,

    I'm using pfSense 2.4.3 with em Intel network interfaces and my WAN is PPPoE based. When the connection drops for any reason, pfSense does not reconnect until a full system reboot is done.

    The same WAN in a domestic router (e.g. TP-Link) reconnects just fine. I found a similar issue on Redmine but it is from six years ago.

    Observing the logs, I could not understand why this is happening.

    Here's two logs:
    Log #1: PPPoE connects OK in the system startup
    Log #2: PPPoE fails to reconnect indefinitely
    (I've changed the IPs to $_WAN_PUBLIC_IP_$ and $_ISP_GATEWAY_$ on both logs)

    Could anyone help me on this issue?

    Thanks!


  • Netgate Administrator

    I assume you have redacted the IPs from those logs?
    That does mean we can't assess whether or not they are reasonable.

    This line appears to be key [wan] IFACE: Adding IPv4 address to pppoe0 failed

    Does the IP it's trying to add conflict? The previous connection still exist somehow maybe?

    Do you see any other errors in the system log at that point?

    Steve



  • @stephenw10

    Hi Steve, thanks for your time.

    Yes, I've redacted the IPs replacing them with $_WAN_PUBLIC_IP_$ where my public IP was logged and $_ISP_GATEWAY_$ for my ISP's gateway. Please note that those two IPs are the only ones logged and they stay the same on both logs (#1 and #2).

    About the conflict, I think that is possible. Since the ISP is giving the same public IP to this PPPoE every time. Here's something I found in the system log:

    Jun 28 15:28:17     ppp         [wan] IPCP: rec'd Configure Request #2 (Ack-Sent)
    Jun 28 15:28:17     ppp         [wan] IPADDR $_ISP_GATEWAY_$
    Jun 28 15:28:17     ppp         [wan] $_ISP_GATEWAY_$ is OK
    Jun 28 15:28:17     ppp         [wan] IPCP: SendConfigAck #2
    Jun 28 15:28:17     ppp         [wan] IPADDR $_ISP_GATEWAY_$
    Jun 28 15:28:17     ppp         [wan] IPCP: rec'd Configure Ack #7 (Ack-Sent)
    Jun 28 15:28:17     ppp         [wan] IPADDR $_WAN_PUBLIC_IP_$
    Jun 28 15:28:17     ppp         [wan] PRIDNS 8.8.8.8
    Jun 28 15:28:17     ppp         [wan] SECDNS 8.8.4.4
    Jun 28 15:28:17     ppp         [wan] IPCP: state change Ack-Sent --> Opened
    Jun 28 15:28:17     ppp         [wan] IPCP: LayerUp
    Jun 28 15:28:17     ppp         [wan] $_WAN_PUBLIC_IP_$ -> $_ISP_GATEWAY_$
    Jun 28 15:28:17     ppp         [wan] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists
    Jun 28 15:28:17     check_reload_status         Rewriting resolv.conf
    Jun 28 15:28:18     php-fpm     61165   /rc.newwanipv6: rc.newwanipv6: Info: starting on pppoe0.
    Jun 28 15:28:18     php-fpm     61165   /rc.newwanipv6: rc.newwanipv6: No IPv6 address found for interface WAN [wan].
    Jun 28 15:28:18     check_reload_status         rc.newwanip starting pppoe0
    Jun 28 15:28:19     php-fpm     3065    /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
    Jun 28 15:28:19     php-fpm     3065    /rc.newwanip: rc.newwanip: on (IP address: ) (interface: WAN[wan]) (real interface: pppoe0).
    Jun 28 15:28:19     php-fpm     3065    /rc.newwanip: rc.newwanip: Failed to update wan IP, restarting...
    Jun 28 15:28:19     check_reload_status         Configuring interface wan
    Jun 28 15:28:20     ppp         Multi-link PPP daemon for FreeBSD
    Jun 28 15:28:20     ppp         process 95029 started, version 5.8 (nobody@pfSense_v2_4_3_amd64-pfSense_v2_4_3-job-08 16:18 16-Mar-2018)
    Jun 28 15:28:20     ppp         waiting for process 13192 to die...
    Jun 28 15:28:20     ppp         caught fatal signal TERM
    Jun 28 15:28:20     ppp         [wan] IFACE: Close event
    Jun 28 15:28:20     ppp         [wan] IPCP: Close event
    Jun 28 15:28:20     ppp         [wan] IPCP: state change Opened --> Closing
    Jun 28 15:28:20     ppp         [wan] IPCP: SendTerminateReq #8
    Jun 28 15:28:20     ppp         [wan] IPCP: LayerDown
    Jun 28 15:28:20     check_reload_status         Rewriting resolv.conf
    Jun 28 15:28:20     ppp         [wan] IFACE: Removing IPv4 address from pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): Can't assign requested address
    

    Does this help at all? This keeps repeating until a full system reboot.

    Thanks a lot!


  • Netgate Administrator

    That just looks like the ppp output that's logged in both places.

    Hmm, this appears to be something specific to your ISP. I have PPPoE WANs and don't see this. Perhaps some timing issue.

    Is that intertelecom.ua ? Someone else might have hit it.

    Steve



  • @stephenw10 It's not intertelecom.ua. We're using a local FTTH ISP.

    The weird thing is that the issue only happens with pfSense. Routers with DD-WRT or OpenWRT can reconnect just fine.

    Is it possible to set a delay between reconnection attempts?


  • Netgate Administrator

    Are you spoofing the WAN MAC? What NIC driver is the WAN using?

    Steve



  • @stephenw10

    Are you spoofing the WAN MAC?

    Yes! Could it be the cause?

    What NIC driver is the WAN using?

    It's a Intel PRO/1000, using the em(4) drivers.

    Thanks!


  • Netgate Administrator

    I did see a similar report that spoofing the MAC broke PPPoE in some cases.
    I wouldn't expect that to be required for PPPoE, can you test it without spoofing?

    Also where are you spoofing it? On the PPPoE interface or on the parent interface?
    I could imagine an issue passing that through to the parent perhaps.

    Steve



  • @stephenw10

    I wouldn't expect that to be required for PPPoE, can you test it without spoofing?

    I can but it's not easy... Probably next weekend - needs to be in office off-hours and my ISP needs to authorize the correct MAC addr. I'll do and return with feedback.

    Also where are you spoofing it? On the PPPoE interface or on the parent interface?

    I'm spoofing directly on the physical interface (em0). I don't see the option to spoof on the pppoe0 interface.

    Thanks a lot!


  • Netgate Administrator

    Great. If that does work we can open a bug report to look into it.

    Interesting though, I see the MAC spoof option here on PPPoE interfaces in 2.4.3p1:

    0_1531062814856_Selection_438.png

    Steve



  • @stephenw10 Sorry for the delay. Unfortunately, the problem persists.

    Here's what I've done since then:

    • Updated to 2.4.3p1;
    • Fixed the authorized MAC address with my ISP. There's no spoofing anymore;
    • Recreated the PPPoE interface from scratch;
    • Tried another network port in the same machine;
    • Used PPPoE with the same ISP but different layer 1 topology. Reconnection works fine when using their wireless uplink. The problem only happens on the FTTH connection;

    Is there anything else I can do?

    Thanks!


  • Netgate Administrator

    Hmm, the logs still show the same thing? Can we see a current PPP log showing the failure?

    Steve



  • @stephenw10 Here's the updated logs. For safety, I've replaced the WAN IP with $_WAN_PUBLIC_IP_$ and my ISP's gateway with $_ISP_GATEWAY_$.

    Log #1 - Connection works after system restart

    Log #2 - Fails to reconnect if the connection dropped for any reason

    This line keeps appearing:

    IFACE: Adding IPv4 address to pppoe2 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists
    

    Thanks!