New PPPoE backend, some feedback

w0w

Possible that another problem found and maybe solved :)
I want to note that the problem shows only with the new if_pppoe; with mpd5 all VIP addresses come up successfully.
I tried to study why on my HA primary sometimes the system boots with VIP addresses on LAN not initialized. It looks like we have again a race condition.

Even if PPPoE is configured at the end using the function

/* setup PPPoE and PPTP (this will bring up if_pppoe) -edited by me */
vpn_setup();

called from /etc/rc.bootup, in my case interfaces often do not have time to get all VIPs. I have many ports, they flap several times (down/up), and this coincides with the moment when PPPoE brings the link up successfully. At this moment it seems the VIP configuration stays unfinished. From experiments I noticed usually only one VIP gets configurated.

I am not sure if this is the correct solution, but I added into /etc/rc.bootup the line

interfaces_vips_configure("");

before

/* setup PPPoE and PPTP (this will bring up if_pppoe) */
vpn_setup();

This tends system to wait interfaces_vips_configure is completed.
After that it looks OK. All VIPs are present and CARP works too.

azalea

When I checked "Use if_pppoe kernel module for PPPoE client" to enable if_pppoe, this fell under case (b). As a result, the ping failed.

I have a correction. When I checked again, I found that, as in the case (a), pppoe0 had linked up and the IPv6 address had been set before "Bootup complete". (ping passed)

I am prompted to reboot after checking "Use if_pppoe kernel module for PPPoE client" to enable if_pppoe. I canceled here and checked the WAN Interfaces Status, and found that the PPPoE session had been automatically closed. After that I rebooted manually and the result was the same as in case (a). The General log also recorded "pppoe0: link state changed to DOWN" just before the reboot began.

Do IPv4 pings (general connectivity) succeed in both cases? Just varies the IPv6 response?

IPv4 ping passes in both cases. Ping results differ for IPv6 only.

So you also see failures in IPv6 when using mpd5/netgraph?

(c) If I reboot pfSense with manually closing the PPPoE session, pppoe0 links up and the IPv6 address is set before "Bootup complete".
(d) If I reboot pfSense without manually closing the PPPoE session, pppoe0 links up and the IPv6 address is set before "Bootup complete".
(e) If I hard reset pfSense without manually closing the PPPoE session, pppoe0 links up and the IPv6 address is set after "Bootup complete".

In case (c), the ping passes.
In case (d), Ping passes most of the time. In rare cases, ping fails because dpinger is stopped. After I transition from step 1 to step 3, ping passes.
In case (e), the ping fails. After I transition from step 1 to step 3, the ping fails.

[step 1]

Gateway address is displayed.
Monitor address is not displayed.
Gateway status is Pending.
"(default)" under "WAN_DHCP6" is not displayed.
Gateway IPv6 is displayed.
dpinger is stopped.
Ping fails.

[step 2]
I start dpinger.

Monitor address is displayed.
Gateway status is Online.
"(default)" under "WAN_DHCP6" is not displayed.
Ping fails.

[step 3]
I save the Default gateway configuration (Automatic) without any changes.

"(default)" under "WAN_DHCP6" is displayed.
In case (d), the ping passes.
In case (e), the ping fails.

(d) is a surprising result. In the case of mpd5, pfSense may automatically close the PPPoE session before rebooting. If so, I assume that "automatically closing of PPPoE sessions before rebooting" would be an effective solution for if_pppoe as well.

Regarding (e), I assumed that mpd5 had automatically closed the PPPoE session before rebooting, so I tried a hard reset to avoid this.

From the ping results for (b) and (e), it seems that ping fails when the IPv6 address is set after "Bootup complete" whether using mpd5 or if_pppoe.

When I look at (d), in the case of mdp5, the Gateway/dpinger status and the ping result change in tandem with the transition from step 1 to step 3. This is natural. On the other hand, when I look at (a), in the case of if_pppoe, the Gateway/dpinger status and the ping result do not have the correct relationship. This is strange.

azalea

In addition to the ping issue, there is also an issue where the Gateway address (Gateway IPv6) is not set.

Am I the only one for whom the Gateway address (Gateway IPv6) is not set when using if_pppoe? If so, I assume it's due to the uniqueness of setting only IPv6 for one PPPoE session.

Specifically, if_pppoe assumes that IPv4 is configured. However, since there is no IPv4 configuration, if_pppoe cannot set the IPv4 Gateway (WAN_PPPOE). It is determined that an error has occurred in the IPv4 Gateway setting, and the IPv6 Gateway (WAN_DHCP6) setting is canceled. Is this guess correct?

louis2

@stephenw10 said in New PPPoE backend, some feedback:

ifconfig pppoe0 -debug

Stephen, it is drama !!! In the past PPOE did work while the gui was showing that it did not work.

Actual status is that the IPV6-connection is not working at all !!!!!!! At least not on my system! No IPV6 at the moment !!

At the moment I have visitors, so I can not / nearly not execute tests which could interrupt or worse pfSense.

Given the long issue history, I think that the (high level) PPOE design needs to be checked.

Perhaps a good idea to define a command which you can stop start and which logs to a file !!

stephenw10

Hmm, no errors logged? Do you see dhcpv6 client requests?

louis2

@stephenw10

I did not deeper in the problem yet. I have visitors and could not permit to have any outage.

As said I would love to have a script to start and stop writing to a debug log to analyze. But perhaps I can find things in the normal logging.

But ti be honest I have little confidence in the actual PPPE-module design. There are just too much issues and solving them just takes too long.

That does not take away that I am willing to help!

I do not have PPOE knowledge however sometimes I really feel that I should have a look at the code especially what is high level happening.

PS. I did add details about the strange effects I saw yesterday in an other thread.

stephenw10

Mmm, I assume it was only the IPv6 that failed?

Yes the if_pppoe module is opt-in currently for a reason.

louis2

@stephenw10

Yes only IPV6, there has never been an issue with IPV4.

Good idea to switched back to the old PPOE. I did that for a moment ... the gui was OK again.

A bit of further testing showed ... could still not ping iPV6 ....
So I did search further ......

I found a ... test rule blocking IPV6 (for my vlan only)
After removing the rule ... IPV6 worked.

So I switched back to the kernel PPPOE .... which also worked ..... with the GUI problem ....

Sorry Sorry !

I stay on the kernel PPPOE version for now.

stephenw10

Ah, a rogue test rule. That will do it!

perrin

@stephenw10 In my case I can also confirm an issue with IPv6 after the interface has been connected for some time.

in my case it was >20days. Had to reconnect the interface and IPv6 never came back up.

[25.07.1-RELEASE][root@pfSense.xxxxx]/root: pppcfg pppoe0
        dev: vtnet0.501 state: session
        sid: 0x189 PADI retries: 0 PADR retries: 0 time: 00:04:59
        sppp: phase network authproto auto authname "xxxx@xxx" peerproto auto
        dns: 89.58.97.56 89.58.97.57
[25.07.1-RELEASE][root@pfSense.xxxx]/root: ifconfig pppoe0
pppoe0: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
        description: WAN
        options=0
        inet 89.58.xx.1 --> 185.232.xx.x1 netmask 0xffffffff
        inet6 fe80::be24:11ff:fea6:745e%pppoe0 prefixlen 64 scopeid 0x21
        groups: pppoec
        nd6 options=121<PERFORMNUD,AUTO_LINKLOCAL,NO_DAD>

pppoe log seems normal in debug mode:

Oct 14 11:59:21 	kernel 		if_pppoe: pppoe0 (8864) state=3, session=0x189 output -> e2:f6:2d:4a:d9:8f, len=16
Oct 14 11:59:21 	kernel 		if_pppoe: pppoe0: lcp output <echo-reply id=0x26 len=8
Oct 14 11:59:21 	kernel 		if_pppoe: pppoe0: got lcp echo req, sending echo rep
Oct 14 11:59:21 	kernel 		if_pppoe: pppoe0: lcp input(opened): <echo-req id=0x26 len=8

don't see any DHCPv6 packets on the interface:

[25.07.1-RELEASE][root@pfSense.xxx]/root: tcpdump -i pppoe0 -vvv '(port 67 or port 68)'
tcpdump: listening on pppoe0, link-type PPP_ETHER (PPPoE), snapshot length 262144 bytes
0 packets captured
32 packets received by filter
0 packets dropped by kernel

rebooting the firewall brings back IPv6 as expected.

stephenw10

Hmm, anything in the dhcp logs during that time? Do you have verbose logging enabled for the dhcpv6 client? If not enable it in Sys > Adv > Networking.

perrin

@stephenw10 nothing in the logs for the DHCP client. I will enable the debug logging and check the logs upon the next reconnect