New PPPoE backend, some feedback

azalea

I'm seeing the IPv6 gateway monitoring issue, too. (2.8.1-RC)

But my situation is a little different. IPv6 address is set but is not reachable.

My ISP uses separate PPPoE sessions for IPv4 and IPv6.
(IPv4 username and IPv6 username are different.)

DHCP6 Client Configuration is the same as Phil2025.

Below is an IPv6-only situation. These are picked out from the perspective of the differences between if_pppoe and mpd5.

Gateways Status

[if_pppoe]
<WAN_DHCP6>
Gateway	:not displayed
Status	:Pending
<WAN_PPPOE>
Gateway	:not displayed
Status	:Pending

[mpd5]
<WAN_DHCP6(default)>
Gateway	:displayed
Status	:Online
<WAN_PPPOE>
Gateway	:not displayed
Status	:Pending

Gateways Widget

[if_pppoe]
<WAN_DHCP6>
Gateway IP Address:not displayed (tilde symbol)
Status:Unknown
<WAN_PPPOE>
Gateway IP Address:not displayed (tilde symbol)
Status:Unknown

[mpd5]
<WAN_DHCP6(Default gateway)>
Gateway IP Address:displayed
Status:Online
<WAN_PPPOE>
Gateway IP Address:not displayed (tilde symbol)
Status:Unknown

Services Status

[if_pppoe]
dpinger:Stopped (can not start)

[mpd5]
dpinger:Running

Interfaces Status

[if_pppoe]
<WAN Interface>
IPv6 Link Local	:displayed
Gateway IPv6	:not exist
<LAN Interface>
IPv6 Address	:displayed (no reachability)

[mpd5]
<WAN Interface>
IPv6 Link Local	:displayed
Gateway IPv6	:displayed
<LAN Interface>
IPv6 Address	:displayed

NDP Table
(The following is the same situation for IPv4-only.)

[if_pppoe]
WAN Interface IPv6 Link Local:not exist

[mpd5]
WAN Interface IPv6 Link Local:displayed

PPP Log

[if_pppoe]
if_pppoe: pppoe0: failed to clear IP address: 49

[mpd5]
No errors

stephenw10

@azalea said in New PPPoE backend, some feedback:

My ISP uses separate PPPoE sessions for IPv4 and IPv6.

Hmm, how exactly is that configured?

azalea

I used an IPv6-only configuration in my first report. This configuration used pppoe0 and the WAN interface for IPv6.

Below is the configuration for IPv4 and IPv6. This configuration uses pppoe0 and the WAN interface for IPv4, and pppoe1 and the OPT1 interface for IPv6.

PPP Configuration

[pppoe0]
Link Type:PPPoE
Link Interface(s):WAN port device (for example, em0)
Username:username for IPv4
Password:password for IPv4

[pppoe1]
Link Type:PPPoE
Link Interface(s):WAN port device (for example, em0)
Username:username for IPv6
Password:password for IPv6

Interface Assignments Configuration

WAN Interface:PPPOE0(em0)
LAN Interface:LAN port device (for example, em1)
OPT1 Interface:PPPOE1(em0)

WAN Interface Configuration

[General Configuration]
IPv4 Configuration Type:PPPoE
IPv6 Configuration Type:None
Others:blank

[PPPoE Configuration]
Username:username for IPv4
Password:password for IPv4
Others:blank or disabled

[Reserved Networks]
Block private networks and loopback addresses:checked
Block bogon networks:checked

LAN Interface Configuration

[General Configuration]
IPv4 Configuration Type:Static IPv4
IPv6 Configuration Type:Track Interface
Others:blank or default

[Static IPv4 Configuration]
IPv4 Address:192.168.0.254/24
IPv4 Upstream gateway:Npne

[Track IPv6 Interface]
IPv6 Interface:OPT1
IPv6 Prefix ID:0

[Reserved Networks]
Block private networks and loopback addresses:unchecked
Block bogon networks:unchecked

OPT1 Interface Configuration

[General Configuration]
IPv4 Configuration Type:PPPoE
IPv6 Configuration Type:DHCP6
Others:blank

[DHCP6 Client Configuration]
Use IPv4 connectivity as parent interface:checked
Request only an IPv6 prefix:checked
DHCPv6 Prefix Delegation size:56
Others:unchecked

[PPPoE Configuration]
Username:username for IPv6
Password:password for IPv6
Others:blank or disabled

[Reserved Networks]
Block private networks and loopback addresses:checked
Block bogon networks:checked

Gateways Configuration

[Default gateway]
Default gateway IPv4:Automatic
Default gateway IPv6:Automatic

Although it is not related to the issue, there is a mismatch in wording.
In the Gateways Status, the status of WAN_PPPOE is displayed as "Pending".
In the Gateways Widget, the status of WAN_PPPOE is displayed as "Unknown".
I think it would be better to align the expression to "Pending".

stephenw10

Hmm that's an interesting setup. I don't think I've ever seen that before. I assume it was working previously with mpd5/negraph?

Can you say which ISP that is?

azalea

I assume it was working previously with mpd5/negraph?

Yes, it is working with mpd5/negraph. And it is working with if_pppoe for IPv4, but not for IPv6.

Can you say which ISP that is?

ISP is "plala" in Japan. To be precise, it is a combination of a network enabler (NTT FLET'S) and an ISP (plala).

The separation of IPv4 and IPv6 is a specification of "NTT FLET'S". The purpose of this specification is to allow users to choose between IPv4 and IPv6 ISPs separately. I choose plala for both IPv4 and IPv6.

stephenw10

Hmm, so I assume the IPv6 only pppoe connection does not have Use IPv4 connectivity as parent interface set?

Do you see anything logged from the dhcp6c client?

I suspect it never gets triggered if there is no IPv4 address ever set.

azalea

I suspect it never gets triggered if there is no IPv4 address ever set.

PPP is a protocol that can support IPv4-only, IPv6-only, IPv4 and IPv6.

Hmm, so I assume the IPv6 only pppoe connection does not have Use IPv4 connectivity as parent interface set?

"Use IPv4 connectivity as parent interface" is required. However, for "IPv6 only pppoe connection", "Use IPv4 connectivity as parent interface" means "Use IPv6 connectivity as parent interface". Therefore, in the PPP configuration for "IPv4 connectivity", set the IPv6 username and IPv6 password.

This PPP configuration is used to construct PPPoE tunnels for IPv6 and not to set IPv4 address. ("Use IPv4 connectivity" reminds you of IPv4 address settings, but that's not the case.)

In fact, IPv6CP is accepted and IPCP is rejected in PPPoE negotiation, so IPv6 Link Local is set, PPPoE is connected, but IPv4 address is not set.

Below is the PPP log where IPCP was rejected.

PPP log

mpd5:
ppp 	23292 	[wan] IPCP: protocol was rejected by peer

if_pppoe:
if_pppoe: pppoe0: lcp input(opened): <proto-rej id=0x1 len=28
if_pppoe: pppoe0: lcp: RXJ+ (proto-rej) for proto 0x8021 (ipcp/req-sent)

For reference, there is information related to "Use IPv4 connectivity as parent interface".

OPNsense (25.1 and later) has revised the architecture, and "IPv6 only pppoe connection" no longer requires "Use IPv4 connectivity as parent interface". "IPv6 only pppoe connection" is directly linked to a PPPoE connection.
-> Restructure PPP to allow complex IPv6-only deployments with all implications

Do you see anything logged from the dhcp6c client?

Below is the log when using if_pppoe. There is no difference from mpd5.

DHCP log

Sep 4 04:16:24 	dhcp6c 	36830 	failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
Sep 4 04:16:24 	dhcp6c 	36830 	failed initialize control message authentication
Sep 4 04:16:24 	dhcp6c 	36830 	skip opening control port
Sep 4 04:16:25 	dhcp6c 	36979 	Sending Solicit
Sep 4 04:16:26 	dhcp6c 	36979 	Sending Request
Sep 4 04:16:26 	dhcp6c 	36979 	dhcp6c Received REQUEST
Sep 4 04:16:26 	dhcp6c 	36979 	add an address [IPv6 Address]/64 on em1

I'll check the PPP log in debug mode, so please take a little time.

w0w

@azalea
This doesn’t look like a problem with PPPoE or DHCPv6 itself.
It seems more like dpinger isn’t starting for some known reason. Your log shows that you successfully obtained an IPv6 address, so I assume that if you go to System → Routing, edit your IPv6 PPPoE gateway, and disable gateway monitoring, your IPv6 should work again. This looks similar to the issue described here: https://forum.netgate.com/topic/198627/struggling-to-get-if_pppoe-kernel-module-working

stephenw10

Hmm, that dhcpv6 log looks like the client is being triggered and that it is pulling a lease. Is that not the case?

louis2

@stephenw10

Since the 'new pfSense versions', which in general are working OK, the status of my IPV6-gateway is shown as unkown. Which is not dramatic since IPV6 is working, but not OK as well.

I still hope it will be fixed.

Note:
Running the latest pfSense+ version

stephenw10

Yup there are more IPv6 fixes coming.

Phil2025

@louis2 Yes several of us are having this issue. A fix seems to be to go to: Status - Gateways - then click the recycle button to restart the Gateway service, it should then switch to online.

azalea

In my first report, I reported "IPv6 Address is set but not reachable", but found a condition to obtain global reachability.
(I checked the global response in pfSense's "Diagnostics > Ping".)

My ISP has a limit on the number of PPPoE sessions available. My ISP won't allow me to open a new PPPoE session until it knows the previous one has been closed. Also, if I disconnect a PPPoE session without manually closing it, my ISP takes about 5 minutes to recognize that the PPPoE session has been closed.

These specifications make the following difference:

(a) If I reboot pfSense with manually closing the PPPoE session, pppoe0 links up and the IPv6 address is set before "Bootup complete".
(b) If I reboot pfSense without manually closing the PPPoE session, pppoe0 links up and the IPv6 address is set after "Bootup complete".

In case (a), the ping passed, but in case (b), the ping failed.

When I checked "Use if_pppoe kernel module for PPPoE client" to enable if_pppoe, this fell under case (b). As a result, the ping failed.

Here are some other facts I have confirmed:
[PPP log]
Each time a PPPoE session is established, the message "failed to set default route 17" may or may not be displayed.

The message is not displayed after bootup.
The message is displayed after executing the "ifconfig pppoe0 down/up" command.
The message is not displayed after clicking the "Disconnect/Connect" button in Interfaces Status.

Every time a PPPoE session is established, the message "failed to clear IP address: 49" is displayed.

[Interfaces Status]
The "Gateway IPv6" item is not always displayed. It is not blank. The item itself does not exist.
The "Gateway IPv6" undisplayed issue is unrelated to the message "failed to set default route 17" displayed/undisplayed issue.
The "Gateway IPv6" undisplayed issue is unrelated to Ping pass/fail. In case (a), Ping passes.

[DHCP log]
Basically, the message "add an address [IPv6 Address]/64 on em1" is displayed each time a PPPoE session is established.

The message is displayed before or after "Bootup complete" and the IPv6 address is set.
The message is displayed after executing the "ifconfig pppoe0 down/up" command and the IPv6 address is set.
However, after clicking the "Disconnect/Connect" button in Interfaces Status, "Sending Solicit" is displayed repeatedly, no message is displayed, and the IPv6 address is not set. After that, even after executing the "ifconfig pppoe0 down/up" command, "Sending Solicit" is displayed repeatedly, no message is displayed, and the IPv6 address is not set. If I reboot again, the message is displayed and the IPv6 address is set.

[Ping]
In case (a), the ping passes regardless of whether the "failed to set default route 17" message is displayed or not.
In case (a), the ping passes regardless of the dpinger Status (Running/Stopped).
In case (a), the ping passes regardless of the Gateways Status (Online/Pending).
In case (a), the ping passes even after executing the "ifconfig pppoe0 down/up" command.

[Gateways/dpinger]
When dpinger Status is Stopped, nothing is displayed in the Gateways Log.
If I set Gateway's IPv6 Link Local to "Monitor IP", the dpinger Status is displayed as Running. Then, "send_interval" and so on is displayed in the Gateways Log.
The dpinger status (Running/Stopped) is unrelated to Ping pass/fail. Ping pass/fail is determined by case (a)/case (b).
The Gateways Status (Online/Pending) is unrelated to Ping pass/fail. Ping pass/fail is determined by case (a)/case (b).

Based on the facts presented, I assume that the issue of "Gateway IPv6" not being set and the issue of ping failure are independent issues.

stephenw10

So you also see failures in IPv6 when using mpd5/netgraph?

Do IPv4 pings (general connectivity) succeed in both cases? Jusr varies the IPv6 response?

perrin

@azalea said in New PPPoE backend, some feedback:

failed to set default route 17

i am having the exact same issue with the if_pppoe failed setting the default route and failed getting an IPv6 config. Never had any problems with this using MPD.

The issue is occurring when you disconnect, reconnect the interface or disable/enable it. upon a clean boot, everything works as expected

the only solution to regain connectivity is to reboot the firewall. it will not work otherwise. did not try to unload/reload the kernel module due to lack of knowledge on BSD.

w0w

In my case, enabling "Request only an IPv6 prefix" gives me a more stable IPv6 connection, and the WAN interface now consistently gets an IPv6 address as well.
There have definitely been some changes on the ISP’s side too, because it suddenly started losing the IPv6 connection while IPv4 was still working fine. Enabling this option fixed the issue — my IPv6 connection is now stable.

And yes, as I’ve mentioned before, I had the same problem: it wouldn’t get an IPv6 address after a reconnection, but everything worked fine after rebooting the firewall.

Here is a screenshot of my final settings.

stephenw10

@perrin said in New PPPoE backend, some feedback:

i am having the exact same issue with the if_pppoe failed setting the default route and failed getting an IPv6 config.

This happens when it first connects? Just at random during runtime?

perrin

@stephenw10 it always happens whenever you manually reconnect (via status/interfaces) the interface or disable / enable the interface. Upon reboot / first connect everything works fine

To me it seems that there is a case where if_pppoe gets in some sort of stale state which can only be solved by rebooting

it came to my attention when i was developing the pppoe_ha package. in that process I was disabling/enabling the interface often and noted the connectivity loss.

stephenw10

What does pppcfg show in that situation?

If you enable debugging (ifconfig pppoe0 debug) what does it show?

If you manually reconnect the parent NIC dies that allow it to connect?

perrin

@stephenw10 I will try that as soon as possible when I can afford some downtime. Was not aware of these debug options. Will keep you posted and update the thread asap.