new if_pppoe Backend - getting HA/CARP to work like in MPD

zjamali

seems like $rows not getting any value and not go into the foreach loop

too soon, its already have the return for no mapping

perrin

@zjamali said in new if_pppoe Backend - getting HA/CARP to work like in MPD:

seems like $rows not getting any value and not go into the foreach loop

yes, this is what i am suspecting as well. can you check your config.xml and look for the <pppoeha> section. it should look something like this:

 <pppoeha>
                        <config>
                                <row>
                                        <enabled>ON</enabled>
                                        <iface>wan</iface>
                                        <vipref>17</vipref>
                                </row>
                        </config>
                </pppoeha>

zjamali

@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:

yes, this is what i am suspecting as well. can you check your config.xml and look for the <pppoeha> section. it should look something like this:

 <pppoeha>
                        <config>
                                <row>
                                        <enabled>ON</enabled>
                                        <iface>wan</iface>
                                        <vipref>17</vipref>
                                </row>
                        </config>
                </pppoeha>

Where i can find this config.xml?

perrin

@zjamali ah sorry. you can either download a backup (under backup/restore) or directly on the box with ssh do something like:

grep -A10 pppoeha /conf/config.xml

zjamali

@perrin

zjamali

said in new if_pppoe Backend - getting HA/CARP to work like in MPD:

@perrin

is it because vipref == 0? my first vhid comes with id = 0

Yup, that cause it. i changed to VHID LAN, id != 0, its appear in log

perrin

@zjamali great, thanks for figuring that out. I need to check why it's not working with vipref=0.

currently i have no idea why this happens. vipref 0 is just the first item in the vips. config_get_path is a standard pfsense function.

Maybe it is something with the way you configured that vip? is your first VIP configured differently ffrom the one you were using later?

zjamali

@perrin Its the first vip configured in both pfsense node. So it gets id = 0.

perrin

@zjamali this is the way the dropdown in the PPPoE-HA GUI looks in my production firewall:

in my case VHID 18 is the one that is configured for failover.

Can you check if yours maybe starts with VHID 0?

zjamali

@perrin Mine also start with VHID 1 but if you go to Firewall -> Virtual IP, there is listing of all VIP, you hover on the edit button for first VHID, the url link will say its id=0

id = 0 is for internal system id for record creation whilst VHID 1 is for user POV, if i am not mistaken.

perrin

@zjamali yep, same here. I'll debug why it is not working on the first VIP later.
Can you temporarely work with a different VIP?

zjamali

@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:

@zjamali yep, same here. I'll debug why it is not working on the first VIP later.
Can you temporarely work with a different VIP?

Should be OK. no issue

perrin

@zjamali said in new if_pppoe Backend - getting HA/CARP to work like in MPD:

@perrin Mine also start with VHID 1 but if you go to Firewall -> Virtual IP, there is listing of all VIP, you hover on the edit button for first VHID, the url link will say its id=0

id = 0 is for internal system id for record creation whilst VHID 1 is for user POV, if i am not mistaken.

I tried reproducing the behaviour by removing all carp VIPs from my test firewall and adding just a single new one so it gets "id=0". In my test VM the script behaved as expected:

Sep 17 07:46:31 	pppoe-ha 	1464 	VHID 1 BACKUP -> DOWN wan (pppoe0)
Sep 17 07:46:31 	pppoe-ha 	1464 	Handle CARP command for 1@vtnet0.510 - BACKUP

and

Sep 17 07:46:35 	pppoe-ha 	52762 	VHID 1 MASTER -> UP wan (pppoe0)
Sep 17 07:46:35 	pppoe-ha 	52762 	Handle CARP command for 1@vtnet0.510 - MASTER

so, there must be some differences in the config between your and my firewall.

this is the Virtual IP definition in the my config XML:

<vip>
			<mode>carp</mode>
			<interface>opt12</interface>
			<vhid>1</vhid>
			[remaining data ommited]
</vip>

so, in my case it starts with vhid = 1 instead of zero. can you confirm this in your installation?

zjamali

@perrin

The record that i having the issue is this

Check if your opt12 with vhid 1, when you hover the edit button, id = 0. that causing the trouble when selected on the mapping. If i using other carp vip, it can reconcile

perrin

@zjamali Yep, in my case my VHID18 is ID=0

but in my case this VIP also works:

Sep 17 08:08:42 	pppoe-ha 	41559 	VHID 18 MASTER -> UP wan (pppoe0)
Sep 17 08:08:42 	pppoe-ha 	41559 	Handle CARP command for 18@vtnet0.510 - MASTER
Sep 17 08:08:39 	pppoe-ha 	28341 	VHID 18 BACKUP -> DOWN wan (pppoe0)
Sep 17 08:08:39 	pppoe-ha 	28341 	Handle CARP command for 18@vtnet0.510 - BACKUP

Can you please confirm that your VIP correctly switches from MASTER to BACKUP?
Also your vhid 2 network overlaps with vhid 101 and vhid 3 with 102

w0w

Hmm...

empty($row['enabled']) || empty($row['vipref']) || empty($row['iface'])) continue;
$vip =

empty($row['vipref']) will then exlude viperf=0 is not it?

I think this must be changed to

if (empty($row['enabled']) || !isset($row['vipref']) || $row['vipref'] === '' || empty($row['iface'])) continue;
$vip = $vips[$row['vipref']] ?? null;

If we want to use 0 from viperf array (or any other not empty value)

perrin

@w0w said in new if_pppoe Backend - getting HA/CARP to work like in MPD:

empty($row['enabled']) || empty($row['vipref'])

you are right. there is a bug in the reconcile_all function.

i will fix that.

perrin

ok, i did a complete refactor of the logic within handle_carp_change and reconcile_all so that in the end both functions use the same logic. tested it on my machine and it seems to work for me.
I uploaded a new pkg version 0.1.1 to github.

Happy to get feedback from your tests.

zjamali

@perrin

Installed new package, now its work for reconcile on vhid 1

w0w

@perrin
If the PPPoE interface is already up and the selected VIP is MASTER, this is what I see in the logsб when pressing "run reconcile now"

2025-09-19 19:04:02.698352+03:00 	rc.gateway_alarm 	94407 	>>> Gateway alarm: WAN_PPPOE (Addr:212xxxx Alarm:1 RTT:.537ms RTTsd:.089ms Loss:22%)
2025-09-19 19:03:53.092946+03:00 	kernel 	- 	pppoe0: link state changed to UP
2025-09-19 19:03:51.202925+03:00 	kernel 	- 	Limiting ICMPv6 destination unreachable output from 116 to 103 packets/sec
2025-09-19 19:03:50.152914+03:00 	kernel 	- 	Limiting ICMPv6 destination unreachable output from 111 to 98 packets/sec
2025-09-19 19:03:49.102945+03:00 	kernel 	- 	Limiting ICMPv6 destination unreachable output from 106 to 100 packets/sec
2025-09-19 19:03:48.052233+03:00 	kernel 	- 	pppoe0: link state changed to DOWN
2025-09-19 19:03:47.887051+03:00 	php-fpm 	4863 	/rc.interfaces_wan_configure: calling interface_dhcpv6_configure.
2025-09-19 19:03:47.846816+03:00 	kernel 	- 	pppoe0: link state changed to DOWN
2025-09-19 19:03:47.842883+03:00 	kernel 	- 	if_pppoe: pppoe0: failed to clear IP address: 49
2025-09-19 19:03:46.631320+03:00 	check_reload_status 	680 	Configuring interface wan
2025-09-19 19:03:46.626441+03:00 	pppoe-ha 	84394 	VHID 5 MASTER - UP wan (pppoe0)
2025-09-19 19:03:46.600614+03:00 	pppoe-ha 	84394 	Reconcile: evaluating 1 mapping(s)

I am not sure is it really necessary to break the connection at all? This ended up with
for both ipv4 and 6

Overall, I can’t say it’s stable for me—I’m not sure why. I also need to fix another bug. It looks like something is preventing the VIPs from starting when the firewall boots. Thats why I used

 $PHP_BIN -r 'require_once "/etc/inc/interfaces.inc"; interfaces_vips_configure();'