new if_pppoe Backend - getting HA/CARP to work like in MPD
-
@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
yes, this is what i am suspecting as well. can you check your config.xml and look for the <pppoeha> section. it should look something like this:
<pppoeha> <config> <row> <enabled>ON</enabled> <iface>wan</iface> <vipref>17</vipref> </row> </config> </pppoeha>Where i can find this config.xml?
-
@zjamali ah sorry. you can either download a backup (under backup/restore) or directly on the box with ssh do something like:
grep -A10 pppoeha /conf/config.xml -
-
said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
is it because vipref == 0? my first vhid comes with id = 0
Yup, that cause it. i changed to VHID LAN, id != 0, its appear in log
-
@zjamali great, thanks for figuring that out. I need to check why it's not working with vipref=0.
currently i have no idea why this happens. vipref 0 is just the first item in the vips. config_get_path is a standard pfsense function.
Maybe it is something with the way you configured that vip? is your first VIP configured differently ffrom the one you were using later?
-
@perrin Its the first vip configured in both pfsense node. So it gets id = 0.
-
@zjamali this is the way the dropdown in the PPPoE-HA GUI looks in my production firewall:
in my case VHID 18 is the one that is configured for failover.Can you check if yours maybe starts with VHID 0?
-
@perrin Mine also start with VHID 1 but if you go to Firewall -> Virtual IP, there is listing of all VIP, you hover on the edit button for first VHID, the url link will say its id=0
id = 0 is for internal system id for record creation whilst VHID 1 is for user POV, if i am not mistaken.
-
@zjamali yep, same here. I'll debug why it is not working on the first VIP later.
Can you temporarely work with a different VIP? -
@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
@zjamali yep, same here. I'll debug why it is not working on the first VIP later.
Can you temporarely work with a different VIP?Should be OK. no issue
-
@zjamali said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
@perrin Mine also start with VHID 1 but if you go to Firewall -> Virtual IP, there is listing of all VIP, you hover on the edit button for first VHID, the url link will say its id=0
id = 0 is for internal system id for record creation whilst VHID 1 is for user POV, if i am not mistaken.
I tried reproducing the behaviour by removing all carp VIPs from my test firewall and adding just a single new one so it gets "id=0". In my test VM the script behaved as expected:
Sep 17 07:46:31 pppoe-ha 1464 VHID 1 BACKUP -> DOWN wan (pppoe0) Sep 17 07:46:31 pppoe-ha 1464 Handle CARP command for 1@vtnet0.510 - BACKUPand
Sep 17 07:46:35 pppoe-ha 52762 VHID 1 MASTER -> UP wan (pppoe0) Sep 17 07:46:35 pppoe-ha 52762 Handle CARP command for 1@vtnet0.510 - MASTERso, there must be some differences in the config between your and my firewall.
this is the Virtual IP definition in the my config XML:
<vip> <mode>carp</mode> <interface>opt12</interface> <vhid>1</vhid> [remaining data ommited] </vip>so, in my case it starts with vhid = 1 instead of zero. can you confirm this in your installation?
-
The record that i having the issue is this
Check if your opt12 with vhid 1, when you hover the edit button, id = 0. that causing the trouble when selected on the mapping. If i using other carp vip, it can reconcile
-
@zjamali Yep, in my case my VHID18 is ID=0
but in my case this VIP also works:
Sep 17 08:08:42 pppoe-ha 41559 VHID 18 MASTER -> UP wan (pppoe0) Sep 17 08:08:42 pppoe-ha 41559 Handle CARP command for 18@vtnet0.510 - MASTER Sep 17 08:08:39 pppoe-ha 28341 VHID 18 BACKUP -> DOWN wan (pppoe0) Sep 17 08:08:39 pppoe-ha 28341 Handle CARP command for 18@vtnet0.510 - BACKUPCan you please confirm that your VIP correctly switches from MASTER to BACKUP?
Also your vhid 2 network overlaps with vhid 101 and vhid 3 with 102 -
Hmm...
empty($row['enabled']) || empty($row['vipref']) || empty($row['iface'])) continue; $vip =empty($row['vipref']) will then exlude viperf=0 is not it?
I think this must be changed to
if (empty($row['enabled']) || !isset($row['vipref']) || $row['vipref'] === '' || empty($row['iface'])) continue; $vip = $vips[$row['vipref']] ?? null;If we want to use 0 from viperf array (or any other not empty value)
-
@w0w said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
empty($row['enabled']) || empty($row['vipref'])
you are right. there is a bug in the reconcile_all function.
i will fix that.
-
ok, i did a complete refactor of the logic within handle_carp_change and reconcile_all so that in the end both functions use the same logic. tested it on my machine and it seems to work for me.
I uploaded a new pkg version 0.1.1 to github.Happy to get feedback from your tests.
-
-
P perrin referenced this topic
-
@perrin
If the PPPoE interface is already up and the selected VIP is MASTER, this is what I see in the logsб when pressing "run reconcile now"2025-09-19 19:04:02.698352+03:00 rc.gateway_alarm 94407 >>> Gateway alarm: WAN_PPPOE (Addr:212xxxx Alarm:1 RTT:.537ms RTTsd:.089ms Loss:22%) 2025-09-19 19:03:53.092946+03:00 kernel - pppoe0: link state changed to UP 2025-09-19 19:03:51.202925+03:00 kernel - Limiting ICMPv6 destination unreachable output from 116 to 103 packets/sec 2025-09-19 19:03:50.152914+03:00 kernel - Limiting ICMPv6 destination unreachable output from 111 to 98 packets/sec 2025-09-19 19:03:49.102945+03:00 kernel - Limiting ICMPv6 destination unreachable output from 106 to 100 packets/sec 2025-09-19 19:03:48.052233+03:00 kernel - pppoe0: link state changed to DOWN 2025-09-19 19:03:47.887051+03:00 php-fpm 4863 /rc.interfaces_wan_configure: calling interface_dhcpv6_configure. 2025-09-19 19:03:47.846816+03:00 kernel - pppoe0: link state changed to DOWN 2025-09-19 19:03:47.842883+03:00 kernel - if_pppoe: pppoe0: failed to clear IP address: 49 2025-09-19 19:03:46.631320+03:00 check_reload_status 680 Configuring interface wan 2025-09-19 19:03:46.626441+03:00 pppoe-ha 84394 VHID 5 MASTER - UP wan (pppoe0) 2025-09-19 19:03:46.600614+03:00 pppoe-ha 84394 Reconcile: evaluating 1 mapping(s)I am not sure is it really necessary to break the connection at all? This ended up with
for both ipv4 and 6Overall, I can’t say it’s stable for me—I’m not sure why. I also need to fix another bug. It looks like something is preventing the VIPs from starting when the firewall boots. Thats why I used
$PHP_BIN -r 'require_once "/etc/inc/interfaces.inc"; interfaces_vips_configure();' -
@w0w said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
I am not sure is it really necessary to break the connection at all?
what is happening on a reconcile is that all interfaces are forced in the state they should be in. For the UP-command the script currently does a 'interface reload' (/usr/local/sbin/pfSctl -c 'interface reload <interface>'), whereas for the down it does a ifconfig <interface> down
It showed up in my tests that using 'ifconfig <interface> up' as a UP-command would not reliably connect the pppoe interface in every case. That is why i opted in to use the reload command. The downside of this is in fact that it breaks the connection. I could add a check to see if the interface is already connected before reloading it to fix this behaviour in reconcile. I might be adding that.
@w0w said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
Overall, I can’t say it’s stable for me—I’m not sure why. I also need to fix another bug. It looks like something is preventing the VIPs from starting when the firewall boots.
great to hear that is is stable! Regarding the VIPs not coming up: I have no idea where this is coming from, probably has nothing to do with my script. On both of my firewalls the VIPs come up as expected. But i am only using VIPs as part of CARP.
-
@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
great to hear that is is stable!
Unfortunately I have to clarify: no — your variant does not work stable on my systems. The bug described above on an already loaded system with PPPoE up seems to start with initialization of your script, and the link falls into packet loss; I don’t even need to press the button. Also please look at the “Enable” checkbox, try to add several interfaces. For me it sometimes disappears at all, sometimes moves around.
