new if_pppoe Backend - getting HA/CARP to work like in MPD
-
@perrin
https://redmine.pfsense.org/issues/16174
I don’t know if this will ever be implemented. For now, I solved the issue with scripts, but they’re so ugly and clumsy…
-
Thanks for the hint towards redmine. Yeah, having priority very low for that issue makes me think that they won't implement a feature like this.
However implementing it the same way as it worked before might not be a good solution. Better would be some sort of GUI option to have the wan interface enable or disable itself depending on the status of a carp interface.I believe that is probably what you've done with your scripts. Attaching your script to the devd carp event and depending on the event do an ifup or ifdown on the wan interface.
Would you mind sharing those scripts? If I find the time maybe I'd create an addon package containing that logic so it is available to everyone else having this issue.
-
@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
Would you mind sharing those scripts?
Sure. Actually I use only one but on both firewalls.
/usr/local/etc/rc.d/run.sh 0755
#!/bin/sh ############################################################################### ############################################################################### # ──────────────────────────────── # Tunables # ──────────────────────────────── LOCKFILE="/var/run/run.sh.lock" # stores PID + role PPPOE_IF="pppoe0" # logical PPPoE interface LAN_IF="ix0" # CARP LAN interface MASTER_VIP_VHID_IPv4_LAN="MASTER vhid 5" # grep token for MASTER PHP_BIN="/usr/local/bin/php" # path to php on pfSense PPPOE_ALREADY_STARTED="false" # runtime flag ############################################################################### # Exit if another copy is already running ############################################################################### check_already_running() { if [ -f "$LOCKFILE" ]; then # shellcheck disable=SC2034 read -r LOCKPID PREV_ROLE < "$LOCKFILE" if [ -n "$LOCKPID" ] && ps -p "$LOCKPID" -o pid= >/dev/null 2>&1; then echo "Another instance is active (PID $LOCKPID) – aborting." exit 1 fi fi } ############################################################################### update_lockfile() { printf "%s\n%s\n" "$$" "$1" > "$LOCKFILE"; } ############################################################################### ############################################################################### # Optional helper ############################################################################### find_pppoe_info() { INTERFACE=$(ifconfig | awk '/pppoe[0-9]+:/ {sub(":", "", $1); print $1; exit}') IP_ADDRESS=$(ifconfig "$INTERFACE" | awk '$1 == "inet" {print $2}') } ############################################################################### # Main monitoring loop ############################################################################### start_monitoring() { echo "Monitoring CARP role (logs only when role flips)" logger "PPPoE_script_message001: CARP watchdog launched" CUR_ROLE="" [ -f "$LOCKFILE" ] && read -r _ CUR_ROLE < "$LOCKFILE" while true; do sleep 30 if ifconfig "$LAN_IF" | grep -q "$MASTER_VIP_VHID_IPv4_LAN"; then NEW_ROLE="MASTER" else NEW_ROLE="BACKUP" fi # No change → continue [ "$NEW_ROLE" = "$CUR_ROLE" ] && continue # Role changed → act if [ "$NEW_ROLE" = "MASTER" ]; then handle_master_carp logger "PPPoE_script_message002: CARP became MASTER – PPPoE handling executed" else handle_non_master_carp logger "PPPoE_script_message003: CARP became BACKUP – PPPoE handling executed" fi CUR_ROLE="$NEW_ROLE" update_lockfile "$CUR_ROLE" done } ############################################################################### # MASTER logic ############################################################################### handle_master_carp() { if [ "$PPPOE_ALREADY_STARTED" != "true" ]; then handle_pppoe_start else echo "PPPoE already started – verifying link" logger "PPPoE_script_message004: PPPoE already started – verifying link" check_pppoe fi } ############################################################################### # BACKUP logic # – Shuts down PPPoE if present # – Restores VHID 5 if it vanished ############################################################################### handle_non_master_carp() { # 1) Bring PPPoE down local existing_pppoe existing_pppoe=$(ifconfig | awk '/pppoe[0-9]+:/ {sub(":", "", $1); print $1; exit}') if [ -n "$existing_pppoe" ]; then sleep 10 ifconfig "$PPPOE_IF" down PPPOE_ALREADY_STARTED="false" logger "PPPoE_script_message005: PPPoE interface DOWN on BACKUP" fi # 2) Ensure VHID 5 still exists – if not, re-install VIPs if ! ifconfig "$LAN_IF" | grep -q "vhid 5"; then logger "PPPoE_script_message006: VHID 5 missing on BACKUP – reconfiguring VIPs" $PHP_BIN -r 'require_once "/etc/inc/interfaces.inc"; interfaces_vips_configure();' fi } ############################################################################### # Bring PPPoE UP (run when we just became MASTER) ############################################################################### handle_pppoe_start() { sleep 130 # allow CARP convergence local existing_pppoe existing_pppoe=$(ifconfig | awk '/pppoe[0-9]+:/ {sub(":", "", $1); print $1; exit}') if [ -z "$existing_pppoe" ]; then logger "PPPoE_script_message007: No PPPoE interface – reloading WAN via pfSctl" /usr/local/sbin/pfSctl -c 'interface reload wan' PPPOE_ALREADY_STARTED="true" else if ifconfig "$PPPOE_IF" | head -n1 | grep -q '<.*UP'; then logger "PPPoE_script_message008: PPPoE '${PPPOE_IF}' already UP – nothing to do" else ifconfig "$PPPOE_IF" up logger "PPPoE_script_message009: PPPoE '${PPPOE_IF}' brought UP" fi fi } ############################################################################### # Verify PPPoE connectivity – reload WAN if iface vanished ############################################################################### check_pppoe() { sleep 180 local iface iface=$(ifconfig | awk '/pppoe[0-9]+:/ {sub(":", "", $1); print $1; exit}') if [ -z "$iface" ]; then logger "PPPoE_script_message010: PPPoE interface lost – reloading WAN" if /usr/local/sbin/pfSctl -c 'interface reload wan'; then PPPOE_ALREADY_STARTED="true" else logger "PPPoE_script_message011: WAN reload failed" return 1 fi fi } ############################################################################### # Entry point ############################################################################### case "$1" in start) check_already_running find_pppoe_info # keeps old behaviour (not strictly required) start_monitoring ;; stop) echo "Stop command received (not implemented)." ;; *) echo "Usage: $0 {start|stop}" exit 1 ;; esac exit 0Ensure that VHID 5 still exists – if not, re-install the VIPs. This was a workaround for cases when the VIP disappeared for some reason. I’m not sure if it’s really needed, but it seemed that after rebooting the firewall, the VIPs were sometimes missing on the LAN. So it was added as a workaround, though you may not need it.
if ! ifconfig "$LAN_IF" | grep -q "vhid 5"; then logger "PPPoE_script_message006: VHID 5 missing on BACKUP – reconfiguring VIPs" $PHP_BIN -r 'require_once "/etc/inc/interfaces.inc"; interfaces_vips_configure();' fi -
@w0w thanks for sharing those scripts.
I did invest some time to create a pfSense package for handling PPPoe High Availability using CARP - more or less the same way it worked before.
My package has a GUI for selecting the PPPoE interface(s) and the corresponding CARP interface. Whenever a CARP event occurs for the selected interfaces, the corresponding PPPoE interface is taken down on BACKUP and brought up on MASTER, just as your scripts do.
As my logic is responding to the CARP event directly, there is no need for a timely approach using a main loop and sleep.
i've uploaded the files and the most recent pkg to GitHub:
https://github.com/perrin-1/pfSense-pppoe-haI'd be happy if someone would test that on their machines.
-
@perrin i have downloaded and try on my env and i encounter that the pppoe interface still up on backup node. I have leave 2 issue into github project.
-
@zjamali Did you try a manual reconcile?
Can you post the log output and your config? -
This post is deleted! -
-
@zjamali i suspect that i might have an old pkg in the github repo. can you please check the file pppoe_ha_even.php (in /usr/local/sbin) and check if the function reconcile_all (line 176ff) looks like this:
function reconcile_all() { $rows = ppha_get_rows(); if (!$rows){ ha_log("Reconcile: no mappings configured"); return; } ha_log("Running reconcile for all configured mappings"); $vips = config_get_path('virtualip/vip', []); foreach ($rows as $row) { if (empty($row['enabled']) || empty($row['vipref']) || empty($row['iface'])) continue; $vip = $vips[$row['vipref']] ?? null; if (!$vip || ($vip['mode'] ?? '')!=='carp') continue; $vhid = (int)($vip['vhid'] ?? -1); if ($vhid<0) continue; $state = get_carp_state_for_vhid($vhid) ?? 'INIT'; $friendly = (string)$row['iface']; $real = real_ifname_for_friendly($friendly); if (!$real){ ha_log("Reconcile: {$friendly} real if not found; skip"); continue; } if ($state==='MASTER'){ ha_log("Reconcile: VHID {$vhid} MASTER - UP {$friendly} ({$real})"); iface_up($real); } elseif ($state==='BACKUP'){ ha_log("Reconcile: VHID {$vhid} BACKUP - DOWN {$friendly} ({$real})"); iface_down($real); } elseif ($state==='INIT'){ ha_log("Reconcile: VHID {$vhid} INIT - DOWN {$friendly} ({$real})"); iface_down($real); } else { ha_log("Reconcile: VHID {$vhid} {$state} - no action"); } } }the script should log the set interface states (UP, DOWN) in the log - thats why I am wondering...
-
@perrin i have compared with yours above, its match
-
seems like $rows not getting any value and not go into the foreach loop
too soon, its already have the return for no mapping
-
@zjamali said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
seems like $rows not getting any value and not go into the foreach loop
yes, this is what i am suspecting as well. can you check your config.xml and look for the <pppoeha> section. it should look something like this:
<pppoeha> <config> <row> <enabled>ON</enabled> <iface>wan</iface> <vipref>17</vipref> </row> </config> </pppoeha> -
@perrin said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
yes, this is what i am suspecting as well. can you check your config.xml and look for the <pppoeha> section. it should look something like this:
<pppoeha> <config> <row> <enabled>ON</enabled> <iface>wan</iface> <vipref>17</vipref> </row> </config> </pppoeha>Where i can find this config.xml?
-
@zjamali ah sorry. you can either download a backup (under backup/restore) or directly on the box with ssh do something like:
grep -A10 pppoeha /conf/config.xml -
-
said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
is it because vipref == 0? my first vhid comes with id = 0
Yup, that cause it. i changed to VHID LAN, id != 0, its appear in log
-
@zjamali great, thanks for figuring that out. I need to check why it's not working with vipref=0.
currently i have no idea why this happens. vipref 0 is just the first item in the vips. config_get_path is a standard pfsense function.
Maybe it is something with the way you configured that vip? is your first VIP configured differently ffrom the one you were using later?
-
@perrin Its the first vip configured in both pfsense node. So it gets id = 0.
-
@zjamali this is the way the dropdown in the PPPoE-HA GUI looks in my production firewall:
in my case VHID 18 is the one that is configured for failover.Can you check if yours maybe starts with VHID 0?
-
@perrin Mine also start with VHID 1 but if you go to Firewall -> Virtual IP, there is listing of all VIP, you hover on the edit button for first VHID, the url link will say its id=0
id = 0 is for internal system id for record creation whilst VHID 1 is for user POV, if i am not mistaken.
