new if_pppoe Backend - getting HA/CARP to work like in MPD
-
Hi,
I really appreciate the time you put into this. Thanks for sharing.I have installed the solution. After analyzing the logs it is clear that
- CARP transition detected
- Slave starts PPPoE session successfully at first
- ISP rejects authentication with Too many sessions. ISP is refusing a second PPPoE login because the old session from my master pfSense is still alive
-Slave keeps retrying repeatedly but still no luck
(I even waited for 2-3 minutes).
So the slave's WAN is never up.
How to fix / work around? Add gui option to add a startup delay on the slave, so that when CARP changes, pfSense will wait 20 seconds before starting PPPoE.
MAC spoofing came also to my mind, but ISP can use a variety of signals to track PPPoE sessions:
- PPP username/session state (most important)
- PPPoE/PPPoE session id on their BRAS
- CPE MAC address / modem association
-
@crl
I have experimented with different variants, and I can say that using a delay is not a good solution, as I mentioned earlier, because the firewall status can change during that delay. The logic needs improvement, but I don’t have enough time to work on it right now.
My script version handles this case much better, but it’s slower and not fully synchronized with status changes.The only approach I see is to avoid breaking the connection immediately when the backup status is detected. Instead, register the status, start a time-based trigger that checks the status again before executing and quits if the current status has not changed or proceeds with the action if it is changed based on the first registered status. The same applies to the master: monitor it using a time-based trigger synchronized with the first status change, and quit if the status is unchanged or perform the action and then exit. This sounds simple but it is not, because we need also to ignore status changes after first change is detected and start it again in some time after all things have happened. And this all makes me think that logic becomes too complicated and too much code used to serve this implementation.
-
@crl said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
ISP rejects authentication with Too many sessions. ISP is refusing a second PPPoE login because the old session from my master pfSense is still alive
-Slave keeps retrying repeatedly but still no luck
(I even waited for 2-3 minutes).Hi,
the same applies to my ISP. I also get a denied login at first when the slave comes up. Only in my case the ISP times out the old master session within a few minutes allowing the slave to connect.Whenever the master fails "badly" it is unable to end the session cleanly and will always result in the slave not able to establish a connection for the first amount of time.
@crl said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
So the slave's WAN is never up.
I did not think about this case when designing the plugin cause from my understanding of PPPoE there is something called LCP keepalive which will time out a stale session at the ISP after some time. My ISP does that within seconds. Maybe your ISP has a quite lengthy setting of that timeout.
You could try to set the same MAC address on both firewalls for the PPPoE interface and see if that helps. The session definitely is still in a different state but maybe it helps with your ISP.
The most elegant solution however would be to syncronize the PPPoE session id, configuration values (IP addresses, gateways and so forth) between master and slave and have the slave pick up the current session. But that won't work without patching the if_pppoe itself which might be out of scope...
-
@perrin
How does your HA pair react if you put the master node into maintenance mode via Status → CARP → Enable Persistent Maintenance Mode (or whatever it’s called)? -
@w0w Enabling the Maintenance Mode on the Master raises its skew thus transitioning MASTER to BACKUP. pppoe-ha picks up the backup state an disables the interface accoringly.
Since i don't have a problem moving the PPPoE session, in my case the failover works as expected.
Maybe @crl should try that and see
a) if if_pppoe correctly closes the session on the master prior to disabling the interface and
b) if his backup can correctly establish a new PPPoE session -
Please check it this workaround:
Github Issue - ISP side 'Too many sessions' keeping backup pfsense's WAN downIt solves only one use case:
-OK: enter and leave carp maintenance mode on manual trigger-Solution requested: if a wan cable is pulled (between the wan switch and any of the pfsense devices) or if the pfsense machine is down:
perform MASTER --> BACKUP transition and connect pppoe on the BACKUP. Should the MASTER come back again, it shall take back the MASTER role and pppoe-reconnect on the MASTER. -
I tried to summarize what is going on during the switchover experiments. This is one example.
-
-
@w0w said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
@crl, @perrin do you both have dual stack pppoe?
In my case yes, dual stack v4 and V6@crl said in new if_pppoe Backend - getting HA/CARP to work like in MPD:
I tried to summarize what is going on during the switchover experiments. This is one example.
Some of these issues might be related to configuration and or default behavior of pfSense (e.g. when pppoe fails and you're expecting a carp switch.)
Do these things work as expected when you are using the old time based scripts? -
Yes, in my setup things work somewhat differently, as you noticed. There are at least a few reasons. Most importantly, every time PPPoE comes up, the VIPs get reconfigured and CARP reinitializes. I suspect this behavior is related to IPv6 and the fact that the LAN uses the Track Interface option to obtain its IPv6 address, but I’m not certain. I’m currently trying to track down the root cause—or perhaps it’s an “incompatible” configuration.
How does this behave on your side? As I understand it, bringing up PPPoE does not trigger VIP reconfiguration/CARP initialization for you, right?
