Configure an PPPoE on an CARP IF

[Beerman]

Hello,

I've read that since version 2.4.3 there is the possibility to configure PPPoE on an CARP Interface (VIP), so it will only be active on the master node.

How is the procedure for this? I tried it, but could not get an successful login with PPPoE if I assign the PPPoE to an CARP VIP.

From my provider I have to use vlan 7, so I assigned the vlan-IF to my WAN IF and I configured an ipv4 IP to this IF. Afterwards I configured an CARP-VIP for this IF. This CARP-VIP I did assign to the PPPoE configuration.

With "ifconfig" on the shell, I see both ipv4 IPs I configures (from IF and the CARP-VIP) and an "empty" pppoe-IF. (So no "login" is done via pppoe)

What is the thing, I am doing wrong?

Thx, for your help!

[TugBoat]

I have exactly the same question. (I am assuming the facility is still available on pfsense 2.4.4).

I can find the reference to the facility added at 2.4.3 (https://redmine.pfsense.org/issues/8184), however, I can't find any explanation on how to use this facility.

In my situation I have a redundant (dual diverse path) main Internet connection that is supposed to be 100% reliable, however, we also have a PPPoE ADSL 'backup' (with a single static address) that I would like connect directly to the pfsense HA pair.

From the references that I have found to this change in pfsense 2.4.3 it looks to be exactly what I need. However, I have not found any information on exactly how to make it work.

Does anyone know how to configure a PPPoE link on a HA pair so that only the master is active?

Thanks in advance,
Tim

[Beerman]

Hi,

I've found out by now, it's actually quite simple.

First, you add an Carp Interface for your DSL COnnection. (on each machine of you HA pair) Configure this CARP IF on the correct VLAN, you need for this.

Add an CARP IP address.

Go to "Interfaces / Interface Assignments" --> PPPs.
Add the PPP Connection and choose the Carp IP as "Link Interface(s)".

This is working for me :)

Hope it helped!

[TugBoat]

Thanks Beerman,

That sounds remarkably straight forward - I will prototype it and see what happens.

Thanks again,
Tim

[TugBoat]

OK,

This procedure does work, although there are a few tricky bits. So here is a slightly more detailed explanation of what Beerman provided (I hope that I did this correctly). I tested this on the 2.4.4-p1 release of pfsense in a lab environment, the switch to production is a few days away:

1. Add an Interface that will connect to the ADSL modem. This is either a direct NIC, or possibly a VLAN. I called my Interface 'ADSLMODEM' (very original..). You need to allocate a static IP address to this interface on both of the HA pair (otherwise you can't add a CARP IP). As far as I can see these static IPs don't figure in any way in the PPPoE, so you just use a subnet that you are not using anywhere else. (In my test I used 192.168.77.2 and .3)
2. Add the CARP IP to the ADSL modem interface. Give this an appropriate IP address, use all the normal CARP settings. (My test was 192.168.77.1).
3. At this stage I checked that the CARP status showed the interfaces as Master on the Master, and Backup on the Backup of the HA pair. So far, so good. If not, check your cabling, switches, look for typographic errors etc.
4. Use Interfaces/Assignments/PPPs and add a PPPoE device. This is the important part: you need to select the CARP IP address (192.168.77.1) as the 'Link Interface'. The IP address will just be in the drop down list.
5. Now on Interfaces/Assignments you can add an interface using the the PPPoE that you have defined. You will find a an entry called 'PPPoE(<somejunkhere>)' in the list of possible 'Network Ports', this is the 'port' to use. After you click '+Add' and 'Save' you click on the 'OPT??' link to edit the new interface - I renamed mine from 'OPT??' to 'ADSL'. Check that the 'IPv4 Configuration Type' is 'PPPoE' - for some reason when I did this step the field was occasionally something else - make sure you select PPPoE and then all the details (login+password) will appear correctly in the interface parameters down the screen. Make sure that you also enable the interface.

At this point it should all be working. I checked a number of things about the configuration at this point:
a) In Status/System Logs/PPP on the master firewall there was logging showing the PPPoE trying to establish.
b) In Status/System Logs/PPP on the backup firewall the logging showed that PPPoE was configured, but nothing was logged about attempting to establish a connection.

Now I connected my ADSL Modem to the appropriate NIC/VLAN and the master firewall immediately established the PPPoE link.

At this point I disabled CARP on the master firewall and the HA pair switched over, and what was the backup brought the PPPoE link up, the disabled master logs showed PPPoE as inactive. Enable CARP on the master firewall and the PPPoE link switched back.

As a final test I added appropriate firewall rules to the 'ADSL' filter and checked some inbound connections using the ADSL circuit. As far as I could see everything worked as expected.

Thanks again to Beerman for the all important and key information on how to do this.

Tim

[netblues]

This works.
I just converted a pppoe interface to this just by reassigning the link to the carp ip.
It is not statefull failover, but in a ppp world is as close as it can get.

[Gabri.91]

Hi @TugBoat and @netblues,
have you tried also the failover to backup CARP node with dynamic PPPOE?

I've found some strange issues:

• 1. main online, backup online => everything is working fine, just main has PPPOE active
• 2. main goes offline => PPPOE is correctly connected on backup
• 3. main back online => PPPOE it's trying to connect on main, but it's not disconnected on backup (so main it's unable to connect)
• 4. after power off and power on again the backup, main connects but backup still tries to connect

Now I have main connected and backup that still tries to connect, but I don't know how to revert back to initial situation...

EDIT: Found the issue in case it could be useful for someone else, on backup node the PPOE was still pointing to the local interface and not to the CARP IP Address

[A Former User]

Hi, sorry to dig this up but it's the best (only?) documentation regarding CARP and PPPoE. I followed the hints of this thread and got a working setup. Everything including multiple VLANs, VLANs routed over vpn work just fine. But...

Every once in a while (every few hours) I get timeouts accessing the internet from the LAN. It's about 5-10 seconds. I checked and the master doesn't switch, the pppoe interface doesn't reconnect. I'm at a loss what's causing it.

I realized the only thing missing from this thread is outbound NAT. How do you have it configured - as the documentation states to go with manual and set a VIP as the translation address which doesn't seem to apply.

Thanks !!

[netblues]

Outbound nat is set manually to the ppp interface assigned (since this is the one that gets the public ip.)
This works and is quite stable for a long time now. I doubt its nat.
Go to status monitoring and see if you have issues with your isp.

[A Former User]

That's how I setup outbound NAT thank you for confirming it's the correct way.

@netblues said in Configure an PPPoE on an CARP IF:

Go to status monitoring and see if you have issues with your isp.

Yes, that's what I checked and nothing. Everything started the day I enabled ha/carp so it's quite a coincidence if it's something else.

[JeGr]

@icesense1701 said in Configure an PPPoE on an CARP IF:

Yes, that's what I checked and nothing. Everything started the day I enabled ha/carp so it's quite a coincidence if it's something else.

Also check for anything other gateway related or if you are using DNS resolver or forwarder, perhaps check the DNS logs, if it isn't the connection you are loosing but DNS being restarted/hanging up.

[netblues]

Also check in system/advanced/networking, uncheck reset all states, just in case.

[A Former User]

Just to follow up I seem to have fixed it with brute force. I rebooted the pfsense vms once more and all the switches. Also some vlan's didn't have the proper dhcp settings (failover ip) - shouldn't matter because the timeouts were on properly setup vlans but I fixed those too. I didn't experience any more issues for the past two days. Thanks all !

[crl]

It works for me but breaks openVPN, see Link.

[deb8]

Hi all,
I followed the guide provided by @TugBoat and this is how my configuration looks like.

i. Interface in "Interfaces / Interface Assignments", named it LAN, set it with static IPv4 IP (192.168.1.4) and assigned it to Network port "em0". IPv4 Upstream gateway is set to none.

ii. Virtual IP in "Firewall / Virtual IPs" of type CARP, with interface LAN and single address 192.168.1.1.

iii. VLAN interface in "Interfaces / VLANs" with tag 10 on the physical interface em0. "VLAN 10 on em0", used for communication with the VDSL Modem.

iv. Interface in "Interfaces / Interface Assignments", named it WAN, set it with static IPv4 IP (192.168.0.4) and assigned it to Network port "VLAN 10 on em0". IPv4 Upstream gateway is set to none.

v. Virtual IP in "Firewall / Virtual IPs" of type CARP, with interface WAN and single address 192.168.0.2.

vi. PPPoE Interface in "Interfaces / PPPs" with link interface "192.168.0.2 (vhid 2) - WAN CARP VIP" with username and password.

vii. Interface in "Interfaces / Interface Assignments", named it WANPPPoE and assigned it to Network port "PPPOE(_vip610...)".

viii. Gateway in "System / Routing / Gateways" with interface "PPPoE", Address family "IPv4", Gateway "dynamic", checked "Use non-local gateway" (public IP assigned is in different subnet from the providers Gateway) and set as "Default gateway IPv4".

ix. Outbound NAT in "Firewall / NAT / Outbound", with interface "PPPoE", Address family "IPv4", Source Type "Network", Source Address Range "192.168.1.0/24" (LAN), Destination "Any", Translation Address "192.168.0.2 (WAN CARP VIP)".

x. DNS Resolver Network Interfaces set to "192.168.1.1 (LAN CARP VIP)", Outgoing Network Interfaces set to "192.168.0.2 (WAN CARP VIP)"

PPPoE is successfully established and public IP is assigned. However, the LAN portion of the configuration is unable to access the public internet.
Any hint on what I am doing wrong?
Thanks in advance.

[wifi75]

@Gabri-91 I have a dynamic connection in pppoe on vlan 835, I have performed all the steps but it doesn't connect using the carp interface.
I double checked all the steps and it seems to be ok, but it doesn't want to connect to the wan...

[netblues]

@wifi75

This never really worked. pppoe running on a carp interface isn't an option.
And as far as natting is concerned, appart from the ppp interface everything else is irrelevant.
pppoe is a layer 2 thing.
Natting works on layer 3.

[JeGr]

@netblues said in Configure an PPPoE on an CARP IF:

This never really worked. pppoe running on a carp interface isn't an option.

It sure is. We have a few customers set up that way and working well - within boundaries. Of course in such a setup the secondary node of a CARP setup won't easily have internet which is/can be a problem and as such the setup isn't really recommended. But it IS working though. It's important to check though that both nodes on it's WAN "carrier" interface are connected to each other and the DSL modem correctly so both have access to dial-in if needed. If that's set up correctly it's a relatively simple setup:

• either node gets the physical interface for the PPPoE connection assigned with its own IP, say 10.12.34.251 and .252
• check pinging from one to the other and back (allow ICMP on that interface first)
• then add a CARP VIP to it, e.g. .254 - that one should now be active on the primary node anad backup on the secondary node. If that is not the case you don't need to proceed with PPPoE stuff. That's basic CARP that should be working first!
• If that's running you can now add the PPPoE interface but as carrier you don't choose your physical interface BUT the NEW CARP VIP you created (yes, that .254 one from above!) This ensures the PPPoE connection switches from node 1 to 2 and back if needed.
• Then set up PPPoE as usual.
• When finished assign that interface (pppoe0) as your WAN_PPPoE or something else like it. THAT one is your actual WAN, the other physical interface and the VIP on it are only a sort-of transfer/carrier network.

Cheers