PPPoE problems after upgrading from 2.4.2 to 2.4.4

Joxxxx

I'm estonished that there are not more posts about this pppoe problem on 2.4.4. Maybe there are not so many people using pppoe after all ?
Since upgrading from 2.4.3 to 2.4.4, my pppoe doesn't work anymore neither. I also get these weird errors like you stating "file not found" or "not allowed to allocate ip address". Reboot doesn't fix it though, sadly.

It's super annoying because I had to revert back to 2.3.3 and I cannot use any package anymore because I get the following error when trying to install one :

WARNING: Current pkg repository has a new PHP major
         version. pfSense should be upgraded before
         installing any new package.
Failed

Did someone found out why the new pppoe connection layer is failing for some and not for others ?

netblues

Are you using vlans for pppoe?

Joxxxx

@netblues No, it's not needed by the provider.

netblues

@joxxxx Im not talking about the provider, just as a means of adding more than one interface on a machine that just has one via a managed ethernet switch
It seems the problem exists when vlans is involved.
And rebooting or just hitting connect seems to fix things.
Perhaps you are facing a slight different problem?

Joxxxx

@netblues Oh ok, I thought you wanted to know if the connection itself had a special vlan (some providers ask for that).
No, I don't use any vlans. I have 2 wan : 1 pppoe ftth and 1 cable. Only the pppoe fails on 2.4.4.

It's maybe another problem, but I tried everything I could, nothing helped. It's really strange... The connection works flawlessly when directly wired to the pc, on vyOS, sophos, pf2.4.3 but not on pf2.4.4 or pf2.4.5. Something changed, but what ?

netblues

@joxxxx Have you tried recreating the ppp connection on pf 2.4.4 from scratch?

Joxxxx

@netblues yes, tried first by updating, then by installing a fresh 2.4.4 and importing the settings and then on a fresh 2.4.4 and manually creating the pppoe connection. Didn't work with the same errors all the time.

For info, I'm running it on esxi 6.5 with a quad gigabit eth card.

netblues

@joxxxx I suppose you have disabled all 3 hardware offload options on advanced/networking.
(meaningless for a virtual network interface, by design...)

And what are the exact errors?

Joxxxx

@netblues Actually, Hardware TCP Segmentation Offloading and Hardware Large Receive Offloading were disabled, but not Hardware Checksum Offloading.

The errors are :

[opt1] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists

[opt1] IFACE: Removing IPv4 address from pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): Can't assign requested address

It then cycles infinitely, each time getting a new wan ip and throwing the same errors.

Thanks a lot for your help !

netblues

@joxxxx This is something else. Have you tried with hardware checksum offloading disabled?

Joxxxx

@netblues Not yet, I'll boot up the 2.4.4 vm and test it out. (On 2.3.3 hardware offloading doesn't change anything).

Joxxxx

Ok, just tried it and still the same, here some more logs (appart of the recurring file exists etc. from above) :

Nov 22 14:00:50 	ppp 		[opt1] IFACE: Up event
Nov 22 14:00:50 	ppp 		[opt1] IFACE: Rename interface ng0 to pppoe0
Nov 22 14:00:50 	ppp 		[opt1] IPCP: rec'd Configure Ack #3 (Ack-Sent)
Nov 22 14:00:50 	ppp 		[opt1] IPADDR 188.XX.XX.XX
Nov 22 14:00:50 	ppp 		[opt1] IPCP: state change Ack-Sent --> Opened
Nov 22 14:00:50 	ppp 		[opt1] IPCP: LayerUp
Nov 22 14:00:50 	ppp 		[opt1] 188.XX.XX.XX -> 10.0.0.1
Nov 22 14:00:50 	ppp 		[opt1] IFACE: Adding IPv4 address to pppoe0 failed(IGNORING for now. This should be only for PPPoE friendly!): File exists

Strange thing is that it talks about ipv6 although I disabled it everywhere I could. It seems like on 2.4.3 it doesn't do that.

Even updated to 2.4.5-dev to see, but still the same.

netblues

Strange as it is it seems it manages to connect, negotiate, auth, get ip and then fail.
Any ideas why a seemingly public ip is being changed to a private one?(10.0.01) Does this happen on other tests?

Joxxxx

@netblues 10.0.0.1 is the pf lan address. I don't know why it does that actually. There are 3 phases to the connection process. First it's 0.0.0.0, then 10.0.0.1 and then the real ip address. Or sometimes starts with 10.0.0.1, then 0.0.0.0 and the the real ip.

10.0.0.1 seems to be taken as the pppoe gateway, maybe that's not "allowed" anymore in 2.4.4 ?
On the dashboard, the gateway for pppoe says 10.0.0.1 and the interface has the correct ip.

Here is the log of the 2.3.3 that works :

Nov 22 14:05:26 	ppp 		[opt1] IFACE: Up event
Nov 22 14:05:26 	ppp 		[opt1] IFACE: Rename interface ng0 to pppoe0
Nov 22 14:05:26 	ppp 		[opt1] IPCP: rec'd Configure Nak #2 (Ack-Sent)
Nov 22 14:05:26 	ppp 		[opt1] IPADDR 188.XX.XX.XX
Nov 22 14:05:26 	ppp 		[opt1] 188.XX.XX.XX is OK
Nov 22 14:05:26 	ppp 		[opt1] IPCP: SendConfigReq #3
Nov 22 14:05:26 	ppp 		[opt1] IPADDR 188.XX.XX.XX
Nov 22 14:05:26 	ppp 		[opt1] IPCP: rec'd Configure Ack #3 (Ack-Sent)
Nov 22 14:05:26 	ppp 		[opt1] IPADDR 188.XX.XX.XX
Nov 22 14:05:26 	ppp 		[opt1] IPCP: state change Ack-Sent --> Opened
Nov 22 14:05:26 	ppp 		[opt1] IPCP: LayerUp
Nov 22 14:05:26 	ppp 		[opt1] 188.XX.XX.XX-> 10.0.0.1

Nov 22 14:05:26 ppp [opt1] 188.XX.XX.XX is OK : This seems to be the line that fails on 2.4.4 with "File exists".

Grimson

@netblues said in PPPoE problems after upgrading from 2.4.2 to 2.4.4:

Any ideas why a seemingly public ip is being changed to a private one?(10.0.01) Does this happen on other tests?

It's not changing the IP, it's trying to add the IP with a default gateway of 10.0.0.1. So check your gateway settings, is the gateway for WAN set to dynamic, do you have any other gateways on your system. Make sure you have either selected the right default gateway or a properly configured gateway group if you have multiple wan gateways.

Edit: and if your PPPoE provider really uses 10.0.0.1 as their gateway address you can't use that on your LAN.

netblues

@joxxxx there is new functionality dealing with gateways for the box.
Please check it.
Its not right to have the lan ip as a ppp gateway.
(in system/routing gateways...)

Joxxxx

@grimson I have indeed 2 WAN. The main cable network that gets the info through DHCP (provider router in gateway mode) and the fiber 1Gbit ftth through pppoe (OPT1).
Gateways are dynamic and both wan are in a gateway group with the fiber set as Tier 1 and the cable as Tier 2.

I don't think the provider uses 10.0.0.1 as gateway, but pfsense sees it that way. The thing is, why does it work on 2.3.3 and not on 2.4.4 and 2.4.5 ?

Grimson

Show the whole PPP log, not just a small part of it. As for why it doesn't work on 2.4.4, mpd5 has received quite a few updates since then and it may very well check the validity of the gateway IP now.

@joxxxx said in PPPoE problems after upgrading from 2.4.2 to 2.4.4:

I don't think the provider uses 10.0.0.1 as gateway, but pfsense sees it that way.

If you get 10.0.0.1 as gateway on 2.3.3 too, and the connection works then yes your provider is using that as the gateway address.

stephenw10

If they are actually using 10.0.0.1 as the gateway, which is possible, then you should not be using that subnet internally.

It may work as it will probably be /32 and hence a more specific route but eliminating that potential conflict would be a good test.

Steve

Joxxxx

Oh god, I feel stupid now... You guys nailed it, the fiber modem somehow uses 10.0.0.1 as the actual pppoe gateway !
I changed the lan network to something else and the connection worked immediately after.

Thanks a thousand times for your help ! Finally it's fixed. I never thought they'd use this network for their actual gateway.

So, good to know, 2.4.4+ makes pppoe more strict ! :)