Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    New PPPoE backend, some feedback

    Scheduled Pinned Locked Moved Development
    155 Posts 12 Posters 8.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • RobbieTTR
      RobbieTT @stephenw10
      last edited by

      @stephenw10 said in New PPPoE backend, some feedback:

      Yup and in fact you don't have to set the parent to 1508. If you set the pppoe interface to 1500 the parent will inherit that and be set to 1508.

      All good now though. 👍

      More things I didn't know!

      I'll probably leave it set as 1508 just to help remind me what I am doing when I am tired and forgetting what I am supposed to be doing...

      ☕️

      1 Reply Last reply Reply Quote 1
      • P
        Phil2025 @RobbieTT
        last edited by Phil2025

        @RobbieTT I've posted my issues over in the reddit forum but I'm not sure its been picked up.

        If you go to Status-> Interfaces - then Disconnect the WAN interface to drop the PPPoE, then refresh the page to get the button to change to Connect, then Connect, does it reconnect? I only get every other connection attempt work, every other one stalls saying its "UP" but no Gateways or connectivity is established, I need to drop and then reconnect again. It's fine switching back to the original PPPoE.

        Also the IPv6 Gateway doesn't always start monitoring correctly, and shows status Unknown, but there is IPv6 connectivity okay. Going into Gateways and into the IPv6 gateway and disabling monitoring, then saving that, then re-enabling monitoring and it starts monitoring and switches back to online.

        With regards to the MTU, I find a great way to check is to use Speedguide.net and from the left hand menu select their TCP/IP Analzyer, snapshot of mine below. I would think you are only seeing 1492 MTU and losing the 8 bytes due to PPPoE. Your MTU can be set to 1508 (1508 being what they call a Baby Jumbo Frame) which is supported via Openreach and most others, so that holds the extra overhead for PPPoE, and when that's stripped of you are left then with the full 1500 bytes.

        898927d0-fb82-448e-b820-97e69559074f-image.png

        aa9dc3c5-b1ff-46f7-994a-7ec5c503f5be-image.png

        RobbieTTR 1 Reply Last reply Reply Quote 0
        • RobbieTTR
          RobbieTT @Phil2025
          last edited by

          @Phil2025 said in New PPPoE backend, some feedback:

          Speedguide.net

          Hi Phil, perhaps lost in the noise but I didn't say I had an issue with the PPPoE MTU, only that I checked it to be sure it was still ok, demonstrated my settings and displayed the test results to prove it. @stephenw10 did have a hiccup with his settings due to his role in testing multiple configurations - all now resolved.

          I did try the speedguide.net link and I note that it does not run on Safari and seems to require a Chromium-based browser. Using Brave browser my results are:

          « SpeedGuide.net TCP Analyzer Results » 
          Tested on: 2025.04.25 04:13 
          IP address: 93.8x.xxx.xx 
          Client OS/browser: Mac OS (Chrome 135.0.0.0) 
           
          TCP options string: 020405b4010303060101080a26e64ea30000000004020000 
          MSS: 1460 
          MTU: 1500 
          TCP Window: 131776 (not multiple of MSS) 
          RWIN Scaling: 6 bits (2^6=64) 
          Unscaled RWIN : 2059 
          Recommended RWINs: 64240, 128480, 256960, 513920, 1027840 
          BDP limit (200ms): 527 Mbps (53 Megabytes/s) 
          BDP limit (500ms): 211 Mbps (21 Megabytes/s) 
          MTU Discovery: ON 
          TTL: 50 
          Timestamps: ON 
          SACKs: ON 
          IP ToS: 00000000 (0) 
          
          

           2025-04-25 at 09.31.19.png

          Nothing leaps out at me but I am unfamiliar with this tool so feel free to run your eyes over it for me.

          Regarding the Status / Interfaces / PPPoE interface 'Disconnect' I do experience the exact same issue you describe and pfSense seems to hang after the first state change command (eg Disconnect WAN) and if left alone it fails to recover IPv6 properly. If you repeat the command it seems to all work fine again, including IPv6.

           2025-04-25 at 09.37.02.png

          So that looks like a bug that needs resolving. It may even be linked to the PPP status logs not working, as noted by others.

          ☕️

          P 1 Reply Last reply Reply Quote 0
          • P
            Phil2025 @RobbieTT
            last edited by

            @RobbieTT Many thanks for trying the connect and disconnect, good to know its not just me, I hope they pick it up and fix it as I think it could mean if the Internet goes down for some reason, it may not come back up again by itself.

            Yes your MTU looks spot on, usually with 1500 set under the WAN it would mean you would only see 1492 as MTU, so I expect your ISP is negotiating 1508 when you connect and overriding the 1500 MTU you have set.

            With other overheads you end up with 1448 bytes of useable data in each packet. You can tweak things to get rid of those overheads and get a bit more throughput, but things like timestamps are useful for retransmission and recovering from errors, so what you lose in maximum throughput you gain in another way, so its all good as it is I would say.

            RobbieTTR w0wW 2 Replies Last reply Reply Quote 0
            • RobbieTTR
              RobbieTT @Phil2025
              last edited by RobbieTT

              @Phil2025

              I have 1508 MTU set on the interface carrying the PPPoE encapsulation (listed as ONT in my examples above).

              The un-molested WAN connection inside this encapsulated and slightly larger MTU pipe is set at the standard 1500 MTU (albeit note @stephenw10's comments above).

              I don't think the ISP or Openreach can negotiate a 1508 MTU if you have a smaller one set. Indeed, there are many users out there with either no appreciation for baby jumbo frames or have inflexible hardware that does not allow for an MTU above 1500 to be set.

              The BT SINs make no mention of being able to negotiate a higher MTU than requested; the exact opposite is stated though, with connectivity warnings. From memory the BT SINs give a maximum MTU of 1530 (previously 1520) on the Openreach network.

              I'm not sure what overheads can be tweaked or removed on my PPPoE connection to get additional throughput though - can you expand on these for me please?

              ☕️

              1 Reply Last reply Reply Quote 1
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by stephenw10

                I think the connection status you see is actually an artifact of the new interface type. I can sort of replicate what you see but in fact it does connect it's just not instant. You can see what the actual state is using the new pppcfg command.

                When you disconnect the interface in the gui it is removed entirely:

                [2.8.0-BETA][admin@pfsense.fire.box]/root: pppcfg pppoe1
                [2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig pppoe1
                ifconfig: interface pppoe1 does not exist
                

                Then as soon as you click connect the interface is created and is shown as UP but the PPP link it still connecting:

                [2.8.0-BETA][admin@pfsense.fire.box]/root: pppcfg pppoe1
                	dev: igb1 svc: BTInfinity state: PADI sent
                	sid: 0x0 PADI retries: 3 PADR retries: 0
                	sppp: phase establish authproto auto authname "bthomehub@btbroadband.com" peerproto auto 
                [2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig pppoe1
                pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
                	description: BT
                	options=0
                	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 tentative scopeid 0x13
                	groups: pppoec
                	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
                

                Then after some time it completes the connection:

                [2.8.0-BETA][admin@pfsense.fire.box]/root: pppcfg pppoe1
                	dev: igb1 svc: BTInfinity state: session
                	sid: 0x1b6d PADI retries: 4 PADR retries: 0 time: 00:00:12
                	sppp: phase network authproto auto authname "bthomehub@btbroadband.com" peerproto auto 
                	dns: 81.139.56.100 81.139.57.100
                [2.8.0-BETA][admin@pfsense.fire.box]/root: ifconfig pppoe1
                pppoe1: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
                	description: BT
                	options=0
                	inet 217.45.X.X --> 172.16.13.252 netmask 0xffffffff
                	inet6 fe80::201:21ff:fe01:6775%pppoe1 prefixlen 64 scopeid 0x13
                	groups: pppoec
                	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>
                

                The actual time taken depends how fast the server responds. For be that be almost instant or it can be several retries and 20-30s.

                If you just refresh the Status page without clicking connect again it should just connect.

                We saw your post on Reddit yesterday and were discussing options to make the age clearer for if_pppoe. Not sure what will happen yet.

                RobbieTTR P 3 Replies Last reply Reply Quote 0
                • w0wW
                  w0w @Phil2025
                  last edited by

                  @Phil2025 said in New PPPoE backend, some feedback:

                  Many thanks for trying the connect and disconnect, good to know its not just me, I hope they pick it up and fix it as I think it could mean if the Internet goes down for some reason, it may not come back up again by itself.

                  I have the same problem. I thought it was just me. Please report it on Redmine.

                  @RobbieTT said in New PPPoE backend, some feedback:

                  It may even be linked to the PPP status logs not working, as noted by others

                  I don't think so, this is some kind of "GUI switching between backends" problem.

                  As for MTU....mine is set to 1492 but I am using MSS also custom
                  da67a5ef-ac36-420d-996e-1f92444e2eb7-{0E1ED290-FD16-4525-ADCE-89C980D30C5C}.png

                  « SpeedGuide.net TCP Analyzer Results » 
                  Tested on: 2025.04.25 05:40 
                  IP address: 84.52.xx.xx 
                  Client OS/browser: Windows 10 (Firefox 137.0) 
                   
                  TCP options string: 020405200103030801010402 
                  MSS: 1312 
                  MTU: 1352 
                  TCP Window: 65280 (not multiple of MSS) 
                  RWIN Scaling: 8 bits (2^8=256) 
                  Unscaled RWIN : 255 
                  Recommended RWINs: 62976, 125952, 251904, 503808, 1007616 
                  BDP limit (200ms): 2611 kbps (261 Kilobytes/s) 
                  BDP limit (500ms): 1044 kbps (104 Kilobytes/s) 
                  MTU Discovery: OFF 
                  TTL: 54 
                  Timestamps: OFF 
                  SACKs: ON 
                  IP ToS: 00000010 (2) 
                      Precedence: 000 (routine)
                      Delay: 0 (normal delay)
                      Throughput: 0 (normal throughput)
                      Reliability: 0 (normal reliability)
                      Cost: 1 (low cost)
                      Check bit: 0 (correct)
                  DSCP (DiffServ): CS0 000000 (0) - class 0, default traffic (RFC 2474).
                  
                  

                  And Empty fields in WAN configuration gives me:

                  « SpeedGuide.net TCP Analyzer Results » 
                  Tested on: 2025.04.25 05:55 
                  IP address: 84.52.xx.xxx 
                  Client OS/browser: Windows 10 (Firefox 137.0) 
                   
                  TCP options string: 020405ac0103030801010402 
                  MSS: 1452 
                  MTU: 1492 
                  TCP Window: 65280 (not multiple of MSS) 
                  RWIN Scaling: 8 bits (2^8=256) 
                  Unscaled RWIN : 255 
                  Recommended RWINs: 63888, 127776, 255552, 511104, 1022208 
                  BDP limit (200ms): 2611 kbps (261 Kilobytes/s) 
                  BDP limit (500ms): 1044 kbps (104 Kilobytes/s) 
                  MTU Discovery: OFF 
                  TTL: 54 
                  Timestamps: OFF 
                  SACKs: ON 
                  IP ToS: 00000010 (2) 
                      Precedence: 000 (routine)
                      Delay: 0 (normal delay)
                      Throughput: 0 (normal throughput)
                      Reliability: 0 (normal reliability)
                      Cost: 1 (low cost)
                      Check bit: 0 (correct)
                  DSCP (DiffServ): CS0 000000 (0) - class 0, default traffic (RFC 2474).
                  
                  

                  I'm pretty sure my ISP doesn't support any RFCs that would allow pushing over 1492, because the connection just fails when I set the MTU to 1500.

                  1 Reply Last reply Reply Quote 0
                  • RobbieTTR
                    RobbieTT @stephenw10
                    last edited by

                    @stephenw10 said in New PPPoE backend, some feedback:

                    pppcfg pppoe1

                    Ok, that makes sense but slightly odd that GUI seems to hang for a while.

                    New pppcfg commands noted too:

                    [25.03-BETA][admin@Router-7.xxxx.me]/root: pppcfg pppoe0
                    	dev: igc0 state: session
                    	sid: 0x1155 PADI retries: 0 PADR retries: 0 time: 14:30:28
                    	sppp: phase network authproto auto authname "FTTP.xxxx@idnet" peerproto auto 
                    	dns: 212.69.40.23 212.69.36.23
                    [25.03-BETA][admin@Router-7.xxxx.me]/root: ifconfig pppoe0
                    pppoe0: flags=1008851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
                    	description: WAN
                    	options=0
                    	inet 93.xx.xxx.xx --> 212.xx.xx.xx netmask 0xffffffff
                    	inet6 fe80::3eec:efff:xxxx:xxxx%pppoe0 prefixlen 64 scopeid 0xf
                    	inet6 2axx:xxxx:feed:xxxx:3eec:efff:xxxx:xxxx prefixlen 64 autoconf pltime 604800 vltime 2592000
                    	groups: pppoec
                    	nd6 options=123<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL,NO_DAD>
                    [25.03-BETA][admin@Router-7.xxxx.me]/root: 
                    
                    

                    ☕️

                    1 Reply Last reply Reply Quote 1
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Just to be clear if you actually have to click connect twice on the Status page to get connectivity that's a bug. But I don't think that's the case, it just shows UP with no IP when it initially reloads but is still connecting in the background and will connect without further interaction.

                      RobbieTTR 1 Reply Last reply Reply Quote 0
                      • P
                        Phil2025 @stephenw10
                        last edited by

                        @stephenw10 said in New PPPoE backend, some feedback:

                        Just to be clear if you actually have to click connect twice on the Status page to get connectivity that's a bug. But I don't think that's the case, it just shows UP with no IP when it initially reloads but is still connecting in the background and will connect without further interaction.

                        I've left it 30 or 40 seconds and its not come back up, and Gateways on the Dashboard continue to show Offline and there is no connectivity, although the WAN connection has an "UP" icon, but no IP address is ever obtained and the uptime never appears. Dropping the connection and trying again it always connects in a few seconds, although still seems a tad slower than the older code, but does connect.

                        In 2.7.2 using the original method, PPPoE never fails and comes up in a few seconds, never takes any longer, so I can't see this being an issue with the ISP. I know it only takes a few seconds as sometimes I need to drop the connection and bring it back up again to change to better routing with my ISP, and can often do this 3 or 4 times in a row in quick succession, and it is always back up in a few seconds.

                        Could it be with IF_PPPoE on dropping the connection its not closing the session nicely with the ISP, so on reconnecting almost straight away the connection is refused and the connection attempt stalls? By the time we've waited a bit, seen its not working, dropped it and tried again the original session goes stale at the ISP side and so that next attempt allows the connection? Do let me know if I can supply any logs to help diagnose this.

                        I also have the issue of the IPv6 Gateway monitoring remaining at Unknown even though IPv6 connectivity is alive and well. The only way I can get it monitoring correctly is to edit the IPv6 Gateway, disable monitoring, then reenable and it starts monitoring okay, perhaps simply restarting of the Gateway service might achieve the same thing, but I've not tried that.

                        1 Reply Last reply Reply Quote 0
                        • RobbieTTR
                          RobbieTT @stephenw10
                          last edited by

                          @stephenw10 said in New PPPoE backend, some feedback:

                          Just to be clear if you actually have to click connect twice on the Status page to get connectivity that's a bug. But I don't think that's the case, it just shows UP with no IP when it initially reloads but is still connecting in the background and will connect without further interaction.

                          Just retested and waited over 2 minutes with nothing happening, no connectivity and my PingPlotter graph (to first hop) is dead. Try again and the PingPlotter graph becomes live almost instantly with the pfSense GUI showing connected a couple of seconds later.

                          Still feels like a bug.

                          ☕️

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            what does pppcfg show at that point, after the initial 'connect' in the gui?

                            RobbieTTR 2 Replies Last reply Reply Quote 0
                            • RobbieTTR
                              RobbieTT @stephenw10
                              last edited by

                              @stephenw10

                              I've been offline for a while as went through the process again just to capture a few screenshots. It took multiple attempts to get a PPPoE connection and the WAN up and running again.

                              Moving from my previous 'bug' opinion to that of 'race condition'.

                              ☕️

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Hmm, does it just show incrementing retries? I've seen it take 5 or 6 retries whch take 30s or so for my connection. But not longer than that.

                                1 Reply Last reply Reply Quote 0
                                • RobbieTTR
                                  RobbieTT @stephenw10
                                  last edited by RobbieTT

                                  @stephenw10 said in New PPPoE backend, some feedback:

                                  what does pppcfg show at that point, after the initial 'connect' in the gui?

                                  It shows nothing and stops taking commands. When it finally works:

                                  [25.03-BETA][admin@Router-7.xxxx.me]/root: pppcfg pppoe0
                                  	dev: igc0 state: session
                                  	sid: 0x11b4 PADI retries: 0 PADR retries: 0 time: 00:46:38
                                  	sppp: phase network authproto auto authname "FTTP.xxxxx@idnet" peerproto auto 
                                  	dns: 212.69.40.23 212.69.36.23
                                  

                                  Not helpful I know.

                                  When the GUI 'hangs' for a bit I then get the ever-helpful busy message:

                                   2025-04-25 at 14.37.31-GUI droped.png

                                  Hopefully the log may be of more help.

                                  ☕️

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Huh, it just returns nothing immediately? Or appears to hang waiting for something?

                                    Returning nothing is what it does when the interface is down, before you click 'connect'. Usually!

                                    RobbieTTR 1 Reply Last reply Reply Quote 0
                                    • RobbieTTR
                                      RobbieTT @stephenw10
                                      last edited by

                                      @stephenw10

                                      As in the SSH connection stalls so nothing to access. I'm not best placed to stick the console cable in etc.

                                      Anything useful in the system log I sent?

                                      ☕️

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by stephenw10

                                        Nothing really jumps out. Except maybe the fact you have a 10M USB NIC connected. 😉 Which makes me twitch but probably isn't related.

                                        During that time you disconnected and attempted to reconnect from the Interfaces Status page? And it failed connect?

                                        RobbieTTR 2 Replies Last reply Reply Quote 0
                                        • RobbieTTR
                                          RobbieTT @stephenw10
                                          last edited by

                                          @stephenw10 said in New PPPoE backend, some feedback:

                                          Nothing really jumps out. Except maybe the fact you have a 10M USB NIC connected. 😉 Which makes me twitch but probably isn't related.

                                          I may be missing something here but there is no USB NIC connected to my pfSense system. Just a DAC and an RJ45:

                                          IMG_2387.jpeg

                                          It's the blue lighting that makes it fast of course. ✨

                                          ☕️

                                          1 Reply Last reply Reply Quote 1
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            It's probably an IPMI device then. If I had to guess.

                                            RobbieTTR 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.