Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra)

    Scheduled Pinned Locked Moved Routing and Multi WAN
    47 Posts 8 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • brookheatherB
      brookheather @stephenw10
      last edited by

      @stephenw10 I increased net.link.ifqmaxlen to 192 and rebooted - checked this was set correctly but still getting WAN out errors - it doesn't seem to have made any difference.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, actually at the same rate? Not reduced at all?

        brookheatherB 1 Reply Last reply Reply Quote 0
        • brookheatherB
          brookheather @stephenw10
          last edited by

          @stephenw10 if anything higher:

          In/out packets 946815/304707 (1.15 GiB/161.23 MiB)
          In/out packets (pass) 946815/304707 (1.15 GiB/161.23 MiB)
          In/out packets (block) 770/0 (69 KiB/0 B)
          In/out errors 0/3250
          Collisions 0

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            OK try setting that to something much larger like 1024. If it still throws errors we know it's not simply bursting up to the buffer.

            You could also try checking the error type again with:
            dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

            Make sure it's still throwing error 55 with the buffer at 192B

            brookheatherB 2 Replies Last reply Reply Quote 0
            • brookheatherB
              brookheather @stephenw10
              last edited by

              @stephenw10 still throwing error 55 with buffer set to 192 - will try 1024 later.

              1 Reply Last reply Reply Quote 0
              • brookheatherB
                brookheather @stephenw10
                last edited by brookheather

                @stephenw10 I increased the setting to 1024, rebooted and still getting loads of errors with code 55.

                In/out packets 382831/240827 (445.27 MiB/265.41 MiB)
                In/out packets (pass) 382831/240827 (445.27 MiB/265.41 MiB)
                In/out packets (block) 1255/0 (168 KiB/0 B)
                In/out errors 0/5242
                Collisions 0

                1 Reply Last reply Reply Quote 2
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Ok, we'll dig deeper.

                  1 Reply Last reply Reply Quote 0
                  • T
                    tlex
                    last edited by

                    Would you have any VIP on the WAN interface by any chance ?

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      There is a patch for that but the symptoms there are generally that the link continually reconnects.

                      But, seperately, @brookheather are you using ALTQ traffic shaping on the pppoe interface? It could be exhausting the queue on that.

                      brookheatherB 1 Reply Last reply Reply Quote 0
                      • brookheatherB
                        brookheather @stephenw10
                        last edited by

                        @stephenw10 Yes I am using the CODELQ traffic shaper on both LAN and WAN. WAN is set to 73Mb/s (for 500/75 connection).

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Aha, and what is the queue length set to on WAN? Try increasing it and see if that reduces the output errors.

                          brookheatherB 1 Reply Last reply Reply Quote 0
                          • brookheatherB
                            brookheather @stephenw10
                            last edited by

                            @stephenw10 do you mean the Queue Limit on the WAN shaper? I don't have anything set for this.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Yes. It's usually set to 50 packets by default. You can check that in Status > Queues.

                              Try setting it to 100 and see if that changes anything.

                              brookheatherB 1 Reply Last reply Reply Quote 0
                              • brookheatherB
                                brookheather @stephenw10
                                last edited by

                                @stephenw10 So I just deleted my WAN shaper (but retained the LAN shaper) and re-enabled if_pppoe - after a reboot I ran a number of speed tests and there were no WAN out errors so it seems to be an issue with having a WAN shaper in conjunction with if_pppoe enabled. I would prefer to have a WAN shaper enabled as it can help when I WFH and max out the 75Mb/s upload capacity during a grid compile. I will leave the config as is for the moment and see how I get on without the WAN shaper.

                                K 1 Reply Last reply Reply Quote 0
                                • K
                                  kprovost @brookheather
                                  last edited by

                                  @brookheather It's entirely normal for a shaper to drop packets (in fact, that's how they do the shaping). if_pppoe counts those packets drops.
                                  The 'error' count does not indicate a bug or even anything going wrong.

                                  brookheatherB 1 Reply Last reply Reply Quote 0
                                  • brookheatherB
                                    brookheather @kprovost
                                    last edited by brookheather

                                    @kprovost yes I think so as well but the old PPPoE code doesn't count these traffic shaper drops as errors so the question is whether the new if_pppoe should count these as errors - I would only want to see "real" errors - btw I don't see any errors on the LAN side due to the traffic shaper which I have retained.

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      Yeah, you won't usually see drops on LAN because the bandwidth limit there is upstream on the pppoe link. It can send packets out to LAN clients much faster than it receives them and hence no need to drop any. But the other way it needs to queue and/or drop packets because 75Mbps is much lower than 1G.

                                      You might be better off using Limiters based shaping instead of ALTQ:
                                      https://docs.netgate.com/pfsense/en/latest/recipes/codel-limiters.html

                                      That can be applied to traffic on the LAN side.

                                      J brookheatherB 2 Replies Last reply Reply Quote 0
                                      • J
                                        jonathan.young @stephenw10
                                        last edited by

                                        @stephenw10 I have a similar problem to this (I posted earlier in this thread). I had traffic shaping enabled on my WAN interface so I disabled it and tried again with the new PPPoE module (if_pppoe). In my case, it still doesn't work. I cannot get to the web interface but can ssh. System.log is filled with messages like these:

                                        Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: rc.newwanip starting pppoe0
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall kernel: ovpnc6: link state changed to DOWN
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall kernel: ovpnc4: link state changed to DOWN
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall kernel: ovpnc3: link state changed to UP
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: rc.newwanip starting ovpnc3
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket

                                        I do have some OpenVPN and WireGuard client connections to VPN servers. Would that make a difference? Disabling the new PPPoE module (if_pppoe) returns everything back to normal. So it's still broken for me.

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Hmm, that looks like PHP stopped responding. But that could be a symptom of whatever check_reload_status is doing.

                                          Does it clear it if you restart php from the console menu?

                                          Is that the first error logged after enabling the shaper?

                                          How is the shaper configured?

                                          1 Reply Last reply Reply Quote 0
                                          • brookheatherB
                                            brookheather @stephenw10
                                            last edited by brookheather

                                            @stephenw10 thanks - I have now moved to using limiters with if_pppoe and these work fine without generating any WAN out errors. I followed the guide - can I ask that you update the floating rule documentation slightly and add a note for IPv6 as the Source should by "Any" instead of "WAN Address" for IPv6 - otherwise the limiter has no effect.

                                            If the expected behaviour for if_pppoe is that it will increment the WAN out error count for ALTQ shaper dropped packets then it would be useful to note this by the if_pppoe checkbox - perhaps state that ALTQ shapers are not recommended and limiters should be used in their place?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.