Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra)

    Scheduled Pinned Locked Moved Routing and Multi WAN
    47 Posts 8 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • brookheatherB
      brookheather @stephenw10
      last edited by brookheather

      @stephenw10 the current size is 128 - I will apply the change and reboot - might be tomorrow now. Why would this only be affecting the new if_pppoe code - how does it differ from the old code? Is it really an error if the buffer is full - doesn't it just wait until it can write the packet?

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        The if_pppoe code is very different so there could be any number of reasons.

        Another interesting question might be why it's only affecting a small number of users. Something unique about your connection maybe.

        brookheatherB 1 Reply Last reply Reply Quote 0
        • brookheatherB
          brookheather @stephenw10
          last edited by

          @stephenw10 I increased net.link.ifqmaxlen to 192 and rebooted - checked this was set correctly but still getting WAN out errors - it doesn't seem to have made any difference.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Hmm, actually at the same rate? Not reduced at all?

            brookheatherB 1 Reply Last reply Reply Quote 0
            • brookheatherB
              brookheather @stephenw10
              last edited by

              @stephenw10 if anything higher:

              In/out packets 946815/304707 (1.15 GiB/161.23 MiB)
              In/out packets (pass) 946815/304707 (1.15 GiB/161.23 MiB)
              In/out packets (block) 770/0 (69 KiB/0 B)
              In/out errors 0/3250
              Collisions 0

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                OK try setting that to something much larger like 1024. If it still throws errors we know it's not simply bursting up to the buffer.

                You could also try checking the error type again with:
                dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

                Make sure it's still throwing error 55 with the buffer at 192B

                brookheatherB 2 Replies Last reply Reply Quote 0
                • brookheatherB
                  brookheather @stephenw10
                  last edited by

                  @stephenw10 still throwing error 55 with buffer set to 192 - will try 1024 later.

                  1 Reply Last reply Reply Quote 0
                  • brookheatherB
                    brookheather @stephenw10
                    last edited by brookheather

                    @stephenw10 I increased the setting to 1024, rebooted and still getting loads of errors with code 55.

                    In/out packets 382831/240827 (445.27 MiB/265.41 MiB)
                    In/out packets (pass) 382831/240827 (445.27 MiB/265.41 MiB)
                    In/out packets (block) 1255/0 (168 KiB/0 B)
                    In/out errors 0/5242
                    Collisions 0

                    1 Reply Last reply Reply Quote 2
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Ok, we'll dig deeper.

                      1 Reply Last reply Reply Quote 0
                      • T
                        tlex
                        last edited by

                        Would you have any VIP on the WAN interface by any chance ?

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          There is a patch for that but the symptoms there are generally that the link continually reconnects.

                          But, seperately, @brookheather are you using ALTQ traffic shaping on the pppoe interface? It could be exhausting the queue on that.

                          brookheatherB 1 Reply Last reply Reply Quote 0
                          • brookheatherB
                            brookheather @stephenw10
                            last edited by

                            @stephenw10 Yes I am using the CODELQ traffic shaper on both LAN and WAN. WAN is set to 73Mb/s (for 500/75 connection).

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Aha, and what is the queue length set to on WAN? Try increasing it and see if that reduces the output errors.

                              brookheatherB 1 Reply Last reply Reply Quote 0
                              • brookheatherB
                                brookheather @stephenw10
                                last edited by

                                @stephenw10 do you mean the Queue Limit on the WAN shaper? I don't have anything set for this.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Yes. It's usually set to 50 packets by default. You can check that in Status > Queues.

                                  Try setting it to 100 and see if that changes anything.

                                  brookheatherB 1 Reply Last reply Reply Quote 0
                                  • brookheatherB
                                    brookheather @stephenw10
                                    last edited by

                                    @stephenw10 So I just deleted my WAN shaper (but retained the LAN shaper) and re-enabled if_pppoe - after a reboot I ran a number of speed tests and there were no WAN out errors so it seems to be an issue with having a WAN shaper in conjunction with if_pppoe enabled. I would prefer to have a WAN shaper enabled as it can help when I WFH and max out the 75Mb/s upload capacity during a grid compile. I will leave the config as is for the moment and see how I get on without the WAN shaper.

                                    K 1 Reply Last reply Reply Quote 0
                                    • K
                                      kprovost @brookheather
                                      last edited by

                                      @brookheather It's entirely normal for a shaper to drop packets (in fact, that's how they do the shaping). if_pppoe counts those packets drops.
                                      The 'error' count does not indicate a bug or even anything going wrong.

                                      brookheatherB 1 Reply Last reply Reply Quote 0
                                      • brookheatherB
                                        brookheather @kprovost
                                        last edited by brookheather

                                        @kprovost yes I think so as well but the old PPPoE code doesn't count these traffic shaper drops as errors so the question is whether the new if_pppoe should count these as errors - I would only want to see "real" errors - btw I don't see any errors on the LAN side due to the traffic shaper which I have retained.

                                        1 Reply Last reply Reply Quote 0
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Yeah, you won't usually see drops on LAN because the bandwidth limit there is upstream on the pppoe link. It can send packets out to LAN clients much faster than it receives them and hence no need to drop any. But the other way it needs to queue and/or drop packets because 75Mbps is much lower than 1G.

                                          You might be better off using Limiters based shaping instead of ALTQ:
                                          https://docs.netgate.com/pfsense/en/latest/recipes/codel-limiters.html

                                          That can be applied to traffic on the LAN side.

                                          J brookheatherB 2 Replies Last reply Reply Quote 0
                                          • J
                                            jonathan.young @stephenw10
                                            last edited by

                                            @stephenw10 I have a similar problem to this (I posted earlier in this thread). I had traffic shaping enabled on my WAN interface so I disabled it and tried again with the new PPPoE module (if_pppoe). In my case, it still doesn't work. I cannot get to the web interface but can ssh. System.log is filled with messages like these:

                                            Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:30 firewall check_reload_status[531]: rc.newwanip starting pppoe0
Jun 21 14:17:30 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall kernel: ovpnc6: link state changed to DOWN
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall kernel: ovpnc4: link state changed to DOWN
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall kernel: ovpnc3: link state changed to UP
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: rc.newwanip starting ovpnc3
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:31 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket
Jun 21 14:17:32 firewall check_reload_status[531]: Could not connect to /var/run/php-fpm.socket

                                            I do have some OpenVPN and WireGuard client connections to VPN servers. Would that make a difference? Disabling the new PPPoE module (if_pppoe) returns everything back to normal. So it's still broken for me.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.