Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra)

    Scheduled Pinned Locked Moved Routing and Multi WAN
    47 Posts 8 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • brookheatherB
      brookheather @patient0
      last edited by

      @patient0 Yes I am having the same problem - strange that there aren't more reports of this issue with enabling if_pppoe.

      S 1 Reply Last reply Reply Quote 0
      • S
        slu @brookheather
        last edited by

        @brookheather said in [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra):

        strange that there aren't more reports of this issue with enabling if_pppoe.

        see also some errors, but without this thread I wouldn't have noticed it at all:

        In/out packets    [..] (144.98 GiB/8.38 GiB) 
        In/out packets (pass)    [..] (144.98 GiB/8.38 GiB) 
        In/out packets (block)    [..] (2.21 MiB/60 KiB) 
        In/out errors    0/226 
        Collisions    0
        

        pfSense Gold subscription

        brookheatherB 1 Reply Last reply Reply Quote 0
        • brookheatherB
          brookheather @slu
          last edited by

          @slu thanks for confirming - @stephenw10 please note these additional reports of WAN out errors with if_pppoe.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            OK if you can try running this at the actual CLI (not the gui command prompt!) and wait for some errors to occur:
            dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'

            I managed to trigger some errors in my local test connection by restarting the server but you should see some pretty quickly with the rate you're seeing.

            It should look something like this:

            2.8.0-RELEASE][admin@m370.stevew.lan]/root: dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'
            dtrace: description 'fbt::if_inc_counter:entry ' matched 1 probe
            
            CPU     ID                    FUNCTION:NAME
              0  43915             if_inc_counter:entry 
                          if_pppoe.ko`sppp_output+0x15e
                          kernel`pf_route+0x8e5
                          kernel`pf_test+0x1093
                          kernel`pf_check_out+0x2e
                          kernel`pfil_mbuf_out+0x38
                          kernel`ip_output+0xbea
                          kernel`rip_send+0x3f4
                          kernel`sosend_generic_locked+0x60f
                          kernel`sosend_generic+0x61
                          kernel`sousrsend+0x5f
                          kernel`kern_sendit+0x144
                          kernel`sendit+0x1a8
                          kernel`sys_sendto+0x4d
                          kernel`amd64_syscall+0x115
                          kernel`0xffffffff812c145b
            

            Use ctl+c to esacpe once you have some traces.

            brookheatherB 1 Reply Last reply Reply Quote 0
            • brookheatherB
              brookheather @stephenw10
              last edited by

              @stephenw10 here you go:

              [2.8.0-RELEASE][admin@pfSense.localdomain]/root: dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'
              dtrace: description 'fbt::if_inc_counter:entry ' matched 1 probe
              CPU     ID                    FUNCTION:NAME
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                3  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                1  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                1  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                1  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`swi_net+0x128
                            kernel`ithread_loop+0x239
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x1ec
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_fwd+0x38
                            kernel`ip_tryforward+0x288
                            kernel`ip_input+0x34a
                            kernel`netisr_dispatch_src+0x1fa
                            kernel`ether_demux+0x194
                            kernel`ether_nh_input+0x32c
                            kernel`netisr_dispatch_src+0x9f
                            kernel`ether_input+0x56
                            kernel`iflib_rxeof+0xbde
                            kernel`_task_fn_rx+0x72
                            kernel`gtaskqueue_run_locked+0x14e
                            kernel`gtaskqueue_thread_loop+0xc2
                            kernel`fork_exit+0x7b
                            kernel`0xffffffff812c1bae
              
              stephenw10S 1 Reply Last reply Reply Quote 1
              • D
                dorabiatto
                last edited by

                [EN_US]

                Just wanted to share my experience that might help others dealing with the same issue described in this thread.

                I was getting a lot of “Errors Out” on the WAN interface when using pfSense with PPPoE on Vivo Fibra, specifically with the Mitrastar ONU provided by the ISP. I tried changing MTU, tuning system settings, updating pfSense — nothing helped. The errors kept increasing, affecting connection stability and performance.

                I decided to replace the Mitrastar ONU with a VSOL V2802RH ONU set to bridge mode — and that completely stopped the errors. Since the switch, pfSense shows at most 1 single error out right after boot, and then nothing more.

                Summary:

                Before (Mitrastar): constant Errors Out, especially under load.

                After (VSOL): only 1 error after boot, then stable with no further errors.

                So it seems the issue is likely related to some kind of compatibility quirk between the Mitrastar ONU and the new if_pppoe module. Switching to a different ONU (VSOL) significantly reduces or eliminates the issue.

                [PT-BR]

                Fica aqui meu relato que pode ajudar outros usuários enfrentando o mesmo problema descrito neste tópico.

                Estava tendo altos “Errors Out” na interface WAN ao usar pfSense com PPPoE da Vivo Fibra, especialmente com a ONU Mitrastar fornecida pela operadora. Nada resolvia: MTU ajustado, tunable, atualizações — e os erros persistiam, afetando a estabilidade e performance.

                Resolvi trocar a ONU por uma VSOL XPON configurada em modo bridge — e os erros pararam completamente. Desde a troca, o pfSense mostra no máximo 1 erro Out isolado após boot, e depois disso a conexão segue estável, sem erros acumulando.

                Resumo:

                Antes (Mitrastar): errors out constantes, especialmente com alto tráfego.

                Depois (VSOL): apenas 1 erro após boot, e mais nada.

                Ou seja, o problema parece estar relacionado à compatibilidade (ou comportamento estranho) da Mitrastar com o módulo PPPoE novo (if_pppoe), e a troca de ONU resolve ou mitiga o problema drasticamente.

                8ad45427-1bee-49c2-8fdd-678055d828b0-image.png

                1 Reply Last reply Reply Quote 1
                • stephenw10S
                  stephenw10 Netgate Administrator @brookheather
                  last edited by

                  @brookheather Did the PPPoE remain up during that time?

                  brookheatherB 1 Reply Last reply Reply Quote 0
                  • brookheatherB
                    brookheather @stephenw10
                    last edited by

                    @stephenw10 Yes those errors are beig produced while the PPPoE connection is up and running.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      OK try:
                      dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

                      If that shows ENOBUFS (55) then it looks like it's exhausting the available buffers on your system for some reason. We can try just increasing it to allow for larger bursts.

                      brookheatherB 1 Reply Last reply Reply Quote 0
                      • brookheatherB
                        brookheather @stephenw10
                        last edited by

                        @stephenw10 here is the output:

                        [2.8.0-RELEASE][admin@pfSense.localdomain]/root: dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'
                        dtrace: description 'fbt::if_sendq_enqueue:return ' matched 1 probe
                        CPU     ID                    FUNCTION:NAME
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                          1  65524          if_sendq_enqueue:return 
                                      if_pppoe.ko`sppp_output+0x1d0
                                      kernel`pf_route+0x8e5
                                      kernel`pf_test+0x1093
                                      kernel`pf_check_out+0x2e
                                      kernel`pfil_mbuf_fwd+0x38
                                      kernel`ip_tryforward+0x288
                                      kernel`ip_input+0x34a
                                      kernel`swi_net+0x128
                                      kernel`ithread_loop+0x239
                                      kernel`fork_exit+0x7b
                                      kernel`0xffffffff812c1bae
                        => 55
                        
                        
                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by stephenw10

                          Ok great so it is exhausting the interface buffer.

                          Check the current buffer size: sysctl net.link.ifqmaxlen

                          Try increasing it by 50%. It's a loader value so create the file /boot/loader.conf.local and add the line:
                          net.link.ifqmaxlen=192

                          Then reboot. And see if that makes any difference.

                          brookheatherB 1 Reply Last reply Reply Quote 0
                          • brookheatherB
                            brookheather @stephenw10
                            last edited by brookheather

                            @stephenw10 the current size is 128 - I will apply the change and reboot - might be tomorrow now. Why would this only be affecting the new if_pppoe code - how does it differ from the old code? Is it really an error if the buffer is full - doesn't it just wait until it can write the packet?

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              The if_pppoe code is very different so there could be any number of reasons.

                              Another interesting question might be why it's only affecting a small number of users. Something unique about your connection maybe.

                              brookheatherB 1 Reply Last reply Reply Quote 0
                              • brookheatherB
                                brookheather @stephenw10
                                last edited by

                                @stephenw10 I increased net.link.ifqmaxlen to 192 and rebooted - checked this was set correctly but still getting WAN out errors - it doesn't seem to have made any difference.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, actually at the same rate? Not reduced at all?

                                  brookheatherB 1 Reply Last reply Reply Quote 0
                                  • brookheatherB
                                    brookheather @stephenw10
                                    last edited by

                                    @stephenw10 if anything higher:

                                    In/out packets 946815/304707 (1.15 GiB/161.23 MiB)
                                    In/out packets (pass) 946815/304707 (1.15 GiB/161.23 MiB)
                                    In/out packets (block) 770/0 (69 KiB/0 B)
                                    In/out errors 0/3250
                                    Collisions 0

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      OK try setting that to something much larger like 1024. If it still throws errors we know it's not simply bursting up to the buffer.

                                      You could also try checking the error type again with:
                                      dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

                                      Make sure it's still throwing error 55 with the buffer at 192B

                                      brookheatherB 2 Replies Last reply Reply Quote 0
                                      • brookheatherB
                                        brookheather @stephenw10
                                        last edited by

                                        @stephenw10 still throwing error 55 with buffer set to 192 - will try 1024 later.

                                        1 Reply Last reply Reply Quote 0
                                        • brookheatherB
                                          brookheather @stephenw10
                                          last edited by brookheather

                                          @stephenw10 I increased the setting to 1024, rebooted and still getting loads of errors with code 55.

                                          In/out packets 382831/240827 (445.27 MiB/265.41 MiB)
                                          In/out packets (pass) 382831/240827 (445.27 MiB/265.41 MiB)
                                          In/out packets (block) 1255/0 (168 KiB/0 B)
                                          In/out errors 0/5242
                                          Collisions 0

                                          1 Reply Last reply Reply Quote 2
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Ok, we'll dig deeper.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.