Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra)

    Scheduled Pinned Locked Moved Routing and Multi WAN
    47 Posts 8 Posters 3.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • patient0P
      patient0
      last edited by

      Maybe it is the same issue as Netgate forum thread: WAN out errors since 2.8 upgrade?

      brookheatherB 1 Reply Last reply Reply Quote 0
      • brookheatherB
        brookheather @patient0
        last edited by

        @patient0 Yes I am having the same problem - strange that there aren't more reports of this issue with enabling if_pppoe.

        S 1 Reply Last reply Reply Quote 0
        • S
          slu @brookheather
          last edited by

          @brookheather said in [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra):

          strange that there aren't more reports of this issue with enabling if_pppoe.

          see also some errors, but without this thread I wouldn't have noticed it at all:

          In/out packets    [..] (144.98 GiB/8.38 GiB) 
          In/out packets (pass)    [..] (144.98 GiB/8.38 GiB) 
          In/out packets (block)    [..] (2.21 MiB/60 KiB) 
          In/out errors    0/226 
          Collisions    0
          

          pfSense Gold subscription

          brookheatherB 1 Reply Last reply Reply Quote 0
          • brookheatherB
            brookheather @slu
            last edited by

            @slu thanks for confirming - @stephenw10 please note these additional reports of WAN out errors with if_pppoe.

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              OK if you can try running this at the actual CLI (not the gui command prompt!) and wait for some errors to occur:
              dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'

              I managed to trigger some errors in my local test connection by restarting the server but you should see some pretty quickly with the rate you're seeing.

              It should look something like this:

              2.8.0-RELEASE][admin@m370.stevew.lan]/root: dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'
              dtrace: description 'fbt::if_inc_counter:entry ' matched 1 probe
              
              CPU     ID                    FUNCTION:NAME
                0  43915             if_inc_counter:entry 
                            if_pppoe.ko`sppp_output+0x15e
                            kernel`pf_route+0x8e5
                            kernel`pf_test+0x1093
                            kernel`pf_check_out+0x2e
                            kernel`pfil_mbuf_out+0x38
                            kernel`ip_output+0xbea
                            kernel`rip_send+0x3f4
                            kernel`sosend_generic_locked+0x60f
                            kernel`sosend_generic+0x61
                            kernel`sousrsend+0x5f
                            kernel`kern_sendit+0x144
                            kernel`sendit+0x1a8
                            kernel`sys_sendto+0x4d
                            kernel`amd64_syscall+0x115
                            kernel`0xffffffff812c145b
              

              Use ctl+c to esacpe once you have some traces.

              brookheatherB 1 Reply Last reply Reply Quote 0
              • brookheatherB
                brookheather @stephenw10
                last edited by

                @stephenw10 here you go:

                [2.8.0-RELEASE][admin@pfSense.localdomain]/root: dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'
                dtrace: description 'fbt::if_inc_counter:entry ' matched 1 probe
                CPU     ID                    FUNCTION:NAME
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  3  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  1  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  1  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  1  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`swi_net+0x128
                              kernel`ithread_loop+0x239
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                  0  43915             if_inc_counter:entry 
                              if_pppoe.ko`sppp_output+0x1ec
                              kernel`pf_route+0x8e5
                              kernel`pf_test+0x1093
                              kernel`pf_check_out+0x2e
                              kernel`pfil_mbuf_fwd+0x38
                              kernel`ip_tryforward+0x288
                              kernel`ip_input+0x34a
                              kernel`netisr_dispatch_src+0x1fa
                              kernel`ether_demux+0x194
                              kernel`ether_nh_input+0x32c
                              kernel`netisr_dispatch_src+0x9f
                              kernel`ether_input+0x56
                              kernel`iflib_rxeof+0xbde
                              kernel`_task_fn_rx+0x72
                              kernel`gtaskqueue_run_locked+0x14e
                              kernel`gtaskqueue_thread_loop+0xc2
                              kernel`fork_exit+0x7b
                              kernel`0xffffffff812c1bae
                
                stephenw10S 1 Reply Last reply Reply Quote 1
                • D
                  dorabiatto
                  last edited by

                  [EN_US]

                  Just wanted to share my experience that might help others dealing with the same issue described in this thread.

                  I was getting a lot of “Errors Out” on the WAN interface when using pfSense with PPPoE on Vivo Fibra, specifically with the Mitrastar ONU provided by the ISP. I tried changing MTU, tuning system settings, updating pfSense — nothing helped. The errors kept increasing, affecting connection stability and performance.

                  I decided to replace the Mitrastar ONU with a VSOL V2802RH ONU set to bridge mode — and that completely stopped the errors. Since the switch, pfSense shows at most 1 single error out right after boot, and then nothing more.

                  Summary:

                  Before (Mitrastar): constant Errors Out, especially under load.

                  After (VSOL): only 1 error after boot, then stable with no further errors.

                  So it seems the issue is likely related to some kind of compatibility quirk between the Mitrastar ONU and the new if_pppoe module. Switching to a different ONU (VSOL) significantly reduces or eliminates the issue.

                  [PT-BR]

                  Fica aqui meu relato que pode ajudar outros usuários enfrentando o mesmo problema descrito neste tópico.

                  Estava tendo altos “Errors Out” na interface WAN ao usar pfSense com PPPoE da Vivo Fibra, especialmente com a ONU Mitrastar fornecida pela operadora. Nada resolvia: MTU ajustado, tunable, atualizações — e os erros persistiam, afetando a estabilidade e performance.

                  Resolvi trocar a ONU por uma VSOL XPON configurada em modo bridge — e os erros pararam completamente. Desde a troca, o pfSense mostra no máximo 1 erro Out isolado após boot, e depois disso a conexão segue estável, sem erros acumulando.

                  Resumo:

                  Antes (Mitrastar): errors out constantes, especialmente com alto tráfego.

                  Depois (VSOL): apenas 1 erro após boot, e mais nada.

                  Ou seja, o problema parece estar relacionado à compatibilidade (ou comportamento estranho) da Mitrastar com o módulo PPPoE novo (if_pppoe), e a troca de ONU resolve ou mitiga o problema drasticamente.

                  8ad45427-1bee-49c2-8fdd-678055d828b0-image.png

                  1 Reply Last reply Reply Quote 1
                  • stephenw10S
                    stephenw10 Netgate Administrator @brookheather
                    last edited by

                    @brookheather Did the PPPoE remain up during that time?

                    brookheatherB 1 Reply Last reply Reply Quote 0
                    • brookheatherB
                      brookheather @stephenw10
                      last edited by

                      @stephenw10 Yes those errors are beig produced while the PPPoE connection is up and running.

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        OK try:
                        dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

                        If that shows ENOBUFS (55) then it looks like it's exhausting the available buffers on your system for some reason. We can try just increasing it to allow for larger bursts.

                        brookheatherB 1 Reply Last reply Reply Quote 0
                        • brookheatherB
                          brookheather @stephenw10
                          last edited by

                          @stephenw10 here is the output:

                          [2.8.0-RELEASE][admin@pfSense.localdomain]/root: dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'
                          dtrace: description 'fbt::if_sendq_enqueue:return ' matched 1 probe
                          CPU     ID                    FUNCTION:NAME
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                            1  65524          if_sendq_enqueue:return 
                                        if_pppoe.ko`sppp_output+0x1d0
                                        kernel`pf_route+0x8e5
                                        kernel`pf_test+0x1093
                                        kernel`pf_check_out+0x2e
                                        kernel`pfil_mbuf_fwd+0x38
                                        kernel`ip_tryforward+0x288
                                        kernel`ip_input+0x34a
                                        kernel`swi_net+0x128
                                        kernel`ithread_loop+0x239
                                        kernel`fork_exit+0x7b
                                        kernel`0xffffffff812c1bae
                          => 55
                          
                          
                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by stephenw10

                            Ok great so it is exhausting the interface buffer.

                            Check the current buffer size: sysctl net.link.ifqmaxlen

                            Try increasing it by 50%. It's a loader value so create the file /boot/loader.conf.local and add the line:
                            net.link.ifqmaxlen=192

                            Then reboot. And see if that makes any difference.

                            brookheatherB 1 Reply Last reply Reply Quote 0
                            • brookheatherB
                              brookheather @stephenw10
                              last edited by brookheather

                              @stephenw10 the current size is 128 - I will apply the change and reboot - might be tomorrow now. Why would this only be affecting the new if_pppoe code - how does it differ from the old code? Is it really an error if the buffer is full - doesn't it just wait until it can write the packet?

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                The if_pppoe code is very different so there could be any number of reasons.

                                Another interesting question might be why it's only affecting a small number of users. Something unique about your connection maybe.

                                brookheatherB 1 Reply Last reply Reply Quote 0
                                • brookheatherB
                                  brookheather @stephenw10
                                  last edited by

                                  @stephenw10 I increased net.link.ifqmaxlen to 192 and rebooted - checked this was set correctly but still getting WAN out errors - it doesn't seem to have made any difference.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Hmm, actually at the same rate? Not reduced at all?

                                    brookheatherB 1 Reply Last reply Reply Quote 0
                                    • brookheatherB
                                      brookheather @stephenw10
                                      last edited by

                                      @stephenw10 if anything higher:

                                      In/out packets 946815/304707 (1.15 GiB/161.23 MiB)
                                      In/out packets (pass) 946815/304707 (1.15 GiB/161.23 MiB)
                                      In/out packets (block) 770/0 (69 KiB/0 B)
                                      In/out errors 0/3250
                                      Collisions 0

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        OK try setting that to something much larger like 1024. If it still throws errors we know it's not simply bursting up to the buffer.

                                        You could also try checking the error type again with:
                                        dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

                                        Make sure it's still throwing error 55 with the buffer at 192B

                                        brookheatherB 2 Replies Last reply Reply Quote 0
                                        • brookheatherB
                                          brookheather @stephenw10
                                          last edited by

                                          @stephenw10 still throwing error 55 with buffer set to 192 - will try 1024 later.

                                          1 Reply Last reply Reply Quote 0
                                          • brookheatherB
                                            brookheather @stephenw10
                                            last edited by brookheather

                                            @stephenw10 I increased the setting to 1024, rebooted and still getting loads of errors with code 55.

                                            In/out packets 382831/240827 (445.27 MiB/265.41 MiB)
                                            In/out packets (pass) 382831/240827 (445.27 MiB/265.41 MiB)
                                            In/out packets (block) 1255/0 (168 KiB/0 B)
                                            In/out errors 0/5242
                                            Collisions 0

                                            1 Reply Last reply Reply Quote 2
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.