Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-1100 - [zone: mbuf_cluster] kern.ipc.nmbclusters limit reached

    Scheduled Pinned Locked Moved
    Hardware
    2
    15
    382
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J
      jvmack22
      last edited by

      Did a fresh install on a SG-1100 with 23.09.1 yesterday.
      Added a client OpenVPN tunnel, and installed Pfblocker.
      This morning the router crashed, in the logfiles it was many times following:

      kernel [zone: mbuf_cluster] kern.ipc.nmbclusters limit reached

      How to finetune the SG-1100 on Mbuf? (This is where it seems to be about seeing the forum)

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        First check what they are set at and how much is used:

        [23.09.1-RELEASE][root@1100-3.stevew.lan]/root: netstat -m
1521/1224/2745 mbufs in use (current/cache/total)
669/591/1260/59793 mbuf clusters in use (current/cache/total/max)
669/584 mbuf+clusters out of packet secondary zone in use (current/cache)
0/12/12/29896 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/8858 9k jumbo clusters in use (current/cache/total/max)
0/0/0/4982 16k jumbo clusters in use (current/cache/total/max)
1731K/1536K/3267K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 sendfile syscalls
0 sendfile syscalls completed without I/O request
0 requests for I/O initiated by sendfile
0 pages read by sendfile as part of a request
0 pages were valid at time of a sendfile request
0 pages were valid and substituted to bogus page
0 pages were requested for read ahead by applications
0 pages were read ahead by sendfile
0 times sendfile encountered an already busy page
0 requests for sfbufs denied
0 requests for sfbufs delayed

        Steve

        J 1 Reply Last reply Reply Quote 0
        • J
          jvmack22 @stephenw10
          last edited by stephenw10

          @stephenw10

          11 hours switched on. Today it crashed again after 24 hours more or less. So 13 hours to go .

          65769/1806/67575 mbufs in use (current/cache/total)
30761/735/31496/59766 mbuf clusters in use (current/cache/total/max)
30761/735 mbuf+clusters out of packet secondary zone in use (current/cache)
1/507/508/29883 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/8854 9k jumbo clusters in use (current/cache/total/max)
0/0/0/4980 16k jumbo clusters in use (current/cache/total/max)
77981K/3949K/81931K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 sendfile syscalls
0 sendfile syscalls completed without I/O request
0 requests for I/O initiated by sendfile
0 pages read by sendfile as part of a request
0 pages were valid at time of a sendfile request
0 pages were valid and substituted to bogus page
0 pages were requested for read ahead by applications
0 pages were read ahead by sendfile
0 times sendfile encountered an already busy page
0 requests for sfbufs denied
0 requests for sfbufs delayed
          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Check the mbuf usage in Status > Monitoring. Is it climbing?

            It's already quite high there. What traffic is using that firewall?

            J 2 Replies Last reply Reply Quote 0
            • J
              jvmack22 @stephenw10
              last edited by

              @stephenw10

              Not much:

              • 2 workstations with VPN connecting to ext.office where servers are running. Nextcloud, Zimbra etc.
              • When workstations are not running, some 2-3 mobile phones at a time max. now and then. Thats it.
              • a satelite receiver connected with CCca*m to decode channels

              The only difference with the fresh install is that i've installed PfBlocker.
              Below the stats after 20 hours with PfBlocker disabled:

              15037/1733/16770 mbufs in use (current/cache/total)
              5391/959/6350/59797 mbuf clusters in use (current/cache/total/max)
              5391/959 mbuf+clusters out of packet secondary zone in use (current/cache)
              1/761/762/29898 4k (page size) jumbo clusters in use (current/cache/total/max)
              0/0/0/8858 9k jumbo clusters in use (current/cache/total/max)
              0/0/0/4983 16k jumbo clusters in use (current/cache/total/max)
              14558K/5395K/19953K bytes allocated to network (current/cache/total)
              0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
              0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
              0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
              0/0/0 requests for jumbo clusters denied (4k/9k/16k)
              0 sendfile syscalls
              0 sendfile syscalls completed without I/O request
              0 requests for I/O initiated by sendfile
              0 pages read by sendfile as part of a request
              0 pages were valid at time of a sendfile request
              0 pages were valid and substituted to bogus page
              0 pages were requested for read ahead by applications
              0 pages were read ahead by sendfile
              0 times sendfile encountered an already busy page
              0 requests for sfbufs denied
              0 requests for sfbufs delayed

              1 Reply Last reply Reply Quote 0
              • J
                jvmack22 @stephenw10
                last edited by jvmack22

                @stephenw10

                Data Summary
                Minimum Average Maximum Last 95th Percentile
                user util. 0.92 % 4.88 % 23.12% 4.54 %
                nice util. 0.00 % 0.11 % 0.75% 0.30 %
                system util. 0.65 % 3.09 % 10.88% 3.48 %
                interrupt 0.00 % 0.23 % 3.59% 0.63 %
                processes 238.00 240.35 247.61 240.00

                Tomorrow I can see if its climbing, though I don't know what is mbuf in these monitoring graphs.

                J 1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  The graphs should show here:

                  Screenshot from 2023-12-14 19-15-56.png

                  There's pretty much no traffic passing that test box though

                  Steve

                  J 2 Replies Last reply Reply Quote 0
                  • J
                    jvmack22 @stephenw10
                    last edited by jvmack22

                    This post is deleted!
                    1 Reply Last reply Reply Quote 0
                    • J
                      jvmack22 @stephenw10
                      last edited by

                      @stephenw10

                      Sorry, just learned this function, never used it before :)

                      Here you can see when installed from scratch, last coupld of days:

                      Screenshot_20231214_203804.png

                      1 Reply Last reply Reply Quote 0
                      • J
                        jvmack22 @jvmack22
                        last edited by jvmack22

                        @jvmack22

                        But there where no RAM issues I think in the SG-1100:
                        (My main box Dell server 16Gb RAM hits 1 million Mbuff max. What are normal numbers?)

                        Screenshot_20231214_204420.png

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          1M is waay higher than it ever needs to be but that's currently the default for x86.

                          The ~50K you see there is the default for the 1100. It does look like something is leaking there though.

                          What do you have configured on that box?

                          J 1 Reply Last reply Reply Quote 0
                          • J
                            jvmack22 @stephenw10
                            last edited by

                            @stephenw10

                            • 2 workstations with VPN connecting to ext.office where servers are running. Nextcloud, Zimbra etc.
                            • When workstations are not running, some 2-3 mobile phones at a time max. now and then. Thats it.
                            • a satelite receiver connected with CCca*m to decode channels. (NOT streaming)

                            When it comes to installed packages it is just PfBlocker, nothing else on the default / fresh configuration.
                            It had many PRI1-2-3-4-5 lists on it. Switching Pfblocker off makes the differences of above.
                            What I'm trying now is just the most important blocklists, some 8-9 lists of PRI 1-2-3.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              So that's the OpenVPN on the firewall?

                              Do you have anything configured like igmpproxy or traffic shaping?

                              Anything less common like PPPoE server or L2TP?

                              J 1 Reply Last reply Reply Quote 0
                              • J
                                jvmack22 @stephenw10
                                last edited by

                                @stephenw10

                                No nothing at all like that, just OpenVPN (and Pfblocker)

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Just to be clear the workstations using a VPN are using the OpenVPN link on firewall? Rather than VPNs from each workstation directly?

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post