Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense stops working after some time: no buffer space available

    Scheduled Pinned Locked Moved General pfSense Questions
    25 Posts 6 Posters 4.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      fabiolanza
      last edited by

      Hi,

      I am going through a strange problem. My pfSense works perfectly for about 20 hours and then it stops routing traffic by itself, without any touches to pfSense configuration or the network.

      After problem kicks in, from the pfSense console:
      ping gives me the following error:

      ping: sendto: No buffer space available
      

      tracert gives me the following error:

      traceroute: sendto: No buffer space available
      

      If I restart the network interfaces, all routing is gone, and network interfaces no longer get an ip address. If I try to add the routes manually, I get the following error:

      route: writing to routing socket: Network is unreachable
      

      System:
      pfSense is running as a virtual machine on Proxmox. The network ports are USB 3.0 -> Ethernet 1Gbps adapters with USB passthrough: AX88179
      The correct drivers are installed, and this adapter is officially supported by FreeBSD.

      The problem is fixed by restarting the system entirely. Below are some screenshots of the problem.

      Any advice on the cause of the problem is greatly appreciated.

      Thanks

      alt text
      alt text
      alt text
      alt text
      alt text
      alt text
      alt text
      alt text

      1 Reply Last reply Reply Quote 0
      • DaddyGoD
        DaddyGo
        last edited by DaddyGo

        @fabiolanza said in pfSense stops working after some time: no buffer space available:

        Sorry, but it's already a weekend
        isn't that just a joke?

        The network ports are USB 3.0 -> Ethernet 1Gbps adapters with USB passthrough

        by the way, Realtek ethernet controllers are also supported by FreeBSD, but I'll tell you they're not really usable ✋ ☺

        I have to say, I didn't even think, that the pfSense would start ... - VM + with USB eth. + USB passthrough

        • huge hurray for the system -

        you really don't think that, I'm making to joke with you, I wouldn't have thought it would work in this form ✋ ✌

        Cats bury it so they can't see it!
        (You know what I mean if you have a cat)

        F 1 Reply Last reply Reply Quote 0
        • F
          fabiolanza
          last edited by

          I found out that, as it seems, the no buffer space available problem is quite common. There is a page about it in the Netgate documentation

          I did what is proposed in this page (set kern.ipc.nmbclusters to 1,000,000): https://docs.netgate.com/pfsense/en/latest/hardware/tuning-and-troubleshooting-network-cards.html

          I see that there are other buffers such as net.inet.tcp.recvspace and net.inet.tcp.sendspace. Should I increase their buffers too?

          This is what my System Information dashboard is showing up:
          alt text

          From console netstat -m:
          alt text

          I will keep monitoring now to see if the system becomes more stable.

          1 Reply Last reply Reply Quote 0
          • F
            fabiolanza @DaddyGo
            last edited by

            @DaddyGo are there any official recommendations not to use pfSense in a virtual environment? Your comment is not helpful at all.

            DaddyGoD 1 Reply Last reply Reply Quote 0
            • F
              fishbone222
              last edited by

              Not really helpful but sadly true. pfSense in an VM can work but is definitely not newby-friendly.
              Many aspects that cause extra trouble and hard to debug if something changes.
              I was running it at home successfully for more than one Year in an Proxmox-VE but at least with decent intel pro/1000 NIC'S.
              One day it just stopped working and i couldn't find the problem.
              Now i have for long time a small Machine with an J3455 and no problems ever again.

              So your Project might be possible but without spending any money in dedicated hardware, not even some PCI-E NIC's it's a hard Job for an unstable result.

              For me it's obvious if no one has patience to help on that.

              But for me i had better results not to passthru the NIC's in my case at PCI-e and i didn't saw a benefit why i should do so.

              F 1 Reply Last reply Reply Quote 1
              • F
                fabiolanza @fishbone222
                last edited by fabiolanza

                @fishbone222 thanks for your thoughts. I have a mini-itx motherboard with 1 PCIe 16x slot that I am using to connect a NVIDIA Quadro card for hardware transcoding in Plex on my Windows Server VM. Since my motherboard has only 1 network port I had to use the USB adapter to expand. It seems a floppy thing but in reality, it's not and FreeBSD is fully compatible with it. The passthrough works correctly and it's like the card was directly attached to the operating system. I have a ZyXEL firewall appliance but I got tired of the OS and limitations, and decided to use my hypervisor environment to move to pfSense. So far this is the only issue.

                bmeeksB NollipfSenseN 2 Replies Last reply Reply Quote 0
                • bmeeksB
                  bmeeks @fabiolanza
                  last edited by

                  @fabiolanza said in pfSense stops working after some time: no buffer space available:

                  @fishbone222 thanks for your thoughts. I have a mini-itx motherboard with 1 PCIe 16x slot that I am using to connect a NVIDIA Quadro card for hardware transcoding in Plex on my Windows Server VM. Since my motherboard has only 1 network port I had to use the USB adapter to expand. It seems a floppy thing but in reality, it's not and FreeBSD is fully compatible with it. The passthrough works correctly and it's like the card was directly attached to the operating system. I have a ZyXEL firewall appliance but I got tired of the OS and limitations, and decided to use my hypervisor environment to move to pfSense. So far this is the only issue.

                  USB NICs are, in general, quite lousy. This is especially true when trying to use them in most UNIX/Linux/BSD setups. So your basic root issue is the use of USB NICs. You state that is your only option for your current hardware platform, so you might just be stuck working around the USB NIC problem by doing the reboot now and then. You can attempt some of the tweaks in the document you linked, but honestly I would not have high hopes of full success with the USB NICs.

                  F 1 Reply Last reply Reply Quote 1
                  • NollipfSenseN
                    NollipfSense @fabiolanza
                    last edited by NollipfSense

                    @fabiolanza I found this USB 3 NIC that uses Intel chipset and would purchase for another project not related to pfSense, but it was sold out however, I thought to share.

                    https://www.ebay.com/itm/Syncwire-USB-3-0-to-Ethernet-Adapter-1000-Mbps-RJ45-LAN-Gigabit-Network-Adapter/113718959682?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2060353.m1438.l2649

                    pfSense+ 23.09 Lenovo Thinkcentre M93P SFF Quadcore i7 dual Raid-ZFS 128GB-SSD 32GB-RAM PCI-Intel i350-t4 NIC, -Intel QAT 8950.
                    pfSense+ 23.09 VM-Proxmox, Dell Precision Xeon-W2155 Nvme 500GB-ZFS 128GB-RAM PCIe-Intel i350-t4, Intel QAT-8950, P-cloud.

                    F 1 Reply Last reply Reply Quote 1
                    • F
                      fabiolanza @NollipfSense
                      last edited by

                      @NollipfSense Hi I saw somewhere that these use Realtek chipsets. I searched on Syncwire website but did not find the page for this product. Would you have a datasheet to share? Thanks for the answer by the way.

                      NollipfSenseN 1 Reply Last reply Reply Quote 0
                      • F
                        fabiolanza @bmeeks
                        last edited by

                        @bmeeks thanks for the feedback. Dou you mean USB NICs in general or it's more related to the ASIX AX88179 chipset? I did not find an Intel one.

                        1 Reply Last reply Reply Quote 0
                        • GertjanG
                          Gertjan
                          last edited by Gertjan

                          af32562f-5c4d-434b-8335-7bfc7a8c7950-image.png

                          Really ??

                          edit : and who is 10.10.10.1 ?

                          Don't tell my you have pfBlockerNG also installed.

                          No "help me" PM's please. Use the forum, the community will thank you.
                          Edit : and where are the logs ??

                          F 1 Reply Last reply Reply Quote 0
                          • F
                            fabiolanza @Gertjan
                            last edited by

                            @Gertjan I have a transit link architecture setup. 10.10.10.2 is the firewall and 10.10.10.1 is my L3 switch.

                            1 Reply Last reply Reply Quote 0
                            • DaddyGoD
                              DaddyGo @fabiolanza
                              last edited by DaddyGo

                              @fabiolanza
                              Sorry again!:

                              I really didn't want to have a say in your ideas, but somehow you have to start pfSense....(and also learning about it)

                              is there possibly a mini-PCie socket on this miniITX MOBO?
                              http://www.commell.com.tw/Product/Peripheral/PCI%20Express%20mini%20card/MPX-350.htm

                              I've been working with pfSense for a long time (also in a VM environment), but so far I wouldn't dare to start a USB NIC installation

                              a good piece of advice, don't rigidly stick to what FreeBSD supports, listen to the advice of more experienced colleagues,
                              (I also wrote that, for example for Realtek is also have driver under FreeBSD, but not even recommended by anyone!)

                              Plus, I know Asix chips well (these chips cannot be used in this environment), I use it in simple portable MODBUS controllers under Windows, that's all it knows, it's not the basis of a firewall interface (ethernet controller):

                              7789e8c4-9495-4ebd-9b22-06049b8fa190-image.png

                              Cats bury it so they can't see it!
                              (You know what I mean if you have a cat)

                              F 1 Reply Last reply Reply Quote 0
                              • F
                                fabiolanza
                                last edited by

                                I just would like to mention that after the mbuf cluster adjustment that I did, the previous network issue that I had is no longer there. However, I introduced another error, in which any HTTP download would get interrupted after about 50MB. I can resume, but it would still fail over and over until completion of the download. I got sick of this and re-enabled my firewall appliance that is not nearly as nice as the pfSense. I guess I will have to buy one of the chinese pfSense appliances.

                                1 Reply Last reply Reply Quote 0
                                • DaddyGoD
                                  DaddyGo
                                  last edited by

                                  Chinese stuff is not a good idea, you will make a similar mistake!

                                  https://www.netgate.com/solutions/pfsense/sg-1100.html
                                  https://www.pcengines.ch/apu4d4.htm

                                  Cats bury it so they can't see it!
                                  (You know what I mean if you have a cat)

                                  F 1 Reply Last reply Reply Quote 0
                                  • F
                                    fabiolanza @DaddyGo
                                    last edited by

                                    @DaddyGo I understand. This is my motherboard: https://www.asrock.com/mb/AMD/X570%20Phantom%20Gaming-ITXTB3/index.asp. I am using the one 1 PCIe for the GPU. Since this mobo has a Thunderbolt 3 I was thinking about using a Thunderbolt PCIe expansion chassis and then installing a PCIe network card. Any thoughts?

                                    1 Reply Last reply Reply Quote 0
                                    • F
                                      fabiolanza @DaddyGo
                                      last edited by

                                      @DaddyGo I am in Italy and there is not a lot of resellers here. I'd go for the https://www.netgate.com/solutions/pfsense/sg-3100.html because I have a 1Gbps WAN link, but can't find where to buy it. It's a pitty Netgate products are not on Amazon.it.

                                      1 Reply Last reply Reply Quote 0
                                      • DaddyGoD
                                        DaddyGo
                                        last edited by

                                        I live here not far from you and there are many opportunities to get good hardware ☺

                                        Cats bury it so they can't see it!
                                        (You know what I mean if you have a cat)

                                        1 Reply Last reply Reply Quote 1
                                        • DaddyGoD
                                          DaddyGo
                                          last edited by DaddyGo

                                          https://www.netgate.com/partners/locator.html#italy

                                          For a 1 Gig ISP you will need: https://www.netgate.com/solutions/pfsense/sg-3100.html

                                          Cats bury it so they can't see it!
                                          (You know what I mean if you have a cat)

                                          1 Reply Last reply Reply Quote 1
                                          • DaddyGoD
                                            DaddyGo
                                            last edited by DaddyGo

                                            @fabiolanza
                                            I understand. This is my motherboard: https://www.asrock.com/mb/AMD/X570 Phantom Gaming-ITXTB3/index.asp. I am using the one 1 PCIe for the GPU. Since this mobo has a Thunderbolt 3 I was thinking about using a Thunderbolt PCIe expansion chassis and then installing a PCIe network card. Any thoughts?

                                            Try to think about dedicated hardware, when you building a firewall and consider the needs of your own system.
                                            These miniITX things are good too, but not my favorites.
                                            Consider consumption, as a firewall operates 24 hours a day. (SOHO)
                                            Don't want to run other things on the "iron" where pfSense is running!

                                            Thunderbolt is not a good starting point either

                                            edit: From the fact that you are going to put a PCIe NIC in an external adapter, even the OP system has to handle Thunderbolt.
                                            This is the same as the USB issue.

                                            Cats bury it so they can't see it!
                                            (You know what I mean if you have a cat)

                                            F 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.