Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    2.4.5.a.20200110.1421 and earlier: High CPU usage from pfctl

    Scheduled Pinned Locked Moved
    Development
    33
    112
    24.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • IsaacFLI
      IsaacFL @jimp
      last edited by

      @jimp I noticed that during boot heavy cpu usage with hyper-v

      The packages I have installed are:
      Acme certificate
      Anahi
      Opnvpn client

      I do have fqcodel limiters set up

      It seems to spend a lot of time 100% cpu just after:

      load_dn_aqm dn_aqm PIE loaded

      EC3386DA-8DAC-4B2E-823F-55C2AE8ECF29.jpeg

      IsaacFLI 1 Reply Last reply Reply Quote 0
      • IsaacFLI
        IsaacFL @IsaacFL
        last edited by

        @jimp

        The update below seemed to fix the boot time delay:
        Current Base System2.4.5.r.20200211.0854
        Latest Base System2.4.5.r.20200212.1633

        This seems to have fixed the speed on boot. Went from about 2 minutes to 1 minute boot time.

        1 Reply Last reply Reply Quote 0
        • G
          Gektor
          last edited by Gektor

          Same troubles after upgrade from 2.4.4 to 2.4.5 under Windows Server 2019 Hyper-V, very long boot, sometimes pfct processl CPU usage almost 100% and system became very slow and unresponsive.

          1 Reply Last reply Reply Quote 1
          • A
            aydinj
            last edited by

            Same issue.

            G 1 Reply Last reply Reply Quote 1
            • G
              Gektor @aydinj
              last edited by

              @aydinj
              Do you have pfBlockerNG package installed? If yes, maybe it related:
              https://forum.netgate.com/topic/151690/increased-memory-and-cpu-spikes-causing-latency-outage-with-2-4-5/19
              https://forum.netgate.com/topic/151726/pfblockerng-2-1-4_21-totally-lag-system-after-pfsense-upgrade-from-2-4-4-to-2-4-5

              1 Reply Last reply Reply Quote 0
              • A
                aydinj
                last edited by

                No. It has a serious problem, even at boot. With a clean install, it is extremely slow and stock for minutes on "Firewall configuration," even on a fresh installed OS. I think this new version has a problem with hyper-v

                1 Reply Last reply Reply Quote 0
                • E
                  estepbrent
                  last edited by

                  Same issue on Hyper-V. Only workaround is drop CPU to 1.

                  1 Reply Last reply Reply Quote 2
                  • M
                    Magma82
                    last edited by

                    I can also confirm Hyper-V server core 2019 (Dell T20 Xeon) after upgrading my 2.4.4 VM to 2.4.5, I now have extremely High CPU when assigning all of my 4 cores to the Pfsense VM at boot. This is also being caused by the pfctl process (it seems) and consumes the boot process (up to 10 minutes to actually boot past configuring interface WAN > LAN etc)
                    After reducing the cores to 2 to the VM, it still uses high CPU at boot (2 minutes to boot), but I can at least use the internet and webgui (CPU under 20% - still high for my setup usually 1 - 3%).

                    I can also observe the new OpenVPN version service when utilising my WAN (4mb/sec down only) the CPU is 23% for the openvpn process in top which is unusually high.

                    It seems the OS is not optimised for Hyper-V usage so I am reverting back to 2.4.4 and hoping this thread on the forum will be updated by Netgate Dev's.

                    G 1 Reply Last reply Reply Quote 1
                    • G
                      Gektor @Magma82
                      last edited by Gektor

                      @Magma82
                      Now i am using almost 2 days with 2 CPU cores pfSense 2.4.5 under Hyper-V Server 2019, no issues with high CPU usage at all after disabling pfBlockerNG GEOip lists.

                      provelsP 1 Reply Last reply Reply Quote 0
                      • provelsP
                        provels @Gektor
                        last edited by

                        @Gektor Check out this thread. Worked for me.
                        https://www.reddit.com/r/pfBlockerNG/comments/fqjdc5/pfblockerngdevel_downloading_lists_but_not_able/flqzkgp/

                        Peder

                        MAIN - pfSense+ 24.03-RELEASE - Adlink MXE-5401, i7, 16 GB RAM, 64 GB SSD. 500 GB HDD for SyslogNG
                        BACKUP - pfSense+ 23.01-RELEASE - Hyper-V Virtual Machine, Gen 1, 2 v-CPUs, 3 GB RAM, 8GB VHDX (Dynamic)

                        1 Reply Last reply Reply Quote 0
                        • K
                          KasselA
                          last edited by

                          I am seeing this too. Running Hyper-V on a Dell R720. VM had 4 CPU assigned. No packages installed. CPU on the VM would spike to 100% for a few minutes at a time, then drop to normal briefly (no more than 30 seconds or so), then back to 100%.

                          Following recommendations earlier in the thread, I dropped the VM down to 1 CPU and that made everything operate normally again, as far as I can tell. Because it's not really a busy firewall, this is no real issue for me to have 1 CPU. Therefore I don't have a really urgent need to roll back to 2.4.4. I'll stay where I am until a patch is issued.

                          Sounds like the problem is specific to multiple CPUs on Hyper-V only.

                          1 Reply Last reply Reply Quote 1
                          • T
                            ThEWbA
                            last edited by ThEWbA

                            I am seeing this too. We are running Qemu 4.1.1, Kernel 5.3 (KVM) and CPU emulation Skylake-Client.

                            Problem started with 2.4.4 and a upgrade to 2.4.5 did not solve the issue.
                            Workaround is to downgrade to one (1) core.

                            1 Reply Last reply Reply Quote 1
                            • M
                              Magma82
                              last edited by

                              Netgate DEVS, the CPU performance in HyperV is definitely broken in 2.4.5 - are there any Hyper-V integration tools or libraries that are perhaps missing in the OS build?

                              Boot up CPU with 4 cores assigned = 100% constant at an early stage of the boot process and is barely accessible once booted.

                              OpenVPN = CPU is also considerably higher under load (as if the CPU isn't optimised for the VM)

                              I have reverted to 2.4.4 and its rock solid and under 3% CPU in use and just feels a lot more optimised.

                              Hardware
                              stable 2.4.4-RELEASE-p3 (amd64)
                              Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz
                              4 CPUs: 1 package(s) x 4 core(s)
                              AES-NI CPU Crypto: Yes (active)
                              Hardware crypto AES-CBC,AES-XTS,AES-GCM,AES-ICM

                              1 Reply Last reply Reply Quote 0
                              • A
                                Auror990
                                last edited by Auror990

                                Same issue, Server 2019 and Hyper-V, no packages installed on custom HW (Ryzen 2700) after upgrade. Pegs CPU upon boot and is basically unusable.

                                Set VM to 1 virtual processor to get it working but it is sub-optimal for OpenVPN clients. Even experimented with just assigning 2 virtual processors - it runs sluggish.

                                Will look to revert to 2.4.4-p3 snapshot in the near future.

                                Edit: since I had nothing to lose and this is in a test lab, I bumped up to 2.5.0 development (2.5.0.a.20200403.1017). 2.5.0 does not seem to have the Hyper-V CPU issue.

                                1 Reply Last reply Reply Quote 0
                                • Cool_CoronaC
                                  Cool_Corona
                                  last edited by

                                  Its the same in a VM on Vsphere. I run 32 cores on a test system and they all go to almost 100% shortly after boot.

                                  I noticed that the server started spinning its fans a lot harder and looked in the hypervisor and sure enough. Almost 100% and not handling traffic at all....

                                  I was running 2.4.4 p3 and no issues until Suricata wont start. Then I had to upgrade and it died....

                                  1 Reply Last reply Reply Quote 0
                                  • kiokomanK
                                    kiokoman LAYER 8
                                    last edited by

                                    i made a clean install on my esxi with 4 cpu
                                    and upgraded from 2.4.4-p3 to 2.4.5 on another server with qemu/kvm with 4 cpu westmere
                                    both have suricata installed, never had such a problem. and i'm unable to reproduce on my test lab, must be some settings

                                    ̿' ̿'\̵͇̿̿\з=(◕_◕)=ε/̵͇̿̿/'̿'̿ ̿
                                    Please do not use chat/PM to ask for help
                                    we must focus on silencing this @guest character. we must make up lies and alter the copyrights !
                                    Don't forget to Upvote with the 👍 button for any post you find to be helpful.

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      timboau 0
                                      last edited by

                                      Same problem here too Hvper V 2016 version 2.4.5
                                      5GB RAM
                                      4 CPU
                                      pfblocker NG

                                      Sits for ages on 'firewall' & Also DHCPv6 before booting really sluggish dropped packets galore

                                      Dropped back to single CPU and all ok on 2.4.5

                                      S 1 Reply Last reply Reply Quote 0
                                      • S
                                        slim2016 @timboau 0
                                        last edited by

                                        Same problem, pfsense 2.4.4 installed on Vmware Esxi. I have suricata, pfblockerng, squid, squidguard and lightsquid installed. After upgrading to 2.4.5 the latency went haywire. However, I've managed resolve my problem, I reduced 8 vcpu to 1vcpu then did the upgrade to 2.4.5. So far everything worked fine except suricata wouldn't start, so i did a Forced pkg Reinstall. Everything worked fine after that, then I added an additional 3vcpu and it's been working fine ever since.

                                        1 Reply Last reply Reply Quote 1
                                        • U
                                          Uncle_Bacon
                                          last edited by

                                          Same problem here but with a Proxmox VM on pfSense 2.4.5.
                                          2 CPU, 2 core
                                          8GB RAM
                                          NUMA disabled

                                          High CPU on "/sbin/pfctl -o basic -f /tmp/rules.debug" effectively killed my networks and VLANS, and both incoming WAN connections. pfSense would often crash and reboot automatically, which produces a crash report.

                                          Dropping to 1 CPU, 1 core fixes it but it's running hard due to my network. 2.4.4_3 ran just peachy!

                                          S 1 Reply Last reply Reply Quote 0
                                          • S
                                            slim2016 @Uncle_Bacon
                                            last edited by

                                            @Uncle_Bacon Have you tried adding cpu later (after the upgrade)? I noticed that maximum vcpu is 4 before it starts going crazy.

                                            Cool_CoronaC U 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post