• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Haproxy 100% cpu usage

Scheduled Pinned Locked Moved Cache/Proxy
47 Posts 7 Posters 4.9k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    maverick_slo @rootbg
    last edited by Jun 3, 2024, 12:37 PM

    Pleeease? :)

    1 Reply Last reply Reply Quote 0
    • C
      coreybrett
      last edited by Jun 5, 2024, 7:19 PM

      I have this issue also. Is there a work-around ?

      M 1 Reply Last reply Jun 6, 2024, 2:34 PM Reply Quote 0
      • S
        stephenw10 Netgate Administrator
        last edited by Jun 6, 2024, 2:29 PM

        Looks like we lost the screenshot from the first post.

        Does this actually cause a kernel panic when you hit it or just uses all the available CPU time?

        Steve

        M L 2 Replies Last reply Jun 6, 2024, 2:33 PM Reply Quote 0
        • M
          maverick_slo @stephenw10
          last edited by Jun 6, 2024, 2:33 PM

          @stephenw10
          Just cpu usage.
          Updated versions resolve the issue
          Also workaround is working and if applied cpu is ok.

          1 Reply Last reply Reply Quote 0
          • M
            maverick_slo @coreybrett
            last edited by Jun 6, 2024, 2:34 PM

            @coreybrett
            Come on dude, read a little it is in the first post I made.

            1 Reply Last reply Reply Quote 0
            • C
              coreybrett
              last edited by Jun 6, 2024, 5:23 PM

              Adding...

              tune.disable-zero-copy-forwarding
              

              did not fix for me.

              M 1 Reply Last reply Jun 6, 2024, 5:33 PM Reply Quote 0
              • M
                maverick_slo @coreybrett
                last edited by maverick_slo Jun 6, 2024, 5:33 PM Jun 6, 2024, 5:33 PM

                @coreybrett
                Did you entered it in correct settings tab?

                1 Reply Last reply Reply Quote 0
                • C
                  coreybrett
                  last edited by Jun 6, 2024, 7:04 PM

                  9865886a-71df-49ae-b063-702e2a2fa9da-image.png

                  984c9778-fa5b-4c3e-8629-ee6b34b07147-image.png

                  1 Reply Last reply Reply Quote 0
                  • C
                    coreybrett
                    last edited by Jun 6, 2024, 7:07 PM

                    3b7600db-31c0-4270-bb5f-be08b1112e02-image.png

                    After a stop/start, the HAP process hit 100% and stops responding to inbound requests.

                    1 Reply Last reply Reply Quote 0
                    • L
                      Luca De Andreis @stephenw10
                      last edited by Jun 10, 2024, 4:43 AM

                      @stephenw10

                      I can confirm. in my case haproxy generates a kernel panic (occasionally). pfsense 24.03 is stable (more than 30 installations in my case) only with the use of haproxy do kernel panics occur. in this case I went back to the previous version.

                      Luca

                      1 Reply Last reply Reply Quote 0
                      • S
                        stephenw10 Netgate Administrator
                        last edited by Jun 10, 2024, 12:09 PM

                        Do you also see the high CPU usage?

                        L M 2 Replies Last reply Jun 10, 2024, 12:41 PM Reply Quote 0
                        • L
                          Luca De Andreis @stephenw10
                          last edited by Jun 10, 2024, 12:41 PM

                          @stephenw10

                          We have two PfSense with HAProxy (dedicated) in 24.03, on the site with the most traffic (a while ago now), we noticed that the VM would reboot going the crash.
                          Both VMs were experiencing abnormal CPU load (not 100%).
                          On the third crash I rolled back to 23.09.1 and it has been working perfectly ever since.
                          The site with less load at the HAProxy level (I left it only for testing in 24.03) never crashed but still has abnormal CPU load (30-40% without doing practically anything).
                          The site in production with a fair number of accesses, stayed UP 3-5 days, then crashed. After downgrade to version 23.09.1.... perfect.

                          All PfSense works in VM and we have about 30 of them, only those with HAProxy gave this problem with 24.03.

                          S 1 Reply Last reply Sep 9, 2024, 4:14 AM Reply Quote 1
                          • M
                            maverick_slo @stephenw10
                            last edited by Jun 10, 2024, 5:32 PM

                            @stephenw10 why are we discussing this when solution is crystal clear?
                            Binary has to be updated, end of story.

                            1 Reply Last reply Reply Quote 1
                            • S
                              stephenw10 Netgate Administrator
                              last edited by Jun 10, 2024, 5:53 PM

                              Yup, raised it internally. But trying to determine if the reported kernel panic is actually related.

                              L 1 Reply Last reply Jun 10, 2024, 6:02 PM Reply Quote 2
                              • L
                                Luca De Andreis @stephenw10
                                last edited by Jun 10, 2024, 6:02 PM

                                @stephenw10

                                For me, yes, it's related.

                                In my case a PfSense 24.03 VM (with HA Proxy 2.9.1) crashed (and therefore auto rebooted) every 3-5 days.
                                With a usage load... typical of a reverse proxy in production.

                                The same VM but with almost no workload, it never crashed apart from an anomalous CPU load.

                                In my opinion yes, the events are correlated, but only when the reverse proxy is used above a certain threshold.

                                In this regard I had also opened a ticket via "professional" support, which was closed with the response... "HAProxy is a third-party package, its update is managed in best effort" .... closed.

                                S 1 Reply Last reply Sep 9, 2024, 4:31 AM Reply Quote 0
                                • S
                                  stephenw10 Netgate Administrator
                                  last edited by Jun 10, 2024, 6:33 PM

                                  Hmm, nothing much TAC can do beyond opening the bug report, which they did.

                                  We have devs looking at it now.

                                  L 1 Reply Last reply Jun 10, 2024, 6:45 PM Reply Quote 0
                                  • L
                                    Luca De Andreis @stephenw10
                                    last edited by Luca De Andreis Jun 10, 2024, 6:45 PM Jun 10, 2024, 6:45 PM

                                    @stephenw10

                                    ahh I forgot.

                                    After opening the support ticket (and before downgrading to version 23.09.1) I applied the "disable zero copy forwarding" workaround.

                                    The CPU returned to normal on the VM with HAProxy in production... unfortunately after a few days the crash and related reboot occurred again.

                                    I then rolled back to 23.09.1 ​​and the system has been completely rock solid ever since.

                                    Luca

                                    1 Reply Last reply Reply Quote 1
                                    • S
                                      stephenw10 Netgate Administrator
                                      last edited by Jun 11, 2024, 11:56 PM

                                      OK the backend haproxy pkg had been updated to current (2.9.7) and looks good in testing here.

                                      You can upgrade it dircetly like: pkg upgrade haproxy

                                      Or reinstall the pfSense HAProxy package to get it.

                                      If this confirms the fix and doesn't introduce any new show stoppers we'll bump the package version so it appears as an upgrade for all users.

                                      Steve

                                      L 1 Reply Last reply Jun 12, 2024, 12:45 PM Reply Quote 2
                                      • L
                                        Luca De Andreis @stephenw10
                                        last edited by Jun 12, 2024, 12:45 PM

                                        @stephenw10

                                        ... upgrade to 2.9.7 in production and... for now... works fine :)

                                        Thanks
                                        Luca

                                        M 1 Reply Last reply Jun 12, 2024, 1:07 PM Reply Quote 2
                                        • M
                                          maverick_slo @Luca De Andreis
                                          last edited by Jun 12, 2024, 1:07 PM

                                          Same here, removed workaround as well
                                          All good last 8 hours ..

                                          1 Reply Last reply Reply Quote 1
                                          15 out of 47
                                          • First post
                                            15/47
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received