Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    IPsec Apply changes time out

    Scheduled Pinned Locked Moved
    IPsec
    4
    20
    1.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • vergilisV
      vergilis @stephenw10
      last edited by

      @stephenw10 Multiple instance types in AWS. Everything works fine, just the Apply function does not work.

      I previously reported this and @jimp wanted me to duplicate this on physical hardware. It looks like @richi44 has this similar issue when there are about 50 tunnels.

      The only error in my log is the time out error:

      2021/02/18 13:38:18 [error] 51390#100109: *64336 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 5.6.7.8, server: , request: "POST /vpn_ipsec_settings.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "fwname.example.com:1234", referrer: "https://fwname.example.com:1234/vpn_ipsec_settings.php"

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Hmm, that's really the only error shown? That's in the system log?

        vergilisV 1 Reply Last reply Reply Quote 0
        • vergilisV
          vergilis @stephenw10
          last edited by

          @stephenw10 Correct. The only error in the log is the one specified from the system log - for me.

          1 Reply Last reply Reply Quote 0
          • R
            richi44 @stephenw10
            last edited by

            @stephenw10 I tried it on different hw setups but our main pfsense router runs on virtual machine (4cores of cpu-xeon e-2236@3,4Ghz, 12gb ram).

            Error1:
            Mar 17 20:24:04 nginx 2021/03/17 20:24:04 [error] 65880#100222: *1386 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.211.3, server: , request: "POST /status_services.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.211.1:8443", referrer: "https://192.168.211.1:8443/status_services.php"

            Error2:
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (vesa, 0xffffffff8140c3e0, 0) error 19
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80765790, 0) error 1
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff807656e0, 0) error 1
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765630, 0) error 1
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff8073dda0, 0) error 1
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff8073dcf0, 0) error 1
            Mar 17 20:32:29 kernel module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff8073dc40, 0) error 1

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Those kernel module errors are unrelated and not a cause for concern.

              Unclear why that timeout happens yet.

              R 1 Reply Last reply Reply Quote 0
              • R
                richi44 @stephenw10
                last edited by

                @stephenw10
                Kernel erros could relate to virtualisation on Proxmox.

                I tried to setup new router and time out problem does not occur if there were only few tunnels. After clean installation I was able to continually setup up to 50 P1 with 50 P2 but after reboot and apply changes the time out problem occurred.

                Could it be related to nginx memory isssue?

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  It seems more likely it's failing to pull the data from vici/strongswan for some reason. nginx shows it is timing out waiting for that data as far as I can see.

                  Is there a specific number of tunnels that seems to trigger the issue?

                  Or is it perhaps hitting a connection number that is failing to parse?

                  The way connections are numbered was changed significantly in 2.5 to allow for VTI tunnels when a large number exists. https://redmine.pfsense.org/issues/9592

                  Steve

                  vergilisV R 2 Replies Last reply Reply Quote 0
                  • vergilisV
                    vergilis @stephenw10
                    last edited by

                    @stephenw10 I currently have 46 tunnels on the failing system.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Try running ipsec statusall if you can. If that fails it would be interesting. If it doesn't look for a connection number that might be hitting a limit, con100000 maybe.

                      Does it fail with 45 tunnels?

                      Steve

                      vergilisV 1 Reply Last reply Reply Quote 0
                      • vergilisV
                        vergilis @stephenw10
                        last edited by

                        @stephenw10 Yes. It fails to Apply with 45 tunnels all the time. ipsec statusall returns results.

                        1 Reply Last reply Reply Quote 0
                        • jimpJ
                          jimp Rebel Alliance Developer Netgate
                          last edited by

                          Can you try a 2.5.1 RC snapshot and see if it's better there?

                          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                          Need help fast? Netgate Global Support!

                          Do not Chat/PM for help!

                          vergilisV 1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Sorry, I mean is there a specific number where it doesn't fail? Is it something that clear cut?

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • vergilisV
                              vergilis @jimp
                              last edited by

                              @jimp The following release is still exhibiting the issue:

                              21.05-DEVELOPMENT (amd64)
                              built on Sat Mar 20 01:04:33 EDT 2021
                              FreeBSD 12.2-STABLE

                              1 Reply Last reply Reply Quote 0
                              • R
                                richi44 @stephenw10
                                last edited by

                                @stephenw10

                                The firs time it showed was when I added 33th tunnel.

                                R 1 Reply Last reply Reply Quote 0
                                • R
                                  richi44 @richi44
                                  last edited by

                                  @richi44 I setup 51 tunnels on Netgate XG-7100 but the problem remains. After Apply changes, which takes more than 4 min Time Out 504 error shows.

                                  Could you help me to solve this problem? This is really bad if I want to make quick changes to my tunnels.

                                  Thank you.

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post