Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-3100 IPsec Performance Issues

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    9 Posts 4 Posters 1.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      rvwadmin
      last edited by

      Greetings everyone!
      I recently purchased an SG-3100 for a satellite office but have been having issues with the speed through the IPsec tunnel connected to our main office. I currently am only able to get about 12-14mb down/up, our internet connection is 400/20 down/up. Because the tunnel speed is synchronous it makes me think I have reached the limit of the processor or I have made a mistake on the configuration. I played around with various crypto/hashing combinations and haven't seen much difference, I landed on AES128-SHA256-ECP256 since elliptic curve is normally faster than RSA.

      Am I missing something? I have also played around with various hardware crypto settings, but I haven’t found an easy way to tell what crypto devices and algorithms are compatible with IPsec, reading in the forums I see conflicting data about what is supported, and it seems that in the last few versions there have been some significant changes. I was considering trying OpenVPN to see if that has different results.

      Setup:
      Main Office:
      Xeon E31230 custom box HA Master
      SG-8860 HA Backup
      Internet 100mb x 100mb (speed test 90mb x 86mb)

      Satellite Office:
      SG-3100
      Internet connection 400mb x 20mb (speed test 450mb x 22mb)

      Things I have tried:
      Swapping Main/Backup Firewalls
      Enabling and Disabling hardware crypto
      Multiple ciphers/hash combos
      MSS Clamping down to 1400

      1 Reply Last reply Reply Quote 0
      • R
        rvwadmin
        last edited by

        I was able to increase the speed slightly by setting the MSS Clamping to 1360 although the ping tests I performed showed that it shouldn't be required. I am still scratching my head as to why it isn't performing faster, I attached iperf3 results below.

        (Server Main Office)
        [ ID] Interval Transfer Bandwidth
        [ 4] 0.00-1.00 sec 1.50 MBytes 12.6 Mbits/sec
        [ 4] 1.00-2.00 sec 1.75 MBytes 14.7 Mbits/sec
        [ 4] 2.00-3.00 sec 2.12 MBytes 17.8 Mbits/sec
        [ 4] 3.00-4.00 sec 1.88 MBytes 15.8 Mbits/sec
        [ 4] 4.00-5.00 sec 2.12 MBytes 17.8 Mbits/sec
        [ 4] 5.00-6.00 sec 2.12 MBytes 17.8 Mbits/sec
        [ 4] 6.00-7.00 sec 1.75 MBytes 14.7 Mbits/sec
        [ 4] 7.00-8.00 sec 2.12 MBytes 17.8 Mbits/sec
        [ 4] 8.00-9.00 sec 1.88 MBytes 15.7 Mbits/sec
        [ 4] 9.00-10.00 sec 2.00 MBytes 16.8 Mbits/sec


        [ ID] Interval Transfer Bandwidth
        [ 4] 0.00-10.00 sec 19.2 MBytes 16.1 Mbits/sec sender
        [ 4] 0.00-10.00 sec 19.2 MBytes 16.1 Mbits/sec receiver

        (Server Satellite Office)
        [ ID] Interval Transfer Bandwidth
        [ 4] 0.00-1.00 sec 640 KBytes 5.24 Mbits/sec
        [ 4] 1.00-2.00 sec 1.38 MBytes 11.5 Mbits/sec
        [ 4] 2.00-3.00 sec 1.50 MBytes 12.6 Mbits/sec
        [ 4] 3.00-4.02 sec 1.88 MBytes 15.5 Mbits/sec
        [ 4] 4.02-5.00 sec 1.62 MBytes 13.9 Mbits/sec
        [ 4] 5.00-6.01 sec 1.25 MBytes 10.4 Mbits/sec
        [ 4] 6.01-7.01 sec 1.38 MBytes 11.5 Mbits/sec
        [ 4] 7.01-8.01 sec 1.38 MBytes 11.5 Mbits/sec
        [ 4] 8.01-9.00 sec 1.75 MBytes 14.8 Mbits/sec
        [ 4] 9.00-10.01 sec 1.88 MBytes 15.6 Mbits/sec


        [ ID] Interval Transfer Bandwidth
        [ 4] 0.00-10.01 sec 14.6 MBytes 12.3 Mbits/sec sender
        [ 4] 0.00-10.01 sec 14.5 MBytes 12.2 Mbits/sec receiver

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          To get the best IPSec performance from the SG-3100 you need to be sure the CESA crypto hardware is being used.
          To do that you need to two things:
          Set 'Cryptographic Hardware' to BSD Crypto Device in System > Advanced > Misc.

          Make sure you;re using a cryto algorithm that CESA supports. That means AES-CBC, 128, 192 or 256 and sha1 or sha256.
          AES-CBC-128 and SHA1 will give you best performance.

          I expect it to fill that 86/22Mbps connection easily based purely on processing.

          Steve

          1 Reply Last reply Reply Quote 0
          • R
            rvwadmin
            last edited by

            @stephenw10 would the dh group prevent hardware offloading?

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by stephenw10

              Hmm, you are using PFS key 28? I'm using 14 and can see connections close to my line rate at 70+Mbps even between London and Austin. I'm not sure what gets off-loaded there. Nothing obvious shown in the driver:
              https://github.com/pfsense/FreeBSD-src/blob/ff7d4801f1b88de656e028209818ff005e8a1353/sys/dev/cesa/cesa.c#L1229

              Are you able to test group14?

              I'm assuming you're using IKEv2. The phase 2 settings are where the speed is decided.

              Steve

              1 Reply Last reply Reply Quote 0
              • R
                rvwadmin
                last edited by

                I just tested it no real change I am seeing some more variability in the results though, I am not seeing dropped packets through the link so I wonder if there is something between the route on these two, they are just down the road a few miles but are using different providers (Main is using Timewarner/Spectrum fiber provided by a third party and the remote office is Timewarner/Spectrum Cable).

                I will contact my providers and see if I can track something down. I have been testing everything I can think of for the last two weeks and am about ready to price out running a wireless point-to-point link from Ubiquiti if I can't figure it out haha.

                Phase 1
                AES_CBC
                HMAC_SHA2_256_128
                PRF_HMAC_SHA2_256
                MODP_2048

                Phase 2
                AES_CBC
                HMAC_SHA2_256_128
                IPComp: none

                1 Reply Last reply Reply Quote 0
                • R
                  rvwadmin
                  last edited by

                  I ran some speed tests yesterday between the two sites outside of the tunnel and was seeing similar results the issue must be the connections. I am reaching out to my providers today to see if they can resolve the problem.

                  1 Reply Last reply Reply Quote 0
                  • DerelictD
                    Derelict LAYER 8 Netgate
                    last edited by

                    We have, on occasion, seen paths that are decidedly unfriendly to ESP but work well using NAT-T (UDP 4500).

                    Since you are seeing similar issues outside the tunnel this is probably moot for you but I wanted to put it out there.

                    Note that there is no way for force NAT-T using IKEv2 but you can using IKEv1.

                    Chattanooga, Tennessee, USA
                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                    1 Reply Last reply Reply Quote 1
                    • B
                      brians
                      last edited by

                      Hi, the ipsec speed is limited by upload speed at remote site due to ACKs.

                      Having said that I have issue with SG-3100 - I cannot get greater than 45 Mbps on a physical 100Mbps fiber link to my home pfsense which is 940Mbps fiber, whereas I can 380Mbps+ to a SG-5100 on physical 500Mbps link, and 180Mbps to a SG-4860 on a 200Mbps physical link. My home is a home-built pfsense running i7. I will open another thread on this.

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post
                      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.