Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    IPSec Issues 2.2.3 and 2.2.4

    Scheduled Pinned Locked Moved IPsec
    32 Posts 5 Posters 5.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      rain
      last edited by

      That's correct, the Core 2 system is at my house, so is the 4860. I've been swapping back and forth between those two systems just to validate that there is nothing upstream or network related.

      The third system is in the other city.

      I'll try 3DES tonight, failing that, I'll put the 2.2.2 on.

      1 Reply Last reply Reply Quote 0
      • R
        rain
        last edited by

        Sorry, didn't get to testing last night, will try tonight.

        1 Reply Last reply Reply Quote 0
        • R
          rain
          last edited by

          Tested 3DES with and without MSS clamping, even worse throughput than AES, about 4.7Mbit/sec.

          Tried AES-256, get about 9.6Mbit/sec.

          Rolling back to 2.2.2 right now.

          1 Reply Last reply Reply Quote 0
          • R
            rain
            last edited by

            So yeah, there is something seriously wrong with builds 2.2.3 and 2.2.4 with respect to IPsec.

            Rolled back to 2.2.2, and my throughput goes back to maxing out the circuit (50 down/3up). I've attached a screenshot proving this.

            From a "just work" on every version of pfSense with these SG boxes sold on the pfsense store, what options do I have here? Do I submit a ticket of some sort? It's easily to reproduce.

            With regards,

            at222.JPG_thumb
            at222.JPG

            1 Reply Last reply Reply Quote 0
            • J
              jwt Netgate
              last edited by

              @rain:

              No, only a aes128-CCM16 (nothing GCM). Otherwise just AES variants, 3DES. Is GCM all that and a bag of chips? I'm not familiar with it.

              No difference with AES-NI disabled, if anything, a bit slower but same behavior (that was the first thing I tested on the new 2.2.4).

              AES-CCM isn't a great mode for IPSec.  In fact, the only support I can find in the FreeBSD kernel for it is in the wireless code, so I'm confused how you've configured to use it.  (AES-CCM gets used a lot in 802.11.)

              If you don't want to use AES-GCM, have you tried AES-CBC-128 with HMAC-SHA1, because that's the bog-standard "best practice" until you get concerned with the strength of SHA1 and a 128-bit key length.

              In face, I can't find any support for using AES-CCM in the IPSec subsystem in FreeBSD.  Here are the auth and encryption tokens that 'setkey' will recognize.  These are copy-pasta straight for the source code.

              /* authentication alogorithm */
              hmac-md5
              hmac-sha1
              keyed-md5
              keyed-sha1
              hmac-sha2-256
              hmac-sha2-384
              hmac-sha2-512
              hmac-ripemd160
              aes-xcbc-mac
              tcp-md5
              null

              /* encryption alogorithm */
              des-cbc
              3des-cbc
              null
              simple
              blowfish-cbc
              cast128-cbc
              des-deriv
              des-32iv
              rijndael-cbc
              aes-ctr
              camellia-cbc

              Not can I find any support in the GUI for AES-CCM.

              BTW, the only modes registered with the AES-NI module are:
              AES-CBC
              AES-ICM
              AES-GCM
              AES-GHASH (128, 192, 256 bit)
              AES-XTS

              That said, AES-NI isn't going to help much for modes with a separate HMAC (basically all but AES-GCM) because the pass over the packet with the HMAC will dominate the time to encode/decode the packet before transmit/reception.

              This is why AES-GCM is a 'win' with AES-NI.

              I have ZERO doubt that 3DES is slower than AES.

              please send the output of "ipsec statusall".  I don't suggest posting it here in the forum.  Since you purchased these from the pfSense store, you have support.  Open a ticket.  If it's a bug that we've somehow missed, then I'll ensure that you don't "use" that ticket.

              1 Reply Last reply Reply Quote 0
              • J
                jwt Netgate
                last edited by

                SG-2220 (yes, they do exist, C2358 2 cores @ 1.7GHz) at home.
                C2758 (8 cores @ 2.4GHz) as VPN gateway at work.
                Both running pfSense software version 2.2.4

                1Gbps link from home, 1Gbps link at work, what happens between those two is good, but not ideal.

                Jims-MacBook-Pro:~ jim$ ping -c 3 nfs4
                PING nfs4.pfmechanics.com (172.27.32.4): 56 data bytes
                64 bytes from 172.27.32.4: icmp_seq=0 ttl=61 time=4.352 ms
                64 bytes from 172.27.32.4: icmp_seq=1 ttl=61 time=4.434 ms
                64 bytes from 172.27.32.4: icmp_seq=2 ttl=61 time=4.860 ms

                –- nfs4.pfmechanics.com ping statistics ---
                3 packets transmitted, 3 packets received, 0.0% packet loss
                round-trip min/avg/max/stddev = 4.352/4.549/4.860/0.223 ms
                Jims-MacBook-Pro:~ jim$ ssh nfs4
                Last login: Sat Aug  1 15:48:30 2015 from 172.21.0.26
                FreeBSD 10.1-RELEASE-p5 (GENERIC) #0: Tue Jan 27 08:55:07 UTC 2015

                [jim@nfs4 ~]$ rm testfile
                [jim@nfs4 ~]$ dd if=/dev/random of=testfile bs=1k count=200k
                204800+0 records in
                204800+0 records out
                209715200 bytes transferred in 6.281192 secs (33387802 bytes/sec)
                [jim@nfs4 ~]$ ls -l testfile
                -rw-r–r--  1 jim  netgate  209715200 Aug  1 15:49 testfile
                [jim@nfs4 ~]$ exit
                logout
                Connection to nfs4 closed.
                Jims-MacBook-Pro:~ jim$ scp nfs4:testfile /tmp/testfile
                testfile                                          100%  200MB  22.2MB/s  00:09   
                Jims-MacBook-Pro:~ jim$ tcsh
                [Jims-MacBook-Pro:~] jim% repeat 10 sftp nfs4:testfile /dev/null
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  25.0MB/s  00:08   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  25.0MB/s  00:08   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  16.7MB/s  00:12   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  16.7MB/s  00:12   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  18.2MB/s  00:11   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  25.0MB/s  00:08   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  28.6MB/s  00:07   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  25.0MB/s  00:08   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  25.0MB/s  00:08   
                Connected to nfs4.
                Fetching /usr/home/jim/testfile to /dev/null
                /usr/home/jim/testfile                            100%  200MB  25.0MB/s  00:08   
                [Jims-MacBook-Pro:~] jim%

                ![Screen Shot 2015-08-01 at 4.06.37 PM.png](/public/imported_attachments/1/Screen Shot 2015-08-01 at 4.06.37 PM.png)
                ![Screen Shot 2015-08-01 at 4.06.37 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2015-08-01 at 4.06.37 PM.png_thumb)

                1 Reply Last reply Reply Quote 0
                • J
                  jwt Netgate
                  last edited by

                  And a little longer test
                  [jim@nfs4 ~]$ dd if=/dev/random of=testfile bs=1k count=2000k
                  2048000+0 records in
                  2048000+0 records out
                  2097152000 bytes transferred in 64.266291 secs (32632224 bytes/sec)
                  [jim@nfs4 ~]$ ls -l testfile
                  -rw-r–r--  1 jim  netgate  2097152000 Aug  1 16:10 testfile
                  [jim@nfs4 ~]$ exit

                  ![Screen Shot 2015-08-01 at 4.21.22 PM.png](/public/imported_attachments/1/Screen Shot 2015-08-01 at 4.21.22 PM.png)
                  ![Screen Shot 2015-08-01 at 4.21.22 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2015-08-01 at 4.21.22 PM.png_thumb)

                  1 Reply Last reply Reply Quote 0
                  • J
                    jwt Netgate
                    last edited by

                    and, now that I've recovered the nuc from last night's 1hr+ power hit at work…

                    Note that running across a LAN is faster, but no VPN.
                    jim@nucatwork:~ % sudo scp jim@nfs4:testfile /usr/local/www/apache24/data/
                    Password for jim@nfs4:
                    testfile                                          100% 2000MB  87.0MB/s  00:23   
                    jim@nucatwork:~ %

                    ![Screen Shot 2015-08-01 at 5.10.57 PM.png](/public/imported_attachments/1/Screen Shot 2015-08-01 at 5.10.57 PM.png)
                    ![Screen Shot 2015-08-01 at 5.10.57 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2015-08-01 at 5.10.57 PM.png_thumb)

                    1 Reply Last reply Reply Quote 0
                    • R
                      rain
                      last edited by

                      @jwt:

                      AES-CCM isn't a great mode for IPSec.  In fact, the only support I can find in the FreeBSD kernel for it is in the wireless code, so I'm confused how you've configured to use it.  (AES-CCM gets used a lot in 802.11.)

                      I didn't, likely my post wasn't clear. Those are options on the other end of the tunnel, a Palo Alto Networks 3000 series. The response was to let CMB know what other options I have available to try. Although I appreciate your long response!

                      I sent an email to the support from the store asking how to use that support ticket, but I have not yet heard back. That was last week Monday when I sent it. I'd love to report back that I got great support on this hardware.

                      1 Reply Last reply Reply Quote 0
                      • R
                        rain
                        last edited by

                        I don't doubt you are getting that on different hardware. I'm getting a lot better on some of my old home built hardware from the scrap heap. But not an apples to apples comparison.

                        Once I rolled back to 2.2.2 I'm getting reasonable performance from the tunnel. With nothing else changing except moving to 2.2.3 or 2.2.4 the tunnel fails to pass traffic and passes it terribly slow respectively. Not exactly "just works".

                        1 Reply Last reply Reply Quote 0
                        • J
                          jwt Netgate
                          last edited by

                          @rain:

                          I don't doubt you are getting that on different hardware. I'm getting a lot better on some of my old home built hardware from the scrap heap. But not an apples to apples comparison.

                          It's pretty close, actually.  I'm quite familiar with the SG-4860.  If anything, the 2220 is slower, and that was the point.  It's really straight-forward to get > 200Mbps using AES-GCM with AES-NI.

                          If I'd wanted to quote lab performance, I've seen > 1.5Gbps using fairly modern Xeons.  But the SG-2220 is slower than what you're using.

                          @rain:

                          Once I rolled back to 2.2.2 I'm getting reasonable performance from the tunnel. With nothing else changing except moving to 2.2.3 or 2.2.4 the tunnel fails to pass traffic and passes it terribly slow respectively. Not exactly "just works".

                          Have you turned off AES-NI?

                          1 Reply Last reply Reply Quote 0
                          • J
                            jwt Netgate
                            last edited by

                            @rain:

                            @jwt:

                            AES-CCM isn't a great mode for IPSec.  In fact, the only support I can find in the FreeBSD kernel for it is in the wireless code, so I'm confused how you've configured to use it.  (AES-CCM gets used a lot in 802.11.)

                            I didn't, likely my post wasn't clear. Those are options on the other end of the tunnel, a Palo Alto Networks 3000 series. The response was to let CMB know what other options I have available to try. Although I appreciate your long response!

                            OK, so send the output of "ipsec statusall", as requested.  Then we'll know what we're dealing with.

                            @rain:

                            I sent an email to the support from the store asking how to use that support ticket, but I have not yet heard back. That was last week Monday when I sent it. I'd love to report back that I got great support on this hardware.

                            I've forwarded an internal request to see what happened here.

                            1 Reply Last reply Reply Quote 0
                            • R
                              rain
                              last edited by

                              statusall on 2.2.2 and 2.2.4? Or just one?

                              1 Reply Last reply Reply Quote 0
                              • J
                                jwt Netgate
                                last edited by

                                both would be interesting, but 2.2.2 would be OK

                                1 Reply Last reply Reply Quote 0
                                • R
                                  rain
                                  last edited by

                                  More information in there then I'm willing to post publicly, so I've PM'd it to you.

                                  This was on 2.2.2 working the way it should.

                                  1 Reply Last reply Reply Quote 0
                                  • C
                                    cmb
                                    last edited by

                                    There's something to this, I'm working on narrowing it down. I also grabbed your support ticket, will reply back there with an update before I wrap up for the day today.

                                    1 Reply Last reply Reply Quote 0
                                    • R
                                      rain
                                      last edited by

                                      Thanks Chris.

                                      Nice to be validated.  ;D I'm a newbie on these forums, but I'm not a newbie with networks.

                                      With regards,

                                      1 Reply Last reply Reply Quote 0
                                      • A
                                        afrojoe
                                        last edited by

                                        I too am experiencing issues with IPSec 2.2.3 and 2.2.4.  My tunnel is fast, stays up for a couple hours and then just disconnects…. Not the same issue as what others are reporting, but I have three tunnels connecting to my IPSec 2.2.3 instance, (two far ends are 2.2.3 and one is 2.2.4)  The 2.2.4 does not stay healthly for more than 4 hours.. deleting both instances at both ends are recreating brings everything back up for another 4 hours and then the tunnel dies again.

                                        1 Reply Last reply Reply Quote 0
                                        • R
                                          rain
                                          last edited by

                                          What kind of hardware?

                                          What kind of tunnel?

                                          CMB is working on my issue for a couple of weeks now but I haven't heard anything recently. I got the impression it was an upstream problem.

                                          1 Reply Last reply Reply Quote 0
                                          • M
                                            mwhitcomb7
                                            last edited by

                                            The firmware version of Intel NICs can play a significant role in these types of problems. I have had identical issues on a LAN. Take a look at that if you would.

                                            afrojoe: I would update your 2.2.3 concentrator to 2.2.4 if you can. I am curious, are these pfSense HW or hand rolled? If hand rolled what are the NICs?

                                            /M

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.