Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Alix with AES-128 - poor throughput - 5 Mbit/s with glxsb, ShrewSoft client

    IPsec
    2
    5
    2.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      pfsense_user_123
      last edited by

      Hi All,
      I want to ask You about ALIX IPSEC AES-128 with glxsb performance,, what is the real throughput that You can achieve on this?
      Regarding to this Benchmark (from official pfsense doc) I should be able to achieve ~21 Mbit/s AES enc. (with GLXSB, not HIFN) am I right ?

      Ok, a can assume that this graph can be a little bit overclocked for PR :-)

      So I've done research, and found that ALIX with Geode (glxsb) can achieve 19.4 Mbps, I've found this info in many posts on this forum.

      My question is, is this true ?, or it is only marketing ?
      Is it available only with site-to-site VPN ?, I'am connected to this alix via ShrewSoft Client 2.2.2 (runing on 8-core Intel with a huge lot of ram and  (AES-NI acceleration)
      This machine can encrypt & decrypt ~3GB/s of AES128 stream.

      I've got couple of alix box's (2D13, 6F2) with same config.

      [2.1.2-RELEASE][admin@pfSense.localdomain]/root(2): dmesg | grep AES
      glxsb0: <amd geode="" lx="" security="" block="" (aes-128-cbc,="" rng)="">mem 0xefff4000-0xefff7fff irq 9 at device 1.2 on pci0</amd>

      Phase 1
      AES128, SHA1
      Phase 2
      AES128, SHA1

      And max what i can push throught this link is 5 Mbit/s
      Link is 20Mbit/s symetic.

      Without IPSEC, I can easy download with 20 Mbit/s and upload 20Mbit/s

      LAN (with FTP server) –------ALIX-----20(Mbit/s)-----WAN---20(Mbit/s)---------CLIENT (SHREW SOFT)

      Is there any tool to check if this cryptodev really works ?, or maybye there is some trick with ShrewSoft ?
      The funny thing is that, when I disable glxsb module, throughput drops to 2Mbit/s (so it looks that it works)

      Some syntetic test.

      [2.1.2-RELEASE][admin@pfSense.localdomain]/root(4): openssl speed -evp aes-128-cbc
      To get the most accurate results, try to run this
      program when this computer is idle.
      Doing aes-128-cbc for 3s on 16 size blocks: 83852 aes-128-cbc's in 0.09s
      Doing aes-128-cbc for 3s on 64 size blocks: 80324 aes-128-cbc's in 0.07s
      Doing aes-128-cbc for 3s on 256 size blocks: 71802 aes-128-cbc's in 0.06s
      Doing aes-128-cbc for 3s on 1024 size blocks: 49527 aes-128-cbc's in 0.08s
      Doing aes-128-cbc for 3s on 8192 size blocks: 11488 aes-128-cbc's in 0.01s
      OpenSSL 0.9.8y 5 Feb 2013
      built on: date not available
      options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
      compiler: cc
      available timing options: USE_TOD HZ=128 [sysconf value]
      timing function used: getrusage
      The 'numbers' are in 1000s of bytes per second processed.
      type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
      aes-128-cbc      14141.64k    70022.01k  295776.27k  597406.71k 13668801.16k

      Thansk for any feedback.

      1 Reply Last reply Reply Quote 0
      • P
        pfsense_user_123
        last edited by

        Ok,
        I've done some test.

        pure SITE-TO-SITE connection, using 2 ALIX box. (glxsb enabled on both sides)
        IPSEC, with 128 AES, (Phase 1, and Phase 2) hmac SHA1.

        Throughput 5Mbit/s with encryption (AES 128)
        Without encryption … 20Mbit/s
        I've also found some testing tools.

        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(71): cryptotest -z 1000
          0.067 sec,    2000    aes crypts,      16 bytes,  475815 byte/sec,    3.6 Mb/sec
          0.068 sec,    2000    aes crypts,      32 bytes,  943702 byte/sec,    7.2 Mb/sec
          0.069 sec,    2000    aes crypts,      64 bytes,  1842708 byte/sec,    14.1 Mb/sec
          0.073 sec,    2000    aes crypts,    128 bytes,  3523211 byte/sec,    26.9 Mb/sec
          0.078 sec,    2000    aes crypts,    256 bytes,  6539704 byte/sec,    49.9 Mb/sec
          0.091 sec,    2000    aes crypts,    512 bytes, 11235832 byte/sec,    85.7 Mb/sec
          0.116 sec,    2000    aes crypts,    1024 bytes, 17639662 byte/sec,  134.6 Mb/sec
          0.168 sec,    2000    aes crypts,    2048 bytes, 24451834 byte/sec,  186.6 Mb/sec
          0.272 sec,    2000    aes crypts,    4096 bytes, 30074415 byte/sec,  229.4 Mb/sec
          0.564 sec,    2000    aes crypts,    8192 bytes, 29059899 byte/sec,  221.7 Mb/sec

        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(72): cryptostats
        1749914 symmetric crypto ops (0 errors, 318 times driver blocked)
        1 key ops (1 errors, 0 times driver blocked)
        1744022 crypto dispatch thread activations
        28611 crypto return thread activations

        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(80): ipsecstats
        ah packets with hmac-md5: 24974
        ah packets with hmac-sha1: 11588
        esp input packets processed: 21419
        esp output packets processed: 15143
        esp packets with rijndael-cbc: 36562
        esp bytes received: 26989824
        esp bytes transmitted: 1405153

        no SA found 273 (output)
        m_clone processing: 0 mbufs + 0 clusters coalesced
        m_clone processing: 0 clusters copied
        m_makespace: 0 mbufs inserted
        header position [front/middle/end]: 0/21419/0

        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(99): kldstat
        Id Refs Address    Size    Name
        1    4 0xc0400000 104a230  kernel
        2    1 0xc3061000 5000    glxsb.ko
        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(100): kldload crypto
        kldload: can't load crypto: File exists
        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(101): kldload cryptodev
        kldload: can't load cryptodev: File exists
        [2.1.2-RELEASE][admin@pfSense.localdomain]/root(102):

        No traffic shaping on both side,
        No NAT.
        No tunning.

        Simple configuration (out-of-box)

        LAN–--WAN----IPSEC------WAN-----LAN

        Any clue ?

        1 Reply Last reply Reply Quote 0
        • P
          pfsense_user_123
          last edited by

          Ok,
          Now I know what is going on.

          AMD GEODE LX with buil-in cryptodev supported by glxsb modul supports only => (AES) - (CBC/ECB),
          SHA1,MD5 is not supported.

          So if You are using IPSEC with AES-128 with glxsb on ALIX with Phase 2 with HMAC auth You will not experience crypto acceleration couse of glxsb.

          There is understatement in PFSENE doc… about IPSEC performance with AES-128,
          You are not able to achieve more than 5Mbit/s througput inside IPSEC tunel with AES 128, and HMAC-SHA1/MD5 etc...
          and You are not allowed to disable HMAC in Phase 2.

          I've found some post .. (made by CMD)

          Re: Alix IPsec benchmarks 1.2 1.2.2 1.2.3 glxsb hifn
          « Reply #3 on: June 23, 2009, 02:22:10 am »

          Thanks!

          I just happened to find this now that I'm messing with glxsb. We added the patch in kern/132622 in March, it's in 1.2.3 snapshots. Thanks much for your work on glxsb, Patrick!  Glad to see you on our forum too.

          We're looking at building glxsb as a module right now, so we can test with and without it, and to get it out of the way when you have a much faster Hifn installed.

          I'm seeing 19.4 Mbps through IPsec with AES-128 on an ALIX with glxsb, and 40 Mbps 3DES with a hifn 7955 (Soekris vpn1411) vs. 8.4 Mbps 3DES without hifn. Nice performance boost with the hifn. Not sure what impact glxsb has yet.

          How did You measure this ?,
          How does the config look ?
          IPSEC.
          Phase 1 ?
          Phase 2 ?

          1 Reply Last reply Reply Quote 0
          • P
            pieterraxis
            last edited by

            Hi,

            Did you figure it out to increase performance?

            I have the same problem!

            1 Reply Last reply Reply Quote 0
            • P
              pieterraxis
              last edited by

              @pieterraxis:

              Hi,

              Did you figure it out to increase performance?

              I have the same problem!

              And i am using  aes 128 cbc

              What does RNG mean in this line?

              $ dmesg | grep AES
              glxsb0: <amd geode="" lx="" security="" block="" (aes-128-cbc,="" rng)="">mem 0xefff4000-0xefff7fff irq 9 at device 1.2 on pci0</amd>

              1 Reply Last reply Reply Quote 0
              • First post
                Last post
              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.