Alix with AES-128 - poor throughput - 5 Mbit/s with glxsb, ShrewSoft client

  • Hi All,
    I want to ask You about ALIX IPSEC AES-128 with glxsb performance,, what is the real throughput that You can achieve on this?
    Regarding to this Benchmark (from official pfsense doc) I should be able to achieve ~21 Mbit/s AES enc. (with GLXSB, not HIFN) am I right ?

    Ok, a can assume that this graph can be a little bit overclocked for PR :-)

    So I've done research, and found that ALIX with Geode (glxsb) can achieve 19.4 Mbps, I've found this info in many posts on this forum.

    My question is, is this true ?, or it is only marketing ?
    Is it available only with site-to-site VPN ?, I'am connected to this alix via ShrewSoft Client 2.2.2 (runing on 8-core Intel with a huge lot of ram and  (AES-NI acceleration)
    This machine can encrypt & decrypt ~3GB/s of AES128 stream.

    I've got couple of alix box's (2D13, 6F2) with same config.

    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(2): dmesg | grep AES
    glxsb0: <amd geode="" lx="" security="" block="" (aes-128-cbc,="" rng)="">mem 0xefff4000-0xefff7fff irq 9 at device 1.2 on pci0</amd>

    Phase 1
    AES128, SHA1
    Phase 2
    AES128, SHA1

    And max what i can push throught this link is 5 Mbit/s
    Link is 20Mbit/s symetic.

    Without IPSEC, I can easy download with 20 Mbit/s and upload 20Mbit/s

    LAN (with FTP server) –------ALIX-----20(Mbit/s)-----WAN---20(Mbit/s)---------CLIENT (SHREW SOFT)

    Is there any tool to check if this cryptodev really works ?, or maybye there is some trick with ShrewSoft ?
    The funny thing is that, when I disable glxsb module, throughput drops to 2Mbit/s (so it looks that it works)

    Some syntetic test.

    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(4): openssl speed -evp aes-128-cbc
    To get the most accurate results, try to run this
    program when this computer is idle.
    Doing aes-128-cbc for 3s on 16 size blocks: 83852 aes-128-cbc's in 0.09s
    Doing aes-128-cbc for 3s on 64 size blocks: 80324 aes-128-cbc's in 0.07s
    Doing aes-128-cbc for 3s on 256 size blocks: 71802 aes-128-cbc's in 0.06s
    Doing aes-128-cbc for 3s on 1024 size blocks: 49527 aes-128-cbc's in 0.08s
    Doing aes-128-cbc for 3s on 8192 size blocks: 11488 aes-128-cbc's in 0.01s
    OpenSSL 0.9.8y 5 Feb 2013
    built on: date not available
    options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
    compiler: cc
    available timing options: USE_TOD HZ=128 [sysconf value]
    timing function used: getrusage
    The 'numbers' are in 1000s of bytes per second processed.
    type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
    aes-128-cbc      14141.64k    70022.01k  295776.27k  597406.71k 13668801.16k

    Thansk for any feedback.

  • Ok,
    I've done some test.

    pure SITE-TO-SITE connection, using 2 ALIX box. (glxsb enabled on both sides)
    IPSEC, with 128 AES, (Phase 1, and Phase 2) hmac SHA1.

    Throughput 5Mbit/s with encryption (AES 128)
    Without encryption … 20Mbit/s
    I've also found some testing tools.

    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(71): cryptotest -z 1000
      0.067 sec,    2000    aes crypts,      16 bytes,  475815 byte/sec,    3.6 Mb/sec
      0.068 sec,    2000    aes crypts,      32 bytes,  943702 byte/sec,    7.2 Mb/sec
      0.069 sec,    2000    aes crypts,      64 bytes,  1842708 byte/sec,    14.1 Mb/sec
      0.073 sec,    2000    aes crypts,    128 bytes,  3523211 byte/sec,    26.9 Mb/sec
      0.078 sec,    2000    aes crypts,    256 bytes,  6539704 byte/sec,    49.9 Mb/sec
      0.091 sec,    2000    aes crypts,    512 bytes, 11235832 byte/sec,    85.7 Mb/sec
      0.116 sec,    2000    aes crypts,    1024 bytes, 17639662 byte/sec,  134.6 Mb/sec
      0.168 sec,    2000    aes crypts,    2048 bytes, 24451834 byte/sec,  186.6 Mb/sec
      0.272 sec,    2000    aes crypts,    4096 bytes, 30074415 byte/sec,  229.4 Mb/sec
      0.564 sec,    2000    aes crypts,    8192 bytes, 29059899 byte/sec,  221.7 Mb/sec

    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(72): cryptostats
    1749914 symmetric crypto ops (0 errors, 318 times driver blocked)
    1 key ops (1 errors, 0 times driver blocked)
    1744022 crypto dispatch thread activations
    28611 crypto return thread activations

    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(80): ipsecstats
    ah packets with hmac-md5: 24974
    ah packets with hmac-sha1: 11588
    esp input packets processed: 21419
    esp output packets processed: 15143
    esp packets with rijndael-cbc: 36562
    esp bytes received: 26989824
    esp bytes transmitted: 1405153

    no SA found 273 (output)
    m_clone processing: 0 mbufs + 0 clusters coalesced
    m_clone processing: 0 clusters copied
    m_makespace: 0 mbufs inserted
    header position [front/middle/end]: 0/21419/0

    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(99): kldstat
    Id Refs Address    Size    Name
    1    4 0xc0400000 104a230  kernel
    2    1 0xc3061000 5000    glxsb.ko
    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(100): kldload crypto
    kldload: can't load crypto: File exists
    [2.1.2-RELEASE][admin@pfSense.localdomain]/root(101): kldload cryptodev
    kldload: can't load cryptodev: File exists

    No traffic shaping on both side,
    No NAT.
    No tunning.

    Simple configuration (out-of-box)


    Any clue ?

  • Ok,
    Now I know what is going on.

    AMD GEODE LX with buil-in cryptodev supported by glxsb modul supports only => (AES) - (CBC/ECB),
    SHA1,MD5 is not supported.

    So if You are using IPSEC with AES-128 with glxsb on ALIX with Phase 2 with HMAC auth You will not experience crypto acceleration couse of glxsb.

    There is understatement in PFSENE doc… about IPSEC performance with AES-128,
    You are not able to achieve more than 5Mbit/s througput inside IPSEC tunel with AES 128, and HMAC-SHA1/MD5 etc...
    and You are not allowed to disable HMAC in Phase 2.

    I've found some post .. (made by CMD)

    Re: Alix IPsec benchmarks 1.2 1.2.2 1.2.3 glxsb hifn
    « Reply #3 on: June 23, 2009, 02:22:10 am »


    I just happened to find this now that I'm messing with glxsb. We added the patch in kern/132622 in March, it's in 1.2.3 snapshots. Thanks much for your work on glxsb, Patrick!  Glad to see you on our forum too.

    We're looking at building glxsb as a module right now, so we can test with and without it, and to get it out of the way when you have a much faster Hifn installed.

    I'm seeing 19.4 Mbps through IPsec with AES-128 on an ALIX with glxsb, and 40 Mbps 3DES with a hifn 7955 (Soekris vpn1411) vs. 8.4 Mbps 3DES without hifn. Nice performance boost with the hifn. Not sure what impact glxsb has yet.

    How did You measure this ?,
    How does the config look ?
    Phase 1 ?
    Phase 2 ?

  • Hi,

    Did you figure it out to increase performance?

    I have the same problem!

  • @pieterraxis:


    Did you figure it out to increase performance?

    I have the same problem!

    And i am using  aes 128 cbc

    What does RNG mean in this line?

    $ dmesg | grep AES
    glxsb0: <amd geode="" lx="" security="" block="" (aes-128-cbc,="" rng)="">mem 0xefff4000-0xefff7fff irq 9 at device 1.2 on pci0</amd>