IPSec's Padlock problem



  • I am having IPSec throughput problems with the latest 1.2.3RC3 (built on Wed Oct 14 05:10:15 UTC 2009). For a 500Mhz VIA C7 system, I used to get 40Mbps AES256 IPSec throughput with release 1.2.1 and 1.2.2. For 1.2.3RC3, it drops to 12Mbps, which indicates that the padlock engine is not engaged.

    However, openssl seems to indicate that the padlock engine is there and working. It is 70 times faster than the cryptodev device.

    openssl speed -evp aes-256-cbc -engine padlock
    ...
    type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
    aes-256-cbc      24268.69k    76073.21k  163404.36k  229121.34k  259595.93k
    ...
    ...
    openssl speed -evp aes-256-cbc -engine padlock
    ...
    type            16 bytes    64 bytes    256 bytes  1024 bytes  8192 bytes
    aes-256-cbc      3490.81k    3653.29k    3703.74k    3717.73k    3722.04k
    ...

    Any ideas?
    Bao


  • Rebel Alliance Developer Netgate

    What does your dmesg output look like? Do any other crypto devices show up?

    I know there is an issue on some systems like ALIX where you have to check the option to not load glxsb when you have a Hifn card installed, or it takes precedence even though it is a slower processor. Perhaps there is something similar going on.

    With OpenVPN you can explicitly set "engine cryptodev" or "engine padlock" but I'm not sure if IPsec has any similar counterpart.



  • Hi Jim,

    It seems very strange. There is no crypto devices, except the software cryptdev. I'll show the complete dmesg at the end of this post.

    First, FreeBSD recognizes the Padlock in the CPU.

    CPU: VIA Esther processor  500MHz (500.02-MHz 686-class CPU)
      Origin = "CentaurHauls"  Id = 0x6a9  Stepping = 9
      Features=0xa7c9bbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,cmov,pat,clflush,acpi,mmx,fxsr,sse,sse2,tm,pbe>Features2=0x181 <sse3,est,tm2>VIA Padlock Features=0x3fcc <rng,aes,aes-ctr,sha1,sha256,rsa>...
    But, there is no padlock device loaded.
    ...
    pfSense:~#  dmesg | grep padlock
    pfSense:~#  dmesg | grep Padlock
      VIA Padlock Features=0x3fcc <rng,aes,aes-ctr,sha1,sha256,rsa>...
    Looking at the modules confirms that there is no padlock device.
    ...
    pfSense:~#  kldstat
    Id Refs Address    Size    Name
    1    5 0xc0400000 aa2de0  kernel
    2    1 0xc0ea3000 6a45c    acpi.ko
    3    1 0xc450b000 5000    glxsb.ko
    ...
    I could not load the padlock by hand, or set padlock_load="YES" in the /boot/locader.conf!
    ...
    pfSense:~#  kldload padlock
    kldload: can't load padlock: No such file or directory
    ...
    I tried to unload the glxsb either by kldunload or use the web GUI. The glxsb if off. But tt still does not help.
    ...
    pfSense:~#  kldstat
    Id Refs Address    Size    Name
    1    5 0xc0400000 aa2de0  kernel
    2    1 0xc0ea3000 6a45c    acpi.ko
    3    1 0xc450b000 5000    glxsb.ko
    pfSense:~#  kldunload glxsb
    pfSense:~#  kldstat
    Id Refs Address    Size    Name
    1    3 0xc0400000 aa2de0  kernel
    2    1 0xc0ea3000 6a45c    acpi.ko
    ...
    Following is the complete dmesg:
    ...
    Copyright (c) 1992-2009 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
    FreeBSD is a registered trademark of The FreeBSD Foundation.
    FreeBSD 7.2-RELEASE-p4 #0: Wed Oct 14 05:09:25 UTC 2009
        sullrich@FreeBSD_7.2_pfSense_1.2.3_snaps.pfsense.org:/usr/obj.pfSense/usr/pfSensesrc/src/sys/pfSense_SMP.7
    Timecounter "i8254" frequency 1193182 Hz quality 0
    CPU: VIA Esther processor  500MHz (500.02-MHz 686-class CPU)
      Origin = "CentaurHauls"  Id = 0x6a9  Stepping = 9
      Features=0xa7c9bbff <fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,cmov,pat,clflush,acpi,mmx,fxsr,sse,sse2,tm,pbe>Features2=0x181 <sse3,est,tm2>VIA Padlock Features=0x3fcc <rng,aes,aes-ctr,sha1,sha256,rsa>real memory  = 1055784960 (1006 MB)
    avail memory = 1019269120 (972 MB)
    ACPI APIC Table: <cn700  awrdacpi="">ioapic0 <version 0.3="">irqs 0-23 on motherboard
    wlan: mac acl policy registered
    kbd1 at kbdmux0
    ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
    cryptosoft0: <software crypto="">on motherboard
    acpi0: <cn700 awrdacpi="">on motherboard
    acpi0: [ITHREAD]
    acpi0: Power Button (fixed)
    acpi0: reservation of 0, a0000 (3) failed
    acpi0: reservation of 100000, 3ede0000 (3) failed
    Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
    acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
    acpi_button0: <power button="">on acpi0
    acpi_button1: <sleep button="">on acpi0
    pcib0: <acpi host-pci="" bridge="">port 0xcf8-0xcff on acpi0
    pci0: <acpi pci="" bus="">on pcib0
    pcib1: <pci-pci bridge="">at device 1.0 on pci0
    pci1: <pci bus="">on pcib1
    vgapci0: <vga-compatible display="">mem 0xf4000000-0xf7ffffff,0xfb000000-0xfbffffff irq 16 at device 0.0 on pci1
    rl0: <realtek 10="" 8139="" 100basetx="">port 0xf400-0xf4ff mem 0xfdfff000-0xfdfff0ff irq 16 at device 5.0 on pci0
    miibus0: <mii bus="">on rl0
    rlphy0: <realtek internal="" media="" interface="">PHY 0 on miibus0
    rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    rl0: Ethernet address: 00:40:48:b1:ab:10
    rl0: [ITHREAD]
    rl1: <realtek 10="" 8139="" 100basetx="">port 0xf200-0xf2ff mem 0xfdffe000-0xfdffe0ff irq 17 at device 6.0 on pci0
    miibus1: <mii bus="">on rl1
    rlphy1: <realtek internal="" media="" interface="">PHY 0 on miibus1
    rlphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    rl1: Ethernet address: 00:40:48:b1:ab:0f
    rl1: [ITHREAD]
    rl2: <realtek 10="" 8139="" 100basetx="">port 0xee00-0xeeff mem 0xfdffd000-0xfdffd0ff irq 18 at device 7.0 on pci0
    miibus2: <mii bus="">on rl2
    rlphy2: <realtek internal="" media="" interface="">PHY 0 on miibus2
    rlphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    rl2: Ethernet address: 00:40:48:b1:ab:0e
    rl2: [ITHREAD]
    atapci0: <via 6420="" sata150="" controller="">port 0xff00-0xff07,0xfe00-0xfe03,0xfd00-0xfd07,0xfc00-0xfc03,0xfb00-0xfb0f,0xf000-0xf0ff irq 20 at device 15.0 on pci0
    atapci0: [ITHREAD]
    ata2: <ata 0="" channel="">on atapci0
    ata2: [ITHREAD]
    ata3: <ata 1="" channel="">on atapci0
    ata3: [ITHREAD]
    atapci1: <via 8237="" udma133="" controller="">port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfa00-0xfa0f at device 15.1 on pci0
    ata0: <ata 0="" channel="">on atapci1
    ata0: [ITHREAD]
    ata1: <ata 1="" channel="">on atapci1
    ata1: [ITHREAD]
    uhci0: <via 83c572="" usb="" controller="">port 0xf900-0xf91f irq 21 at device 16.0 on pci0
    uhci0: [GIANT-LOCKED]
    uhci0: [ITHREAD]
    usb0: <via 83c572="" usb="" controller="">on uhci0
    usb0: USB revision 1.0
    uhub0: <via 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usb0
    uhub0: 2 ports with 2 removable, self powered
    uhci1: <via 83c572="" usb="" controller="">port 0xf800-0xf81f irq 21 at device 16.1 on pci0
    uhci1: [GIANT-LOCKED]
    uhci1: [ITHREAD]
    usb1: <via 83c572="" usb="" controller="">on uhci1
    usb1: USB revision 1.0
    uhub1: <via 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usb1
    uhub1: 2 ports with 2 removable, self powered
    uhci2: <via 83c572="" usb="" controller="">port 0xf700-0xf71f irq 21 at device 16.2 on pci0
    uhci2: [GIANT-LOCKED]
    uhci2: [ITHREAD]
    usb2: <via 83c572="" usb="" controller="">on uhci2
    usb2: USB revision 1.0
    uhub2: <via 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usb2
    uhub2: 2 ports with 2 removable, self powered
    uhci3: <via 83c572="" usb="" controller="">port 0xf600-0xf61f irq 21 at device 16.3 on pci0
    uhci3: [GIANT-LOCKED]
    uhci3: [ITHREAD]
    usb3: <via 83c572="" usb="" controller="">on uhci3
    usb3: USB revision 1.0
    uhub3: <via 1="" 9="" uhci="" root="" hub,="" class="" 0,="" rev="" 1.00="" 1.00,="" addr="">on usb3
    uhub3: 2 ports with 2 removable, self powered
    ehci0: <via vt6202="" usb="" 2.0="" controller="">mem 0xfdffc000-0xfdffc0ff irq 21 at device 16.4 on pci0
    ehci0: [GIANT-LOCKED]
    ehci0: [ITHREAD]
    usb4: EHCI version 1.0
    usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
    usb4: <via vt6202="" usb="" 2.0="" controller="">on ehci0
    usb4: USB revision 2.0
    uhub4: <via 1="" 9="" ehci="" root="" hub,="" class="" 0,="" rev="" 2.00="" 1.00,="" addr="">on usb4
    uhub4: 8 ports with 8 removable, self powered
    isab0: <pci-isa bridge="">at device 17.0 on pci0
    isa0: <isa bus="">on isab0
    pci0: <multimedia, audio="">at device 17.5 (no driver attached)
    acpi_tz0: <thermal zone="">on acpi0
    speaker0: <pc speaker="">port 0x61 on acpi0
    sio0: configured irq 4 not in bitmap of probed irqs 0
    sio0: port may not be enabled
    sio0: configured irq 4 not in bitmap of probed irqs 0
    sio0: port may not be enabled
    sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
    sio0: type 16550A
    sio0: [FILTER]
    sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
    sio1: type 16550A
    sio1: [FILTER]
    atkbdc0: <keyboard controller="" (i8042)="">port 0x60,0x64 irq 1 on acpi0
    atkbd0: <at keyboard="">irq 1 on atkbdc0
    kbd0 at atkbd0
    atkbd0: [GIANT-LOCKED]
    atkbd0: [ITHREAD]
    psm0: <ps 2="" mouse="">irq 12 on atkbdc0
    psm0: [GIANT-LOCKED]
    psm0: [ITHREAD]
    psm0: model IntelliMouse Explorer, device ID 4
    cpu0: <acpi cpu="">on acpi0
    est0: <enhanced speedstep="" frequency="" control="">on cpu0
    est: CPU supports Enhanced Speedstep, but is not recognized.
    est: cpu_vendor CentaurHauls, msr 406050604000506
    device_attach: est0 attach returned 6
    p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
    pmtimer0 on isa0
    orm0: <isa option="" rom="">at iomem 0xd0000-0xd3fff pnpid ORM0000 on isa0
    ppc0: <parallel port="">at port 0x378-0x37f irq 7 on isa0
    ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode
    ppbus0: <parallel port="" bus="">on ppc0
    ppbus0: [ITHREAD]
    plip0: <plip network="" interface="">on ppbus0
    plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag
    lpt0: <printer>on ppbus0
    lpt0: Interrupt-driven port
    ppi0: <parallel i="" o="">on ppbus0
    ppc0: [GIANT-LOCKED]
    ppc0: [ITHREAD]
    sc0: <system console="">at flags 0x100 on isa0
    sc0: VGA <16 virtual consoles, flags=0x300>
    vga0: <generic isa="" vga="">at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
    Timecounter "TSC" frequency 500023847 Hz quality 800
    Timecounters tick every 1.000 msec
    IPsec: Initialized Security Association Processing.
    ad2: 1911MB <transcend 20070831="">at ata1-master PIO4
    GEOM: ad2: partition 4 does not start on a track boundary.
    GEOM: ad2: partition 4 does not end on a track boundary.
    GEOM: ad2: partition 1 does not start on a track boundary.
    GEOM: ad2: partition 1 does not end on a track boundary.
    WARNING: Expected rawoffset 0, found 20480
    Trying to mount root from ufs:/dev/ad2s4a</transcend></generic></system></parallel></printer></plip></parallel></parallel></isa></cpu></enhanced></acpi></ps></at></keyboard></pc></thermal></multimedia,></isa></pci-isa></via></via></via></via></via></via></via></via></via></via></via></via></via></via></via></ata></ata></via></ata></ata></via></realtek></mii></realtek></realtek></mii></realtek></realtek></mii></realtek></vga-compatible></pci></pci-pci></acpi></acpi></sleep></power></cn700></software></version></cn700 ></rng,aes,aes-ctr,sha1,sha256,rsa></sse3,est,tm2></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,cmov,pat,clflush,acpi,mmx,fxsr,sse,sse2,tm,pbe></rng,aes,aes-ctr,sha1,sha256,rsa></rng,aes,aes-ctr,sha1,sha256,rsa></sse3,est,tm2></fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,cmov,pat,clflush,acpi,mmx,fxsr,sse,sse2,tm,pbe>



  • Looks like padlock was removed from the kernel. You could try copying the module in from a stock FreeBSD 7.2 box and set padlock_load=YES.



  • The padlock engine driver is not included in the default build, but it does appear that openssl can somehow access the engine directly without the kernel module being loaded (my testing without the module loaded matches bao's).

    I have built a custom build of the latest 1.2.3 that includes the padlock module in the build.  When I manually load the module it does work, and the performance of the cryptodev engine in the openssl test also improves.  What I have found though is certain situations crash the entire box (deleting a SA on the other IPSec endpoint for one seems to bring this about fairly regularly).  The crash is severe enough that the box doesn't boot upon the reboot.  The file system is corrupt (/tmp is missing, etc) and it requires a fresh reinstall.

    Has anyone successfully used the Padlock driver on recent versions of pfSense or FreeBSD 7.2?

    Regards,
    Ron



  • @dotdash:

    Looks like padlock was removed from the kernel. You could try copying the module in from a stock FreeBSD 7.2 box and set padlock_load=YES.

    Does this mean that the pfsense team removed it from our kernel?  If so, is there a reason?


  • Rebel Alliance Developer Netgate

    It's been gone for a while, apparently at some point it did not compile. It was disabled in December of 08. It might compile now, if someone wanted to try a custom build.

    You could open a redmine ticket about it, but it might be too late for 1.2.3.


  • Rebel Alliance Developer Netgate

    I have not tested this, but I did a run on my builder with the padlock module back in the list. If someone is feeling brave they can try it:

    http://pingle.org/files/pfSense-Full-Update-1.2.3-RC3-padlock.tgz

    (I don't have a system with padlock in it to try, myself)


  • Rebel Alliance Developer Netgate

    Padlock was added back into the 2.0 builds, for those interested.

    Did anyone try the padlock image I built?



  • It should be back in all builds actually, should be in 1.2.3 snapshots from 20091020 on.



  • It works perfectly with the latest snapshot: 20091102-0130. I got 45Mps IPSec AES256 throughput measured by iperf on a 500Mhz VIA C7, compared to a miserable 12Mbps without Padlock.

    Thank you both to Jim and Chris!



  • yes the latest snapshot works fine..

    but the hardware crypto (padlock) isnt listed on the main webpage like the HiFN one is..


Log in to reply