IPSec performance using 1 gigabit /second WAN

[Tacoma]

I am interested in IPSec performance using 1 gigabit/sec WAN connections.
My initial testing is run on the bench using spare computers.
Ultimately I was hoping to use the 4 core SG-4860 devices in our applications, but only if I am certain I can get the full 1 gigabit throughput through the IPSec tunnel.

In my testing there are 4 computers used.
2 of the computers have dedicated pfsense installations with IPSec tunnels connecting them on the wan side.
The IPSec tunnel is setup for AES 256 phase 1, and 2.

The 2 other computers are used in a file transfer test from LAN side to LAN side across the tunnel.
When the two computers are setup on the same subnet as a benchmark baseline, the file transfer rates are at the full 1 gigabit / second speed.
However, when using the IPSec tunnel to transfer the files, the transfer rate drops to around 80-100 mbit / sec
These test devices are all dedicated to this test.

Attached are activity performance screen shots.  The two pfsense computers are mostly idle expect for the interrupt task and they show free memory still available.  On the pfsense computer #1 utilization runs around 50-65% on the interrupt routine and on pfsense computer #2 shows around 30% utilization on the interrupt routine.  Since utilization is well less than 100%, I am wondering why the throughput isn't better?
Are there any settings or recommendations that might increase the speed?
Can anyone show me results from a pair of SG-4860's that show they can handle the full 1 gigabit speed?

See performance attachments for:
pfsense computer 1
pfsense computer 2

[eri--]

You should start by loading AESNI module.

In pfSense 2.2.x surely its confirmed you can get 800Mbit/s with lower boxes with AES-GCM.
In 2.3 its improved a bit more.

Can you please post your numbers and what ipsec configuration you are using?

[Tacoma]

@ermal:

You should start by loading AESNI module.

In pfSense 2.2.x surely its confirmed you can get 800Mbit/s with lower boxes with AES-GCM.
In 2.3 its improved a bit more.

Can you please post your numbers and what ipsec configuration you are using?

For AESNI modules, it looks like a 64 bit installation is required?
These are 32 bit installs, so I may have to start over and with a 64 bit installation.

Wasn't sure what "your numbers" were, but see attached jpg documents for phase 1 and phase 2 test configs.

[Tacoma]

@ermal:

You should start by loading AESNI module.

In pfSense 2.2.x surely its confirmed you can get 800Mbit/s with lower boxes with AES-GCM.
In 2.3 its improved a bit more.

Can you please post your numbers and what ipsec configuration you are using?

Re-testing with AES-128 and I can see that computer #1 (the less powerful of the 2 pfsense computers) is showing much higher loads on the interrupt than on the first snapshots taken, seems like the interrupt is ranging between 70-90% of utilization