On Verizon Fios Gigabit port, I have:
Dual-six core Intel 5650, 8GB RAM (HP DL360 - overkill, I realize)
4 X Broadcom NetExtreme 1000Base-SX (WAN, LAN and 2 spare)

On the Centurylink Gigabit fiber port, I have:
2 vCPU Intel 5650, 2GB RAM - Running as a VMware VM
2 X Intel E1000 vNIC's in the VM (WAN and LAN)

I recognize the difference in provisioning, but I have never seen the VM become even 20% utilized with RAM or CPU.

The IPsec VPN is: Blowfish (256 bits)/SHA1 -  Blowfish (256 bits)/SHA1  (very stable)

I have recently been trying to optimize throughput, and been somewhat mystified by the results. When I run:
iperf -c 192.168.100.29                              I might get: [SUM]  0.0-13.6 sec  89.2 MBytes  55.2 Mbits/sec
iperf -c 192.168.100.29 -P 40 -t 60            I might get: SUM]  0.0-63.4 sec  739 MBytes  97.8 Mbits/sec

I had previously thought those were pretty good results for any VPN, but then I thought to set-up three clients and three servers and was surprised to see the VPN gracefully handle all of the traffic! Running three of each of the previous tests, I saw almost identical results (simultaneously). In other words: the VPN was pushing 300 Mbps  for 60 seconds at one point. BTW, in all cases iperf is running from CentOS 6 X64m as I never trust Windows TCP/IP stack.

Now here is the problem: The VPN is for disaster Recovery and the source and destination Proxies are Windows Server 2008 X64 and I never get throughput greater than 80-100 Mbps.

The question: If the VPN can push 300 Mbps (or more) across the WAN, why does a single connection from one IP (at the source) to one IP (at the destination) seem to be limited to 60-100 Mbps?

THX,
-J