Hello Everyone, I have ,what is quite possibly, a really stupid question. My network has 2 sites that are seperated by about 3 miles. Our IP blocks are 22.214.171.124/28 and 126.96.36.199/28 at these 2 sites, each at 100Mbps sync, and only 1 hop that is the providers router between
Currently we use a combination of juniper and pfsense (vmware) to handle our firewall/routing.
We have the junipers handling the vpn traffic between the 2 sites and other 3rd party providers. All non vpn traffic has the source nat done here. There us a /27 subnet inside the router that contains 1 vlan interface on the pfsense and a few other things that we dont want inside out networks. These are numbered routed tunnels, not policy based.
The pfsense has all NAT disabled, no firewall rules except any/any/any and 15ish vlans at the sites (this was a quick inexpensive way to get a 10gb router). We are able to fully saturate and sustain the 10gbe interfaces on the pfsense box both local to each hypervisor and also going across our 2x 10gbe nics in LACP to our switch cluster and across our network so pfsense is doing its job perfectly.
The issue is traffic across the tunnel is only hitting 80% on average for throughput across the vpn while getting a full 100% when going to the internet. The following is my diagnostic process
1. In the vlan that exists between the juniper and pfsense traffic is at 100% and performs as expected
2. In and all vlans pfsense routes to my vpn it is around 80% consistantly.
3. If i disable the vpn and directly route all traffic going between the 2 junipers to the other site without encryption and source nat disabled(i know this is bad but was just testing) the performance is 100%
4. I copied the vlan and ip setup on pfsense to another juniper and used the origional configuration where i function on a day to day basis except juniper router to juniper firewall as opposed to pfsense router to pfsense firewall, and throughput is 100%
5. I reconfigured the tunnel to the absolute lowest possible encryption overhead to get the juniper from 10% cpu load to under 5% just to rule out the unlikely overhead issue.
I can find absolutely nothing that could be causing this. I like to think of myself as a fairly efficient at troubleshooting, however this has me stumped.
Thank You in advance for your input.