You don't use static routes for IPsec unless you're using VTI.

There is no way to have more than one mobile configuration at this time. Though I have yet to see the strongSwan app fail to connect, since pfSense uses strongSwan for the server. Odds are that it's just a client misconfiguration. Share more details about the configuration and error messages / logs and it can likely be solved.

No problem.

You label only one tunnel there as VTI but whether the routing is dynamic or static if you are routing it must be VTI. If the tunnel to Oracle is policy based then do you have the correct P2s to carry traffic sourced from the AWS VPC? It sounds like they may be missing. Steve

@Velociraptor That PDF is dated 1999, things have moved on since then, i.e IKEv2 which was proposed in 2005 in RFC4306. https://tools.ietf.org/html/rfc4306

So I found my issue. Apparantly "Network Manager" did not safe the password properly or something happened there. So usually commandline the PSK/EAP secrete of the client is stored in /etc/ipsec.secrets where my password was correct but Network Manager also stores it in its own location under /etc/NetworkManager/system-connections/<CONNECTION_NAME> and in the second file a wrong password was given under the section [vpn-secrets] \n password=<PSK>. I dont know why and how because a definitly have change and give the correct password but thats how it is. So user error. -> Check your configs

UPDATE: It seems that the ixl driver has been ported to iflib and is available in FreeBSD 12. As pfSense is adopting FreeBSD 12 for its 2.5 release, I decided to try a development snapshot of pfSense 2.5 (pfSense-CE-memstick-2.5.0-DEVELOPMENT-amd64-20191031-1313.img). Below are the updated results for both pfSense 2.4.4-p3 and pfSense 2.5.0-dev: +------------------+---------------+---------------+ | MTU | streams | pfSense 2.4.4 | pfSense 2.5.0 | +------------------+---------------+---------------+ | | 1 | 1.30 Gbps | 1.66 Gbps | | 1500 |-----------+---------------+---------------+ | | 8 | 1.81 Gbps | 2.52 Gbps | +----------------- +---------------+---------------+ | | 1 | 6.13 Gbps | 2.92 Gbps*| | 9000 |-----------+---------------+---------------+ | | 8 | 8.76 Gbps | 9.77 Gbps | +------------------+---------------+---------------+ *The result for pfSense 2.5.0 with 9000 MTU and 1 stream is suspect. Performance decreased dramatically with pfSense 2.5.0 for this test, while performance improved with pfSense 2.5.0 for all of the other tests. I did see some curious behavior during this specific tests as compared to the others. In all other tests the throughput was relatively constant on a second-by-second basis. During this test the throughput each second varied wildly from 1 to 6 Gbps. The interrupt processing definitely improved with the move to pfSense 2.5.0. Below is a snapshot of the output from "top" on the left pfSense server during the 1500 MTU test. The interrupt processing and CPU usage seems to be distributed nicely across all of the cores. In fact, most of the CPUs seem to be underutilized. So I am not sure now what the bottleneck might be that is limiting performance. [2.5.0-DEVELOPMENT][admin@left]/root: top -P last pid: 46970; load averages: 1.65, 1.10, 0.97 up 0+16:56:20 08:44:13 49 processes: 2 running, 47 sleeping CPU 0: 3.5% user, 0.0% nice, 15.7% system, 3.1% interrupt, 77.6% idle CPU 1: 2.4% user, 0.0% nice, 14.2% system, 2.0% interrupt, 81.5% idle CPU 2: 0.8% user, 0.0% nice, 23.1% system, 2.0% interrupt, 74.1% idle CPU 3: 2.7% user, 0.0% nice, 18.0% system, 5.5% interrupt, 73.7% idle CPU 4: 3.5% user, 0.0% nice, 20.4% system, 2.4% interrupt, 73.7% idle CPU 5: 1.6% user, 0.0% nice, 23.1% system, 4.3% interrupt, 71.0% idle CPU 6: 2.4% user, 0.0% nice, 16.9% system, 1.6% interrupt, 79.2% idle CPU 7: 7.5% user, 0.0% nice, 52.5% system, 0.8% interrupt, 39.2% idle CPU 8: 5.1% user, 0.0% nice, 24.7% system, 1.6% interrupt, 68.6% idle CPU 9: 2.7% user, 0.0% nice, 16.9% system, 2.4% interrupt, 78.0% idle CPU 10: 3.1% user, 0.0% nice, 16.1% system, 3.1% interrupt, 77.6% idle CPU 11: 2.7% user, 0.0% nice, 24.3% system, 2.4% interrupt, 70.6% idle CPU 12: 4.7% user, 0.0% nice, 21.2% system, 2.4% interrupt, 71.8% idle CPU 13: 2.4% user, 0.0% nice, 22.0% system, 2.0% interrupt, 73.7% idle CPU 14: 0.0% user, 0.0% nice, 100% system, 0.0% interrupt, 0.0% idle CPU 15: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 77M Active, 39M Inact, 522M Wired, 69M Buf, 15G Free Swap: 3979M Total, 3979M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 8645 root 17 52 0 70M 20M sigwai 7 8:08 111.07% charon 26414 root 2 92 0 18M 6380K CPU0 0 6:52 99.59% ntpd

What kind of transfer? What is the latency across the tunnel? How exactly are you measuring throughput? AESGCM will generally be faster overall than AES. Change the Phase 2 to AESGCM without a hash.

So, I've installed OPNsense and it seems to work properly there with a basic NAT port forward rule. I'm gonna do a couple more tests with different hardware on pfsense, but I suspect there're some issues with port forwarding rules on pfsense regardin IPSEC. Edit: I've reset pfsenseA to factory defaults and set it up again. It was basically the same as the previous install but something went different this time. Now it seems to work properly.

@enthu19 Hello Show phase 2 settings and rules on Lan and IPSEC interfaces There is no need to configure NAT OUTBOUND for IPSec tunnel (/Firewall/NAT/Outbound) There is no need to configure NAT Reflection for IPSec tunnel.

@Konstanti Thanks mate. Much appreciate it.

@ngoehring123 Yeah. Can't help you with the latency. Glad it helped.

Ok, all that blocked traffic you're seeing is TCP flagged traffic that is out of state. It's either blocked because the states have already closed, probably the case on that :PA traffic, ot because the states were never opened, usually due to rouet asymmetry. https://docs.netgate.com/pfsense/en/latest/firewall/troubleshooting-blocked-log-entries-for-legitimate-connection-packets.html You should be trying to find out why that is happening not just trying to pass the traffic anyway. Remove any floating rules you added there. You should not be seeing asymmetric traffic if this is setup correctly. I assume pings work fine from the policy routed clients? If you run a packet capture do you see both requests and replies at all points in the path? Steve

So I managed to fix it in the end, seemed the issue was the IKEv2 P1 connection on the XG1541's side. I added the public IP as an attribute in the RSA cert and it seemed to then hookup fine - weird how there was no report of this failure though... James.