I know this is an old topic but I got here from searching the error message. In our case the person adding the VPN didn't use the .ps1 file from pfSense to do it, and Windows 11 24H2 still apparently uses weak algos by default.

@itBJA said in Mobile Clients loosing connectivity akter 60 minutes:

But as I understood, the initial PFS Group might be different from a later rekeying, what would be very weird.

That would be wierd, but my understanding is that the initial setup can succeed even if there is a PFS mismatch between the client and server (might even be disabled at the client side). However during later rekey attempts it will certainly fail...

@michmoor I would love to give your reply a thumbs up, but apparently you have to have 5 something, and no clue on how to get that.

Anyway, I'm going to look at wireguard; however, i upped my p1 timeout, rekey, and expiry times to 7 days then 10 under for rekey and 2 under for expiry and i've gone ahead and upped the p2 to 1 day and rekey at 5 minutes under.

That was at 13:44 and we are now at 16:17 and we haven't had a drop yet.

@tompark said in IPSec to USG behind NAT:

It looks to be as if the connection between the USG and the PFSense I am connecting too, timesout. Is there a way that I can easierly check the traffic is being forwarded by the PFSense firewall?

You can check pftop to see the state table. Doc is here

@diegoavelinogomes
You need to add an IPSec phase 2 for the OpenVPN tunnel network:
A:
local: A clients subnet
remote: OpenVPN tunnel subnet
B:
local: OpenVPN tunnel subnet
remote: A clients subnet

In the OpenVPN access server settings you have to add the "A clients subnet" to the "Local networks".

However, OpenVPN clients might block access from the remote site by their own firewalls by default. You will have to configure their firewalls accordingly.
Possibly masquerading the source IP with the OpenVPN interface IP can circumvent firewall restrictions.

@viragomann firstly thanks for the reply, forgive me for the dense paragraph. I did setup a P2 per subnet. In any case I have in the mean time found a solution (which happens to coincide with what you suggested):

Firstly I switched from BiNAT to simple NAT for each target subnet and then I NATted on a single address per each subnet like this:

Local Subnet Remote Subnet NATed IP
172.20.48.x/24 10.10.x.x/16 10.10.12.201/24
172.20.48.x/24 10.11.x.x/16 10.10.12.202/24
172.20.2.x/24 10.10.x.x/16 10.10.12.203/24
172.20.2.x/24 10.11.x.x/16 10.10.12.204/24

This way the packets from both subnets are routed and NATed through the IPSec tunnel correctly. However, even though I also added a local static route as per https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/access-firewall-over-ipsec.html to be able to be pinged by the other end for monitoring, the remote pings do not return. This may be a limitation of the above NAT setup in the IPSec. At the moment this isn't a major issue but I should find a solution for the remote monitoring setup.

Notes:
The BiNAT solution works only for a primary Network (the first P2 Network encountered in the list), any subsequent different specified P2 network subnet is ignored - packets reach the firewall LAN interface but are not routed to the IPSec interface.

I have not attempted a routed VTI IPSec approach which I suppose will work fine also, but requires a more elaborate configuration to setup the relative IPSec enabled interfaces, firewall rules and NAT/routing. I may take a shot at this in the future as with the above setup it's hard if not impossible to have a fallback/secondary IPSec gateway configured if the main one dies - this would require the VTI routed approach if I'm not mistaken.

Some additional references to IPSec / NAT issues and workarounds which seem to be relevant:
https://forum.netgate.com/topic/155132/problems-with-routed-ipsec-vti/6
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248474

Cheers

@crosstheroad said in IPSec problem with one-way traffic flow:

On the 172.16.136.14 VM I have added a static route in the OS for traffic destined for 10.12.105.0/24 to route through 172.16.136.20 (instead of the default gateway 172.16.136.1)
I have also added a static route on the 10.12.105.10 VM since it is connected to multiple networks, but can still only reach the site B pfSense LAN IP address and nothing else on that subnet.

You should at least be able to reach 172.16.136.14, which has the static route set.
If the machine doesn't respond though, it probably blocks access from outside of its local subnet by its own firewall.

Update 2: Fixed it. It is not so clear that vti interfaces ip addresses have to be routed also. To make it simple: use single /24 subnet for all vti tunnels and add this subnet to "Static routes" at every site

@TheStormsOfFury Okay - that means we should totally disregard the IPsec numbers and tests for now. Since we can’t reach Gbit in the public/public test, then we won’t be able to through IPsec either.

When doing 4 streams?

Are the numbers more or less the same in both directions when the client is in site 1 and server in site 2 (test from client with and without -R) Are the numbers more less the same if you run the server on site 1 and test both directions with the client on site 2? Is the packetloss still showing up in the “ramp up” intial phase of all 4 session tests above? (this causes TCP to not scale further) What it the PING latency between the sites?

Anyways - like I said, these things are sometimes impossible to diagnose properly because there is so many factors in play.
I’m guessing you probably have to do a packetcapture of the first 2 secs of an iPerf run to see if it’s just packetloss that causes TCP not to scale further, or if it is out-of-order delivery. I have started seeing the latter become a more and more prevalent issue on the Internet. I don’t know if ISPs are starting to use some new multipath loadbalancing mechanisms that are really suceptible to in-path latency causing a lot of out-of-order issues end-to-end.

The point here is - your speedtests to internet does not pass the routers and paths that the test between your sites does. So it’s likely something in that path that is causing the problem - be it latency, out-of-order or packetdrops. Generally ISP’s focus and calibrate bandwidth and latency on vertical traffic (that is down/upload to the wider internet). Horisontal traffic between end user sites is of very little to no priority at all depending on the subscriptions used in both ends.

@mcury

Copy. Will either write it up on the plane today or when I get to my destination.

@lc63
The answer seems to be no. I have switched to Policy-based mode for tunnels, which allows failover automatically.

@lmassera

Hey, I'm the person who made the original thread that you referenced. I can confirm that setting the interface address manually via console is a working solution for 24.11. However, under "system -> routing" the gateway address changed from 0.0.0.0 to "dynamic", which is not correct. This still makes me think it's a bug.

I would like to stress again that routing via interface has been working flawlessly since at least CE 2.6.0 (from 2022), even though I understand it's not quite supported explicitly in strongswan. Setting a static address for an interface that is then ignored seems... unnecessary.

@patient0 Fixed, I forgot to add the remote ID :).
Thanks for help.