No duplicates appeared over the weekend. Seems split connections is the way to go.

@cto_frank Changing the loglevel is only reloading configuration and not restarting services. So it's not rebulding anything but could theoretically correct an unwanted state. But the reason it works could as well be a special alignment of the planets... 👽 Anyways. Glad it's working 😏

Thank you for your response, but i am still on the same thing.
in log
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 10[ENC] received fragment #1 of 2, waiting for complete IKE message
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 10[MGR] checkin IKE_SA (unnamed)[2]
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 10[MGR] checkin of IKE_SA successful
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[MGR] checkout IKEv2 SA by message with SPIs d026565155dc83e0_i 76c33ab5aa293765_r
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[MGR] IKE_SA (unnamed)[2] successfully checked out
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[NET] received packet: from fd00:4444:5555:6666::2[4500] to fd00:1111:2222:3333::2[4500] (772 bytes)
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[ENC] parsed IKE_AUTH request 1 [ EF(2/2) ]
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[ENC] received fragment #2 of 2, reassembled fragmented IKE message (1920 bytes)
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[ENC] parsed IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ IDr AUTH SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_4_ADDR) N(ADD_6_ADDR) N(ADD_6_ADDR) N(ADD_6_ADDR) N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] received cert request for "C=CZ, ST=Czech republic, L=FrydekMistek, O=OU, OU=OU, CN=My CA, E=sasinka.martin@gmail.com"
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] received end entity cert "C=CZ, ST=Czech republic, L=FrydekMistek, O=OU, OU=OU, CN=computer2, E=sasinka.martin@gmail.com"
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[CFG] looking for peer configs matching fd00:1111:2222:3333::2[computer1]...fd00:4444:5555:6666::2[C=CZ, ST=Czech republic, L=FrydekMistek, O=OU, OU=OU, CN=computer2, E=sasinka.martin@gmail.com]
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[CFG] no matching peer config found
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] peer supports MOBIKE
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] got additional MOBIKE peer address: 192.168.88.246
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] got additional MOBIKE peer address: 192.168.1.48
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] got additional MOBIKE peer address: fd00:1111:2222:3333:70cf:7fe3:26c5:7345
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] got additional MOBIKE peer address: fd00:9999:8888:7777:9d4:e1dd:7a3d:502a
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[IKE] got additional MOBIKE peer address: fd00:4444:5555:6666:7379:3fd6:6178:f67d
Jul 17 15:20:12 martin-Legion-5-15IAH7H charon: 11[ENC] generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]
So the problem is no matching peer config found

computer1 ipsec.conf
conn hosttohost
left=fd00:1111:2222:3333::2
leftcert=server-cert.pem
leftsubnet=fd00:1111:2222:3333::/64
leftid=@computer1
right=fd00:4444:5555:6666::2
rightsubnet=fd00:4444:5555:6666::/64
rightid=@computer2
keyexchange=ikev2
ike=aes256-sha256-modp3072
esp=aes256gcm16-sha384-modp3072
authby=rsasig
type=tunnel
auto=add

computer2 ipsec.conf
conn hosttohost
left=fd00:4444:5555:6666::2
leftcert=client-cert.pem
leftsubnet=fd00:4444:5555:6666::/64
leftid=@computer2
right=fd00:1111:2222:3333::2
rightid=@computer1
keyexchange=ikev2
ike=aes256-sha256-modp3072
esp=aes256gcm16-sha384-modp3072
authby=rsasig
type=tunnel
auto=add
Any ideas please?

Hello @viragomann

Thanks for the responses and attempts to help. We found the problem. On the client side he needed to enable nat-t. After adjustment, communication worked normally. Thank you very much

@rolytheflycatcher Really interesting - or rather sad - that this bug/issue has been there for so many years.
Suggests that FreeBSD is seeing less and less use in large installations/organisations - or that the FreeBSD community is starved for people with knowledge on how to fix core issues like this.

Such a fundamental problem does not go unnoticed in bigger installations, so it would seem policy based IPsec tunneling sees very little use when based on FreeBSD.

@viragomann yes, i did this setting, but it had no effect.

@viragomann thank you, that did the trick

@mrsunfire When using QAT there is very very little difference between the ciphers for mobile clients (which ususally has a bit of latency). Your settings are fine and very secure - you will not see noticable better throughput on other ciphers. Perhaps a little ekstra if you attempt to maximise throughput with 10 iPerf3 sessions, but for real world use - no difference.

@jeffsmith82 It requires nothing special. You just setup Mobile IPsec in pfSense pr. Any available guide - with authentication using Radius.
On the Radius you install the Azure MFA plugin and register that for MFA authentication in the wanted Azure AD tennant. The two things work completely independant of each other - the trick is that Radius will only complete the authencation when the user has approved in their authenticator app.
The only “non-standard” setup in pfSense is that you will need to configure the Mobile Radius auth part with a long timeout as it usually takes a little while for users to get the notification and login/approve on their phone.

Yeah, my other sites not using PFSense stay up all the time no issues, I feel like it is a little configuration setting I have not found yet.
Thanks.

@rcoleman-netgate
Thank you for the suggestion, Unfortunately, this is way out of my knowledge base (doing packet captures, other than on the firewall) and then definately out of my knowledge base on reading pcaps. What should I use to capture packets on the remote side when the devices are not friendly with that? I'm in St. Louis, the "remote end" is in Seattle. The ShoreTel switch in Seattle doesn't provide that capability (not that I can find/document) nor do the ShoreTel ip phones. I have a notebook computer there running Windows 10 but the network switch is not managed so there's no pervasive mode option (just a layer-2 Linksys switch). I provided PCAPs from the firewall to ShoreTel and they said there was no "RTP traffic" coming over the IPSEC tunnel from Seattle to St. Louis and claim it's network/routing issue. The ShoreTel switch does not provide any routing capabilities, it just has a default gateway setting, which is set to the pfSense firewall.

@yeahmagnets
You have to policy route the VoIP traffic to the remote VPN endpoint. But this is not possible with policy based IPSec. I think, it can be done with routed IPSec (VTI), but I never set this up by myself.

You can policy route the traffic with OpenVPN or Wireguard though.

@michmoor Yeah, we'll ask to change that iprange

But indeed when working remote, this looses internet, also at the HQ.

When I was trying at the HQ locally this morning, it did not get lost.... so weird

And just now on the branch it did seem to work when I left I came home and added a second P2 to the tunnel (and somehow I also saw the ipsec gateway was down in a glitch) it went down again....

it does not like me when I try to do thing from home apparently

When the local IT guy disabled the ipsec tunnel, internet was working again

@patrick-pesegodinskiHow I currently do it, I have graylog set up. All my logs from all systems get sent there. I also have routing turned up over the tunnel. When the routing protocol neighbor goes down, a syslog is created, sent to graylog where i have a flow set up that I get an email when this happens. Doesn't always indicates that the tunnel goes down but its informational.
Another twist on this is to use a monitor IP for the other end of the tunnel. When there is loss or high latency a syslog gets created and sent to graylog where I have a flow set up to send me an email. An example of this email is below
9951eb1b-d096-438f-86c7-da868a807d49-image.png

edit
Here is a screen shot of my routing neighborship going down
bb6a82c1-200c-4a77-80dd-c2ee795ab93b-image.png

These examples are just evidence that something is going on on the path the VPN travels between sites. As there could be quite a few hops and the quality of the links could be suspect it indicates trouble but i cant really do much about it.
All of this is just informational but I did manage to spot a few problems and resolve so your mileage may vary.

@jacquesh for some reason everything is working fine this morning. i changed nothing so i really have no idea what fixed this problem..

I FIGURED IT OUT!

Here is what I had to do, before I built the phase 2.

On IPSEC Firewall Rules.

I created the following rules:

Source: MY_Network --- Destination: MOMs_Network Source: MOMs_Network --- My_Network Source: *(ANY) Destination: *(ANY) <--- Disabled Source: Guest_Network Destination: Any Source: Any Destination: Guest_Network

Once that was done I was able to create a phase to allowing:

Source: Guest Network Remote Network: 0.0.0.0/0

I confirmed with mom her internet is still up, I can still access PFSense remotely and the Guest Wifi is not routing through my internet.

I just wanted updated so if anyone else runs into the same issue this will give them the direction to go in.

:) ~ ForrestExplorer~