@michmoor Thanks so much for the response. After further digging, it turns out that I was just being dumb and not paying attention to what I was doing. I was jumping from adapter to adapter, connection to connection running around the building with my laptop. I didn't realize that the dock I had plugged into left me wireless.

Once I figured out I was chasing my tail and wired myself, I was able to all but max out the connection at 35MB/s. I also looked and one site is 500/500 and the other is 300/300., so the 35MB/s makes complete sense.

With that said, I'm going to bump up the second site to 500/500 on Monday. Any suggestions on how to pair the IPsec settings with the settings for the cryptographic hardware?

@aryanrai
Did you add firewall rules to the IPSec interfaces to allow access from the other site?

Or do you try to ping the LAN device in the other network? In this case you have to ensure that the device also allows access from a remote network. For testing disable its firewall.

Thanks @jimp - I found that out with additional reading, so changed tack and now am using OpenVPN with ESET Secure Authentication, which works well and provides convenient push authentication.

Just want to reply here my discoveries, to save people the hassle of attempting this to find out it does not work, there are two types of GRE tunnels, GRETAP and GRETUN, one supports layer 2 features such as broadcast/multicast and one does not, the PFSense implementation appears to use the later which does not support this feature, please see the following article to show the difference

https://developers.redhat.com/blog/2019/05/17/an-introduction-to-linux-virtual-interfaces-tunnels#:~:text=While%20GRE%20tunnels%20operate%20at,header%20in%20the%20inner%20header.

You would need a local UDP relay instead (on the client side) to instead allow the client to relay these broadcast message as unicast to a specific host, I struggled with this for Windows File Sharing (WS-Discovery) broadcast packets and ended up resorting to a script that auto maps all network drives on successful client connection, perhaps someone could get this working with a L2TP on top of Wireguard?

https://github.com/sparky3387/automapwireguard - Shameless plug of the automap script if someone else also needs this.........

That is expected and noted in the GUI:

c93e825d-26f0-4859-99eb-5d883a0f76d3-image.png

There is also more detail in the docs:

https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/configure-p1.html#ike-endpoint-configuration

@gorberus is the local unit able to route to internet using the second card?

@Topogigio the problem persists. After a few days pfSense stops binding the IP address on the established tunnel interface.
I've started to build a new opnSense gateway, but if there is some pfSense solution I'll be happy

@jto82 You may use OSPF, not static, than it will be easy

@Moses_Kabungo said in Setting up a site-to-site IPSec tunnel with a Vendor who needs to reach us via a public IP other than the WAN address:

no CHILD_SA built

From this error message, I"d assume that there is something wrong with the phase 2 configuration. But I don't know, what you're set there.

@HKFEVER
Fail, if I try to connect Office's pfsense IPsec from WIN11 through Home router gateway with NordVPN on!
OK, if I connect Office's pfsense IPsec from WIN11 through Home router gateway with NordVPN off :)

But then after connected:

WIN11's gateway becomes Office's pfsense default gateway. which don't exit out through Office's pfSense's NordVPN setup! If I un-checked "Use default gateway on remote network" in WIN11's ADVANCE TCP/IP Setting, then the gateway will become WIN11's NIC gateway. Which in theory, I can use NordVPN app in WIN11. I didn't try yet, as too busy :(

Here is the new question:
How can I set the WIN11's Internet request to go through "home or some cafeshop's" gateway to Office's pfSense and exit out to internet through Office pfsense's NordVPN setup?

I have send too long to figure out the rules in pfSense and still no go. May be need to find professional help :(

Okay, I took jimp's advice, and after some struggling with syntax, I was able to get past the NO_PROP error (to run into a different error right behind it).

Anyway, to help someone else with the NO_PROP error, I'll document what I did.

I looked in the /var/etc/ipsec/swanctl.conf file on the Netgate 4100 and found these two lines: proposals = aes256-sha256-modp1024 esp_proposals = aes256-sha1,aes256-sha256,aes256-sha384,aes256-sha512

Note that the syntax is very different from what was shown in the log file such as "AES_CBC_128".

I copied these into the corresponding fields in the network-manager-strongswan VPN settings.

On Ubuntu 22.04, it is in this location:

VPN Settings > Identity tab > Algorithms at the bottom

Check the box "Enable custom algorithm proposals"

In the IKE text input, I put:

aes256-sha256-modp1024

In the ESP text input, I put:

aes256-sha1;aes256-sha256;aes256-sha384;aes256-sha512

NOTE THAT THE COMMAS WERE REPLACED WITH SEMICOLONS! This caused me a bit of frustration until I accidentally mouse-overed the input label and saw that it said the list must be semi-colon-separated.

Anyway, with these changes, I now no longer get the NO_PROP error.

Now, I get a missing public key on the SSL certificate. If I can't solve that, I'll start a new thread.

Thanks, @jimp !

More logs still no success.

2100 Logs
Aug 28 23:10:56 charon 25380 05[NET] <201> received packet: from 24.51.235.3[4500] to 99.255.178.179[4500] (304 bytes)
Aug 28 23:10:56 charon 25380 05[ENC] <201> parsed IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
Aug 28 23:10:56 charon 25380 05[IKE] <201> local endpoint changed from 99.255.178.179[500] to 99.255.178.179[4500]
Aug 28 23:10:56 charon 25380 05[IKE] <201> remote endpoint changed from 24.51.235.3[500] to 24.51.235.3[4500]
Aug 28 23:10:56 charon 25380 05[CFG] <201> looking for peer configs matching 99.255.178.179[99.255.178.179]...24.51.235.3[172.24.0.233]
Aug 28 23:10:56 charon 25380 05[CFG] <201> no matching peer config found
Aug 28 23:10:56 charon 25380 05[IKE] <201> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
Aug 28 23:10:56 charon 25380 05[ENC] <201> generating IKE_AUTH response 1 [ N(AUTH_FAILED) ]
Aug 28 23:10:56 charon 25380 05[NET] <201> sending packet: from 99.255.178.179[4500] to 24.51.235.3[4500] (80 bytes)
Aug 28 23:10:56 charon 25380 05[IKE] <201> IKE_SA (unnamed)[201] state change: CONNECTING => DESTROYING
Aug 28 23:11:01 charon 25380 10[CFG] vici client 603 connected
Aug 28 23:11:01 charon 25380 10[CFG] vici client 603 registered for: list-sa
Aug 28 23:11:01 charon 25380 10[CFG] vici client 603 requests: list-sas
Aug 28 23:11:01 charon 25380 06[CFG] vici client 603 disconnected
Aug 28 23:11:07 charon 25380 07[CFG] vici client 604 connected
Aug 28 23:11:07 charon 25380 07[CFG] vici client 604 registered for: list-sa
Aug 28 23:11:07 charon 25380 07[CFG] vici client 604 requests: list-sas
Aug 28 23:11:07 charon 25380 09[CFG] vici client 604 disconnected
Aug 28 23:11:12 charon 25380 13[CFG] vici client 605 connected
Aug 28 23:11:12 charon 25380 13[CFG] vici client 605 registered for: list-sa
Aug 28 23:11:12 charon 25380 07[CFG] vici client 605 requests: list-sas
Aug 28 23:11:12 charon 25380 12[CFG] vici client 605 disconnected

1100 Logs

Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> initiating IKE_SA con1[221] to 99.255.178.179
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> IKE_SA con1[221] state change: CREATED => CONNECTING
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> configured proposals: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> sending supported signature hash algorithms: sha256 sha384 sha512 identity
Aug 28 23:16:56 charon 80583 06[ENC] <con1|221> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
Aug 28 23:16:56 charon 80583 06[NET] <con1|221> sending packet: from 172.24.0.233[500] to 99.255.178.179[500] (464 bytes)
Aug 28 23:16:56 charon 80583 06[NET] <con1|221> received packet: from 99.255.178.179[500] to 172.24.0.233[500] (472 bytes)
Aug 28 23:16:56 charon 80583 06[ENC] <con1|221> parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(CHDLESS_SUP) N(MULT_AUTH) ]
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> received FRAGMENTATION_SUPPORTED notify
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> received SIGNATURE_HASH_ALGORITHMS notify
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> received CHILDLESS_IKEV2_SUPPORTED notify
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> selecting proposal:
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> proposal matches
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> received proposals: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> configured proposals: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> selected proposal: IKE:AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> received supported signature hash algorithms: sha256 sha384 sha512 identity
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> local host is behind NAT, sending keep alives
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> reinitiating already active tasks
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> IKE_CERT_PRE task
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> IKE_AUTH task
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> authentication of '172.24.0.233' (myself) with pre-shared key
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> successfully created shared key MAC
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> proposing traffic selectors for us:
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> 192.168.2.0/24|/0
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> proposing traffic selectors for other:
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> 192.168.1.0/24|/0
Aug 28 23:16:56 charon 80583 06[CFG] <con1|221> configured proposals: ESP:AES_GCM_16_128/NO_EXT_SEQ, ESP:AES_CBC_256/HMAC_SHA2_256_128/NO_EXT_SEQ
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> establishing CHILD_SA con1{206} reqid 1
Aug 28 23:16:56 charon 80583 06[ENC] <con1|221> generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
Aug 28 23:16:56 charon 80583 06[NET] <con1|221> sending packet: from 172.24.0.233[4500] to 99.255.178.179[4500] (304 bytes)
Aug 28 23:16:56 charon 80583 06[NET] <con1|221> received packet: from 99.255.178.179[4500] to 172.24.0.233[4500] (80 bytes)
Aug 28 23:16:56 charon 80583 06[ENC] <con1|221> parsed IKE_AUTH response 1 [ N(AUTH_FAILED) ]
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> received AUTHENTICATION_FAILED notify error
Aug 28 23:16:56 charon 80583 06[CHD] <con1|221> CHILD_SA con1{206} state change: CREATED => DESTROYING
Aug 28 23:16:56 charon 80583 06[IKE] <con1|221> IKE_SA con1[221] state change: CONNECTING => DESTROYING
Aug 28 23:17:17 charon 80583 06[KNL] creating acquire job for policy 172.24.0.233/32|/0 === 99.255.178.179/32|/0 with reqid {1}

Fixed by change IPSec to OpenVPN ( so even speed increased )

I've had no issues with IPSec on pf Plus at least, don't have a 2.7 system to test right now though, but that NAT setting normally shouldn't have to be adjusted.

Just out of curiosity, are you seeing any MAC auth errors?

I had an issue a while back, still not sure if it's solved or not (made a post with no responses) and haven't been able to test, but for some reason I was getting a ton of auth issues after updating pfSense to a newer version when it comes to IPSec, turned out that for some reason the option of using "My IP Address" wasn't properly authenticating and I had to manually specify the IP.

Anyway, seems like that's not related to your issue but just wanted to double check since it was something I ran across and never managed to solve.

Edit: here is that post I made: https://forum.netgate.com/topic/176502/had-to-manually-specify-identifier-ip-address-no-nat-involved-bug

Another edit: this does appear to have been resolved, just got it working when before it wouldn't.

Thank you @NOCling .

I've tried everything and I still have performance problems and packet loss.

Now the biggest problem is that after a while or the size of the traffic data, the traffic is lost and the stats = 0.

I realized that when it reaches 5Mb of traffic data it restarts phase 2 and resets the traffic, not letting it travel anymore. It is necessary to restart the VPN to resume traffic.

I don't know what else to change to solve this.

@mralvi22244
As I wrote, the above with BINAT in IPSec is meant for policy-based tunnel.

The last one is how I think, it has to be configured with VTI.
However, I'm unsure if it will work with the stated local / remote addresses, 192.168.227.253 / 10.10.10.10. Accordingly to the pfSense docs both addresses have be within a (transit) network. But yours obviously aren't. Don't think, that IPSec can do PPP.
But these are the data you stated.