Also, this should not be possible, right?

b59dd3a3-ac9d-4c42-89f7-6bf3dbd29f62-image.png

172.17.1.27 is a Server on the IPsec-Side, not an OVPN-client.
Why did this appear as src on the ovpns1 Interface...

@steveits Thanks for your reply. I am aware of the changes. I was initially on 2.4.5, then went to 21.02-p1, and am currently on 21.02.2.

As mentioned, the issues started after the 'upgrade' from 2.4.5. But a few details that I will add since I was thinking back:

Very rarely, the speed on transfers does go up to the expected speed of around 40 MB/s and lasts a few minutes, but then returns to the around 8 MB/s. Unfortunately didn't catch the logs when this happened.

Also, the logs look normal, just the dashboard checking the IPsec status in the normal fashion.

CPU usage does rise when Async is turned off, but speeds stay basically the same regardless.

The issue is not like what is mostly mentioned by others, where the tunnel does not stay active. Mine does stay active, and is rather stable, its just that the speed is much slower than previous, with the same settings.

i updated an test pfsense to 2.6 dev version and the problem is solved.

@operator2024 Hi
I have same situation, no matter what I do I can't get a second phase 2 to come up when it uses a subnet that doesn't directly exist on a local interface.
could you please tell me what exactly you did so i can compare with my conf

in my case i have
Palo Alto --- IPsec ---- Pfsense --- IPsec --- AWS

Pfsense --- IPsec ---- Pfsense --- IPsec --- AWS

both don't work
could you please help

@werter OSPF - это уже лишнее в данной ситуации. Вопрос этот я решил через дополнительную фазу 2

@mainzelman Thanks for the reply.

Site B IPSec firewall rules were empty (I assumed this to be ok because Site A and Site B hosts can talk no problems)

I added the rule for Site B and it appears to be now working!
dd6e54f6-fa74-4b38-bf03-a8b3e6c04ec9-image.png

I knew it had to be something simple I missed, thank you!

hi all can i ask if is that possible when you used ipsec vpn in pfsense . im using vmware workstation in my laptop. when i tried to connect to another pfsense which is located to another site it doesn't work for me please help

My laptop connect to isp then i installed vmware workstation at my laptop then setup pfsense server.

Hard to see how that could be. The packet is arriving over the IPSec. TCP Syn packets are tiny anyway. But if you've seen something similar before I guess....

But that pass rule should match and clearly isn't. IP Options on it or something odd?

Steve

@alexandre-angeli said in IPSEC perdendo conexão:

A IPSEC fica offline enquanto não usa, e comprovei o correto funcionamento, quando pingo ela "levanta" novamente.

Hmmmm, mas:
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/keep-alive.html

It's not a fatal error, just annoying log spam. No way to suppress it currently.

Sorted it.

On line 96 of /etc/inc/ipsec.auth-user.php it reads:

$userGroups = getUserGroups($username, $authcfg, array());

Where it should read:

$userGroups = getUserGroups($username, $authcfg, $attributes = array());

To abide by PHP referenced variable.

You would use separate P2 entries for each subnet.

Though you could combine the 172.x.x.x as 172.16.0.0/14 which would cover both 172.17 and 172.18, so long as it doesn't conflict with anything else you are doing.

Alternately, use routed IPsec then you don't need to worry about tunnel mode policies at all.

@CodeNinja said in How to setup a second local network for an IPSec connection?:

I'm also curious if its preferred/best practice to use "supernet" or this "multiple tunnel" construction like i currently do.

In many bigger scenarios, I see "supernets" or bigger CIDR masks to simplify tunnel deployments. Especially in centralized structures with one or two "main" sites with big uplinks and many small/branch offices network design often tends to do sth. along these lines:

Roll out big network structure on main(1) -> e.g. multiple 172.19.x.0/24 networks for security segmentation Dial Up / RAS VPN uses IP ranges either from an upper 172.19.x segment or another IP range altogether (e.g. 192.168.vvv.0/24) Branch offices use separate range -> e.g. 10.10.bbb.0/24 for office 1, 10.20.bbb.0/24 for office 2 (or 10.11.bbb.0 if you have a whole lot of branch offices).

With that setup, you can easily do tunnels from "main" to "site a" with <172.19.0.0/16> <-> <10.10.0.0/16> and have no problem whatsoever to grow in either space. If you have the need for new networks on site or on in the main location - just add another VLAN with /24 and as your tunnel is set up with /16 it already includes the new networks.

So yeah, pretty common to use CIDR ranges bigger than your local network to add some "space to grow" lateron.

I also noticed this morning that one of the connection had 8 tunnels where i expected only 4. 5 are duplicates from eachother and 1 is missing..

That seems strange. A duplicate can (and will) happen at times, when rekeying gets near and the lifetime is about to expire. Then it's pretty normal to sometimes see every phase with a second entry as the old one gets "disabled" (but not disconnected) and the new one takes over so the rekey/lifetime turnaround goes smooth. You then see new traffic accumulate on the newer P2 and the old one won't get any more and after expiry should vanish a few seconds/minutes later. But having the same phase 5 times is strange. And some were brought up only seconds after another. Weird. I'd disconnect the whole bunch and reestablish the tunnel and check if that happens again. Perhaps something with the edgerouter on the other site? Maybe setting the split option in P1 of the connection could help if pfsense tries to group the connection but the edgerouter doesn't support it (fully) - but that's just a guess.

More details are in the attached file. I cannot seem to add it here, because it's supposed spam.
A little frustrating.

MoreDetails.txt

I could not find my previous post, I thought it was not posted properly, now I found it but can not remove this one... please Admin, remove it and pardon my mistake

You could send some of that /27 across OpenVPN to the other site if the /27 is routed to you.

If the interface is a /27 that's going to be much more difficult.

@Konstanti said in IPsec VTI pfSense to UniFi:

@kriechmaden
Hi
ifconfig does not show that the vti tunnel is up (There is no vti tunnel in the list of interfaces, ipsec1000, for example)
This is the output of ifconfig on my PFSense .
enc0: flags=41<UP,RUNNING> metric 0 mtu 1536
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: enc
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
pflog0: flags=100<PROMISC> metric 0 mtu 33160
groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
groups: pfsync
syncpeer: 224.0.0.240 maxupd: 128 defer: on
syncok: 1
ipsec1000: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1400
tunnel inet 10.3.100.1 --> 10.3.100.100
inet6 fe80::a00:27ff:fe02:c8c1%ipsec1000 prefixlen 64 scopeid 0x7
inet 10.6.106.1 --> 10.6.106.2 netmask 0xfffffffc
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
reqid: 1000
groups: ipsec

[2.4.4-RELEASE][root@fw1]/root: ifconfig vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE> ether 02:00:00:ef:85:e6 hwaddr 02:00:00:ef:85:e6 inet6 fe80::ff:feef:85e6%vtnet0 prefixlen 64 scopeid 0x1 inet6 XXXX:XXXX:XXXX:XXXX::3 prefixlen 64 inet6 XXXX:XXXX:XXXX:XXXX::2 prefixlen 64 vhid 3 inet XXX.XXX.XXX.114 netmask 0xfffffff0 broadcast XXX.XXX.XXX.XXX inet XXX.XXX.XXX.115 netmask 0xfffffff0 broadcast XXX.XXX.XXX.XXX vhid 5 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T <full-duplex> status: active carp: MASTER vhid 3 advbase 1 advskew 0 carp: MASTER vhid 5 advbase 1 advskew 0 vtnet1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE> ether 26:88:14:13:f6:c0 hwaddr 26:88:14:13:f6:c0 inet6 fe80::2488:14ff:fe13:f6c0%vtnet1 prefixlen 64 scopeid 0x2 inet6 fd60:fef5:50c0:e3fc::2 prefixlen 64 inet6 fd60:fef5:50c0:e3cf::1 prefixlen 64 vhid 4 inet 10.0.0.2 netmask 0xfffffc00 broadcast 10.0.3.255 inet 10.0.0.1 netmask 0xfffffc00 broadcast 10.0.3.255 vhid 1 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T <full-duplex> status: active carp: MASTER vhid 1 advbase 1 advskew 0 carp: MASTER vhid 4 advbase 1 advskew 0 vtnet2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=c00b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE> ether 72:df:07:2c:37:6b hwaddr 72:df:07:2c:37:6b inet6 fe80::70df:7ff:fe2c:376b%vtnet2 prefixlen 64 scopeid 0x3 inet6 fdf5:3371:813a:5aac::2 prefixlen 64 inet6 fdf5:3371:813a:5aac::1 prefixlen 64 vhid 7 inet 10.0.8.2 netmask 0xfffffc00 broadcast 10.0.11.255 inet 10.0.8.1 netmask 0xfffffc00 broadcast 10.0.11.255 vhid 6 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T <full-duplex> status: active carp: MASTER vhid 6 advbase 1 advskew 0 carp: MASTER vhid 7 advbase 1 advskew 0 enc0: flags=41<UP,RUNNING> metric 0 mtu 1536 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: enc lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo pflog0: flags=100<PROMISC> metric 0 mtu 33160 groups: pflog pfsync0: flags=41<UP,RUNNING> metric 0 mtu 1500 groups: pfsync pfsync: syncdev: vtnet1 syncpeer: 10.0.0.3 maxupd: 128 defer: off syncok: 1 ovpns1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> inet6 fe80::10d4:df6d:5e75:438d%ovpns1 prefixlen 64 scopeid 0x8 inet6 fd75:6d19:84ae:d2c9::1 prefixlen 64 inet 10.0.252.1 --> 10.0.252.2 netmask 0xffffffff nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: tun openvpn Opened by PID 87483 ovpns2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> inet6 fe80::10d4:df6d:5e75:438d%ovpns2 prefixlen 64 scopeid 0x9 inet6 fd9f:17e9:b703:fb61::1 prefixlen 64 inet 10.0.248.1 --> 10.0.248.2 netmask 0xffffffff nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: tun openvpn Opened by PID 92953 ovpns3: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> inet6 fe80::10d4:df6d:5e75:438d%ovpns3 prefixlen 64 scopeid 0xa inet6 fd27:dd3e:7e8e:d32e::1 prefixlen 64 inet 10.0.244.1 --> 10.0.244.2 netmask 0xffffffff nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: tun openvpn Opened by PID 11769 ovpns4: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> inet6 fe80::10d4:df6d:5e75:438d%ovpns4 prefixlen 64 scopeid 0xb inet6 fd7d:a519:4cbf:b745::1 prefixlen 64 inet 10.255.243.241 --> 10.255.243.242 netmask 0xffffffff nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: tun openvpn Opened by PID 18031 ipsec1000: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500 tunnel inet XXX.XXX.XXX.114 --> XXX.XXX.XXX.253 inet6 fe80::10d4:df6d:5e75:438d%ipsec1000 prefixlen 64 scopeid 0xc inet 10.252.243.242 --> 10.252.243.243 netmask 0xfffffff0 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> reqid: 1000 groups: ipsec

Very strange the tunnel is now up and it seems to be working.
Niw the problem is, that if we ping the other site no response from the host is coming. But on a tcpdum we see, that the ICMP reuqest was received and the echo is send.

Ping from UniFi to pfSense:

tcpdump on the pfSense

[2.4.4-RELEASE][root@fw1]/root: tcpdump -i ipsec1000 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ipsec1000, link-type NULL (BSD loopback), capture size 262144 bytes 16:09:14.055108 IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51345, length 44 16:09:14.148584 IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 997, length 8 16:09:14.248284 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 27647, seq 10, length 64 16:09:14.248326 IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 27647, seq 10, length 64 16:09:14.388477 IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51346, length 44 16:09:14.568088 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 223, length 64 16:09:14.568148 IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 19207, seq 223, length 64 16:09:14.659358 IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 998, length 8 16:09:15.055940 IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51348, length 44 16:09:15.169418 IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 999, length 8 16:09:15.249018 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 27647, seq 11, length 64 16:09:15.249038 IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 27647, seq 11, length 64 16:09:15.389457 IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51349, length 44 16:09:15.569286 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 224, length 64 16:09:15.569363 IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 19207, seq 224, length 64 16:09:15.690172 IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 1000, length 8 16:09:16.057022 IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51351, length 44 16:09:16.227717 IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 1001, length 8 16:09:16.248121 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 27647, seq 12, length 64 16:09:16.248179 IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 27647, seq 12, length 64 16:09:16.390299 IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51352, length 44 16:09:16.578063 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 225, length 64 16:09:16.578156 IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 19207, seq 225, length 64 16:09:16.764230 IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 1002, length 8 ^C 24 packets captured 24 packets received by filter 0 packets dropped by kernel [2.4.4-RELEASE][root@fw1]/root: tcpdump -i enc0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enc0, link-type ENC (OpenBSD encapsulated IP), capture size 262144 bytes 16:09:22.578118 (authentic,confidential): SPI 0xc24b31ec: IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 231, length 64 16:09:22.578170 (authentic,confidential): SPI 0xc7901687: IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 19207, seq 231, length 64 16:09:23.049116 (authentic,confidential): SPI 0xc7901687: IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 1014, length 8 16:09:23.054739 (authentic,confidential): SPI 0xc7901687: IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51372, length 44 16:09:23.247988 (authentic,confidential): SPI 0xc24b31ec: IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 27647, seq 19, length 64 16:09:23.248049 (authentic,confidential): SPI 0xc7901687: IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 27647, seq 19, length 64 16:09:23.388236 (authentic,confidential): SPI 0xc7901687: IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51373, length 44 16:09:23.563408 (authentic,confidential): SPI 0xc7901687: IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 1015, length 8 16:09:23.577872 (authentic,confidential): SPI 0xc24b31ec: IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 232, length 64 16:09:23.577917 (authentic,confidential): SPI 0xc7901687: IP 10.252.243.242 > 10.252.243.243: ICMP echo reply, id 19207, seq 232, length 64 16:09:24.055579 (authentic,confidential): SPI 0xc7901687: IP ip114.ip-51-75-157.eu > 10.1.0.2: ICMP echo request, id 34846, seq 51375, length 44 16:09:24.077415 (authentic,confidential): SPI 0xc7901687: IP 10.252.243.242 > 10.252.243.243: ICMP echo request, id 2268, seq 1016, length 8 ^C 12 packets captured 12 packets received by filter 0 packets dropped by kernel

ping command on the UniFi site:

device-admin@USG-PRO1:~$ ping 10.252.243.242 PING 10.252.243.242 (10.252.243.242) 56(84) bytes of data. ^C^C --- 10.252.243.242 ping statistics --- 31 packets transmitted, 0 received, 100% packet loss, time 30050ms device-admin@USG-PRO1:~$ ping 10.252.243.243 PING 10.252.243.243 (10.252.243.243) 56(84) bytes of data. 64 bytes from 10.252.243.243: icmp_req=1 ttl=64 time=0.146 ms 64 bytes from 10.252.243.243: icmp_req=2 ttl=64 time=0.125 ms 64 bytes from 10.252.243.243: icmp_req=3 ttl=64 time=0.148 ms 64 bytes from 10.252.243.243: icmp_req=4 ttl=64 time=0.113 ms ^C --- 10.252.243.243 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3000ms rtt min/avg/max/mdev = 0.113/0.133/0.148/0.014 ms

Ping from pfSense to UniFi:

tcpdump on UniFi:

root@USG-PRO1:~# tcpdump -i vti64 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on vti64, link-type RAW (Raw IP), capture size 262144 bytes 16:11:21.689119 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 350, length 64 16:11:22.688719 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 351, length 64 16:11:23.689212 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 352, length 64 16:11:24.688916 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 353, length 64 16:11:25.690202 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 354, length 64 16:11:26.699178 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 355, length 64 16:11:27.699096 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 356, length 64 16:11:28.699099 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 357, length 64 16:11:29.709125 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 358, length 64 16:11:30.709099 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 359, length 64 16:11:31.719095 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 360, length 64 16:11:32.720917 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 361, length 64 16:11:33.729182 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 362, length 64 16:11:34.739091 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 363, length 64 16:11:35.739018 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 364, length 64 16:11:36.739108 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 365, length 64 16:11:37.749104 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 366, length 64 16:11:38.749143 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 367, length 64 16:11:39.749049 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 368, length 64 16:11:40.759056 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 369, length 64 16:11:41.759098 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 370, length 64 16:11:42.759097 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 371, length 64 16:11:43.759065 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 372, length 64 16:11:44.759087 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 373, length 64 16:11:45.759073 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 374, length 64 16:11:46.759099 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 375, length 64 16:11:47.760307 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 376, length 64 16:11:48.772532 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 377, length 64 16:11:49.779243 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 378, length 64 16:11:50.789094 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 379, length 64 16:11:51.789082 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 380, length 64 16:11:52.789099 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 381, length 64 16:11:53.789073 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 382, length 64 16:11:54.789129 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 383, length 64 16:11:55.788908 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 384, length 64 16:11:56.788971 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 385, length 64 16:11:57.789097 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 386, length 64 16:11:58.789057 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 387, length 64 16:11:59.789097 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 388, length 64 16:12:00.789103 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 389, length 64 16:12:01.789119 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 390, length 64 16:12:02.789123 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 391, length 64 16:12:03.789111 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 392, length 64 16:12:04.789085 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 393, length 64 16:12:05.789154 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 394, length 64 16:12:06.789099 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 395, length 64 16:12:07.789096 IP 10.252.243.243 > 10.252.243.242: ICMP echo request, id 19207, seq 396, length 64 ^C 47 packets captured 47 packets received by filter 0 packets dropped by kernel

Ping command on pfSense:

[2.4.4-RELEASE][root@fw1]/root: ping 10.252.243.243 PING 10.252.243.243 (10.252.243.243): 56 data bytes ^C --- 10.252.243.243 ping statistics --- 10 packets transmitted, 0 packets received, 100.0% packet loss [2.4.4-RELEASE][root@fw1]/root: ping 10.252.243.242 PING 10.252.243.242 (10.252.243.242): 56 data bytes ^C --- 10.252.243.242 ping statistics --- 15 packets transmitted, 0 packets received, 100.0% packet loss

A brief update on the issue:
I did a several tcpdumps to see how traffic traveled through the machine.
pings from the Raspberry Pi 10.0.1.253 to 172.17.20.130 are sent to the ipsec interface and receive replies:
tcpdump -i lagg0.100 icmp

14:22:27.875175 IP 10.0.1.253 > 172.17.20.130: ICMP echo request, id 8761, seq 14, length 64 14:22:27.899031 IP 172.17.20.130 > 10.0.1.253: ICMP echo reply, id 8761, seq 14, length 64

These in turn are encapsulated:
tcpdump -i enc0 icmp

14:23:23.969153 (authentic,confidential): SPI 0x82bee771: IP 10.0.1.253 > 172.17.20.130: ICMP echo request, id 8761, seq 70, length 64 14:23:23.992999 (authentic,confidential): SPI 0xc030c198: IP 172.17.20.130 > 10.0.1.254: ICMP echo reply, id 21640, seq 70, length 64

We see the requests going out as 10.0.1.253 and replies coming in for 10.0.1.254, which are then translated back to finally be delivered to the Pi.

Now, when repeating this check with an OpenVPN client, we see different behavior:
tcpdump -i ovpns7 icmp

14:27:13.375762 IP 192.168.248.2 > 172.17.20.130: ICMP echo request, id 13598, seq 1, length 64 14:27:14.383797 IP 192.168.248.2 > 172.17.20.130: ICMP echo request, id 13598, seq 2, length 64

Requests go out, but nothing returns. Let's see where the requests go:
tcpdump -i enc0 icmp yields nothing, so let's try WAN:
tcpdump -i lagg0.4090 icmp

14:30:10.318176 IP 192.168.248.2 > 172.17.20.130: ICMP echo request, id 30820, seq 3, length 64 14:30:11.326380 IP 192.168.248.2 > 172.17.20.130: ICMP echo request, id 30820, seq 4, length 64

Now it's getting interesting: the pings from OpenVPN clients are sent to the default route (WAN). If I change the NAT rule's interface from IPsec to WAN, the translation also shows up in the tcpdump output:

interface: WAN address family: ipv4 protocol: any source: 192.168.248.0/24 destination: 172.17.0.0/16 NAT address: INT_IP (10.0.1.254) Pool options: default

Let's try the tcpdump again:
tcpdump -i lagg0.4090 icmp

14:33:12.248168 IP 10.0.1.254 > 172.17.20.130: ICMP echo request, id 15552, seq 1, length 64 14:33:13.251668 IP 10.0.1.254 > 172.17.20.130: ICMP echo request, id 15552, seq 2, length 64

So the firewall is correctly performing outbound NAT translation for 192.168.248.0/24->172.17.0.0/16 as 10.0.1.254, but the traffic is routed from the OpenVPN interface to the default gateway (WAN interface / lagg0.4090) rather than to the IPsec tunnel for encapsulation.

The whole point of wanting to use outbound NAT for traffic to 172.17.0.0/16 is so either side only needs to define one P2 entry, but we can set up multiple OpenVPN tunnels (with each a different subnet) that can all reach that remote network.

Would anybody know a trick / way to force traffic from 192.168.248.0 to 172.17.0.0/16 to go via the IPsec interface (enc0) rather than the default gateway (WAN/lagg0.4090)? I've tried creating a bogus gateway under System/Routing/Gateways and creating a static route under System/Routing/Static Routes for 172.17.0.0/16 via this bogus gateway, but this only resulted in all traffic from ovpns7 meant for this subnet to get put through a loop until the TTL expired.

@dragoangel
Maybe https://forum.netgate.com/category/30/bounties
If you really need it and are willing to pay for it.
Else the best you can do is hope that it will come some time...

I believe that I have a similar issue with the IPsec tunnel. My setup is as follow:
main office
Pfsense with Dual WAN. They are configured in a Failover group (Tier1 and 2 ). I have static public IP addresses for both of the WANs.

Branch office:
Pfsense with one WAN. It has a static public IP address.

I configured one tunnel in the main office that points towards the branch office from the WAN failover group. On the branch office, I set up 2 tunnels one point towards the main WAN in the main office and the other points towards the secondary WAN.

When the main WAN in the main office goes offline, The tunnel will be recreated with the secondary WAN. But when The main WAN comes back online there will be 2 active tunnels from the main and secondary WAN to the branch office. I don't know if this is an issue with my configuration since there is one tunnel in the main firewall and 2 tunnels in the other firewall, or it is only a limited configuration.

I try change mode from tunnel IPv4 to Route (VTI) but after change IPsec not connect.

For anyone who is interested (n00b here), i got it to work (branch to pfsense only):

Phase 1 remote subnet on pfsense has to be 0.0.0.0 with responder only option checked.

on Huawei Side, the following command had to be configured:

ipsec authentication sha2 compatible enable

the result is:

22accdc1-de10-456f-beb1-06c813df2382-image.png

The problem now is that pfsense does not direct traffic with destination to remote subnet (i.e. 10.2.20.0) through IPSec, it uses WAN0 for that. any ideas?

[update] working now, was pinging from the wrong device.

BGP over policy based IPSec is just asking for trouble IMO. It's possible to make it work but you need P2s to cover all traffic being routed and the BGP traffic itself. Since AWS only allows 2 P2s on each P1 that's often a problem.

Steve

you can have multiple tunnel configured, i don't see why not

Thanks Pablo. Good to have in case we ever move to an HA setup with Google VPN. For anyone else that reads this, my posts were for the Classic Google VPN setup (non HA).

One note I wanted to add, in the BGP settings in my instructions above, don't change the setting for "Redistribute connected networks" to Yes. When set to Yes this advertised our WAN network to Google and caused issues with hitting public facing servers we had in Google. Since we only have a few networks locally, I just manually defined those along with the BGP network 169.254.10.0/30 in the fields below that setting.

The other option may be to change the setting to Yes and somehow mark it to ignore the WAN network, but I haven't looked into that.