IPSEC VTI Tunnels
-
I already searched in Redmine, but I couldn't find one. Your explanation explains why there is no. (Due to it being a bug in FreeBSD IPSec implementation which must be fixed by FreeBSD Developers)
I'm not familiair with reporting bugs. Will do some research how to submit one. Hopefully this can be resolved in the future.
-
Bumping this because I'm experiencing a similar problem.
I have 2 IPsec Tunnels to an ISP.
IPSec Phase 1 works good. When Phase 2 would come up for a short period of time, then fail. and the whole tunnel would collapse and restart.
I observed a bunch of logs stating
Nov 19 19:40:41 charon 14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found Nov 19 19:40:41 charon 14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found Nov 19 19:40:41 charon 14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not found Nov 19 19:40:41 charon 14[KNL] <con2000|3> querying policy 10.0.255.248/30|/0 === 10.0.255.248/30|/0 in failed, not foundIf I had the VTI settings to retain /0 on the remote, that's when my tunnels would constantly fail and restart after about a minute or so. So I executed the PFShell commands above to set /30. After doing so the IPSec/VTI 'appear' stable. But I can see my Security Association Database creating tons of new SPIs. So something still, just isn't right.
I've setup my interfaces, OPT1 and OPT2, but I'm baffled as to why I don't see any Firewall Rule options for OPT1 like I saw in @ScottS post.
I guess I'm curious if I should expect to see OPT1/OPT2 under my firewall rules and further, and further wonder if my accumulation of SPIs is an indication that things aren't quite right.
I can PING
Any help, or insights would be appreciated.
-
I also have a similar issue with the same error message - the tunnel comes up fine, routing is working bi-directionally but if I attempt to do an iperf across the tunnel stops transferring packets almost immediately.
Tried changing the MTU down to 1300, but didn't change anything.I have 2 SG-3100's setup with ikev2 VTI.
$ tracert 192.168.11.10
Tracing route to 192.168.11.10 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 192.168.1.1
2 35 ms 20 ms 31 ms 10.255.255.2
3 18 ms 28 ms 32 ms 192.168.11.10$ tracert 192.168.1.150
Tracing route to 192.168.1.150 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 192.168.11.1
2 18 ms 23 ms 29 ms 10.255.255.1
3 25 ms 35 ms 29 ms 192.168.1.150Nov 19 21:37:02 charon 13[KNL] <con2000|2> querying policy 10.255.255.2/32|/0 === 10.255.255.1/32|/0 in failed, not found
$ iperf3 -c 192.168.11.10
Connecting to host 192.168.11.10, port 5201
[ 4] local 192.168.1.150 port 51858 connected to 192.168.11.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.01 sec 256 KBytes 2.07 Mbits/sec
[ 4] 1.01-2.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 6.00-7.01 sec 0.00 Bytes 0.00 bits/sec
[ 4] 7.01-8.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
[ 4] 9.00-10.02 sec 0.00 Bytes 0.00 bits/sec
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.02 sec 256 KBytes 209 Kbits/sec sender
[ 4] 0.00-10.02 sec 7.97 KBytes 6.52 Kbits/sec receiverThe_Saint - check that your local and remote assignment is different, in the screenshot it looks like you have both local and remote as the same /32 (10.0.255.248)?
-
@bradlay
Thank you for your reply Bradly. This is my config
I used PFShell to edit the fields manually though it doesn't seem to take the$config['ipsec']['phase2'][0]['remoteid']['type'] = "network"; $config['ipsec']['phase2'][1]['remoteid']['type'] = "network";
For your issue, does your FW actually have interfaces for your VTI (besides the IPSec)?
Also, do you have MSS settings enabled? -
Is the VTI patch needed since 4.2.2 patch2, seeing an odd issue where SSH hangs looks like an MTU issue but it doesn't seem to make a blind bit of difference as to what I set the MSS to in the Ipsec settings.
This was on a new deploy, though it was an LRO issue with vmware, but since upgraded my other firewall from 4.2.2 p1 with the VTI patch to 4.2.2 p2 and are now having the same issue (that one runs in proxmox)
Edit:
Setting the MTU on the VTI interface slightly higher than the MSS on IPSEC > Advanced and lower than the physical interface seemed to fix it
MTU on PHY: 1500
MTU on VTI: 1450
MSS on IPSEC > Advanced 1400It took me way to long to get to the bottom of that, i'm off to grab a coffee
-
@dragon2611 any update? I have been having a reoccurring issue, both ssh issues, and over all routing and connectivity issues through the VTI tunnel. I am also have the phase2 reestablish and not kill off old session/id.
I followed the hangouts setup verbatim. I know its the VTI as communication via openvpn and client if flawless. -
Setting 1400 for MSS (advanced settings on the IPSEC config) and then setting MTU of 1450 on the VTI interface itself (once you've assigned it via assign interface you should then be able to change it's MTU) seemed to fix it for me.
This is a tunnel over a link that has an MTU of 1500, if you have a tunnel going over something with a lower MTU (E.g PPPoE) you might have to adjust the values downwards slightly.
-
It sounds like there is a Bug that causes the MTU set on the VTI interfaces to revert on a reboot/restart.
Looks like it's already in but as a feature request - https://redmine.pfsense.org/issues/9111 - I'd argue it's a bug because it breaks things by causing MTU issues on the tunnel. -
Has anyone found a way to use Gateway Monitor with Routed IPsec (VTI)?
I'm attempting to add the Routed IPsec (VTI) tunnel to a Gateway Group, but if/when the IPSEC tunnel goes down, I need the traffic to traverse an alternate gateway.
Thanks! -
I have not, I have had a really tough time with routing BGP over the VTI, and even static over VTI. Regular IPSEC seems to work fine. There is something broken with VTI, so be aware.
-
@mountainlion said in IPSEC VTI Tunnels:
I have not, I have had a really tough time with routing BGP over the VTI, and even static over VTI. Regular IPSEC seems to work fine. There is something broken with VTI, so be aware.
Routed IPSec (VTI) works great for me. Within the firewall rules, I can force certain hosts or entire interfaces to route all their traffic over the IPSec tunnel by forcing the gateway. But what I can't figure out is how to implement gateway monitoring for that 'gateway'. I need the ability for that traffic to failover to a secondary gateway, should the IPSec tunnel go down. Thanks in advance for anyone who can assist.
-
@AndrewBucklin Did you go to {system/ routing / Gateways ?
Then look for the field that says "Monitor Gateway"
Let me know if that works. -
@mountainlion No, it doesn’t work. No matter what IP I enter for the Monitor IP, the status will never change to “up”. I can only force it up by disabling gateway monitoring.
-
@AndrewBucklin
Assuming you have your ACL for that interface setup properly.... -
@mountainlion Do you mean Firewall Rules? The interface doesn't actually appear under Firewall Rules, but I do have an Allow rule for all traffic under the default "IPsec" firewall rule group:
-
@jimp Would you please elaborate a bit more on NAT doesn't work with VTI?
I've setup 2 VM's with latest pfSense for simplicity (trying to mimic our office setup between a Huawei 4G modem with CGNAT IP and a HQ pfSense firewall with public static ip). Here is what i found (not sure it is related to what you said about NAT & VTI?)
Pfsense1 had Responder only ticked, and remote Gateway in P1 set to 0.0.0.0 didn't work, it would error with
/rc.newipsecdns: The command '/sbin/ifconfig 'ipsec1000' create reqid '1000'' returned exit code '1', the output was 'ifconfig: create: bad value'and it will have the symptoms of having Inbound Traffic in IPSec interface (with packet capture), nothing on the VTI interface.
if i change the remote Gateway in P1 (in pfSesnse1) to the WAN address of pfSesnse2, it will work.
Confusingly, reverting P1 remote gateway to 0.0.0.0 would still work and persist over ipsec service restart. it will only stop working if i delete P2 & P1 entry, (along with VTI interface) and recreate them again with P1's remote gateway set to 0.0.0.0 (or a reboot after setting to 0.0.0.0).
I got P1's remote gateway set as 0.0.0.0 to work with IPv4 Tunnel Mode, but am trying to achieve it with VTI since it's easier to manage with static routing rather than ACL's (on Huawei end) and avoid multiple P2 entries on pfSense. The reason behind HQ pfSense remote gateway set to 0.0.0.0 is that branch office with 4G modem is CGNATed as i said earlier
Any suggestion would be very much appreciated
! -
VTI and NAT not working means that you can't NAT to other addresses as traffic exits a VTI inside the tunnel -- it has nothing to do with establishing the tunnel as is your case. VTI won't be possible without a static remote peer, or at least dyndns. It needs to know the remote peer address. That's a topic for another thread, though.
-
@jimp when you say static remote peer or dyndns, you mean a public IP and hence a modem with CGNAT won't work? (given that Huawei modem supports ipsec on logical interfaces.)
-
Maybe an example of a running Cisco pfSense VTI tunnel connection with dynmaic routing helps:
Cisco-pfSense with VTI
Unfortunately in German but ist pretty self explaining.
