[IPSEC site-to-site] Subnets Connectivity



  • Hi,

    I've configured IpSec Site-to-Site using PfSense from AWS Marketplace (on EC2 instance). But can not ping Remote Private Network from Local Private Network.

    Details:

    b9550251-29c1-4c89-b557-81acaa746731-image.png

    I have 2 elastic network interfaces on instance with pfSense. 1st is in public subnet and 2nd in private subnet (as it is described in pfsense docs)

    I've Used Routed Phase2 Mode => ipsec2000 interface has been created => Static Route has been added

    I've configured Outbound NAT for 172.19.5.0/24 to translate to 172.16.17.30 address.

    IpSec is Up and I can ping remote tunnel endpoint and hosts in remote private network from pfsense itself:

    PING 192.168.179.117 (192.168.179.117) from 172.16.17.30: 56 data bytes
    64 bytes from 192.168.179.117: icmp_seq=0 ttl=252 time=280.156 ms
    64 bytes from 192.168.179.117: icmp_seq=1 ttl=252 time=280.335 ms
    64 bytes from 192.168.179.117: icmp_seq=2 ttl=252 time=280.528 ms
    
    --- 192.168.179.117 ping statistics ---
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 280.156/280.340/280.528/0.152 ms
    

    75fc869f-bca9-4772-bdeb-bbf529e80feb-image.png

    Local Private Subnet - 172.19.5.0/24
    xn1 interface address - 172.19.5.254
    xn0 interface address - 172.19.1.148
    ipsec2000 interface address - 172.16.17.30/30

    Remote Private Subnet - 192.168.179.0/24
    Remote Tunnel Endpoint - 172.16.17.29/30

    When I ping 192.168.179.117 from 179.19.5.219 I can not get ICMP replies back.
    But they are coming to VTI ipsec2000:

    [2.4.4-RELEASE][admin@pfSense.localdomain]/root: tcpdump -i ipsec2000 | grep 192.168.179.117
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ipsec2000, link-type NULL (BSD loopback), capture size 262144 bytes
    10:35:05.341401 IP 192.168.179.117 > pfSense.localdomain: ICMP echo reply, id 29800, seq 31, length 64
    10:35:06.061043 IP pfSense.localdomain > 192.168.179.117: ICMP echo request, id 29800, seq 32, length 64
    10:35:06.341897 IP 192.168.179.117 > pfSense.localdomain: ICMP echo reply, id 29800, seq 32, length 64
    10:35:07.061127 IP pfSense.localdomain > 192.168.179.117: ICMP echo request, id 29800, seq 33, length 64
    10:35:07.341692 IP 192.168.179.117 > pfSense.localdomain: ICMP echo reply, id 29800, seq 33, length 64
    10:35:08.061225 IP pfSense.localdomain > 192.168.179.117: ICMP echo request, id 29800, seq 34, length 64
    10:35:08.341818 IP 192.168.179.117 > pfSense.localdomain: ICMP echo reply, id 29800, seq 34, length 64
    10:35:09.061677 IP pfSense.localdomain > 192.168.179.117: ICMP echo request, id 29800, seq 35, length 64
    
    

    So the problem is ICMP replies don't get to Host which initially sends requests.

    Please advise, what could be the cause of the issue and how can I resolve this to establish connectivity between private networks?



  • @nomatter
    Hello
    and if you run tcpdump on PFSense LAN interface ?
    tcpdump -i xn1 icmp and host 192.168.179.117

    Is there any reply packets ?



  • @Konstanti
    Hi,

    No, just requests:

    [2.4.4-RELEASE][admin@pfSense.localdomain]/root: tcpdump -i xn1
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on xn1, link-type EN10MB (Ethernet), capture size 262144 bytes
    16:07:33.981401 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 14, length 64
    16:07:34.981459 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 15, length 64
    16:07:35.981537 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 16, length 64
    16:07:36.981581 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 17, length 64
    16:07:37.981629 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 18, length 64
    16:07:38.981679 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 19, length 64
    16:07:39.981731 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 20, length 64
    16:07:40.981778 IP 172.19.5.219 > 192.168.179.117: ICMP echo request, id 3386, seq 21, length 64
    ^C
    8 packets captured
    8 packets received by filter
    0 packets dropped by kernel
    


  • @nomatter

    Are there any floating rules ?

    If not, you can try to delete the tunnel from the PFSense side and create it again



  • @Konstanti
    No, no Floating Rules.

    Do you mean delete all Ipsec Config and setup it again?



  • @nomatter

    You can make a backup copy of the firewall settings, and then try to configure the tunnel again .
    And show the rules on LAN (xn1) and IPSEC interfaces



  • @Konstanti

    Recreated IpSec - same results.

    Here are Firewall Rules

    #System aliases
      
    loopback = "{ lo0 }"
    WAN = "{ xn0 }"
    IPSEC = "{ ipsec2000 }"
    LAN = "{ xn1 }"
    IPsec = "{ enc0 }"
    OpenVPN = "{ openvpn }"
    
    # Outbound NAT rules (manual)
    nat on $IPSEC inet from 172.19.5.0/24 to any port 500 -> 172.16.17.30/32  static-port
    nat on $IPSEC inet from 172.19.5.0/24 to any -> 172.16.17.30/32 port 1024:65535 
    nat on $WAN inet from 127.0.0.0/8 to any port 500 -> 172.19.1.148/32  static-port # Auto created rule for ISAKMP - localhost to WAN
    nat on $WAN inet from 127.0.0.0/8 to any -> 172.19.1.148/32 port 1024:65535  # Auto created rule - localhost to WAN
    nat on $WAN inet6 from ::1/128 to any port 500 -> (xn0)  static-port # Auto created rule for ISAKMP - localhost to WAN
    nat on $WAN inet6 from ::1/128 to any -> (xn0) port 1024:65535  # Auto created rule - localhost to WAN
    nat on $WAN inet from 172.25.53.0/24 to any port 500 -> 172.19.1.148/32  static-port # Auto created rule for ISAKMP - IPsec client to WAN
    nat on $WAN inet from 172.25.53.0/24 to any -> 172.19.1.148/32 port 1024:65535  # Auto created rule - IPsec client to WAN
    nat on $WAN inet from 172.16.17.29 to any port 500 -> 172.19.1.148/32  static-port # Auto created rule for ISAKMP - IPsec VTI: SX site-to-site to WAN
    nat on $WAN inet from 172.16.17.29 to any -> 172.19.1.148/32 port 1024:65535  # Auto created rule - IPsec VTI: SX site-to-site to WAN
    
    # Outbound NAT rules (automatic)
    
    # Subnets to NAT 
    table <tonatsubnets> { 127.0.0.0/8 ::1/128 172.19.5.0/24 172.25.53.0/24 172.16.17.29 }
    nat on $WAN inet from <tonatsubnets> to any port 500 -> 172.19.1.148/32  static-port
    nat on $WAN inet6 from <tonatsubnets> to any port 500 -> (xn0)  static-port
    nat on $WAN inet from <tonatsubnets> to any -> 172.19.1.148/32 port 1024:65535 
    nat on $WAN inet6 from <tonatsubnets> to any -> (xn0) port 1024:65535 
    
    anchor "userrules/*"
    pass  in  quick  on $IPsec inet proto icmp  from any to any tracker 1574329796 keep state  label "USER_RULE"
    pass  in  quick  on $IPsec inet proto tcp  from any to any tracker 1574273174 flags S/SA keep state  label "USER_RULE"
    pass  in  quick  on $IPsec inet from any to any tracker 1565007607 keep state  label "USER_RULE: Default allow IPsec to any rule"
    pass  in  quick  on $IPsec inet6 from any to any tracker 1565007607 keep state  label "USER_RULE: Default allow IPsec to any rule"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto udp  from any to any port 4500 tracker 1565015851 keep state  label "USER_RULE"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto udp  from any to any port 500 tracker 1565015834 keep state  label "USER_RULE"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto esp  from any to any tracker 1565015776 keep state  label "USER_RULE: IPsec"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto icmp  from any to 172.19.1.148 tracker 1565007607 keep state  label "USER_RULE: Default ICMP rule"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto tcp  from any to 172.19.1.148 port 22 tracker 1565007607 flags S/SA keep state  label "USER_RULE: Default SSH rule _replace_src_with_mgmtnet_"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto tcp  from any to 172.19.1.148 port 443 tracker 1565007607 flags S/SA keep state  label "USER_RULE: Default HTTPS rule _replace_src_with_mgmtnet_"
    pass  in  quick  on $WAN reply-to ( xn0 172.19.1.1 ) inet proto tcp  from any to 172.19.1.148 port 80 tracker 1565007607 flags S/SA keep state  label "USER_RULE: Default HTTP rule _replace_src_with_mgmtnet_"
    pass  in log  quick  on $LAN inet from 172.16.17.28/30 to 172.19.5.0/24 tracker 1574333773 keep state  label "USER_RULE"
    pass  in log  quick  on $LAN inet from 172.19.5.0/24 to any tracker 1574331909 keep state  label "USER_RULE"
    
    # VPN Rules
    pass out   route-to ( xn0 172.19.1.1 )  proto udp from (self) to [REMOTE_IP] port = 500 tracker 1000106361 keep state label "IPsec: SX site-to-site - outbound isakmp"
    pass in  on $WAN  reply-to ( xn0 172.19.1.1 )  proto udp from [REMOTE_IP] to (self) port = 500 tracker 1000106362 keep state label "IPsec: SX site-to-site - inbound isakmp"
    pass out   route-to ( xn0 172.19.1.1 )  proto udp from (self) to [REMOTE_IP] port = 4500 tracker 1000106363 keep state label "IPsec: SX site-to-site - outbound nat-t"
    pass in  on $WAN  reply-to ( xn0 172.19.1.1 )  proto udp from [REMOTE_IP] to (self) port = 4500 tracker 1000106364 keep state label "IPsec: SX site-to-site - inbound nat-t"
    pass out   route-to ( xn0 172.19.1.1 )  proto esp from (self) to [REMOTE_IP] tracker 1000106365 keep state label "IPsec: SX site-to-site - outbound esp proto"
    pass in  on $WAN  reply-to ( xn0 172.19.1.1 )  proto esp from [REMOTE_IP] to (self) tracker 1000106366 keep state label "IPsec: SX site-to-site - inbound esp proto"
    
    

    Hope I showed all relevant rules.



  • @nomatter said in

    And if to refuse NAT OUTBOUND for a network 172.19.5.0 / 24 on the IPSEC interface and to add a static route on CISCO for this network ?

    Do you have firewall rules that are not quite necessary

    pass in log quick on $LAN inet from 172.16.17.28/30 to 172.19.5.0/24 tracker 1574333773 keep state label "USER_RULE"
    pass in log quick on $LAN inet from 172.19.5.0/24 to any tracker 1574331909 keep state label "USER_RULE"

    pass in quick on $IPsec inet proto icmp from any to any tracker 1574329796 keep state label "USER_RULE"
    pass in quick on $IPsec inet proto tcp from any to any tracker 1574273174 flags S/SA keep state label "USER_RULE"

    pass in quick on $IPsec inet from any to any tracker 1565007607 keep state label "USER_RULE: Default allow IPsec to any rule"



  • Yes, I know they are not neccessary. I've added them to track packets in mid of troubleshooting and forget to remove.

    Unfortunately I have no access to the remote side. I can just ask their network guy to add it.

    Without NAT there are no ICMP replies even to ipsec2000. They recommended to setup NAT so I believe they don't have any config for my private network.

    Actually I thought if ICMP replies are coming to pfsense than there is some misconfiguration on my side.



  • @nomatter

    Try to move the NAT OUTBOUND to manual mode
    88228526-9b29-4900-8c30-1d6fe5e1611a-image.png



  • @Konstanti

    Did that. No luck. Replies are returning to ipsec2000.

    Feels like pfsense can not route them back to 172.19.5.219 or operation reverse to NAT doesn't take place.



  • @nomatter
    And if you try to do so ?
    What will be the result ?

    ebea7cfe-75aa-4cd8-be8f-4fecb95ee5ad-image.png



  • @Konstanti

    PING 192.168.179.117 (192.168.179.117) from 172.19.5.254: 56 data bytes
    
    --- 192.168.179.117 ping statistics ---
    3 packets transmitted, 0 packets received, 100.0% packet loss
    

    It's ok only when Source address is VTI

    PING 192.168.179.117 (192.168.179.117) from 172.16.17.30: 56 data bytes
    64 bytes from 192.168.179.117: icmp_seq=0 ttl=252 time=280.394 ms
    64 bytes from 192.168.179.117: icmp_seq=1 ttl=252 time=280.279 ms
    64 bytes from 192.168.179.117: icmp_seq=2 ttl=252 time=280.492 ms
    
    --- 192.168.179.117 ping statistics ---
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 280.279/280.388/280.492/0.087 ms
    


  • @nomatter Your initial tcpdump of ipsec2000 doesn't reveal what the actual IP if, suggest running with -n option.
    On the surface this looks like a routing problem.
    Check the routing table on the C3945 and make sure it has routes to 172.19.1.0/24 and 172.19.5.0/24 via the tunnel.



  • @awebster

    Tried to tcpdump with -n option same results:

    tcpdump -n -i ipsec2000 | grep 192.168.179.117
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ipsec2000, link-type NULL (BSD loopback), capture size 262144 bytes
    18:02:04.472813 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 12413, seq 105, length 64
    18:02:04.752956 IP 192.168.179.117 > 172.16.17.30: ICMP echo reply, id 12413, seq 105, length 64
    18:02:05.473363 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 12413, seq 106, length 64
    18:02:05.753784 IP 192.168.179.117 > 172.16.17.30: ICMP echo reply, id 12413, seq 106, length 64
    18:02:06.472929 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 12413, seq 107, length 64
    18:02:06.753029 IP 192.168.179.117 > 172.16.17.30: ICMP echo reply, id 12413, seq 107, length 64
    18:02:07.472973 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 12413, seq 108, length 64
    18:02:07.753102 IP 192.168.179.117 > 172.16.17.30: ICMP echo reply, id 12413, seq 108, length 64
    18:02:08.473044 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 12413, seq 109, length 64
    ^C66 packets captured
    66 packets received by filter
    0 packets dropped by kernel
    
    

    I'll ask guys from other side to add routes. I'm sure they don't have them. But will need to wait until Monday at least.
    Can I do something relevant on my side while waiting?



  • @nomatter Thanks, that dump is showing the VTI interface pinging the destination host.
    The VTI on the subnet /30 shared between both devices, so it is normal that the C3945 knows how to get back to it, but if there are no routes, the C3945 has no way to know that it has to send traffic for 172.19.1.0/24 and 172.19.5.0/24 to 172.16.17.30.



  • @awebster
    Ah, yes. I missed that on initial dump there is pfsense.localdomain instead of address. You're right.
    I asked other side to add 2 routes 172.19.5.0/24 and 172.19.1.0/24 using 172.16.17.30 as Gateway. Will wait.
    I'll return to this thread after their reply.
    Thank you guys for now



  • @awebster
    Turned out that other side is unable to set static routes to 172.19.X.X networks as there will be overlap with their local networks :(

    Is there any way to work around setting routes on their side?

    Actually, I think routes on their side is needed if traffic originates out there.

    For example to ping google.com we don't need google to setup routes.

    There must be some misconfiguration on my side.



  • Turned out that other side is unable to set static routes to 172.19.X.X networks as there will be overlap with their local networks :(
    Is there any way to work around setting routes on their side?

    @nomatter, one possible way to work around an overlapping subnet problem is to use 1:1 NAT, but before you do that, ask the other side what subnets that you could present to them that won't cause overlap, the you can setup the pfSense to NAT 172.19.1.x and 172.19.5.x into something that both sides can agree on.
    For example 172.19.1.0/24 <=> 10.19.1.0/24 and 172.19.5.0/24 <=> 10.19.5.0/24
    On their side they would need to put routes for 10.19.1.0/24 and 10.19.5.0/24
    When you ping their system with their address (no change), it looks to them like it is coming from a 10.19.x.x address
    When they ping your system, they use 10.19.x.x address and it looks to you like it is coming from their address (no change).

    Alternatively if you don't want to use 1:1 NAT, depending on how much work is involved, you could rebuild the AWS setup using different subnets.



  • @awebster

    SO if I understand correctly at first 172.19.5.0/24 is translated to 10.19.5.0/24 and then outbound NAT is taking place translating 10.19.5.0/24 to 172.16.17.30 which is 'allowed' to communicate with remote side?



  • @nomatter Not quite... but you do bring up another way that this could be accomplished, albeit in a limited fashion.
    If you don't need the ability to reach the AWS instances from the remote side, you can hide NAT everything behind the 172.16.17.30 address, but this will only allow one-way initiated connections, unless you port-forward certain ports inside AWS. Functional, but limited; I wouldn't recommend it.

    Here's an example of what happens with 1:1 NAT based on your original diagram...
    Lets look at what happens when Instance 172.19.5.219 pings instance 192.168.179.119

    • Ping packet has SRC=172.19.5.219 and DST=192.168.179.119
    • Packet arrives at pfSense on 172.19.5.254 and gets NATted, so now looks like this: SRC=10.19.5.219 DST=192.168.179.119.
    • The routing table of pfSense says to reach 192.168.179.119 you need to send the packet to 172.16.17.29 via the VTI interface of the IPSec tunnel.
    • Packet crosses the IPSec tunnel and arrives at C3945 on its tunnel interface.
    • Router looks up packet DST=192.168.179.119, which is an attached network and forwards it to 192.168.179.119.
    • The instance 192.168.179.119 receives the ping request, it is from 10.19.5.219, and replies to the packet with ping reply. The reply packet has SRC=192.168.179.119 and DST=10.19.5.219 - so far so good, but we're only half way there.
    • Packet arrives at the C3945 router and DST=10.19.5.219 address is looked up in the C3945 routing table which says to reach 10.19.5.0/24 you need to forward it to 172.16.17.30 via the tunnel interface.
    • Packet is received by pfSense on 172.16.17.30 and DST address 10.19.5.219 is NATted to 172.19.5.219. The packet looks like this: SRC=192.168.179.119 and DST=172.19.5.219
    • Packet is sent out interface 172.19.5.254 to host 172.19.5.219 which sees ping reply packet with SRC=192.168.179.119.

    The above routing and NAT does not take into consideration the firewall aspects of this scenario. If the remote side has firewalls, it needs to be aware that the AWS instances are behind 10.19.x.x, not 172.19.x.x.
    Similarly, the pfSense needs to be aware that the AWS instances will be accessed using 10.19.x.x addresses.
    Maybe someone with more in-depth knowledge of pfSense internals can chime in on whether inbound firewall rules on IPSEC interface are pre-NAT or post-NAT. Its easy enough to test it out with some logging to see which one works properly.



  • @awebster

    Thank you for such detailed explanation!

    Seems like adding NAT 1:1 is making things more complicated than they should be.
    I'll setup VPC with another CIDR block instead.

    But I can not understand is it really necessary for them to have route to my private subnet 172.19.5.0/24 if stick to Routed Phase 2 mode.
    I looked to ICMP replies captured at VTI interface in wireshark and they have dest addr 172.16.17.30

    Internet Protocol Version 4, Src: 192.168.179.118, Dst: 172.16.17.30
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 84
    Identification: 0x305c (12380)
    Flags: 0x4000, Don't fragment
    Time to live: 252
    Protocol: ICMP (1)
    Header checksum: 0x12a3 [validation disabled]
    [Header checksum status: Unverified]
    Source: 192.168.179.118
    Destination: 172.16.17.30

    Shouldn't they have dest address 172.19.5.219?
    Looks like from their perspective everything is ok. C3945 receives ICMP requests from 172.16.17.30 and sends replies...
    Maybe pfsense doesn't handle those replies properly not redirecting them to original source?

    Also it's strange for me why I can not ping 172.16.17.30 from 172.19.5.219 which is one of the router interfaces...



  • @nomatter
    Freebsd does not always work correctly on a virtual machine. In particular, IP packet checksums are not always correctly calculated by network card drivers . Try to enable this option
    /System/Advanced/Networking

    41711858-fc86-4d32-8db5-1a0c6cce69c3-image.png



  • @nomatter said in [IPSEC site-to-site] Subnets Connectivity:

    Shouldn't they have dest address 172.19.5.219?

    Only if that is where the ping initiated from, but I suspect that there might be some confusion from the configuration of the VPN.
    In the IPSEC Phase 2, it is possible to configure it to ping a remote destination to keep the tunnel up. In this case it will ping from 172.16.17.30 to that remote destination, which will be replying, I suspect this is what you are actually seeing.



  • @Konstanti

    Enabled but nothing significant happens.

    There was misconfiguration on AWS VPC Routing table layer that's why I couldn't ping 172.16.17.30 which is pfsense's VTI.
    Now I'm able to ping 172.16.17.30 from 172.19.5.219

    Should I be able to ping 172.16.17.29 which is remote tunnel endpoint? 172.16.17.28/30 is directly attached to pfsense so if i understand correctly in normal circumstances I should be able to ping 172.16.17.29 from any host in 172.19.5.0/24
    For now I can not.
    Is it normal?



  • @awebster
    I don't have checkmark on "automatically ping host" in Phase 2 settings but nevertheless there is ICMP traffic between 172.16.17.30 and 172.16.17.29 all the time and it's not related to my ICMP requests from host in private subnet.

    tcpdump -nni ipsec2000 icmp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ipsec2000, link-type NULL (BSD loopback), capture size 262144 bytes
    >>>07:41:53.453287 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 64007, seq 16, length 64
    07:41:53.668029 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 5860, seq 13119, length 8
    07:41:53.668077 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 48500, seq 54233, length 8
    >>> 07:41:53.732895 IP 192.168.179.117 > 172.16.17.30: ICMP echo reply, id 64007, seq 16, length 64]
    07:41:53.947746 IP 172.16.17.29 > 172.16.17.30: ICMP echo reply, id 5860, seq 13119, length 8
    07:41:53.947762 IP 172.16.17.29 > 172.16.17.30: ICMP echo reply, id 48500, seq 54233, length 8
    07:41:54.209324 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 48500, seq 54234, length 8
    07:41:54.209372 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 5860, seq 13120, length 8
    >>> 07:41:54.453365 IP 172.16.17.30 > 192.168.179.117: ICMP echo request, id 64007, seq 17, length 64
    07:41:54.488891 IP 172.16.17.29 > 172.16.17.30: ICMP echo reply, id 48500, seq 54234, length 8
    07:41:54.488906 IP 172.16.17.29 > 172.16.17.30: ICMP echo reply, id 5860, seq 13120, length 8
    07:41:54.724422 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 5860, seq 13121, length 8
    07:41:54.724473 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 48500, seq 54235, length 8
    >>> 07:41:54.733203 IP 192.168.179.117 > 172.16.17.30: ICMP echo reply, id 64007, seq 17, length 64
    07:41:55.004501 IP 172.16.17.29 > 172.16.17.30: ICMP echo reply, id 5860, seq 13121, length 8
    07:41:55.004517 IP 172.16.17.29 > 172.16.17.30: ICMP echo reply, id 48500, seq 54235, length 8
    07:41:55.265786 IP 172.16.17.30 > 172.16.17.29: ICMP echo request, id 48500, seq 54236, length 8
    


  • Should I be able to ping 172.16.17.29 which is remote tunnel endpoint?

    @nomatter, you will not be able to ping 172.16.17.29 from 172.19.5.0/24 subnet because the C3945 doesn't know how to get back to 172.19.5.0.



  • @awebster

    So I if understand you correctly, you 100% sure that issue is caused by missing routes on the other side?
    And there is no possibility that I missed something in configuration on my side?



  • @nomatter 100%



  • @awebster
    So other side refused to set routes for my private network :) They say that's my private network should be NATed to virtual tunnel endpoint. And it's working like that for all other parties.

    I've set lab environment with ipsec tunnel between 2 pfSenses.
    Of course we need routes on both sides with opposite virtual IPs and endpoints. That's confirmed.
    Everything is working between 2 pfsenses until...

    Turned that there is another issue:
    When trying to apply Outbound NAT on VTI interface connectivity between hosts in remote private networks is lost.
    And I have exactly the same situation which was discussed previously in this topic:
    ICMP replies are getting back to VTI interface but don't going further to source host.

    Are there any issues or limitations with outbound NAT to VTI in pfsense?



  • Summary:

    1. For unknown reason src-nat doesn't work properly with VTI. Observed behavior is described above.
    2. Routed VTI can be used and everything is ok without using NAT to tunnel interface. Setup is simple. Netgate even has video on this.
    3. My issue was solved using software (infrastructure is in AWS) from another vendor.


  • @nomatter Thanks for the followup.
    I've not experimented with VTI source NAT before, and I'm surprised to see that it doesn't in fact work.


Log in to reply