IPSEC between 2x pfsense



  • Hi,

    I have the following situation:

    Location 1
    pfsense01 direct on public internet connection (fiber connection)
    Ipsec connection to pfsense02 (location 2)
    LAN: 139.64.24.253/24

    Location 2
    pfsense02 direct on public internet connection (fiber connection)
    Ipsec connection to pfsense01 (location 1)
    LAN: 192.168.31.253/24

    The connection is ok according to pfsense. I'm able to access (ping, webinterface) pfsense1 and pfsense02 from both sides.

    I'm unable to connect to an host in the remote network (behind the pfsense) from both sides.
    On both sides i have an firewall rule on the ipsec tab to allow traffic any to any on any protocol
    On the trafficgraphs I see data going over the ipsec connection.

    Outbound NAT mode is automatic

    Could someone point me in the right direction?



  • It looks like the problem is in the routing on the remote side.

    Location1:
    traceroute from desktop(139.64.24.176) behind pfsense01 to an existing ip(192.168.31.252) on location2:

    Tracing route to 192.168.31.252 over a maximum of 30 hops

    1    <1 ms    <1 ms    <1 ms  pfSense01.kennis.local [139.64.24.253]
      2    *        *        *    Request timed out.
      3    *        *        *    Request timed out.
      4    *        *        *    Request timed out.

    packet capture pfsense02 IPSEC interface:
    10:42:46.235413 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176 > 192.168.31.252: ICMP echo request, id 1, seq 13949, length 72
    10:42:46.670158 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176.51668 > 192.168.31.253.443: tcp 1
    10:42:46.670209 (authentic,confidential): SPI 0xc70274fc: IP 192.168.31.253.443 > 139.64.24.176.51668: tcp 0
    10:42:50.233990 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176 > 192.168.31.252: ICMP echo request, id 1, seq 13950, length 72
    10:42:54.235494 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176 > 192.168.31.252: ICMP echo request, id 1, seq 13951, length 72
    10:42:55.186775 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176.51667 > 192.168.31.253.443: tcp 1
    10:42:55.186829 (authentic,confidential): SPI 0xc70274fc: IP 192.168.31.253.443 > 139.64.24.176.51667: tcp 0
    10:42:56.687121 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176.51668 > 192.168.31.253.443: tcp 1
    10:42:56.687167 (authentic,confidential): SPI 0xc70274fc: IP 192.168.31.253.443 > 139.64.24.176.51668: tcp 0
    10:42:58.241166 (authentic,confidential): SPI 0xc0e8a4fd: IP 139.64.24.176 > 192.168.31.252: ICMP echo request, id 1, seq 13952, length 72

    The data is only going in one direction.

    Am I missing an firewall rule or something?



  • Setting apart the (serious) issue that your LAN network resides in public IP space (and I don't think you really own the segment), this has to work right away from a networking point of view. It seems that pfsense02 is delivering the packets to the destination server but that one is not answering. Are you sure that you don't have some setting on the remote machine that is blocking connections from a different subnet? (Windows firewall is known to do this by default). Furthermore, the incoming connections looks like coming from the open internet to a properly configured device so they are even more likely to get blocked.



  • hi,

    Thnx for the reply.

    The remote device (192.168.31.252) is an Linksys Accespoint. ICMP is replying from the remote side (pfsense2 to AP)

    I have changed the subnet so both are in the private space.

    Location 1
    pfsense01 direct on public internet connection (fiber connection)
    Ipsec connection to pfsense02 (location 2)
    LAN: 10.9.8.253/24

    Location 2
    pfsense02 direct on public internet connection (fiber connection)
    Ipsec connection to pfsense01 (location 1)
    LAN: 192.168.31.253/24

    The result is exactly the same. i could reach the webinterfaces from pfsense01 and pfsense02 from both sides but i am not able to access the network behind the pfsense routers.

    any idea?



  • First run a packet capture on the LAN interface of the pfsense02 to see if the packets make it through, and check the firewall logs for possible blocks. Post your IPsec settings and firewall rules and we can take a look



  • Hi,

    Attached some screenshots of the config and statusses.

    Packet capture from pfsense02 (while ping running from desktop behind pfsense01(10.9.8.50) to AP behind pfsense02(192.168.31.252) )
    Interface: IPSEC:

    
    09:51:15.177623 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:15.177642 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:15.177651 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:15.217442 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 464
    09:51:15.217501 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:15.217992 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 1391
    09:51:15.218049 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 1391
    09:51:15.218095 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 1391
    09:51:15.218139 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 1391
    09:51:15.218182 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 1391
    09:51:15.218225 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 281
    09:51:15.231405 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:15.231423 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:15.231431 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:15.348134 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:15.823748 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:16.323778 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54794: tcp 0
    09:51:16.538989 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:16.964200 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54825: tcp 0
    09:51:17.823769 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:18.315154 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50.54842 > 192.168.31.253.443: tcp 0
    09:51:18.315209 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:18.315274 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:18.655405 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:18.719075 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54831: tcp 0
    09:51:19.157498 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:19.923761 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:20.072621 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50 > 192.168.31.252: ICMP echo request, id 1, seq 471, length 40
    09:51:20.123754 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:21.310036 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:23.823757 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:23.923800 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:25.086094 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50 > 192.168.31.252: ICMP echo request, id 1, seq 472, length 40
    09:51:25.323765 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54781: tcp 0
    09:51:26.019737 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54784: tcp 0
    09:51:26.383605 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54828: tcp 0
    09:51:26.924188 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54772: tcp 0
    09:51:27.864830 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54842: tcp 0
    09:51:29.889065 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54824: tcp 0
    09:51:30.086799 (authentic,confidential): SPI 0xc6046aeb: IP 10.9.8.50 > 192.168.31.252: ICMP echo request, id 1, seq 473, length 40
    09:51:30.624938 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54782: tcp 0
    09:51:31.161659 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54841: tcp 0
    09:51:32.542764 (authentic,confidential): SPI 0xc205fae6: IP 192.168.31.253.443 > 10.9.8.50.54827: tcp 0
    
    

    Interface LAN:

    
    09:59:21.123793 IP6 fe80::1:1 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
    09:59:21.818791 STP 802.1w, Rapid STP, Flags [Forward], bridge-id 8000.78:48:59:84:45:8e.8018, length 47
    09:59:23.818086 STP 802.1w, Rapid STP, Flags [Forward], bridge-id 8000.78:48:59:84:45:8e.8018, length 47
    09:59:24.191495 IP6 fe80::1:1 > ff02::1: ICMP6, router advertisement, length 56
    09:59:25.079800 IP 10.9.8.50 > 192.168.31.252: ICMP echo request, id 1, seq 573, length 40
    09:59:25.817932 STP 802.1w, Rapid STP, Flags [Forward], bridge-id 8000.78:48:59:84:45:8e.8018, length 47
    09:59:27.818044 STP 802.1w, Rapid STP, Flags [Forward], bridge-id 8000.78:48:59:84:45:8e.8018, length 47
    09:59:29.031582 ARP, Request who-has 172.16.55.1 tell 172.16.55.10, length 46
    09:59:29.426520 IP6 fe80::1:1 > ff02::1: ICMP6, router advertisement, length 56
    09:59:29.523812 IP6 fe80::1:1 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
    09:59:29.817964 STP 802.1w, Rapid STP, Flags [Forward], bridge-id 8000.78:48:59:84:45:8e.8018, length 47
    09:59:30.089640 IP 10.9.8.50 > 192.168.31.252: ICMP echo request, id 1, seq 574, length 40
    09:59:31.323769 IP6 fe80::1:1 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
    
    

    As far as i can see the packets arrive but the response will not arrive back.

    When i ping 192.168.31.252 from the lan behind pfsense02 i get an reply.

    I hope you can help me solving the problems..

    screenshots.zip



  • The packets are seen on the LAN interface, so indeed they are most likely reaching the AP just fine. Is pfsense02 the default gateway of the AP? Isn't there a setting on the AP to block connections from a different subnet?



  • Thnx for the tip.

    For now I can ping to both sides (network devices like AP's).

    i am still not able to ping windows hosts on the remote side. it looks like this problem:
    https://superuser.com/questions/1087392/windows-firewall-blocking-ssh-to-secondary-subnet?noredirect=1&lq=1

    The windows firewall is disabled. To be sure i've added the any to any rule but without success.

    The ping is arriving on the remote side (LAN interface) but i think windows is not responding because the traffic comes from an different subnet.

    isn't it easier to translate the traffic to the local subnet on both sides?