2.3.1_1 IPSEC tunnel up, but IP Traffic between subnets is not working



  • Hi,

    IPSEC stopped working for me after upgrading to 2.3   
    Since then I made a bunch of changes trying to get it to work. 
    Both Firewalls are now on V2.3.1_1
    According to this post by cmb, all is well in version 2.3.1_1  …so I can't blame the tools anymore, it's something I screwed up.

    Below is a composite image of Status-> IPSEC on my two firewalls.
    The only tunnel traffic is a few pings from a 192.168.10.x host to a 192.168.3.x host

    This is as close as I have gotten to narrowing down the issue. 
    As my arrows indicate, it looks to me like:

    • The ping starts at the 192.168.10.x host

    • The ping leaves the x.x.x.66 firewall

    • The ping reaches the x.x.x.163 firewall

    • The ping reaches the 192.168.3.x host

    • The echo leaves the x.x.x.163 firewall

    • But the echo never reaches the x.x.x.66 firewall

    I checked the firewall logs, I did not find an entry showing the ping being blocked.
    I checked the IPSEC logs, I found the following 5 lines repeating exactly every 10s

    
    May 29 17:00:28	charon		08[IKE] <con2000|144>sending DPD request
    May 29 17:00:28	charon		08[ENC] <con2000|144>generating INFORMATIONAL_V1 request 3777181309 [ HASH N(DPD) ]
    May 29 17:00:28	charon		08[NET] <con2000|144>sending packet: from x.x.x.66[500] to x.x.x.163[500] (92 bytes)
    May 29 17:00:28	charon		08[NET] <con2000|144>received packet: from x.x.x.163[500] to x.x.x.66[500] (92 bytes)
    May 29 17:00:28	charon		08[ENC] <con2000|144>parsed INFORMATIONAL_V1 request 3270230140 [ HASH N(DPD_ACK) ]
    May 29 17:00:38	charon		05[IKE] <con2000|144>sending DPD request
    May 29 17:00:38	charon		05[ENC] <con2000|144>generating INFORMATIONAL_V1 request 2760215261 [ HASH N(DPD) ]
    May 29 17:00:38	charon		05[NET] <con2000|144>sending packet: from x.x.x.66[500] to x.x.x.163[500] (92 bytes)
    May 29 17:00:38	charon		05[NET] <con2000|144>received packet: from x.x.x.163[500] to x.x.x.66[500] (92 bytes)
    May 29 17:00:38	charon		05[ENC] <con2000|144>parsed INFORMATIONAL_V1 request 3152045256 [ HASH N(DPD_ACK) ]</con2000|144></con2000|144></con2000|144></con2000|144></con2000|144></con2000|144></con2000|144></con2000|144></con2000|144></con2000|144> 
    

    I am not a pfsense expert, is that normal?
    Either way: I am out of tricks, does anyone have any suggestions what I should try/check next?



  • Norsak, sorry to bother you with another question, but here it goes: are you able to make any changes on your IPsec P1/P2 phases?

    L.E.: I'm askig because my problem started exactly like this. Then I've spoted a missing P2 phase configured, wanted to add it and the surprize: the web-gui won't save my configuration :)



  • @sebyp:

    Norsak, sorry to bother you with another question, but here it goes: are you able to make any changes on your IPsec P1/P2 phases?

    L.E.: I'm askig because my problem started exactly like this. Then I've spoted a missing P2 phase configured, wanted to add it and the surprize: the web-gui won't save my configuration :)

    Hi,

    I can make changes to my IPSEC config's, for me the GUI is working as expected.
    Changes are saved, and I get prompted to Apply Changes.  It doesn't look like I have exactly the same issues as you do.



  • OK, thanks for the heads-up.



  • I have now given up on fixing this problem in situ.
    I have installed pfsense +12x times in the last 2 weeks, building physical and virtual test environments.

    Fair or not fair, I conclude the following from my limited testing:

    • IPSEC works in 2.3.1 & 2.3.1_1
    • But upgrading from earlier versions to 2.3 can break IPSEC (and subsequent upgrades to 2.3.1_1 do not remedy the problem)

    The only difference between my Firewalls where IPSEC works and those where IPSEC does NOT work:
    All the ones that do NOT work were upgraded when 2.3 first came out; all other settings were identical.

    I am now shipping out freshly formatted 2.3.1_1 firewalls to my branch offices, it's the only way I can get IPSEC site-to-site VPN working again.

    :(



  • There is absolutely no IPsec config file diff between systems upgraded to 2.3.x and those that start on 2.3.x.

    The screenshots in the original post could be a number of diff things depending on specifics. Judging by the description, probably the most likely cause is ESP from the .163 to .66 system getting dropped somewhere in between. At least that's the vast majority of support case resolutions with symptoms matching what's shown and described there, and the best first thing to confirm or deny. More than likely the .163 system is sending that traffic since its byte counters are incrementing, and since they are 0 on the opposite side, not likely getting back to that end.



  • Probably not the problem, but thought I would mention the following.  I'm pretty sure this was fixed in 2.3.1, but some people that upgraded to 2.3 from 2.2 with ipsec configs, had an issue where more than one instance of ipsec/strongswan/charon was running.  I had the problem.  2.3.1 seems to have fixed it, but for fun you may want to take a peek at the process list and see if you happen to have more than one instance of the ipsec daemon running.

    Thread with more details here –> https://forum.pfsense.org/index.php?topic=109908.0



  • @moterpent:

    Probably not the problem, but thought I would mention the following.  I'm pretty sure this was fixed in 2.3.1, but some people that upgraded to 2.3 from 2.2 with ipsec configs, had an issue where more than one instance of ipsec/strongswan/charon was running.  I had the problem.

    That's definitely fixed in 2.3.1 and newer. For those who hit that, it impacted new configs exactly the same as upgraded ones.