PfSense (Proxmox) to Fortigate IPSEC tunnel fragmentation problem 2.4.3_x



  • Hello forum,

    I have been analyzing for a week a wierd behaviour with an IPSEC tunnel. Similar to the one described in this post: https://forum.pfsense.org/index.php?topic=87610.0.

    Current configuration and settings overview:

    Established tunnel with a declared mtu in the remote side of 1438 bytes. Our enc0 (IPSEC interface) mtu in pfSense is 1535 bytes and we can not modify it.

    A 64byte ping flows flowlessly through the tunnel. ALWAYS, any time. From pfSense itself and from any server behind pfsense that go through the tunnel.

    A 1477byte ping from a server behind pfsense flows during certain time and then stops flowing. Capturing the traffic in pfSense, we can see the response from the other side is getting to us but now it comes fragmented. When the ping flows correclty, the capture do not show any fragmentation.
    When fragmentation appears the ping won't flow and will stay like this during minutes.
    The same ping from pfSense flows all the time, with no disruption.

    When the IPSec tunnel renegotiation take place, the ping starts to flow again in the server behind pfSense, and in the packet capture there is no fragmentation at all again. The fragmented packets come from the other endpoint, so… it seems that somehow the other endpoint "thought" that in "some moment" it should start fragmenting the same packets it was sending unfragmented seconds ago... After some minutes, it starts working again, and so on.
    The ping from pfSense works all the time (with fragmented packets and with unfragmented packets). So.. I should think that the tunnel is correctly established and it manages ok the fragmentation.

    The first thoughts were it was a problem with mtu… but when the traffic (ICMP ping bigger than 1476bytes) from pfSense works flawlessly (all the time!) make us think that somehow pfSense has some problems routing?/ressemble? correctly those packets to the servers behind him but not for himself.

    Mainly our doubts are (from a server behind pfSense point of view):

    • Why the other end starts fragmenting packets it wasn't seconds ago? (always a ping with a fixed size)

    • Why the packets bigger than a certaing length go through the tunnel flawlessly during certaing time and then stop working in the servers behind pfSense and the same packets from pfSense itself still flowing?

    • Why when IPSec renegotiation routines take places the servers behind pfSense's pings start to flow again when all the time pfSense has been able to deliver that kind of packets?

    • Has anyone faced something like this against fortiOS with their latests updates?

    Any hint appreciated.

    Thanks in advanced!