Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Conenction to linux box dies over ipsec

    Scheduled Pinned Locked Moved General pfSense Questions
    10 Posts 2 Posters 685 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J
      jca1981
      last edited by

      set up a new department with new pfsense and at first it seemed all is working.
      first i give will you a overview of our net structure, we have a server net 10.0.0.0/16 with a pfsense firewall and from there all departments are connected via internet over ipsec.
      the new department is 10.5.0.0/16
      all seamed to work fine at first until i set up my a unifi ap, i could not talk to the unifi controller on the server net, it gives timeout errors, i can ping unifi from the ap and ping the ap from the controller.
      ive set any any rules on ipsec interface, seems nothing is blocked.
      next thing i tried to ssh from a server 10.0.0.204 into a linux box 10.5.1.30 and ecery time i connect via ssh and run a command, eg ls the connection dies.
      i have included wireshark dumps from both the 1.5.0.0 pfsense box and 10.0.0.0 pfsense box, where i try to ssh and run ls and it dies. i can see tcp retransmission errors in the logs.

      hope someone will help, there is some free beer to the one who solves it :)

      frompfsense10.0.0.0.pcap
      from10.5.0.0.pcap

      1 Reply Last reply Reply Quote 0
      • J
        jca1981
        last edited by

        could it be a MTU issue, or maybe MTU over ipsec?

        1 Reply Last reply Reply Quote 0
        • JKnottJ
          JKnott
          last edited by

          @jca1981:

          could it be a MTU issue, or maybe MTU over ipsec?

          IP is designed to tolerate different MTUs, especially with TCP, as would be used with SSH.  When a TCP connection is set up, the MTU is depended on the smallest MSS at either end.  Then, if a packet is still too small, it will be fragmented on IPv4.

          PfSense running on Qotom mini PC
          i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
          UniFi AC-Lite access point

          I haven't lost my mind. It's around here...somewhere...

          1 Reply Last reply Reply Quote 0
          • J
            jca1981
            last edited by

            soo, you are saying that it most likley is not MTU.
            any idea what i could try to resolve it?

            1 Reply Last reply Reply Quote 0
            • JKnottJ
              JKnott
              last edited by

              One very useful technique is to examine the traffic, to see what's happening.  This can be done the the pfSense packet capture or Wireshark.  Once you understand what the problem is and where, you can fix it.

              PfSense running on Qotom mini PC
              i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
              UniFi AC-Lite access point

              I haven't lost my mind. It's around here...somewhere...

              1 Reply Last reply Reply Quote 0
              • J
                jca1981
                last edited by

                yea, i have included downloaderble wireshark pcap in this thread, would you mind looking at them to see if you can figure out what is happening, ive tried and like i wrote i found retransmision and malformed packet, but i am not a wireshark master :)

                1 Reply Last reply Reply Quote 0
                • JKnottJ
                  JKnott
                  last edited by

                  In the from 10.5.0.0 I see some 1512 byte packets, with a 1448 packet size.  What is the MTU on IPSec?  You can try pinging with different size packets, to see where it fails.  Use the -s option for this.  Also, I see the do not fragment flag is set, which means path MTU discovery is used, instead of fragmenting.  This should generate some "too big" ICMP messages, but I don't see those.  Have they been filtered out?

                  PfSense running on Qotom mini PC
                  i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
                  UniFi AC-Lite access point

                  I haven't lost my mind. It's around here...somewhere...

                  1 Reply Last reply Reply Quote 0
                  • J
                    jca1981
                    last edited by

                    im have not set mtu on ipsec so i guess it is using 1400, to set mtu on ipsec mss clamping needs to be enabled, right?
                    biggest MTU i can get now with ping -s is 1410.
                    ive now tried enabling "do not fragment" and "firewall scrub" under system/advanced/firewall & nat, not sure if there is a specific place only for ipsec.

                    1 Reply Last reply Reply Quote 0
                    • J
                      jca1981
                      last edited by

                      ok, so tried setting mss clamping to 1300 and now it works, thanks, triet upping it to 1400 and it stopped working, found 1380 to be working after trying som more,

                      1 Reply Last reply Reply Quote 0
                      • JKnottJ
                        JKnott
                        last edited by

                        Linux normally uses PMTUD to set packet size.  Do you see the ICMP "too big" messages?  I'm not sure about IPSec settings, as I haven't used IPSec with pfSense.  The MSS is normally used when setting up a TCP connection to tell the other end the maximum supported packet size.  It has nothing to do with any router, including pfSense.  It is PMTUD that's used to determine the maximum packet size that will fit the smallest MTU along the path.

                        PfSense running on Qotom mini PC
                        i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
                        UniFi AC-Lite access point

                        I haven't lost my mind. It's around here...somewhere...

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.