Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    I can ping both directions but only access servers one way...?

    Scheduled Pinned Locked Moved IPsec
    18 Posts 2 Posters 1.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G
      Gblenn
      last edited by

      I have three sites, two of which are running pfsense (2.7.2 and 24.03 respectively) and one of them is a Sophos XG firewall.

      Subnets are
      Site A : 172.168.1.0/24 Sophos XG v20 (tunnel subnets 10.10.1.3 and 10.10.2.3)
      Site B : 192.168.4.0/24 pfsense 2.7.2 (tunnel subnet 10.10.1.2/30)
      Site C : 192.168.1.0/24 pfsense 24.03 (tunnel subnet 10.10.2.2/30)

      I managed to set up a Routed (VTI) tunnel between sites A and B without too much hassle. I then copied the setup to use for A to C tunnel on the Sophos machine. Only changing to site C FQDN and adjusting the tunnel IP for the interface that was automatically created (somewhat similar to VTI on pfsense). Firewall rules were also amended with the subnet of site C.

      I then walked through all the steps to replicate the pfsense setup from site B onto site C. Only difference being the tunnel subnet.

      Site A to B is fully working but site A to C does not. I can ping from either side to any device on either of the subnets. I can access the Sophos fw and other servers from site C but not the other way around...
      Meaning ICMP goes across in both directions and I can see that in pcap on site C, both directions. A trace shows the exact same and correct information regardless of which one I try A to B / C or B / C to A.

      Running iperf server and testing from a client at site C gives strange results in that the very first interval shows a very slow speed (50-150 Kbps) and all following intervals are 0 = 0 bytes and 0 bits/sec.
      The other way around gives me nothing... meaning that iperf just times out.

      I have gone through my settings, as well as deleting and recreating the setup so many times and just can't find anything wrong, anywhere...

      Anyone who has succeded in setting up a tunnel between pfsense and sophos xg who can guide me? Any ideas how to move on?

      V 1 Reply Last reply Reply Quote 0
      • V
        viragomann @Gblenn
        last edited by

        @Gblenn
        Possibly the packet size exceeds the limit?
        If that you can try System > Advanced > Firewall & NAT > Enable Maximum MSS.

        I had such an issue between pfSense and OPNsense, even there were already other IPSec connections successfully running on pfSense.

        You can try out the possible packet size by playing with the data bytes option in ping.

        G 1 Reply Last reply Reply Quote 0
        • G
          Gblenn @viragomann
          last edited by

          @viragomann Thanks for the tip, turned it on set to default but no change. What do you mean playing with packet size in ping? What will that give me?
          If I try ping -s 1400 I get Frag needed and DF set (mtu = 1400)
          If I do -s 1500 I just get the ping times, no additional info.

          V 1 Reply Last reply Reply Quote 0
          • V
            viragomann @Gblenn
            last edited by

            @Gblenn
            For testing you can try a low value like 1100.If this doesn't work, there might be another reason for the problem.

            G 1 Reply Last reply Reply Quote 0
            • G
              Gblenn @viragomann
              last edited by Gblenn

              @viragomann I tried various settings but no difference that I can see...

              I did some PCAP when trying to access Site C from Site A and this is what it looks like in pfsense.
              The problem is the same regardless of which server I'm trying to acces, be it pfsense itself or in this case a switch. I just chose it for simplicity and cleanliness of the pcap.

              2a907b40-9a3c-4995-986b-d4d692166a0b-image.png

              And the downloaded file from within Wireshark

              108daa83-9ca0-40a5-ac50-0eacace78c52-image.png

              If i google pfsense blocking SYN on port 80/443 I find an old redmine issue that clearly has been resolved since way back? So it must be something different, or? Why do I not see this in the 2.7.2 setup?

              I should add that in the firewall log I see a PASS on the IPSec interface of 10.10.2.3:PORT to 10.10.177.5:80
              31e288be-acdb-4aef-8388-972b4b5b195a-image.png

              V 1 Reply Last reply Reply Quote 0
              • V
                viragomann @Gblenn
                last edited by

                @Gblenn
                Are the routes set properly at C?

                G 1 Reply Last reply Reply Quote 0
                • G
                  Gblenn @viragomann
                  last edited by

                  @viragomann Yes, site C being the pfsense side, the only thing to do there is to set the static route 172.168.1.0/24 to the IPSecVTI Gateway that was automagically created.

                  32c84c45-f5fb-44aa-acc1-84269e7a8809-image.png

                  On Sophos it looks pretty similar and it's been quite obvious when setting this up a few times that whenever I forgot this part, I don't get near where I am now...

                  G 1 Reply Last reply Reply Quote 0
                  • G
                    Gblenn @Gblenn
                    last edited by

                    I have been trying to go through the logs in Sophos as well and everything I can find on that side shows green as in all packets are passed by the correct fw and nat rules for this.

                    G 1 Reply Last reply Reply Quote 0
                    • G
                      Gblenn @Gblenn
                      last edited by

                      When trying to log in via SSH on a Linux machine on Site C , I get the following...
                      Simlar to before but it looks like there is a response initally from 192.168.1.31?

                      cadb9ec4-5d8f-45ea-a4be-6f9971facb9a-image.png

                      V 1 Reply Last reply Reply Quote 0
                      • V
                        viragomann @Gblenn
                        last edited by viragomann

                        @Gblenn
                        So even if the destination machine was sending an ACK, the SYN packet is re-transmitted.

                        I'd suspect, that the ACK packet does not arrive at the machine, which initiated the connection. But why?

                        BTW: I noticed, that the LAN at A is a public network range.
                        However, this shouldn't matter if the routes are added properly.

                        G 1 Reply Last reply Reply Quote 0
                        • G
                          Gblenn @viragomann
                          last edited by Gblenn

                          @viragomann said in I can ping both directions but only access servers one way...?:

                          @Gblenn
                          So even if the destination machine was sending an ACK, the SYN packet is re-transmitted.

                          It appears that way doesn't it... however having run the same test again several times I don't see the ACK coming back. So perhaps it was an anomaly?

                          I'd suspect, that the ACK packet does not arrive at the machine, which initiated the connection. But why?

                          BTW: I noticed, that the LAN at A is a public network range.
                          However, this shouldn't matter if the routes are added properly.

                          Ah you are right, it is in the public IP range... I made the simplest possible change from 192 to 172 and I should have changed to 16 as well. But I don't suppose pfsense would care? And it's working towards site B anyway...

                          I started thinking there was some sync issue since site A and C are connected via a switch. Site A is for testing purposes at the moment. But latency is below 1 ms when pinging between them. So I added a limiter in order to add a bit of latency to get it closer to what I have between the other sites, but that doesn't seem to change things.

                          G 1 Reply Last reply Reply Quote 0
                          • G
                            Gblenn @Gblenn
                            last edited by

                            I have sort of given up on getting this to work, for now at least. Ended up deploying Wireguard on a VM on the Sophos LAN and have a connection going that way instead...

                            G 1 Reply Last reply Reply Quote 0
                            • G
                              Gblenn @Gblenn
                              last edited by Gblenn

                              @viragomann
                              So this is definitely a 24.03 thing since a change to 2.7.2 solved the issue immediately... No change in the configuration on either side, just a switch to CE.
                              And I guess I have to see if the patches I just found out about for IPsec VTI will fix the issue.

                              G 1 Reply Last reply Reply Quote 0
                              • G
                                Gblenn @Gblenn
                                last edited by

                                And now it's working also on 24.03 after implementing patches for Redmine #15449, #15430 and #15606...

                                V 1 Reply Last reply Reply Quote 0
                                • V
                                  viragomann @Gblenn
                                  last edited by

                                  @Gblenn
                                  Great info. Thanks.
                                  I never had this issue, even running multiple IPSec site-to-site connections to other routers than pfSense and didn't apply IPSec patches.

                                  G 1 Reply Last reply Reply Quote 0
                                  • G
                                    Gblenn @viragomann
                                    last edited by

                                    @viragomann Well, first time I have used IPSec since Sophos doesn't do WG...

                                    One thing to note though, this is Routed VTI connection, which is different from "Tunnel IPv4" that you have as default Mode in pfsense...

                                    V 1 Reply Last reply Reply Quote 0
                                    • V
                                      viragomann @Gblenn
                                      last edited by

                                      @Gblenn
                                      Ah ya, you mentioned above.
                                      I never set up a VTI to be honest, I just have some small knowledge about from the docs.

                                      G 1 Reply Last reply Reply Quote 0
                                      • G
                                        Gblenn @viragomann
                                        last edited by

                                        @viragomann Well, since I couldn't get the default one to work, I tried VTI and it worked. Not sure what I did wrong with the other method but I did find VTI a bit more like the WG tunnels I have set up in the past. With the gateway and routing settings at least...

                                        1 Reply Last reply Reply Quote 0
                                        • G Gblenn referenced this topic on
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.