Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    VPN - Routing Issue - Only Linux Hosts

    Scheduled Pinned Locked Moved General pfSense Questions
    40 Posts 9 Posters 5.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      DungaBee
      last edited by

      Hello All,

      I have a very strange issue happening on our local work network.  We have our local network at our office and then an IPSec tunnel that connects us to our main data center where most of our gear is kept.

      Our local network routes all the internet traffic out of our office and should route only the VPN traffic through the VPN.  When using Windows machines and also my iPhone, this works fine.  But, a local Linux laptop will not "find" anything on the other side of the tunnel and neither will any Android devices.

      Our local network is:
      172.26.0.1 - 172.26.255.254

      The network on the other side of the tunnel is:
      172.25.0.1 - 172.25.255.254

      I can use/ping/traceroute any of the 172.25.. from my windows laptop without issue.

      But, when trying to do the same from the Linux laptop, it fails each time.

      I had thought my colleagues Android issues were just some oddity on his devices, but now seeing the same thing play out from a fresh Linux Mint install, it has to be some sort of routing issue, but I am not sure how to troubleshoot or solve it.

      Can anyone offer some help on this?

      Thanks much!

      1 Reply Last reply Reply Quote 0
      • D
        divsys
        last edited by

        Is it possible the problem devices have a local firewall in place, blocking the external subnet?

        -jfp

        1 Reply Last reply Reply Quote 0
        • D
          DungaBee
          last edited by

          Thank you for the reply.  I was thinking the same thing so I've been doing some sort of "throw a dart" troubleshooting to try and figure it out.

          I did check and it does not seem there is any firewall active on the linux machine.

          Here are some tests I did:

          Just as a refresher…the Linux laptop is on 172.26.x.x

          My windows laptop can connect to everything on the VPN subnet without issue.

          Windows Laptop on same subnet –> PING --> Linux Laptop = OK
          Linux Laptop --> PING --> Windows Laptop on same subnet = OK

          This seems to indicate that ping itself is OK to/from the linux machine.

          I then used a RDP session from my laptop to a server on the VPN subnet of 172.25.x.x.

          Server –> PING --> Linux Laptop = NOT OK

          Here is where it gets sort of interesting...

          If I try and PING a device on the other subnet from the Linux laptop, I will get 1 good reply reply and then that's it.  And I only get that 1 reply the first time I try and ping the device.  After that, I get no replies at all until I reboot.  After a reboot, I can again get 1 response and then that's it.  Similarly, if I try and access a web page on the VPN subnet, the browser will seem to indicate that a connection was made because it will say "waiting for 172.25.10.231" but then nothing happens.  If I ping 172.25.10.231 and get the 1 reply and then try and hot the web page, it immediately fails to load the page in the borwser.

          Also, I tried using NMAP on the laptop and it seemed to locate devices properly on the VPN subnet.  But, after trying to ping the a devices, NMAP suddenly can't see any of the ones I've pinged any longer.

          It would seem logical that the issue lies on the Linux machine, but the fact that Android devices are similarly unable to talk with anything on the other end of the VPN makes me think there is some setting on pfSense that may overcome whatever the issue is.  After all, the Android device should just be able to be on the LAN here and "see" the other stuff on the VPN without issue.

          Any added thoughts from the crowd?

          Thanks again.

          1 Reply Last reply Reply Quote 0
          • A
            aGeekhere
            last edited by

            Is ufw installed or running?

            Never Fear, A Geek is Here!

            1 Reply Last reply Reply Quote 0
            • D
              DungaBee
              last edited by

              I put the gufw package on there to check that and it was off.  I activated it and then told it to allow all and still had the same results unfortunately.

              1 Reply Last reply Reply Quote 0
              • K
                Klaws
                last edited by

                Yesm this is a dumb question, but…did you remember to disable dead peer detection?

                1 Reply Last reply Reply Quote 0
                • D
                  DungaBee
                  last edited by

                  Dead Peer Detection is active on the IPSEC setup.

                  Would that cause an issue somehow?

                  Keep in mind all Windows hosts on my end of the tunnel can access everything on the other side without issue, if that matters.

                  Thank you for your reply.  Hoping to get this figured out.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Something that can impact Linux (including Android) but not Windows is partial IPv6 connectivity. Linux can attempt to use IPv6 if it appears to be available even if no external route is possible.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • D
                      DungaBee
                      last edited by

                      I do not seem to have IPv6 activated anyplace but can you tell me where I should look, just so I can confirm?  Or, is there some option I need to select to handle IPv6 requests?

                      Thanks!

                      1 Reply Last reply Reply Quote 0
                      • K
                        Klaws
                        last edited by

                        @DungaBee:

                        Dead Peer Detection is active on the IPSEC setup.

                        Would that cause an issue somehow?

                        Keep in mind all Windows hosts on my end of the tunnel can access everything on the other side without issue, if that matters.

                        Sorry, my fault - I somehow assumed that the Linux machines used one IPsec tinnel and the Windows boxes an other one. Had i read your initial post correctly, I would have noted that all machines use the same tunnel.

                        DPD can, in some cases, cause the tunnel to disconnect for no apparent reason. Obviously, with the tunnel completly going down, all machines would be affacted.

                        What does

                        sudo route -n
                        netstat
                        ip route list
                        

                        show on a Linux machine? (That are three separate commands)

                        The Windows version is

                        route print
                        
                        1 Reply Last reply Reply Quote 0
                        • D
                          DungaBee
                          last edited by

                          Thank you for the follow up.  Here is the info.  I omitted all the misc connection info from netstat as I assumed that was not relevant.

                          Windows Machine

                          route print
                          
                          IPv4 Route Table
                          ===========================================================================
                          Active Routes:
                          Network Destination        Netmask          Gateway       Interface  Metric
                                    0.0.0.0          0.0.0.0    172.26.10.254     172.26.10.50     20
                                  127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
                                  127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
                            127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
                                 172.26.0.0      255.255.0.0         On-link      172.26.10.50    276
                               172.26.10.50  255.255.255.255         On-link      172.26.10.50    276
                             172.26.255.255  255.255.255.255         On-link      172.26.10.50    276
                                  224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
                                  224.0.0.0        240.0.0.0         On-link      192.168.56.1    276
                                  224.0.0.0        240.0.0.0         On-link      172.26.10.50    276
                            255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
                            255.255.255.255  255.255.255.255         On-link      192.168.56.1    276
                            255.255.255.255  255.255.255.255         On-link      172.26.10.50    276
                          ===========================================================================
                          Persistent Routes:
                            None
                          

                          Linux Machine

                          sudo route -n
                          Kernel IP routing table
                          Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
                          0.0.0.0         172.26.10.254   0.0.0.0         UG    0      0        0 wlan0
                          172.26.0.0      0.0.0.0         255.255.0.0     U     9      0        0 wlan0
                          
                          netstat
                          Active Internet connections (w/o servers)
                          Proto Recv-Q Send-Q Local Address           Foreign Address         State      
                          tcp6       0      0 ip6-localhost:45710     ip6-localhost:ipp       ESTABLISHED
                          tcp6       0      0 ip6-localhost:ipp       ip6-localhost:45710     ESTABLISHED
                          tcp6       1      0 ip6-localhost:45708     ip6-localhost:ipp       CLOSE_WAIT 
                          
                          ip route list
                          default via 172.26.10.254 dev wlan0  proto static 
                          172.26.0.0/16 dev wlan0  proto kernel  scope link  src 172.26.10.152  metric 9 
                          
                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            I doubt this is applicable here but just in case. In this thread, example, the issue turned out to be an interface that had it's IPv6 type set to 'track interface' instead of 'none'. I guess you could check the VPN interface for something similar.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • D
                              DungaBee
                              last edited by

                              Unfortunately that did not help.  My IPv6 configuration was already set to "None".  I changed it and then changed it back, but no luck.

                              1 Reply Last reply Reply Quote 0
                              • DerelictD
                                Derelict LAYER 8 Netgate
                                last edited by

                                Pinging IPv4 addresses directly shouldn't involve IPv6 at all.

                                Are both sides pfSense?

                                What version?

                                What's on the IPsec tab of the firewall rules at both ends?

                                Chattanooga, Tennessee, USA
                                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                1 Reply Last reply Reply Quote 0
                                • D
                                  DungaBee
                                  last edited by

                                  Only my side is pfSense.  The other side is a Cisco ASA.

                                  My end is 2.1.5.

                                  I do not know much about the ASA other than I told the corporate firewall guys that I didn't want one  :)

                                  To me, it seems the issue has to be on my end because the windows hosts (and my iPhone) operate just fine through the tunnel.

                                  Also, just to mention it again, the FIRST time I ping a host on the other end of the tunnel from the Linux laptop, I get ONE reply back and then all others fail.

                                  All following communications to that same host on the other side fail.  If I try another host on the other end of the tunnel from the Linux machine, I will again get a reply on the FIRST ping.  All other pings fail and all other attempts to communicate with that host fail, until I reboot the linux machine.

                                  Thanks again for your help in figuring out this mystery.

                                  1 Reply Last reply Reply Quote 0
                                  • D
                                    DungaBee
                                    last edited by

                                    While reading another thread, I noticed a suggestion to use packet capture.  I had forgotten about that being in pfSense so I did that today.

                                    I pinged a host and captured the following.  You can see that one good ping reply followed by nothing.  But, I am not sure how to really interpret these results so I am hoping someone on here can help in that regard.

                                    Thank you again.

                                    12:34:15.423806 IP 172.26.10.153 > 172.25.10.11: ICMP echo request, id 3515, seq 1, length 64
                                    12:34:15.424004 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36
                                    12:34:15.448867 IP 172.25.10.11 > 172.26.10.153: ICMP echo reply, id 3515, seq 1, length 64
                                    12:34:16.425303 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:17.424494 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:18.424525 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:19.424416 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:20.424455 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:20.432494 ARP, Request who-has 172.26.10.254 tell 172.26.10.153, length 46
                                    12:34:20.432512 ARP, Reply 172.26.10.254 is-at 00:10:18:03:75:7f, length 28
                                    12:34:21.424495 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:22.424698 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:23.424586 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    12:34:24.424355 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • johnpozJ
                                      johnpoz LAYER 8 Global Moderator
                                      last edited by

                                      Why would you arp for something that is not on your network?

                                      12:34:16.425303 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                      12:34:17.424494 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                                      12:34:18.424525 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46

                                      Your arping for 25.10.11 from 26.10.253

                                      looks like 10.253 redirect your icmp request, and it sent you back a reply.. but clearly this seems to be different network because your not getting arp back.

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                                      1 Reply Last reply Reply Quote 0
                                      • D
                                        DungaBee
                                        last edited by

                                        172.26.10.253 is my pfSense firewall.

                                        172.26.10.153 is the linux machine that gets 1 ping reply and then none after that.

                                        172.26.0.0\16 is my local LAN

                                        172.25.0.0/16 is the other side of the tunnel.

                                        I know that didn't exactly solve the issue, but does that help in your figuring out why traffic is not being routed?

                                        Thank you.

                                        1 Reply Last reply Reply Quote 0
                                        • D
                                          DungaBee
                                          last edited by

                                          Wait a minute…..

                                          172.26.10.253 is my wireless router.

                                          .254 is pfSense.

                                          It would see that the wireless router (being used as just an access point) is somehow trying to do more than just drop the wireless clients on to the LAN.

                                          Could it being trying to find the route itself for some reason?

                                          1 Reply Last reply Reply Quote 0
                                          • DerelictD
                                            Derelict LAYER 8 Netgate
                                            last edited by

                                            Unplug it, get everything else working, then add it back properly configured.  I'm starting to smell a duplicate IP address somewhere.

                                            Chattanooga, Tennessee, USA
                                            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.