Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    VPN - Routing Issue - Only Linux Hosts

    Scheduled Pinned Locked Moved General pfSense Questions
    40 Posts 9 Posters 6.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      DungaBee
      last edited by

      Only my side is pfSense.  The other side is a Cisco ASA.

      My end is 2.1.5.

      I do not know much about the ASA other than I told the corporate firewall guys that I didn't want one  :)

      To me, it seems the issue has to be on my end because the windows hosts (and my iPhone) operate just fine through the tunnel.

      Also, just to mention it again, the FIRST time I ping a host on the other end of the tunnel from the Linux laptop, I get ONE reply back and then all others fail.

      All following communications to that same host on the other side fail.  If I try another host on the other end of the tunnel from the Linux machine, I will again get a reply on the FIRST ping.  All other pings fail and all other attempts to communicate with that host fail, until I reboot the linux machine.

      Thanks again for your help in figuring out this mystery.

      1 Reply Last reply Reply Quote 0
      • D
        DungaBee
        last edited by

        While reading another thread, I noticed a suggestion to use packet capture.  I had forgotten about that being in pfSense so I did that today.

        I pinged a host and captured the following.  You can see that one good ping reply followed by nothing.  But, I am not sure how to really interpret these results so I am hoping someone on here can help in that regard.

        Thank you again.

        12:34:15.423806 IP 172.26.10.153 > 172.25.10.11: ICMP echo request, id 3515, seq 1, length 64
12:34:15.424004 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36
12:34:15.448867 IP 172.25.10.11 > 172.26.10.153: ICMP echo reply, id 3515, seq 1, length 64
12:34:16.425303 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:17.424494 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:18.424525 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:19.424416 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:20.424455 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:20.432494 ARP, Request who-has 172.26.10.254 tell 172.26.10.153, length 46
12:34:20.432512 ARP, Reply 172.26.10.254 is-at 00:10:18:03:75:7f, length 28
12:34:21.424495 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:22.424698 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:23.424586 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:24.424355 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
        1 Reply Last reply Reply Quote 0
        • johnpozJ
          johnpoz LAYER 8 Global Moderator
          last edited by

          Why would you arp for something that is not on your network?

          12:34:16.425303 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
          12:34:17.424494 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
          12:34:18.424525 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46

          Your arping for 25.10.11 from 26.10.253

          looks like 10.253 redirect your icmp request, and it sent you back a reply.. but clearly this seems to be different network because your not getting arp back.

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.8, 24.11

          1 Reply Last reply Reply Quote 0
          • D
            DungaBee
            last edited by

            172.26.10.253 is my pfSense firewall.

            172.26.10.153 is the linux machine that gets 1 ping reply and then none after that.

            172.26.0.0\16 is my local LAN

            172.25.0.0/16 is the other side of the tunnel.

            I know that didn't exactly solve the issue, but does that help in your figuring out why traffic is not being routed?

            Thank you.

            1 Reply Last reply Reply Quote 0
            • D
              DungaBee
              last edited by

              Wait a minute…..

              172.26.10.253 is my wireless router.

              .254 is pfSense.

              It would see that the wireless router (being used as just an access point) is somehow trying to do more than just drop the wireless clients on to the LAN.

              Could it being trying to find the route itself for some reason?

              1 Reply Last reply Reply Quote 0
              • DerelictD
                Derelict LAYER 8 Netgate
                last edited by

                Unplug it, get everything else working, then add it back properly configured.  I'm starting to smell a duplicate IP address somewhere.

                Chattanooga, Tennessee, USA
                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Some of this traffic is going over wifi?
                  That packet capture was on the pfSense LAN interface I assume?
                  Are you using static IPs or DHCP? Check the DHCP leases are coming from pfSense if you are.

                  .253 is not actually shown. I think that's just a misread of .153. Your wifi access point does not appear to be involved at all.

                  Try running a similar packet capture while pinging from a Windows client for comparison.

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    What's that ICMP redirect doing?
                    It appears, to my untrained eyes, to be pfSense(172.26.10.254) telling your client(172.26.10.153) that to reach the remote host(172.25.10.11) there's a better router going directly via 172.25.10.11.  :-\

                    1 Reply Last reply Reply Quote 0
                    • D
                      DungaBee
                      last edited by

                      Here is a ping from my laptop (172.26.10.50) to a host across the VPN (172.25.10.11)

                      DHCP is in use, but I am certain only pfSense is giving out addresses.  I reviewed the wireless router setup numerous times and it looks good in that regard:

                      Good Ping from Windows

                      14:41:21.359361 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 417, length 40
14:41:21.359526 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36
14:41:21.384430 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 417, length 40
14:41:22.359116 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 418, length 40
14:41:22.359274 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36
14:41:22.383116 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 418, length 40
114:41:23.364131 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 419, length 40
14:41:23.364276 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36
14:41:23.388422 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 419, length 40

                      Failed Ping to Same hose from Linux machine (172.26.10.153)

                      14:43:50.070739 IP 172.26.10.153 > 172.25.10.11: ICMP echo request, id 2305, seq 1, length 64
14:43:50.070924 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36
14:43:50.099853 IP 172.25.10.11 > 172.26.10.153: ICMP echo reply, id 2305, seq 1, length 64
14:43:51.072299 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
14:43:52.070287 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
14:43:53.070345 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
14:43:54.088953 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
14:43:55.086226 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
14:43:56.086409 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
                      1 Reply Last reply Reply Quote 0
                      • johnpozJ
                        johnpoz LAYER 8 Global Moderator
                        last edited by

                        And again your ARPing for a IP that is NOT on your network!!!

                        14:43:51.072299 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46

                        You get a redirect from 10.254 ???  Who is that?  You say you pfsense is .253
                        14:43:50.070924 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36

                        And now your client at 26.10.153 is arping for that IP vs sending it out to its gateway.  No shit its never going to get an answer to that.

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        1 Reply Last reply Reply Quote 0
                        • D
                          DungaBee
                          last edited by

                          172.26.10.254 is pfSense.

                          I misspoke when I said it was .253 earlier, my fault.

                          So, to be clear.

                          | pfSense | 172.26.10.254 |
                          | Windows Machine | 172.26.10.50 |
                          | Linux Machine | 172.26.10.153 |
                          | Host on other end of Tunnel | 172.25.10.11 |

                          So, the initial redirect by pfSense seems to be correct, but then what would trigger the ARPing?

                          I am not even sure the function of that, so I am pretty lost  :)

                          Thanks again for your help!

                          1 Reply Last reply Reply Quote 0
                          • johnpozJ
                            johnpoz LAYER 8 Global Moderator
                            last edited by

                            Why is doing a redirect? A redirect normally can happen when there a better route..

                            "The interface on which the packet comes into the router is the same interface on which the packet gets routed out."
                            "The subnet or network of the source IP address is on the same subnet or network of the next-hop IP address of the routed packet."

                            This is when cisco routers would send a redirect.

                            Do you have some issues with your masks on your interfaces..  How exactly do you have this site to site setup, are you not using a transient network?

                            I ping a vpn client from a box on my lan and this is what a capture looks like on the pfsense lan

                            15:17:15.135118 IP 192.168.1.100 > 10.0.200.6: ICMP echo request, id 1, seq 1, length 40
                            15:17:15.333586 IP 10.0.200.6 > 192.168.1.100: ICMP echo reply, id 1, seq 1, length 40
                            15:17:16.142803 IP 192.168.1.100 > 10.0.200.6: ICMP echo request, id 1, seq 2, length 40
                            15:17:16.320914 IP 10.0.200.6 > 192.168.1.100: ICMP echo reply, id 1, seq 2, length 40

                            You don't know what a arp is?

                            You could turn off redirects I would think  net.inet.ip.redirect set to 0

                            What does the traceroute look like?

                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                            If you get confused: Listen to the Music Play
                            Please don't Chat/PM me for help, unless mod related
                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              I suspect pfSense is sending the redirect all the time but Windows and IOS are ignoring it.
                              Disabling redirects in pfSense should at least prove this but why is it sending them at all? I assume it must be some misconfiguration in the VPN setup.

                              Steve

                              1 Reply Last reply Reply Quote 0
                              • D
                                DungaBee
                                last edited by

                                Turning OFF redirects in the "System Tunables" worked!!

                                net.inet.ip.redirect set to 0

                                But, do you think there is a setup issue in the VPN that is really the culprit?

                                I'd like to fix the root cause and learn from this, if possible.

                                Thanks again and let me know what you think.

                                1 Reply Last reply Reply Quote 0
                                • johnpozJ
                                  johnpoz LAYER 8 Global Moderator
                                  last edited by

                                  We don't have anything of worth to work with here, other than saying he has a vpn connection to this other network.  We don't have routing table off the pfsense box, etc.

                                  Makes no sense that pfsense would send a redirect when it should be routing the traffic down the tunnel.  Is the mask wrong on the network in pfsense?  And it thinks that network is local?

                                  Really needs some more details on how pfsense vpn is setup, off what interface?  Routing table off pfsense would help for sure.

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                                  1 Reply Last reply Reply Quote 0
                                  • DerelictD
                                    Derelict LAYER 8 Netgate
                                    last edited by

                                    As would a diagram properly documented with network and interface addresses.

                                    Chattanooga, Tennessee, USA
                                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      DungaBee
                                      last edited by

                                      Thanks guys.

                                      I'm heading out of office for the day but will post a diagram with details tomorrow and you can tell me what else to add to help figure it out.

                                      Hopefully as I document it, perhaps something will jump out.

                                      For now at least it works, even if I've just sort of put a band-aid on it.

                                      Thanks again and talk to you tomorrow!

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Yes, seeing how your vpn interface is configured should be revealing.
                                        One thing that seems like it can cause this is having both subnets on the same interface. I'm struggling to see how that might apply here though.

                                        Steve

                                        1 Reply Last reply Reply Quote 0
                                        • P
                                          phil.davis
                                          last edited by

                                          From a quick scan of this thread, I would guess that the netmask/CIDR on pfSense has been set (accidentally) to cover both the 25 and 26 networks - 172.26.10.254/15 (or smaller) would cover all that and cause pfSense to think that 172.25.n.n is on its LAN and thus send a redirect message back to the client.

                                          As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                          If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                          1 Reply Last reply Reply Quote 0
                                          • johnpozJ
                                            johnpoz LAYER 8 Global Moderator
                                            last edited by

                                            ^ Agreed, I never understand how people come in here asking without some diagram.. I can not believe a company that has multiple locations and a site to site vpn do not have a network drawing??

                                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                                            If you get confused: Listen to the Music Play
                                            Please don't Chat/PM me for help, unless mod related
                                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.