VPN - Routing Issue - Only Linux Hosts
-
Something that can impact Linux (including Android) but not Windows is partial IPv6 connectivity. Linux can attempt to use IPv6 if it appears to be available even if no external route is possible.
Steve
-
I do not seem to have IPv6 activated anyplace but can you tell me where I should look, just so I can confirm? Or, is there some option I need to select to handle IPv6 requests?
Thanks!
-
Dead Peer Detection is active on the IPSEC setup.
Would that cause an issue somehow?
Keep in mind all Windows hosts on my end of the tunnel can access everything on the other side without issue, if that matters.
Sorry, my fault - I somehow assumed that the Linux machines used one IPsec tinnel and the Windows boxes an other one. Had i read your initial post correctly, I would have noted that all machines use the same tunnel.
DPD can, in some cases, cause the tunnel to disconnect for no apparent reason. Obviously, with the tunnel completly going down, all machines would be affacted.
What does
sudo route -n netstat ip route listshow on a Linux machine? (That are three separate commands)
The Windows version is
route print -
Thank you for the follow up. Here is the info. I omitted all the misc connection info from netstat as I assumed that was not relevant.
Windows Machine
route print IPv4 Route Table =========================================================================== Active Routes: Network Destination Netmask Gateway Interface Metric 0.0.0.0 0.0.0.0 172.26.10.254 172.26.10.50 20 127.0.0.0 255.0.0.0 On-link 127.0.0.1 306 127.0.0.1 255.255.255.255 On-link 127.0.0.1 306 127.255.255.255 255.255.255.255 On-link 127.0.0.1 306 172.26.0.0 255.255.0.0 On-link 172.26.10.50 276 172.26.10.50 255.255.255.255 On-link 172.26.10.50 276 172.26.255.255 255.255.255.255 On-link 172.26.10.50 276 224.0.0.0 240.0.0.0 On-link 127.0.0.1 306 224.0.0.0 240.0.0.0 On-link 192.168.56.1 276 224.0.0.0 240.0.0.0 On-link 172.26.10.50 276 255.255.255.255 255.255.255.255 On-link 127.0.0.1 306 255.255.255.255 255.255.255.255 On-link 192.168.56.1 276 255.255.255.255 255.255.255.255 On-link 172.26.10.50 276 =========================================================================== Persistent Routes: NoneLinux Machine
sudo route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 172.26.10.254 0.0.0.0 UG 0 0 0 wlan0 172.26.0.0 0.0.0.0 255.255.0.0 U 9 0 0 wlan0 netstat Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp6 0 0 ip6-localhost:45710 ip6-localhost:ipp ESTABLISHED tcp6 0 0 ip6-localhost:ipp ip6-localhost:45710 ESTABLISHED tcp6 1 0 ip6-localhost:45708 ip6-localhost:ipp CLOSE_WAIT ip route list default via 172.26.10.254 dev wlan0 proto static 172.26.0.0/16 dev wlan0 proto kernel scope link src 172.26.10.152 metric 9 -
I doubt this is applicable here but just in case. In this thread, example, the issue turned out to be an interface that had it's IPv6 type set to 'track interface' instead of 'none'. I guess you could check the VPN interface for something similar.
Steve
-
Unfortunately that did not help. My IPv6 configuration was already set to "None". I changed it and then changed it back, but no luck.
-
Pinging IPv4 addresses directly shouldn't involve IPv6 at all.
Are both sides pfSense?
What version?
What's on the IPsec tab of the firewall rules at both ends?
-
Only my side is pfSense. The other side is a Cisco ASA.
My end is 2.1.5.
I do not know much about the ASA other than I told the corporate firewall guys that I didn't want one :)
To me, it seems the issue has to be on my end because the windows hosts (and my iPhone) operate just fine through the tunnel.
Also, just to mention it again, the FIRST time I ping a host on the other end of the tunnel from the Linux laptop, I get ONE reply back and then all others fail.
All following communications to that same host on the other side fail. If I try another host on the other end of the tunnel from the Linux machine, I will again get a reply on the FIRST ping. All other pings fail and all other attempts to communicate with that host fail, until I reboot the linux machine.
Thanks again for your help in figuring out this mystery.
-
While reading another thread, I noticed a suggestion to use packet capture. I had forgotten about that being in pfSense so I did that today.
I pinged a host and captured the following. You can see that one good ping reply followed by nothing. But, I am not sure how to really interpret these results so I am hoping someone on here can help in that regard.
Thank you again.
12:34:15.423806 IP 172.26.10.153 > 172.25.10.11: ICMP echo request, id 3515, seq 1, length 64 12:34:15.424004 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 12:34:15.448867 IP 172.25.10.11 > 172.26.10.153: ICMP echo reply, id 3515, seq 1, length 64 12:34:16.425303 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:17.424494 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:18.424525 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:19.424416 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:20.424455 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:20.432494 ARP, Request who-has 172.26.10.254 tell 172.26.10.153, length 46 12:34:20.432512 ARP, Reply 172.26.10.254 is-at 00:10:18:03:75:7f, length 28 12:34:21.424495 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:22.424698 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:23.424586 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 12:34:24.424355 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 -
Why would you arp for something that is not on your network?
12:34:16.425303 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:17.424494 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
12:34:18.424525 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46Your arping for 25.10.11 from 26.10.253
looks like 10.253 redirect your icmp request, and it sent you back a reply.. but clearly this seems to be different network because your not getting arp back.
-
172.26.10.253 is my pfSense firewall.
172.26.10.153 is the linux machine that gets 1 ping reply and then none after that.
172.26.0.0\16 is my local LAN
172.25.0.0/16 is the other side of the tunnel.
I know that didn't exactly solve the issue, but does that help in your figuring out why traffic is not being routed?
Thank you.
-
Wait a minute…..
172.26.10.253 is my wireless router.
.254 is pfSense.
It would see that the wireless router (being used as just an access point) is somehow trying to do more than just drop the wireless clients on to the LAN.
Could it being trying to find the route itself for some reason?
-
Unplug it, get everything else working, then add it back properly configured. I'm starting to smell a duplicate IP address somewhere.
-
Some of this traffic is going over wifi?
That packet capture was on the pfSense LAN interface I assume?
Are you using static IPs or DHCP? Check the DHCP leases are coming from pfSense if you are..253 is not actually shown. I think that's just a misread of .153. Your wifi access point does not appear to be involved at all.
Try running a similar packet capture while pinging from a Windows client for comparison.
Steve
-
What's that ICMP redirect doing?
It appears, to my untrained eyes, to be pfSense(172.26.10.254) telling your client(172.26.10.153) that to reach the remote host(172.25.10.11) there's a better router going directly via 172.25.10.11. :-\ -
Here is a ping from my laptop (172.26.10.50) to a host across the VPN (172.25.10.11)
DHCP is in use, but I am certain only pfSense is giving out addresses. I reviewed the wireless router setup numerous times and it looks good in that regard:
Good Ping from Windows
14:41:21.359361 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 417, length 40 14:41:21.359526 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:41:21.384430 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 417, length 40 14:41:22.359116 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 418, length 40 14:41:22.359274 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:41:22.383116 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 418, length 40 114:41:23.364131 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 419, length 40 14:41:23.364276 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:41:23.388422 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 419, length 40Failed Ping to Same hose from Linux machine (172.26.10.153)
14:43:50.070739 IP 172.26.10.153 > 172.25.10.11: ICMP echo request, id 2305, seq 1, length 64 14:43:50.070924 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:43:50.099853 IP 172.25.10.11 > 172.26.10.153: ICMP echo reply, id 2305, seq 1, length 64 14:43:51.072299 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:52.070287 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:53.070345 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:54.088953 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:55.086226 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:56.086409 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 -
And again your ARPing for a IP that is NOT on your network!!!
14:43:51.072299 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
You get a redirect from 10.254 ??? Who is that? You say you pfsense is .253
14:43:50.070924 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36And now your client at 26.10.153 is arping for that IP vs sending it out to its gateway. No shit its never going to get an answer to that.
-
172.26.10.254 is pfSense.
I misspoke when I said it was .253 earlier, my fault.
So, to be clear.
| pfSense | 172.26.10.254 |
| Windows Machine | 172.26.10.50 |
| Linux Machine | 172.26.10.153 |
| Host on other end of Tunnel | 172.25.10.11 |So, the initial redirect by pfSense seems to be correct, but then what would trigger the ARPing?
I am not even sure the function of that, so I am pretty lost :)
Thanks again for your help!
-
Why is doing a redirect? A redirect normally can happen when there a better route..
"The interface on which the packet comes into the router is the same interface on which the packet gets routed out."
"The subnet or network of the source IP address is on the same subnet or network of the next-hop IP address of the routed packet."This is when cisco routers would send a redirect.
Do you have some issues with your masks on your interfaces.. How exactly do you have this site to site setup, are you not using a transient network?
I ping a vpn client from a box on my lan and this is what a capture looks like on the pfsense lan
15:17:15.135118 IP 192.168.1.100 > 10.0.200.6: ICMP echo request, id 1, seq 1, length 40
15:17:15.333586 IP 10.0.200.6 > 192.168.1.100: ICMP echo reply, id 1, seq 1, length 40
15:17:16.142803 IP 192.168.1.100 > 10.0.200.6: ICMP echo request, id 1, seq 2, length 40
15:17:16.320914 IP 10.0.200.6 > 192.168.1.100: ICMP echo reply, id 1, seq 2, length 40You don't know what a arp is?
You could turn off redirects I would think net.inet.ip.redirect set to 0
What does the traceroute look like?
-
I suspect pfSense is sending the redirect all the time but Windows and IOS are ignoring it.
Disabling redirects in pfSense should at least prove this but why is it sending them at all? I assume it must be some misconfiguration in the VPN setup.Steve
