VPN - Routing Issue - Only Linux Hosts
-
What's that ICMP redirect doing?
It appears, to my untrained eyes, to be pfSense(172.26.10.254) telling your client(172.26.10.153) that to reach the remote host(172.25.10.11) there's a better router going directly via 172.25.10.11. :-\ -
Here is a ping from my laptop (172.26.10.50) to a host across the VPN (172.25.10.11)
DHCP is in use, but I am certain only pfSense is giving out addresses. I reviewed the wireless router setup numerous times and it looks good in that regard:
Good Ping from Windows
14:41:21.359361 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 417, length 40 14:41:21.359526 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:41:21.384430 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 417, length 40 14:41:22.359116 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 418, length 40 14:41:22.359274 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:41:22.383116 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 418, length 40 114:41:23.364131 IP 172.26.10.50 > 172.25.10.11: ICMP echo request, id 1, seq 419, length 40 14:41:23.364276 IP 172.26.10.254 > 172.26.10.50: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:41:23.388422 IP 172.25.10.11 > 172.26.10.50: ICMP echo reply, id 1, seq 419, length 40
Failed Ping to Same hose from Linux machine (172.26.10.153)
14:43:50.070739 IP 172.26.10.153 > 172.25.10.11: ICMP echo request, id 2305, seq 1, length 64 14:43:50.070924 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36 14:43:50.099853 IP 172.25.10.11 > 172.26.10.153: ICMP echo reply, id 2305, seq 1, length 64 14:43:51.072299 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:52.070287 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:53.070345 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:54.088953 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:55.086226 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46 14:43:56.086409 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
-
And again your ARPing for a IP that is NOT on your network!!!
14:43:51.072299 ARP, Request who-has 172.25.10.11 tell 172.26.10.153, length 46
You get a redirect from 10.254 ??? Who is that? You say you pfsense is .253
14:43:50.070924 IP 172.26.10.254 > 172.26.10.153: ICMP redirect 172.25.10.11 to host 172.25.10.11, length 36And now your client at 26.10.153 is arping for that IP vs sending it out to its gateway. No shit its never going to get an answer to that.
-
172.26.10.254 is pfSense.
I misspoke when I said it was .253 earlier, my fault.
So, to be clear.
| pfSense | 172.26.10.254 |
| Windows Machine | 172.26.10.50 |
| Linux Machine | 172.26.10.153 |
| Host on other end of Tunnel | 172.25.10.11 |So, the initial redirect by pfSense seems to be correct, but then what would trigger the ARPing?
I am not even sure the function of that, so I am pretty lost :)
Thanks again for your help!
-
Why is doing a redirect? A redirect normally can happen when there a better route..
"The interface on which the packet comes into the router is the same interface on which the packet gets routed out."
"The subnet or network of the source IP address is on the same subnet or network of the next-hop IP address of the routed packet."This is when cisco routers would send a redirect.
Do you have some issues with your masks on your interfaces.. How exactly do you have this site to site setup, are you not using a transient network?
I ping a vpn client from a box on my lan and this is what a capture looks like on the pfsense lan
15:17:15.135118 IP 192.168.1.100 > 10.0.200.6: ICMP echo request, id 1, seq 1, length 40
15:17:15.333586 IP 10.0.200.6 > 192.168.1.100: ICMP echo reply, id 1, seq 1, length 40
15:17:16.142803 IP 192.168.1.100 > 10.0.200.6: ICMP echo request, id 1, seq 2, length 40
15:17:16.320914 IP 10.0.200.6 > 192.168.1.100: ICMP echo reply, id 1, seq 2, length 40You don't know what a arp is?
You could turn off redirects I would think net.inet.ip.redirect set to 0
What does the traceroute look like?
-
I suspect pfSense is sending the redirect all the time but Windows and IOS are ignoring it.
Disabling redirects in pfSense should at least prove this but why is it sending them at all? I assume it must be some misconfiguration in the VPN setup.Steve
-
Turning OFF redirects in the "System Tunables" worked!!
net.inet.ip.redirect set to 0
But, do you think there is a setup issue in the VPN that is really the culprit?
I'd like to fix the root cause and learn from this, if possible.
Thanks again and let me know what you think.
-
We don't have anything of worth to work with here, other than saying he has a vpn connection to this other network. We don't have routing table off the pfsense box, etc.
Makes no sense that pfsense would send a redirect when it should be routing the traffic down the tunnel. Is the mask wrong on the network in pfsense? And it thinks that network is local?
Really needs some more details on how pfsense vpn is setup, off what interface? Routing table off pfsense would help for sure.
-
As would a diagram properly documented with network and interface addresses.
-
Thanks guys.
I'm heading out of office for the day but will post a diagram with details tomorrow and you can tell me what else to add to help figure it out.
Hopefully as I document it, perhaps something will jump out.
For now at least it works, even if I've just sort of put a band-aid on it.
Thanks again and talk to you tomorrow!
-
Yes, seeing how your vpn interface is configured should be revealing.
One thing that seems like it can cause this is having both subnets on the same interface. I'm struggling to see how that might apply here though.Steve
-
From a quick scan of this thread, I would guess that the netmask/CIDR on pfSense has been set (accidentally) to cover both the 25 and 26 networks - 172.26.10.254/15 (or smaller) would cover all that and cause pfSense to think that 172.25.n.n is on its LAN and thus send a redirect message back to the client.
-
^ Agreed, I never understand how people come in here asking without some diagram.. I can not believe a company that has multiple locations and a site to site vpn do not have a network drawing??
-
you create docs/schematics ?
some of us seem to have the luxury of collegues and spare time ….i only know people who get abused by their employer todo a 5-man-job ; on their own :D
-
Yes, unfortunately it doesn't surprise me at all. And in fact i'd go further and say that very often network issues can be caused by an existing network diagram that's out of date or just plain wrong. I have always found it prudent to assume nothing. Perhaps just my own experience. ::)
Steve
-
Thanks again everyone that's helped. Comments about lack of documentation duly noted as well. I am guessing there is a more elaborate network diagram with the main office guys that sort of support the network, but it is likely not fully up to date as well. We're not a very large company so we do not have a fully dedicated group or person that supports the network. If we did, they might have tried to force a Cisco ASA on me some time ago. The fact I can more or less support what I've got has helped me and pfSense is really the reason I can support it, because it's straightforward to use.
The person that originally reconfigured our company network decided to set up the main office and my office with very large LAN subnets for some reason. So, you will see in the image that the main office is 172.25.0.0 - 172.25.255.255 and my office is 172.26.0.0 - 172.26.255.255. We likely could/should have been all on 172.25 with the next digit being the assigned to each office and the last being left for all the hosts within the office. But, no matter, that is how it is set up.
When I first set up this remote office, we had no VPN connectivity at all. I think I started with some Linux firewall distribution and then later used monowall and that lead me to pfSense. I think it's been here since one of the very, very early releases. All that being said, I'm a middling sort of network person so mistakes in the setup would not exactly be surprising. Part of what is awesome about pfSense is the traffic shaping which has been huge for me because I use hosted VoIP for my office phone system.
I've attached a very basic image that describes some of what I've mentioned along with the relevant pfSense screens (parts of them anyway), so you can see the setup. I'm guessing one of you experts will notice something right away, which is appreciated.
Thank you again for your help on this.
![pfSense VPN Info.JPG](/public/imported_attachments/1/pfSense VPN Info.JPG)
![pfSense VPN Info.JPG_thumb](/public/imported_attachments/1/pfSense VPN Info.JPG_thumb) -
This does not look right - see attached.
You have the gateway setup for the remote 25 network as your lan interface 26.10.254 on pfsense???
Where is your phase 1 details when you setup the tunnel? You wan interface is normally your endpoint for the tunnel.
-
When you mentioned routing, I started looking around and found a specific routing entry for the VPN. I am not sure why it's there, but it is. I did some reading on VPN setup in pfSense and it seems that the routing over the tunnel takes care of itself so no specific routing entry like this should be needed.
So, I removed this, re-enabled the redirect setting in system tunables back to the default and rebooted the linux machine (to be safe). I can still ping IPs on the other end of the tunnel, so that's great!
That introduced a new issue with DNS resolution over the VPN for our domain. I figured that out with some searching but will post the details here so it might help another person later.
Basically in the DNS forwarder where you can specify a domain override, I had to also specify the LAN IP of pfSense (172.26.10.254 in my case) as the "Source IP" on the domain override configuration. Once I did that, lookups for our domain worked perfectly again.
So, at the end of the day, the issue was the static route that I added and then the IP on the DNS domain override. I assume I did the route entry to try and "tell" pfSense to send traffic for the remote VPN someplace. And oddly, it worked until now.
But, it now seems that all is well and I've only got the configuration in place that is needed.
Thanks again everyone!
-
Basically in the DNS forwarder where you can specify a domain override, I had to also specify the LAN IP of pfSense (172.26.10.254 in my case) as the "Source IP" on the domain override configuration.
You usually have to do that when the DNS server that services the domain in question is over a VPN, because otherwise the source IP of the request (from the pfSense, across the VPN to the DNS server) will be some IP address of a VPN tunnel endpoint, or some internal tunnel address. The remote DNS server typically won't have a route back to that and so the reply to those DNS queries would never make it back.