OpenVPN tunnel established, one side's traffic gets lost
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
Are both endpoint the default gateways in their respective LAN?
No, the WAN interface is the default on both pfSenses.
The point is if both VPN endpoints, i.e. both pfSense boxes, are the default gateways for the LAN devices or if there is another router in use.
I should mention that the remote side I am currently testing over a connection with CGN (added as WAN in pfSense directly), but since it is a client and the tunnel is established, I guess this is irrelevant?
Yes, if it's on the client side it doesn't matter. And your connection is already established as you said.
-
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
The point is if both VPN endpoints, i.e. both pfSense boxes, are the default gateways for the LAN devices or if there is another router in use.
Oh, no, these pfSense routers are the only routers for these LAN devices, on both sides, 1 per side. On the relevant part of the network at least, there's another one downstream, unrelated. The pfSense machines themselves are the only gateways, but the VPN endpoint is not the default gateway inside the router, the WAN connection is (just to clarify).
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
There's two, don't know the difference, I assume one is the interface I added, but the second one is also named the same but with "OpenVPN client:" prepended. I never got past pinging tunnel endpoints so I didn't test much further.
"OpenVPN" is an interface group which is implicitly created by pfSense when you set up an OpenVPN instance.
Therefore it's not wise to call an self assigned interface as well "OpenVPN".On local side I tried pinging with the default source address and both the VPN ones, none of them work.
At least it should work with the default source.
So possibly the destination device itself is blocking the access if it is coming from outside its network.
-
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
"OpenVPN" is an interface group which is implicitly created by pfSense when you set up an OpenVPN instance.
Ah interface group, that's why there's a shared one.
There's a separate interface for the VPN as well though, so the interface I added, the VPN that's added by pfSense and the interface group OpenVPN, 3 of them. But nevermind that part.The problem is the same when I ping the pfSense interface on LAN the same way, same happens:
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
On local side I tried pinging with the default source address and both the VPN ones, none of them work.
On remote side I tried the same, it works with all three. -
This post is deleted! -
Ok, so, from pfSense I can ping both endpoints from both sides, but from a machine on the network I can ping all except from remote LAN to local tunnel endpoint.
Same when I try pinging pfSense's LAN address on local side from remote pfSense - all dropped. This looks like a problem with pfSense. -
I've checked the traffic on "OpenVPN Server: Site-to-site VPN" interface and the equivalent client interface; when I try to ping the local TEP from the remote side (from within pfSense), I can see the ICMP packets on the client side at the tunnel endpoint interface, and at the equivalent server TEP interface. But when I do it from remote LAN, I can see the packet at the LAN interface, and the remote TEP interface with 1 subtracted from TTL, but it doesn't appear at the server TEP interface. What would this indicate?
-
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
Use the pfSense Diagnostics > Ping tool, from the source drop-down select OpenVPN and try a ping to a local computer. Does this work as well?
This simple test could shed some light. Do it on both sides and provide what you get.
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
On local side I tried pinging with the default source address and both the VPN ones, none of them work.
On remote side I tried the same, it works with all three.I did this already!
Unless you wanted me to ping machines local to the router on both sides, in which case it doesn't work on either side (pfSense ping to a machine on its own LAN). On local side I get packets dropped, on the remote side I get the following:PING 192.168.130.101 (192.168.130.101) from 192.168.240.2: 56 data bytes 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 b167 0 0000 3f 01 d688 192.168.240.2 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 7741 0 0000 3f 01 10af 192.168.240.2 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 29e5 0 0000 3f 01 5e0b 192.168.240.2 192.168.130.101 --- 192.168.130.101 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet lossThis looks like it tried to send the packet over VPN instead of locally and got the packet handed back to it by the other TEP and then it got lost. It looks as if routing isn't working anywhere.
The machine I am pinging on both sides definitely responds to out-of-subnet (if there are any) pings because I can ping it from other LANs, 192.168.91.0/24 for example.Also I am occasionally getting this error in OpenVPN log on the client side:
Jul 17 16:26:46 openvpn 60908 ERROR: FreeBSD route add command failed: external program exited with error status: 1 Jul 17 16:26:46 openvpn 60908 ERROR: FreeBSD route add command failed: external program exited with error status: 1 -
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
Also I am occasionally getting this error in OpenVPN log on the client side:
Jul 17 16:26:46 openvpn 60908 ERROR: FreeBSD route add command failed: external program exited with error status: 1
Jul 17 16:26:46 openvpn 60908 ERROR: FreeBSD route add command failed: external program exited with error status: 1Occasionally?
Is this all of the adding route issue you can find in the log?To troubleshoot this, need to know all subnets on the router and also the OpenVPN configuration. The pfSense routing tables would be helpful.
-
@viragomann Let's say it's all the time, I was troubleshooting and removed the "remote networks" in the remote side config and I think that's when it went away, but the server "local networks" was set all the time, there's an issue with something here but I don't see what. The error repeats each time the tunnel is connected.
This is all I see on the remote side, except transient error when disconnecting Internet. Local side has no errors that I can see except transients.@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
To troubleshoot this, need to know all subnets on the router and also the OpenVPN configuration. The pfSense routing tables would be helpful.
Tunnel is using 192.168.240.0/28, with assigned TEPs 192.168.240.1 for local and 192.168.240.2 for remote.
Remote network LAN is 192.168.130.0/24, WAN is an Android hotspot with 192.168.219.0/24 network.
Local network LAN is 192.168.1.0/24, WAN is direct Internet address.I've described what I did in the configuration in the first post, tell me if you need anything else.
As mentioned, I have provisional static routes in place for the opposite LANs, with local side having
192.168.130.0/24 with gateway 192.168.240.2
and remote side having
192.168.1.0/24 with 192.168.240.1.
If I remove this, the 192.168.1.0/24 route disappears from routing table.Local side routing table (the relevant part, there's more downstream subnets that are not relevant here):
Destination Gateway Flags Use Mtu Netif Expire default [ISP gateway] UGS 10266 1492 pppoe0 1.0.0.1 [ISP gateway] UGHS 32874 1492 pppoe0 1.1.1.1 [ISP gateway] UGHS 33231 1492 pppoe0 [Internet address] link#15 UHS 6 16384 lo0 127.0.0.1 link#6 UH 798349 16384 lo0 [ISP gateway] link#15 UH 60857 1492 pppoe0 192.168.1.0/24 link#1 U 146973429 1500 em0 192.168.1.250 link#1 UHS 0 16384 lo0 192.168.2.0/24 link#2 U 60651 1500 vmx0 192.168.130.0/24 192.168.240.2 UGS 0 1500 ovpns3 192.168.240.0/28 192.168.240.2 UGS 0 1500 ovpns3 192.168.240.1 link#18 UHS 0 16384 lo0 192.168.240.2 link#18 UH 65134 1500 ovpns3Remote side routing table:
Destination Gateway Flags Use Mtu Netif Expire default 192.168.219.216 UGS 3898 1400 ue0 127.0.0.1 link#4 UH 12126 16384 lo0 192.168.1.0/24 192.168.240.1 UGS 0 1500 ovpnc1 192.168.130.0/24 link#1 U 18522729 1500 vmx0 192.168.130.250 link#1 UHS 0 16384 lo0 192.168.219.0/24 link#7 U 1483 1400 ue0 192.168.219.134 link#7 UHS 13 16384 lo0 192.168.240.0/28 192.168.240.1 UGS 0 1500 ovpnc1 192.168.240.1 link#8 UH 1467 1500 ovpnc1 192.168.240.2 link#8 UHS 0 16384 lo0 -
@issuehaver
So I assume, the add routes issue was coming from having static routes in place, while OpenVPN tries to add a route for same route network.You shouldn't set static routes for networks across the VPN. This should be done by OpenVPN only. Use the "Remote Networks" box in the OpenVPN settings on both sites for setting the routes properly.
After removing the static routes and setting it in the OpenVPN you should see the equal routes when the connection is established.Further it's recommended to use a /30 tunnel subnet for a site-to-site VPN.
-
I removed the manual static route on both ends. Now the remote side has only one error:
"ERROR: FreeBSD route add command failed: external program exited with error status: 1"Okay, now the routing tables haven't changed compared to with static routes. I don't know why this wasn't working properly before, I had it without static routes and they were missing in the table.
Nothing changed regarding connectivity though; still can't get data through.
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
Further it's recommended to use a /30 tunnel subnet for a site-to-site VPN.
I read in the instructions that /30 tunnels are used when they're the only planned remote site, to use larger than /30 if you plant to have more sites. So that's why I changed it a larger subnet. Also says it behaves differently with /30 subnet, with some things unavailable, like pushing routes and settings to clients, but most importantly the only one client part.
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
I removed the manual static route on both ends. Now the remote side has only one error:
"ERROR: FreeBSD route add command failed: external program exited with error status: 1"Have you entered a network on the local side at "Local Network"?
For a site-to-site I recommend to keep this empty and put the servers LANs into the "Remote Networks" box on the client.@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
On local side I tried pinging with the default source address and both the VPN ones, none of them work.
On remote side I tried the same, it works with all three.I did this already!
Unless you wanted me to ping machines local to the router on both sides, in which case it doesn't work on either side (pfSense ping to a machine on its own LAN). On local side I get packets dropped, on the remote side I get the following:This seems quite strange to me at all.
To be clear, does the ping at least work when with the default source? -
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
Have you entered a network on the local side at "Local Network"?
Hold on, I have entered the local networks into the local networks on server and remote networks on client and remote networks into local networks on client and remote networks on the server. Was I supposed to put each network into only one of those?
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
This seems quite strange to me at all.
To be clear, does the ping at least work when with the default source?Tell me about it, I started with the premise that I did something very stupid, because if it wasn't stupid it'd be easier to find.
As it is right now with default source, from the local 192.168.1.0/24 pfSense I can ping the local machine on LAN, (192.168.1.101) pinging remote pfSense interface on LAN and remote machine on LAN doesn't work.
From the 192.168.130.0/24 remote side I can ping local 192.168.1.101 machine, the local pfSense LAN interface and the remote LAN machine (192.168.130.101) -
Removing the local network CIDR from "local networks" on local side seems to have gotten rid of the second route add error on the remote side. No change to connectivity though.
-
@issuehaver
Yes, I didn't presume that this will resolve your routing / access issue, but the add-route error in the OpenVPN log.As it is right now with default source, from the local 192.168.1.0/24 pfSense I can ping the local machine on LAN, (192.168.1.101) pinging remote pfSense interface on LAN and remote machine on LAN doesn't work.
From the 192.168.130.0/24 remote side I can ping local 192.168.1.101 machine, the local pfSense LAN interface and the remote LAN machine (192.168.130.101)This isn't what I asked for. The point of interest is how it behaves when you use the Diagnostics > Ping tool a) with default source and b) when the source is OpenVPN (server or client).
If I understood you correctly, you only need access from remote to the local, right?
-
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
This isn't what I asked for. The point of interest is how it behaves when you use the Diagnostics > Ping tool a) with default source and b) when the source is OpenVPN (server or client).
Apologies for misunderstanding.
On the local side, OpenVPN source:
Pinging remote LAN interface, remote LAN machine and local LAN machine doesn't work. Pinging local LAN interface works.On the local side, default source:
Pinging remote LAN interface, remote LAN machine doesn't work. Pinging local LAN machine and local LAN interface works.On the remote side, OpenVPN source:
Pinging remote LAN machine doesn't work with the following messages:PING 192.168.130.101 (192.168.130.101) from 192.168.240.2: 56 data bytes 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 450b 0 0000 3f 01 42e5 192.168.240.2 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 78c2 0 0000 3f 01 0f2e 192.168.240.2 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 7152 0 0000 3f 01 169e 192.168.240.2 192.168.130.101 --- 192.168.130.101 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet lossPinging remote LAN interface, local LAN machine and local LAN interface works.
On the remote side, default source:
Pinging remote LAN interface, remote LAN machine, local LAN machine and local LAN interface works.@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
If I understood you correctly, you only need access from remote to the local, right?
I need access both ways, i.e. site-to-site with servers on both ends, both hosting stuff on the Internet as well as internal-only stuff.
Edit: Post content flagged as spam.
-
@issuehaver
I suspect that the traffic from the local machine isn't routed to pfSense.Can you check that, please?
For instance try a ping from the local machine to a remote LAN IP, while you capture the packets on pfSense LAN interface. You can set the filter to ICMP and the Host to the remote IP you're pinging to prevent much noise. -
@viragomann
On local LAN machine (192.168.1.101) I ping remote LAN machine (192.168.130.101).
ICMP request exists on local pfSense LAN interface. ICMP request exists on local pfSense VPN interface (192.168.204.1). ICMP request doesn't exist on remote pfSense VPN interface (192.168.204.2).