OpenVPN tunnel established, one side's traffic gets lost
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
I removed the manual static route on both ends. Now the remote side has only one error:
"ERROR: FreeBSD route add command failed: external program exited with error status: 1"Have you entered a network on the local side at "Local Network"?
For a site-to-site I recommend to keep this empty and put the servers LANs into the "Remote Networks" box on the client.@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
On local side I tried pinging with the default source address and both the VPN ones, none of them work.
On remote side I tried the same, it works with all three.I did this already!
Unless you wanted me to ping machines local to the router on both sides, in which case it doesn't work on either side (pfSense ping to a machine on its own LAN). On local side I get packets dropped, on the remote side I get the following:This seems quite strange to me at all.
To be clear, does the ping at least work when with the default source? -
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
Have you entered a network on the local side at "Local Network"?
Hold on, I have entered the local networks into the local networks on server and remote networks on client and remote networks into local networks on client and remote networks on the server. Was I supposed to put each network into only one of those?
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
This seems quite strange to me at all.
To be clear, does the ping at least work when with the default source?Tell me about it, I started with the premise that I did something very stupid, because if it wasn't stupid it'd be easier to find.
As it is right now with default source, from the local 192.168.1.0/24 pfSense I can ping the local machine on LAN, (192.168.1.101) pinging remote pfSense interface on LAN and remote machine on LAN doesn't work.
From the 192.168.130.0/24 remote side I can ping local 192.168.1.101 machine, the local pfSense LAN interface and the remote LAN machine (192.168.130.101) -
Removing the local network CIDR from "local networks" on local side seems to have gotten rid of the second route add error on the remote side. No change to connectivity though.
-
@issuehaver
Yes, I didn't presume that this will resolve your routing / access issue, but the add-route error in the OpenVPN log.As it is right now with default source, from the local 192.168.1.0/24 pfSense I can ping the local machine on LAN, (192.168.1.101) pinging remote pfSense interface on LAN and remote machine on LAN doesn't work.
From the 192.168.130.0/24 remote side I can ping local 192.168.1.101 machine, the local pfSense LAN interface and the remote LAN machine (192.168.130.101)This isn't what I asked for. The point of interest is how it behaves when you use the Diagnostics > Ping tool a) with default source and b) when the source is OpenVPN (server or client).
If I understood you correctly, you only need access from remote to the local, right?
-
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
This isn't what I asked for. The point of interest is how it behaves when you use the Diagnostics > Ping tool a) with default source and b) when the source is OpenVPN (server or client).
Apologies for misunderstanding.
On the local side, OpenVPN source:
Pinging remote LAN interface, remote LAN machine and local LAN machine doesn't work. Pinging local LAN interface works.On the local side, default source:
Pinging remote LAN interface, remote LAN machine doesn't work. Pinging local LAN machine and local LAN interface works.On the remote side, OpenVPN source:
Pinging remote LAN machine doesn't work with the following messages:PING 192.168.130.101 (192.168.130.101) from 192.168.240.2: 56 data bytes 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 450b 0 0000 3f 01 42e5 192.168.240.2 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 78c2 0 0000 3f 01 0f2e 192.168.240.2 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 7152 0 0000 3f 01 169e 192.168.240.2 192.168.130.101 --- 192.168.130.101 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss
Pinging remote LAN interface, local LAN machine and local LAN interface works.
On the remote side, default source:
Pinging remote LAN interface, remote LAN machine, local LAN machine and local LAN interface works.@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
If I understood you correctly, you only need access from remote to the local, right?
I need access both ways, i.e. site-to-site with servers on both ends, both hosting stuff on the Internet as well as internal-only stuff.
Edit: Post content flagged as spam.
-
@issuehaver
I suspect that the traffic from the local machine isn't routed to pfSense.Can you check that, please?
For instance try a ping from the local machine to a remote LAN IP, while you capture the packets on pfSense LAN interface. You can set the filter to ICMP and the Host to the remote IP you're pinging to prevent much noise. -
@viragomann
On local LAN machine (192.168.1.101) I ping remote LAN machine (192.168.130.101).
ICMP request exists on local pfSense LAN interface. ICMP request exists on local pfSense VPN interface (192.168.204.1). ICMP request doesn't exist on remote pfSense VPN interface (192.168.204.2). -
@issuehaver
Did you set the tunnel mask to /30 now? -
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
Did you set the tunnel mask to /30 now?
No. Still at /28.
Due to:
"If x.x.x.x/30 is entered for the IPv4 Tunnel Network then the server will use a peer-to-peer mode much like Shared Key operates: It can only have one client, does not require client-specific overrides or iroutes, but also cannot push routes or settings to clients. If an IPv4 Tunnel Network larger than that is used, such as x.x.x.x/24, the server will accept multiple clients and can push settings, but does require iroutes."
I chose to use larger network. I want to be able to add more sites in the future. -
@issuehaver
If you have a larger tunnel subnet to be able to access multiple clients you have to set the routes with CSO on the server, even when only one client is connected. -
@viragomann What is CSO? I don't find anything under docs, was following the official pfSense documentation.
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
What is CSO?
VPN > OpenVPN > Client Specific Overrides
It the part on pfSense which does the iroute. -
@viragomann Geez, if only they wrote that segment more clearly and explained it more. I read it before, but since I don't understand how OpenVPN works under the hood I thought iroute was either the local or the remote networks box in the main configuration. Guess I'll first go read about it and figure out how that works then. Must be the only thing wrong in the configuration then. Thanks!
-
@issuehaver
iroute (=internal route) set the routes inside OpenVPN. You can read more of it in the OpenVPN docs. -
@issuehaver
There is a chapter in the docs which describes a similar setup:
Configuring a Single Multi-Purpose OpenVPN Instance -
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
iroute (=internal route) set the routes inside OpenVPN. You can read more of it in the OpenVPN docs.
Ah thanks, that explains it.
@viragomann said in OpenVPN tunnel established, one side's traffic gets lost:
There is a chapter in the docs which describes a similar setup:
Configuring a Single Multi-Purpose OpenVPN InstanceOkay so I've corrected the tunnel network, server now has 192.168.240.0/24 and I've set the client side to 192.168.240.0/30.
Now I can additionally ping remote LAN interface from local side, the rest is the same as in this post:
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
On the local side, OpenVPN source:
Pinging remote LAN interface, remote LAN machine and local LAN machine doesn't work. Pinging local LAN interface works.
On the local side, default source:
Pinging remote LAN interface, remote LAN machine doesn't work. Pinging local LAN machine and local LAN interface works.
On the remote side, OpenVPN source:
Pinging remote LAN machine doesn't work with the following messages:
PING 192.168.130.101 (192.168.130.101) from 192.168.240.2: 56 data bytes
92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2)
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 450b 0 0000 3f 01 42e5 192.168.240.2 192.168.130.10192 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2)
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 78c2 0 0000 3f 01 0f2e 192.168.240.2 192.168.130.10192 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2)
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 7152 0 0000 3f 01 169e 192.168.240.2 192.168.130.101--- 192.168.130.101 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet lossPinging remote LAN interface, local LAN machine and local LAN interface works.
On the remote side, default source:
Pinging remote LAN interface, remote LAN machine, local LAN machine and local LAN interface works.It looks as if the machines aren't pingable from a different subnet, but I checked this by pinging them from a different subnet on the same side and that works, so I don't understand what else is wrong. I will look at it tomorrow, things don't make sense right now.
Thanks a lot for your help so far!
-
@issuehaver said in OpenVPN tunnel established, one side's traffic gets lost:
Okay so I've corrected the tunnel network, server now has 192.168.240.0/24 and I've set the client side to 192.168.240.0/30.
In an access sever setup it's recommended to let the server push the VPN interface settings to the clients. So you should leave the clients tunnel network blank and only set it on the server in the CSO.
A /30 subnet should only be used when the servers topology setting is 30net as well, otherwise use a single IP (/32) for the client in CSO. -
Well I checked with packet capture and this time the packets were traversing the VPN tunnel, which they did not before that last fix, but still no response from remote host, so I checked and it wouldn't respond from a different subnet, the local host would.
Naturally, I made a mistake during installation: I installed it in 192.168.1.0/24 originally and then moved it to 192.168.130.0/24. Changed the address and subnet mask but forgot to change the gateway as that was in a completely different place in the interface. So that explains that last part. Luckily I did the packet capture prior to this or I would've thought it was this all along. :)Well, the tunnel works properly now, except for pinging local-to-it machine from OpenVPN source. I should've first checked the example configurations for what I was doing, as you have linked, instead of just reading the main instructions. So basically that client specific override was the issue.
So I need to fill all the networks on the host and then leave the networks on the client side empty and the use the client override to set it - is that correct way?
And I changed the tunnel /32.Also why do I now see traffic coming from 192.168.240.0, that would be the network address for 192.168.240.0/24 and not usable, no?
Edit: okay, my bad, this is the /32 address, and the server has .1, I guess that is my bad for chosing .0/32.
I also see some kind of gateway on 192.168.240.2 now, this must be why it wasn't working properly before, 240.2 got assigned to the first client.
Edit2: Oh, this is the OpenVPN interface that I've added before. Doesn't look like this works now. Will have to do some reading on that.When I try to ping a local machine from OpenVPN source, I get no response (it works with default source), and packets never cross from 192.168.240.0/24 to 192.168.1.0/24.
On the remote side I get the redirect messages and the packets start ringing within the OpenVPN interface until TTL reaches 0 and I don't see anything on remote LAN:PING 192.168.130.101 (192.168.130.101) from 192.168.240.0: 56 data bytes 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 3f 01 5fcb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 3d 01 61cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 3b 01 63cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 39 01 65cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 37 01 67cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 35 01 69cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 33 01 6bcb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 31 01 6dcb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 2f 01 6fcb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 2d 01 71cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 2b 01 73cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 29 01 75cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 27 01 77cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 25 01 79cb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 7c5f 0 0000 3f 01 0b93 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 23 01 7bcb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 7c5f 0 0000 3d 01 0d93 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 2827 0 0000 21 01 7dcb 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID Flg off TTL Pro cks Src Dst 4 5 00 0054 7c5f 0 0000 3b 01 0f93 192.168.240.0 192.168.130.101 92 bytes from 192.168.240.1: Redirect Host(New addr: 192.168.240.2) Vr HL TOS Len ID
-
Okay, I got to the bottom of it, the settings were wrong, but one of the recipes in their docs had an exact example.
That previous example was for a deprecated net30 topology option, which doesn't work properly otherwise. I was getting an error about adding routes again, and it did not show up right off, it was intermittent for some reason.https://docs.netgate.com/pfsense/en/latest/recipes/openvpn-s2s-tls.html
So for anyone having a similar issue, the client tunnel and network settings need to be empty, so basically just authentication on the client side. On the server side in local networks you need to list all the networks, both server and client(s), then in remote only the client(s)'s networks, and the tunnel in server options is the only palce where you specify the tunnel CIDR, I used /24 tunnel.
The client override does not include anything in the tunnel and in the remote networks you need to enter the remote network which is local to that client. And that's all that needs to be there, server will configure the correct tunnel on the client(s) remotely.I think mostly the issue was wrong tunnel network entered for/at the client. The software has so many different possbilities and the pfSense GUI doesn't/can't validate the settings properly so there's plenty (infinite?) of mistakes possible.
After that just ensure the firewall rules allow whatever you need.
I am now getting 15/16 of those prior ping combinations through, only combo dropping is the one that's returning redirects (from a previous post), but it might be normal, it doesn't affect any connectivity, I don't know.
Big thanks to @viragomann for helping! I was a bit lost in the documentation I guess. I should have read a lot more than the page where they describe the settings as that did not tell the whole story and I did not realize this at the time.
Thanks a lot for your time! -
@issuehaver
Thanks for the feedback. Glad that you get it working at last.