Clients cannot communicate with each other.
-
First I set up an OpenVPN [Remote Access (SSL/TSL)] server on each of the routers. Then I created two clients to the other routers. Now I can ping any client from any other. It took me about an hour to get it working. Do you happen to know a good tutorial that explains and exemplifies all this?
-
I have found something that might be a clue to what goes wrong. In the "OpenVPN" tab of my system logs, I found this entry:
ERROR: FreeBSD route add command failed: external program exited with error status: 1It seems that for some reason FreeBSD refuses to add the new route. I took care of the problem by adding the routes manually. But why does this happen? Does anyone have any idea why? What is the solution?
NB: Took care of the problem by creating static routes to LANs.
-
@scilek
I've seen the FreeBSD route add command failed error before also. I think in my situation, it was related to it trying to add the same route twice. I had to look above in the log to see the same route numbers added twice. I think I had something in Client Specific Overrides that duplicated some other setting elsewhere. -
@yumcheese
I hope the issue will be solved in the next version. -
@scilek said in Clients cannot communicate with each other.:
First I set up an OpenVPN [Remote Access (SSL/TSL)] server on each of the routers. Then I created two clients to the other routers. Now I can ping any client from any other. It took me about an hour to get it working. Do you happen to know a good tutorial that explains and exemplifies all this?
Huh? You made one server on every router with two clients to every other server? How is that supposed to work? Doesn't make any sense to me. Per your logic you have:
- HQ: One server, two clients to Site A and B
- Site A: One server (with connection from site B and HQ) but ALSO two clients TO HQ and side B?
- see above
That logic makes no sense at all and isn't anything like a "Mesh".
The simple way to do what you described in your OP would be:
- Check which site (A/B) has the bigger or more stable connection
- HQ: create two shared keys OVPN site2site VPN servers
- Site A: create a client S2S with the shared key from HQ's ovpns1 connection
- Site A: create a server for connection with Side B, again shared key S2S setup like the two in HQ
- Site B: create two clients S2S, one with HQ, one with Site A's server.
Now you've got your triangle complete. If you don't need SiteA-SiteB interconnect but want to route it through HQ, you can skip the server/client setup between Site A and B, just add the appropriate routes to the HQ tunnel and be done (but keep in mind your HQ line will get the traffic twice, inbound and outbound if you have traffic from A to B).
That's it. Simple and smooth. Setups with RemoteAccess Setup are more complicated as the routes must be mapped to the appropriate client, interface etc. because RA server type isn't a 1:1 Site2Site connection with clear endpoints. So you're making your life harder than it has to. I've got clients running setups like this with around two dozens of remote offices and sites around the globe, some only to HQ, some with each other without any problems. So nothing that "has to be fixed in the next version". Just use the tools at hand the way they work :)
-
Actually, I figured out that the reason why my original setup had failed was because there is something wrong with the pfSense 2.4.4_p3 OpenVPN tab; it won't just create the necessary routes as defined in the configuration; or maybe something wrong the FreeBSD route creation command. I solved the problem by creating static routes and now my hub-and-spoke topology works fine. Thanks for the advice, though. Appreciated...
-
@scilek said in Clients cannot communicate with each other.:
because there is something wrong with the pfSense 2.4.4_p3 OpenVPN tab; it won't just create the necessary routes as defined in the configuration; or maybe something wrong the FreeBSD route creation command. I solved the problem by creating static routes and now my hub-and-spoke topology works fine. Thanks for the advice, though. Appreciated...
I'm telling you that either pfSense or FreeBSD do route creation with OpenVPN just fine. And I could post several configurations from our clients (if I were free to do so) doing what you'd like to do in your first post without any problems with route creation at all. So as I have multiple clients running multi-site-to-site tunnels without a hitch, I'm leaning towards saying "you have an error in your configuration". But as long as you don't post anything to help, that's as far as we can come. You say you set it up right and pfsense has some sort of routing bug. I claim multiple clients of our company running dozens of site2site tunnels even in mesh/web-like configurations without any routing problems so the error would be somewhere in your setup. So I agree to disagree ;)
If you'd like to solve your problem - help us find the error. If you'd like to manually set routes that should be set up automatically if configured the right way - feel free. But that make break the next time you change your setup in any way or perhaps with a future update of OVPN in pfSense. :)
-
I did exactly as I explained in my original post and it did not work. I followed the example demonstrated by the Youtuber but what worked for him failed for me. I don't mind the administrative overhead.
-
You didn't explain nor show any configuration just what you think you did. Also some example from some YT says nothing about if that tutorial is actually worth a cent. There are numerous pointless videos out there of things that are either obsolete, not needed anymore or plain wrong. So without anything factual about your config - and without you posting some screens or config samples - there's simply no way to check what is wrong with your setup. That's it. So as long as there's no config, screenshots or any tangible data at all - sorry, I'm out. Nothing I can do besides reading a magic 8 ball. Just telling you, it is not a general bug or it would be all over the forums.
-
Exactly.
Show us your VPN configuration.
Show us your routing table.
This works pretty much flawlessly. If OpenVPN cannot/does not install the routes in the routing table it is misconfigured.
What you have SAID you have done looks right. But if it was right it would be working.
-
You've got a point there. However, I have already told the owner that the system is ready and therefore cannot risk disrupting their workflow. If I ever get a chance to do the same, I will definitely be more verbose.
-
I would fix it. If you are using manual static routes that is an invalid way to configure OpenVPN. It will only bite you in mysterious, hard to find ways later.
-
@scilek said in Clients cannot communicate with each other.:
Actually, I figured out that the reason why my original setup had failed was because there is something wrong with the pfSense 2.4.4_p3 OpenVPN tab; it won't just create the necessary routes as defined in the configuration; or maybe something wrong the FreeBSD route creation command. I solved the problem by creating static routes and now my hub-and-spoke topology works fine. Thanks for the advice, though. Appreciated...
There is nothing wrong with pfsense and the OpenVPN tab. You are doing something wrong.
I have a dozen of these pipes all over the place working just fine.
-
I just configured the OP's exact scenario and it worked immediately. I had two existing tunnels connected, so I was basically acting as HQ. All that was done was to add the remote site's LAN to the IPv4 Local network(s) section in each site's OpenVPN config and verifying that the firewall rules on the OpenVPN tab are configured to allow the incoming traffic. As soon as the OpenVPN configs were saved at each site, each branch could ping the other through me acting as HQ.
So, as others have already stated, there's nothing wrong with OpenVPN tab. There's something missing (or misconfigured) in the OP's config or firewall rules.
I would start reviewing routing tables, firewall rules, firewall logs and possibly conflicting static routes at each site.
-
Actually, something interesting happened the other day. It was very early in the morning and I decided to run a power failure test. I rebooted all three routers and ROUTER_B did not come online through the tunnel. Fortunately, I had left the management ports open on WAN and was able to log back in. Although everything seemed to be in order, the OpenVPN client did not seem to connect to the server. Acting on a hunch, I opened the
ovpnc1interface tab (https://router_b_address/interfaces.php?if=opt1) and pressed the "Save" button. When I opened the main page, I found out that it had connected – I was able to ping any other host at any other site from any interface. Then I rebooted the router again, just to see what would happen. Again, it would not connect. So I went to the interface configuration page and saved again, and it came online.I was baffled and decided to run yet another test. First, on one of the production routers I have access to, I started an OpenVPN Peer-to-Peer (Shared Key) server. Then, I set up a new router. I configured a client on the new router (without the manual route entries) and it connected immediately. I rebooted several times to see what would happen. Each time it would connect and ping without a problem.
So yes, you're right, it is not the OpenVPN tab. But what is it? What could possibly be causing this? Why won't it connect after a reboot? Why will it connect when I hit the "Save" button on the interface configuration page?
-
Impossible to say without you providing more details and undoing your known-incorrect configuration including static routes, and starting from scratch using pfSense/Netgate documentation.
You can look at the openvpn logs on the site that did not connect for insight into what it did not like as well.
It is quite possible the system attempted to install the static route into the routing table before the OpenVPN client was running so the destination address did not exist and adding the route failed.
That is why you let the OpenVPN process install the routes by adding the networks to the Remote Networks field.
All of this works correctly when you configure it correctly.
-
@Derelict said in Clients cannot communicate with each other.:
It will only bite you in mysterious, hard to find ways later.
Manually added static routes are not how openvpn expects to operate. If you chose that you would have to make sure no subnets are defined in the OpenVPN config (except the tunnel subnet) as it will try to add the routes, they will conflict and it will fail.
The error you were previously seeing:
ERROR: FreeBSD route add command failed: external program exited with error status: 1is usually caused by a route conflict of that type. The OpenVPN daemon is trying to add the routes it is configured with and cannot. Either because it conflicts with an existing route or it's not a valid subnet, not the network address.
Steve
-
This morning, knowing that there would very little or no need to access the Internet, I decided to remotely configure all the three routers from scratch. I removed all the static routes and then the clients and servers. After that, I redefined OpenVPN servers and then the clients. But this time, I rebooted the routers after I configured them, which I had not done previously as the users could not afford even a minute of downtime. After each reboot, the VPN routes appeared to be offline for about 20 seconds or so, then got online and then I was able to connect to remote networks. So it appears that the OpenVPN was not to blame after all.
I think settles the matter for good. I have learned through hardship that it is a good idea to reboot your router after configuring router OpenVPN clients/servers and deploy them only when you know you have configured them properly. This was my first VPN job, so I think I am entitled to making mistakes.
-
@scilek said in Clients cannot communicate with each other.:
I have learned through hardship that it is a good idea to reboot your router after configuring router OpenVPN clients/servers
A reboot is not necessary. Only stating this so future readers will know.
Glad you got past whatever problem it is you were having.
