Multiple Sites Routing with Site to Site and Road Warrior
-
You're running 2 separate site-to-site instances, as I understand, right?
yes, there are 3 OpenVPN Server instances actually, 2 are site to site, and one is RR all three are on SPH.RGB and PVG are Site to Site Clients to SPH.
I have added the routes as perscribed:
On PVG client enter: <sph-lan>,<rgb-lan>, <rr-tunnel network="">On RGB client: <sph-lan>,<pvg-lan>, <rr-tunnel network="">Then go to the road-warrior server settings and add all networks the clients should be able to reach to the "Local networks" box: <sph-lan>,<rgb-lan>, <pvg-lan>The ROADWARRIOR routes perfectly now. –Thank you very much!However the site to sites still do not route right....
SPH --> RGB = WORKS
SPH --> PVG = WORKSRGB ---> SPH --> PVG = FAILS
PVG ---> SPH --> RGB = FAILSRGB ---> SPH = WORKS
PVG ---> SPH = WORKSThe firwall rule on SPH, RGB and PVG allows any traffic from all to all on the OpenVPN interface at each end point.
But I am not able to hit anything on PVG from RGB going through SPH. I am also not able to go the other way, not able to hit anything on RGB from PVG going through SPH.
I am able to hit anything in PVG from SPH, as well as being able to hit anything in RGB from SPH, so that is working.
Do I need static route on RGB pointing to PVG with SPH as gateway, as well as a static route on PVG pointing to RGB with SPH as gateway?
Not sure what else I am missing.....all looks like it should work.</pvg-lan></rgb-lan></sph-lan></rr-tunnel></pvg-lan></sph-lan></rr-tunnel></rgb-lan></sph-lan>
-
Is there an outbound NAT rule defined on openVPN interface of RGB or PVG?
-
There is one single rule on all 3 sites OpenVPN interface under Rules, to allow any OpenVPN traffic to pass in either direction in and out.
See screenshot attached….
There is PURE NAT enabled with auto outbound rule creation.
see other screenshot...
-
Okay, your outbound NAT is set in automatic mode, but are there rules for the OpenVPN interface listed at the bottom? For instance, if you assign a gateway to a VPN connection, pfSense may generate an outbound NAT rule for that interface.
If not, maybe it helps to assign an interface to each of the site-to-site servers on SPH. Sometimes packets get miss-routed without this.
At least I would go to troubleshooting with packet capture. On SPH take captures on the different OpenVPN interfaces (you've assigned above) to see if the packets are routed correctly and have their proper IP addresses.
-
Post the server1.conf and server2.conf from SPH and the client1.conf from each client site.
After that, I would add any/any rules to both the LAN interface and the OpenVPN interface on all sites until basic IP connectivity is established. I would then disable the software firewall on the endpoints you are testing from.
-
using FIND there is no server1.conf, server2.conf or client1.conf in any directory.
When I add in interface, for OpenVPN, then none of the routing works, I am not able to ping ANYTHING but SPH's LAN interface, as soon as I remove the OpenVPN interface,what routing that was works before comes back.
SPH sees traffic from PVG and RGB, it just is not routing from PVH through SPH to RGB or vice versa.
There is NO software firewalls at any ENDPOINT. Only via Pfsense.
LAN Rules are ALL all between OpenVPN and LAN, block out outbound on WAN, except for ports that are Forwarded to a LAN host.
-
The OpenVPN config files are in "/var/etc/openvpn". What syntax did you use for the find command? Because it works for me:
[2.3.4-RELEASE][admin@example.home]/: find / -name "server*.conf"
/var/etc/openvpn/server1.conf
/var/etc/openvpn/server3.conf
/var/etc/openvpn/server2.conf
/var/etc/openvpn/server4.conf
/var/etc/openvpn/server6.conf
/var/etc/openvpn/server8.confYou can also use the GUI:
Diagnostics -> Edit File -> Browse to "/var/etc/openvpn"
LAN Rules are ALL all between OpenVPN and LAN, block out outbound on WAN, except for ports that are Forwarded to a LAN host.
I don't want to make assumptions on what I think you're saying here, so it would be better to post screen shots of the rules on each interface.
-
I don't want to make assumptions on what I think you're saying here, so it would be better to post screen shots of the rules on each interface.
The LAN rules are the defaults when pfsense is installed, no changes, but I can post here with the conf files.
OpenVPN Rule Screenshot posted already…..above first screenshot, this rule is the same OpenVPN rule on ALL sites.
The OpenVPN config files are in "/var/etc/openvpn". What syntax did you use for the find command? Because it works for me:
used find "server.conf"
I'll try your command and post the files......
--Thanks and stand by....might be a day or two....just got a few other fires on other projects I need to stomp out first.....like Employee Mid-year reviews! :P
-
If I were you I would switch the site-to-site networks to SSL/TLS so you can just push the necessary routes to the site-to-site clients as your needs change.
But:
SPH Site-to-Site server to PVG:
IPv4 Remote Networks: 10.0.32.0/19
SPH Site-to-Site server to RGB:
IPv4 Remote Networks: 10.0.96.0/19
SPH Remote Access Server:
IPv4 Local Networks: 10.0.160.0/19, 10.0.96.0/19, 10.0.32.0/19 (OR Redirect Gateway set, which will, of course, also send this traffic through the tunnel)
PVG Client to SPH:
IPv4 Remote Networks: 10.0.160.0/19, 10.0.96.0/19, SPH Remote Access Tunnel Network (as yet unspecified in thread) unless included in 10.0.160.0/19
RGB Client to SPH:
IPv4 Remote Networks: 10.0.160.0/19, 10.0.32.0/19, SPH Remote Access Tunnel Network (as yet unspecified in thread) unless included in 10.0.160.0/19
-
I have tried SSL/TLS it was SO SLOW that the connections within the VPN would constantly drop.
As per requested the conf files….
PVG-client1.conf.txt
RBG-client1.conf.txt
SPH-server1.conf.txt
SPH-server3.conf.txt -
Here are the WAN rules on SPH
 -
Here are the LAN rules, which are the SAME on all sites.
The openVPN rules are on another post previously.
 -
If I were you I would switch the site-to-site networks to SSL/TLS so you can just push the necessary routes to the site-to-site clients as your needs change.
But:
SPH Site-to-Site server to PVG:
IPv4 Remote Networks: 10.0.32.0/19
SPH Site-to-Site server to RGB:
IPv4 Remote Networks: 10.0.96.0/19
SPH Remote Access Server:
IPv4 Local Networks: 10.0.160.0/19, 10.0.96.0/19, 10.0.32.0/19 (OR Redirect Gateway set, which will, of course, also send this traffic through the tunnel)
PVG Client to SPH:
IPv4 Remote Networks: 10.0.160.0/19, 10.0.96.0/19, SPH Remote Access Tunnel Network (as yet unspecified in thread) unless included in 10.0.160.0/19
RGB Client to SPH:
IPv4 Remote Networks: 10.0.160.0/19, 10.0.32.0/19, SPH Remote Access Tunnel Network (as yet unspecified in thread) unless included in 10.0.160.0/19
So you mean add
SPH Site-to-Site server to PVG:
IPv4 Remote Networks: 10.0.96.0/19
SPH Site-to-Site server to RGB:
IPv4 Remote Networks: 10.0.32.0/19
As the way you specify:
SPH Site-to-Site server to PVG:
IPv4 Remote Networks: 10.0.32.0/19
SPH Site-to-Site server to RGB:
IPv4 Remote Networks: 10.0.96.0/19
These are the site PVG and RGB subnets that are LOCAL already to the sites listed. Are you meaning add the other sites non routing sites to REMOTE NETWORKS on their mated sites?
I've attached the SPH to PVG Server screenshot….and it already has 10.0.32.0/19 on REMOTE NETWORKS, so do you mean add 10.0.96.0/19 as well?
 -
I explained what Remote networks need to be where. I cannot tell exactly what needs to be added to what is there because I do not know what is already there.
Adding something to IPv4 Remote Networks for a shared-key OpenVPN server or client creates a kernel ROUTE into that OpenVPN instance for those networks. Those routes are visible in Diagnostics > Routes.
For traffic to be directed into OpenVPN there has to be a ROUTE telling pfSense to send it there.
For traffic to be allowed IN there has to be a RULE passing it (or an already-existing state allowing the traffic, which would include reply traffic).
I've attached the SPH to PVG Server screenshot….and it already has 10.0.32.0/19 on REMOTE NETWORKS, so do you mean add 10.0.96.0/19 as well?
If you want RGB to send traffic to SPH for delivery to PVG then RGB needs the PVG networks added as remote networks. Why would you want to route RGB networks to PVG? That makes no sense.
PVG 10.0.32.0/19 –-> SPH 10.0.160.0/19 ---> RGB 10.0.96.0/19 = FAILS
That tells you exactly what you need:
PVG needs 10.0.160.0/19 and 10.0.96.0/19 as IPv4 Remote Networks
RGB 10.0.96.0/19 –-> SPH 10.0.160.0/19 ---> PVG 10.0.32.0/19 = FAILS
And RGB needs 10.0.160.0/19 and 10.0.32.0/19 as IPv4 Remote Networks
It is just interfaces, rules, and routes. Not really different from anything else.
-
If you want RGB to send traffic to SPH for delivery to PVG then RGB needs the PVG networks added as remote networks. Why would you want to route RGB networks to PVG? That makes no sense.
We have application servers in all three sites that need to talk to each other. I need routing to all three sites. I am trying to avoid having VPNs from PVG to RGB, first off because PVG have dynamic IP from the ISP so setting them up as servers is almost impossible. Secondly only SPH and RGB have a static IP assigned to them so those are the only ones that could possible be OpenVPN Servers.
We also have clients at all three sites that need to talk to servers in all three sites as well.
This is why we need routing between PVG, SPH and RGB to work.
Prior to creating this thread I tried, adding the remote sites to REMOTE NETWORKS, however the VPN went down, and would not come back up….just kept trying to reconnect but never would. I will try adding them again and see what happens. I thought I was missing something but it appears maybe not.....I maybe didn't wait long enough.....for the VPN to establish. I will attempt it again and see it if it works.
-
"first off because PVG have dynamic IP from the ISP so setting them up as servers is almost impossible."
I really don't buy this… So your saying ISP changes the IPs on those how often? I have dynamic IP from my isp - and its been the same for well over 2 years. As long as you renew your lease and your equipment is not offline for extended periods with how dhcp normally works your IP should rarely change. Are you saying your ISP forces your IP to change - if so how often?
And why can you not just use some sort of dynamic dns to allow have the same fqdn point to whatever your IP might be, etc..
-
SSL/TLS vs. Shared Key performs generally the same. It is a matter of authentication, not transport.
The real issue is whether the server transport itself is UDP (preferred) or TCP (can be pretty poor).
You don't seem to be grasping something. Not sure how else to explain it.
-
I really don't buy this… So your saying ISP changes the IPs on those how often? I have dynamic IP from my isp - and its been the same for well over 2 years. As long as you renew your lease and your equipment is not offline for extended periods with how dhcp normally works your IP should rarely change. Are you saying your ISP forces your IP to change - if so how often?
My ISP changes the IP every 7 days. Sometimes several times a week. They are constantly pushing firmware which reboots the modem. Normally about 3 am every Sunday.
And why can you not just use some sort of dynamic dns to allow have the same fqdn point to whatever your IP might be, etc..
My ISP blocks Dynamic DNS so this does not work.
TCP (can be pretty poor)
We use TCP because is it a guaranteed connections, UDP is not, it is pretty much send and forget, where TCP is send an make sure it gets there otherwise resend/
You don't seem to be grasping something. Not sure how else to explain it.
I am having issue with is how and where to place my routing....I have read just about every topic on here about OpenVPN routing, some read that one needs to use "push route" or iroutes in the client, others read in the server, then some read, place your networks in the REMOTE NETWORK on the server, some read place your networks in the REMOTE NETWORK of the client.....and others read place your networks in REMOTE NETWORKS on both client and server. Then others read that static routes are how it should be done. So I am confused.
I am trying to figure out which is correct and which I should be implementing.
I'd rather not just do trial and error, as that just creates a bouncing network in general. I just need to get all my sites to route to each other.
I do not want to switch to SSL/TLS for my Site to Site links, as that would mean I would have to tear down all the existing links and rebuild them. I had a hard enough time getting SSL/TLS working for RoadWarrior and that took me a while to get it stable for my roaming users. I am not wishing to go through that process for each site to site and end up also impacting the now WORKING Roadwarror.
I am sure someone has this set-up, I just need to know how they got their routing to work with Site to Site.
-
I have already told you exactly what you need where. You do not need to push anything. You do not need iroutes. You just need to look at every site and put the networks you want to reach FROM THAT SITE ON THAT OPENVPN INSTANCE in IPv4 Remote Networks there.
UDP is better for OpenVPN transport. You still have TCP on TCP connections inside the tunnel for guaranteed delivery where required.
One writeup: http://sites.inka.de/bigred/devel/tcp-tcp.html
