Basic static route question, doesn't seem to be working.
-
OK, I have a Ubuntu server on my local LAN running OpenVPN. I also have a remote Ubiquiti Edgerouter connecting to my Ubuntu OpenVPN with no issue, port forwarding etc in local pfSense working fine tunnel up etc. Current Symptoms:
-
Local Ubuntu server can ping and SSH into multiple things on remote LAN
-
Remote Edgerouter and a linux server on remote LAN can ping the Ubuntu server local LAN IP, but can't reach anything else on local LAN
-
Nothing else on local LAN can reach remote LAN
So, sounds like I need to add a static route to pfSense to point to the local Ubunto VPN server to allow local LAN devices to reach out to the remote LAN. Right? This is what I did and it doesn't seem to be doing anything.
System / Routing / Gateways
-
Interface: LAN
-
Address Family IPv4
-
Gateway; the Ubuntu Server LAN IP 10.0.1.6
-
Default Gateway not checked; I don't think I want this to be the LAN default gateway…
-
Disable Monitoring not checked
-
Monitor IP, blank. Ubuntu server will ping
-
In pfSense dashboard the gateway shows UP
System / Routing / Static Routes
-
Destination Network: remote VPN virtual IP entered as "10.80.0.0" drop down /24
-
Gateway; Selected the above created gateway
Using a computer on my local LAN I can't seem to get anything on tracert past pfSense, IMO pfSense should be sending this to my Ubuntu server at 10.0.1.6 but it isn't. What am I missing?
tracert 10.0.3.1 Tracing route to 10.0.3.1 over a maximum of 30 hops 1 1 ms <1 ms 2 ms pfSense.localdomain [10.0.1.1] 2 * * * Request timed out. 3 * * * Request timed out. 4 * * * Request timed out.Edit: I'm using these guides
https://community.openvpn.net/openvpn/wiki/RoutedLans
https://secure-computing.net/wiki/index.php/Graph -
-
If the VPN endpoint is within the LAN, a static route on the edge router cannot resolve the routing issue.
You rahter need static routes on each LAN device pointing to the Ubuntu server.Why do you not run the OpenVPN server on pfSense?
-
If the VPN endpoint is within the LAN, a static route on the edge router cannot resolve the routing issue.
You rahter need static routes on each LAN device pointing to the Ubuntu server.I don't think you understood me. I put a static route in pfsense, which is the default gateway for everything on my local LAN. It should see that traffic and send it to the VPN.
More findings, changing the static route entry:
System / Routing / Static Routes
-
Destination Network: remote network entered as "10.0.3.0" drop down /24
-
Gateway; Selected the above created gateway
This allows anything on my local LAN to communicate to anything on my remote LAN, great
But, things on the remote LAN can't reach anything on my local LAN. This baffles me.
Why do you not run the OpenVPN server on pfSense?
I could not wrap my head around the GUI to make OpenVPN do what I wanted. I have decent experience with this at work doing site to site between endpoints that are the default gateway but we aren't routing to the server LAN.
https://forum.pfsense.org/index.php?topic=145034.msg789391#msg789391 -
-
Correction; This entry
System / Routing / Static Routes
-
Destination Network: remote network entered as "10.0.3.0" drop down /24
-
Gateway; Selected the above created gateway
Does allow local LAN devices to ping remote LAN devices all day long.
But once I SSH into a remote server and tell it to ping something on my local LAN this works great for a little while and then I get disconnected. The VPN tunnel is not dropping.
Edit, more info. This looks like what I want to see going on from the LAN.
tracert 10.0.3.3 Tracing route to 10.0.3.3 over a maximum of 30 hops 1 1 ms <1 ms 2 ms pfSense.localdomain [10.0.1.1] 2 1 ms 1 ms <1 ms ubuntuserver.localdomain [10.0.1.6] 3 45 ms 42 ms 40 ms 10.80.0.11 4 44 ms 41 ms 41 ms 10.0.3.3user@10.0.3.3's password: Linux raspberrypi3 4.9.59-v7+ #1047 SMP Sun Oct 29 12:19:23 GMT 2017 armv7l The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Fri Mar 16 14:17:23 2018 from 10.0.1.54 user@raspberrypi3:~ $ ping 10.0.1.1 PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data. 64 bytes from 10.0.1.1: icmp_seq=1 ttl=62 time=76.1 ms 64 bytes from 10.0.1.1: icmp_seq=2 ttl=62 time=40.0 ms 64 bytes from 10.0.1.1: icmp_seq=3 ttl=62 time=44.1 ms 64 bytes from 10.0.1.1: icmp_seq=4 ttl=62 time=42.9 ms 64 bytes from 10.0.1.1: icmp_seq=5 ttl=62 time=41.4 ms 64 bytes from 10.0.1.1: icmp_seq=6 ttl=62 time=39.8 ms 64 bytes from 10.0.1.1: icmp_seq=7 ttl=62 time=39.0 ms 64 bytes from 10.0.1.1: icmp_seq=8 ttl=62 time=42.7 ms 64 bytes from 10.0.1.1: icmp_seq=9 ttl=62 time=40.8 ms 64 bytes from 10.0.1.1: icmp_seq=10 ttl=62 time=39.8 ms 64 bytes from 10.0.1.1: icmp_seq=11 ttl=62 time=42.8 ms 64 bytes from 10.0.1.1: icmp_seq=12 ttl=62 time=40.3 ms 64 bytes from 10.0.1.1: icmp_seq=13 ttl=62 time=44.2 ms 64 bytes from 10.0.1.1: icmp_seq=14 ttl=62 time=42.8 ms 64 bytes from 10.0.1.1: icmp_seq=15 ttl=62 time=40.8 ms 64 bytes from 10.0.1.1: icmp_seq=16 ttl=62 time=43.6 ms 64 bytes from 10.0.1.1: icmp_seq=17 ttl=62 time=42.9 ms 64 bytes from 10.0.1.1: icmp_seq=18 ttl=62 time=42.6 ms 64 bytes from 10.0.1.1: icmp_seq=19 ttl=62 time=44.1 ms 64 bytes from 10.0.1.1: icmp_seq=20 ttl=62 time=42.7 msputty session disconnected…
-
-
I don't think you understood me. I put a static route in pfsense, which is the default gateway for everything on my local LAN. It should see that traffic and send it to the VPN.
???
So, if a device on the LAN wants to send a packet to the other end of the VPN, it sends it to pfSense, which is supposed to route it back out the interface it came in on, to get to the VPN elsewhere on the local LAN? That's not the way routers work. You'll need to add the specific route to all the devices that want to send traffic to the VPN.
-
I don't think you understood me. I put a static route in pfsense, which is the default gateway for everything on my local LAN. It should see that traffic and send it to the VPN.
???
So, if a device on the LAN wants to send a packet to the other end of the VPN, it sends it to pfSense, which is supposed to route it back out the interface it came in on, to get to the VPN elsewhere on the local LAN? That's not the way routers work. You'll need to add the specific route to all the devices that want to send traffic to the VPN.
Yes, exactly. Do you think I am interpreting this wrong?
https://secure-computing.net/wiki/index.php/Graph
https://community.openvpn.net/openvpn/wiki/RoutedLans#ROUTESTOADDOUTSIDEOFOPENVPN
C:\Users\me>tracert 10.0.3.1 Tracing route to 10.0.3.1 over a maximum of 30 hops 1 1 ms <1 ms 2 ms pfSense.localdomain [10.0.1.1] 2 1 ms 1 ms <1 ms ubuntuserver.localdomain [10.0.1.6] 3 47 ms 41 ms 45 ms 10.0.3.1 Trace complete. C:\Users\me>ipconfig ~ IPv4 Address. . . . . . . . . . . : 10.0.1.54 -
From the 2nd link you provided:
the annoying work-around would be to add the route to every box on the LAN, in which case step 3 above would work.
This is exactly what I said, when I said to add specific routes to each device.
-
From the 2nd link you provided:
the annoying work-around would be to add the route to every box on the LAN, in which case step 3 above would work.
This is exactly what I said, when I said to add specific routes to each device.
Yes, that's a work around. If you can't do the thing it mentions before that:
That means in our example: 10.10.2.1 must know that for 10.10.1.x 10.10.3.x and the vpn internal network (for example, 10.8.0.x), it sends the traffic to 10.10.2.10 This is true for any number of lans you want to connect, whether server or client.
-
Another possible workaround is to set up an transit network between the pfSense and the VPN server, if you don't move the vpn server to pfSense.
Maybe you can set up the transit network as additional VLAN on the existing LAN interfaces. Then add a static route to pfSense which points to the VLAN IP of the Ubuntu server. -
Maybe I'm not expressing the problem right, at the moment everything can reach everything else. See these traceroutes below. BUT when I reach out from my local LAN to those remote devices I can only stay connected for a minute or two. I haven't done anything in the pfSense firewall yet either, maybe that's the issue??
This is my remote raspberry Pi reaching back to some random local LAN device
user@raspberrypi:~ $ traceroute 10.0.1.3 traceroute to 10.0.1.3 (10.0.1.3), 30 hops max, 60 byte packets 1 10.0.3.1 (10.0.3.1) 0.598 ms 0.524 ms 0.478 ms 2 10.80.0.1 (10.80.0.1) 80.190 ms 80.152 ms 80.372 ms 3 10.80.0.1 (10.80.0.1) 3049.530 ms !H 3089.671 ms !H 3089.633 ms !H pi@raspberrypi3:~ $ ifconfig eth0: flags=4163<up,broadcast,running,multicast>mtu 1500 inet 10.0.3.3 netmask 255.255.255.0 broadcast 10.0.3.255</up,broadcast,running,multicast>This is my local chromebook reaching a remote LAN device.
crosh> ping 10.0.3.2 PING 10.0.3.2 (10.0.3.2) 56(84) bytes of data. 64 bytes from 10.0.3.2: icmp_seq=1 ttl=62 time=49.3 ms 64 bytes from 10.0.3.2: icmp_seq=2 ttl=62 time=44.9 ms 64 bytes from 10.0.3.2: icmp_seq=3 ttl=62 time=43.4 ms ^C --- 10.0.3.2 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 43.487/45.909/49.310/2.482 ms crosh> tracepath 10.0.3.2 1?: [LOCALHOST] pmtu 1500 1: pfSense.localdomain 1.393ms 1: pfSense.localdomain 1.054ms 2: ubuntuserver.localdomain 1.252ms asymm 1 3: 10.80.0.11 50.313ms asymm 2 4: 10.0.3.2 45.226ms reached Resume: pmtu 1500 hops 4 back 3 -
ping and traceroute maybe do well. ICMP is a stateless protocol. The problems with that come if you establish a stateful connection.
So I'd try one of the suggestions.
-
ping and traceroute maybe do well. ICMP is a stateless protocol. The problems with that come if you establish a stateful connection.
So I'd try one of the suggestions.
OK, thanks for bearing with me! That one got through, I can just about picture how that makes a difference with how things are flying around.
