Routing via a LAN client like it's a Gateway
-
Hi there,
I'm using Pfsense as the main router/firewall on my home network. I also wanted to try Wireguard VPN, but know it's not anywhere near ready on FreeBSD, and didn't want to go hacking away at my pfsense install. I've instead got a VM running on my LAN with another open source command line based Router distro ( based on Linux ) and have it setup with it's LAN interface & a Wireguard interface. I also tried this with a vanilla linux VM and IP Forwarding + iptables setup with same results as below.If I am on a client PC on the LAN, and I manually set that client PC to use the Wireguard VM's LAN IP as the default gateway ( instead of Pfsense ), it works perfectly. So the Wireguard host is forwarding, NAT'ing etc very nicely, and there is no problem. I know Wireguard is up and working, and that VM is forwarding traffic + NATing.
However, I wanted to use this via Pfsense, so created a new Gateway, and set it's IP to the LAN IP of the Wireguard router VM, and then created a LAN Firewall rule to route whatever hosts/alias over that 'gateway'.
It works - kinda, it's not as snappy and fast as the first method, and on www.speedtest.net, The upload test fails every time with socket errors, Download does work. After a few minutes, the www.speedtest.net site won't load at all. Some sites continue to work, others just won't load. It's flaky, and definitely something is wrong.
I didn't set up any Outbound NAT rules on pfsense, as I assume that would be double NAT'ing? ( the wireguard router VM does NAT on it's external interface ).
How can I debug this? why would the Upload be failing, but not download? I know it's the way I've setup pfsense that is causing the issue and likely i've made some newbie mistake, as when I go direct from a client machine via the Wireguard Router VM, it works perfectly.
TL;DR, I'd like the LAN host to be a Gateway that I can use on LAN firewall rules, and route traffic to it.
Any advice on things I could try would be greatly appreciated.
Seb
-
You may have an asymmetric routing issue.
To solve, configure a VLAN on pfSense and put that VM in this VLAN. Update the gateway setting for that VM.
Then you can route traffic to it by polivy routing rules. -
@viragomann Thanks for this. It seems to have done the trick.
- Created VLAN with ID 100
- Assigned it Interface WG0 with static IP of 192.168.100.1
- Created Gateway on WG0 interface called WGGW and pointed it to 192.168.100.2 ( the Wireguard gateway VM )
- Setup Outbound NAT rule on WAN interface for 192.168.100.2/32 only ( to setup the tunnel over the plain WAN )
At first it wasn't working, but after closing browser and re-opening, it all worked like magic - including upload speed test - no more socket errors.
Thanks again!
Cheers
Seb
-
@Sebastiaan76 would you mind sharing some more details (screenshots maybe?) on how you accomplished this? I'm in a similar situation and am struggling to get things working correctly. Thank you!
-
Sure, happy to help if I can,
Although can you clarify which parts you are having trouble with? The points above should be fairly self explanatory.Other things to consider when using another LAN client to forward traffic through are to also have IP Forwarding enabled on that machine/VM! That nearly tripped me up in the beginning. Or you can use a Routing OS ( i used VyOS in the end for the LAN/Wireguard Gateway - primarily because Wireguard isn't supported on pfsense, and is a bit sketchy on FreeBSD anyway ).
Which parts are you getting stuck on?
-
The technical term for what you created there is a transit network. And yes any time you would be using a downstream router you would need that to be on an isolated transit network.. Hosts that were on your old transit, ie your lan would end up with asymmetrical traffic flow and since a stateful firewall is also involved you could run into state issues as well.
Here maybe the drawing will help
So when on the same network as hosts.. You would have this traffic flow
Client wants to get to somewhere outside pfsense, pfsense says oh need to send that to .2, traffic goes out from .2 to internet but when it comes back to .2, vas sending back to pfsense he says oh this is for .100 he is on my same network - let me send that directly to him.. This would be asymmetrical and never good.
You could fix it by setting .100 to directly go to .2 when wanted to get to xyz outside.. But that is pita, and not scalable, etc..
So you put the downstream router on its own transit network - then you end up with symmetrical traffic flow
So now .100 still sends his traffic to .1 to get to anything off his network. But pfsense says hey you want to go there, I need to send that to 100.2, when the traffic come back to 100.2 he has to send it back to pfsense to get to the 192.168.1/24 network.
Now normally the downstream router doesn't route back through pfsense to get somewhere or tunnel, etc. But yes asymmetrical causes new users to networking lots of grief, even though in routing its a very basic concept.
Another way around the asymmetrical routing problem would be to source nat the traffic from so when sending on to .2 vs the source being .100, you would source nat it so it looks like it came from pfsense .1 address... But this also has its own issues.. Transit network with downstream routers is always the better option.
-
@Sebastiaan76 @johnpoz Thanks for getting back to me so quickly.
Let me quickly describe my setup: I have pfsense (192.168.1.1) running on my home network (192.168.1.0/24). I'm also using ZeroTier to create a virtual network comprised of a number of remote VPSes. For simplicity they are assigned IP addresses from the same subnet as my home network (192.168.1.40-192.168.1.49) and then I have a VM on my home network that bridges the two networks together.
One of the ZeroTier connected machines (192.168.1.47) is a cheap VPS configured as a VPN to forward traffic out. ZeroTier offers clients for IOS and laptops that lets you easily toggle on full tunnel/default route mode that will direct all traffic through this exit node which works as expected.
I'm trying to replicate the same thing the clients do but on pfsense now so that I can route all my home traffic through the VPN. I'm not super experienced with VLANs so I guess I am confused on how to go about placing the VPN machine in the new VLAN.
@johnpoz said in Routing via a LAN client like it's a Gateway:
Another way around the asymmetrical routing problem would be to source nat the traffic from so when sending on to .2 vs the source being .100, you would source nat it so it looks like it came from pfsense .1 address... But this also has its own issues.. Transit network with downstream routers is always the better option.
I did make an attempt at the alternative solution above, but I think I'm still running into the original issue of asymmetric routing - the traffic goes out through the VPN but I think it's getting lost coming back, upload on speedtest.net won't work, and things just generally seem flaky. Here's what I attempted:
I know this is a bit convoluted, and understand if it's a bit out of scope - but any thoughts or tips would be greatly appreciated!
-
@johnpoz Sorry to bump this thread but I have related question. Does this downstream router need to be on its own network, or can it just stay on a VLAN different from the clients I need to route?
Let's say I already have a SERVICES VLAN, none of the hosts on this VLAN will be routed via this Wireguard gateway. Would placing this downstream router on this VLAN solve the asymmetric routing issue you explained?
