Policy-based Routing (outbound) and port forwarding (inbound) through WG tunnel
-
I just switched from IPsec to WireGuard with pfsense 2.5 on both sides for my site-to-site tunnel. Traffic from the subnets on either side to the subnets on either side work just fine. Port forwarding (for inbound connections) and reply-to works beautifully as well. Gateway monitoring also works so the WG gateways on both sides are ONLINE.
My problem is with PBR. It seems to be a hit and miss. As a test, on the local end I created a LAN rule for one client source IP to any destination and set it to use the WG gateway and on the remote end I set an outbound NAT rule for the whole source subnet (of the local end) on the remote end's WAN interface.
I did a packet capture on the local end WG interface and saw the packets reaching that interface so that means that the packets are being routed correctly. However, when I do a packet capture on the remote end's WG interface with the same source IP, I don't see any results.
This was all working properly with OpenVPN. Any ideas?
-
just a guess: did you allow the interresting traffic in the correct firewall rules section for wireguard?
-
@pete35 said in Policy-based Routing through WG tunnel:
just a guess: did you allow the interresting traffic in the correct firewall rules section for wireguard?
Oops, forgot to mention that yes I did. I have an any-any rule in the WG interface rules tab AND left the WG group tab rules empty. This is the recommended way of doing it so that reply-to's will work.
-
So I'm not sure if it's a PBR problem on 2.5 or a WG issue. I saw a recent thread in reddit regarding PBR issues for OpenVPN as well after upgrading to 2.5, if that matters.
-
Do you guys have any remote ideas for this? I'd really want to make this work just like OpenVPN.
-
@kevindd992002 I haven't set up a site-to-site Wiregurad tunnel yet so I'm not going to be much help. But I thought I'd pass this along in case you haven't seen it. Hope it helps!
https://docs.netgate.com/pfsense/en/latest/recipes/wireguard-s2s.html -
@kevindd992002 I have another thread on here about my problems with WG on PF. I have a PBR rule but I can't even get the tunnel stable enough to test the rule. If I do I'll come report how it goes.
G
-
@kevindd992002 Tom over at Lawrence Systems just put out a YouTube video for Wireguard site to site.
https://youtu.be/ZY49EAMnniY
-
@dma_pf said in Policy-based Routing through WG tunnel:
@kevindd992002 I haven't set up a site-to-site Wiregurad tunnel yet so I'm not going to be much help. But I thought I'd pass this along in case you haven't seen it. Hope it helps!
https://docs.netgate.com/pfsense/en/latest/recipes/wireguard-s2s.htmlThanks but I already read all the official documentation while I was setting this up. Coming from OpenVPN, then IPsec, and now WireGuard, I can say that WireGuard is very straightforward to setup so the chance of messing something up is lesser.
@xxgbhxx said in Policy-based Routing through WG tunnel:
@kevindd992002 I have another thread on here about my problems with WG on PF. I have a PBR rule but I can't even get the tunnel stable enough to test the rule. If I do I'll come report how it goes.
G
Thanks. I'll check your thread and maybe I can help. I have everything stable with WG except for PBR's.
@hypnosis4u2nv said in Policy-based Routing through WG tunnel:
@kevindd992002 Tom over at Lawrence Systems just put out a YouTube video for Wireguard site to site.
https://youtu.be/ZY49EAMnniY
I just watched the video. I have the exact same setup as his because one side is behind a CGNAT and the other side has a static public IP. I have the same settings except for a few things:
-
In the Peer WG Address field of each side's peer settings, I make sure to specify a single IP address without the CIDR (/32) notation. It works both ways because /32 is basically a single IP address also but it just makes it more simple if you use a single IP and this is what's documented in the official pfsense WG S2S article anyway.
-
I do not have keep alive on both sides enabled and it works just fine because of two things:
a. If the IP you set in the Peer WG Address field is included in the IP subnet settings of the actual WG interface, a gateway is automatically created and gateway monitoring is enabled by default. The monitoring itself is already the keep alive. This is documented here.
b. In the side with a static public IP, if you keep the Endpoint field blank this pfsense box WILL NOT initiate traffic to the remote peer until the remote peer sends traffic. This is documented here. -
For gateway monitoring to work, the peer settings Allowed IPs field should contain the IP of the remote peer's WG interface on top of the subnets in the remote peer side that you want to route.
-
-
I edited my OP and included a diagram of my setup. It's fairly a very basic setup and I have some additional weird observations with PBR and now port forwarding not working properly.
So again, for the topic related to PBR, here's what I have:
- Site B LAN tab rules
- Site A Outbound NAT rules
* Outbound NAT rule on WAN interface to translate packets with source IP = 192.168.20.0/24 to have a source IP = WAN address interface (static public IP)Observations:
- 192.168.20.10 gets routed out Site A's WAN interface properly without any issues
- 192.168.20.11 does not get routed properly. I see packets reaching the Site B WG0 interface but I don't see anything in Site A's WG interface
Those PBR's are all for outbound. Now let's do inbound (port forwarding). Here's what I have on the topic related with port forwarding:
- Site A Port Forward rules
Observations:
- I tested first using my usual external open port test site (tool1). Everything works as expected! I can reach both 192.168.20.10 and 192.168.20.11 (Site B clients) through Site A's WAN interface. So I thought everything was working properly.
- I then tested with another external open port test site (tool2) and now it's not working. The same exact behavior happens like the PBR issue above and that is the inbound packets reaches Site A's WG0 interface but stops there. Site B's WG0 interface never sees these packets.
So from my observations above, I can conclude that, for some odd reason, both PBR and port forwarding work with the "first" source IP and they don't with succeeding source IP's, if that even makes sense. The routing for both PBR and port forwarding is basically the same but only reverse of each other. This also proves that both pfsense boxes exhibit the problem.
@jimp @stephenw10 do you have any ideas?
-
Workaround for the PBR issue
Workaround for the port forwarding issue
I created WG0 interface outbound NAT rules to source translate inbound packets destined to 192.168.20.10:8989/192.168.20.11:8990 (from any) and it worked.These should totally be unnecessary with WireGuard since it supports reply-to. It worked for the first local source IP (PBR issue) and external source IP test (port forwarding issue) which proves that reply-to does work. So there's got to be a bug somewhere here.
-
@kevindd992002 I have exactly the same issue here - https://forum.netgate.com/topic/160335/unsolved-possible-bug-wireguard-routing-weirdly/29
I'll try your fix and see if it works.