WAN to WireGuard to LAN reply-to bug
-
WAN-to-WireGuard-to-LAN
reply-to not working
After doing much testing and trying to do several sanity checks I am concluding that I have found a bug. The bug I'm 99% sure is an issue with pfSense/WireGuard (or I finally lost my mind in the networking world and nothing makes sense anymore).
I have tested this scenario flipping all kinds of checkboxes and switches. I have been pulling my hair out for the better part of a week.
Note, the following scenario works with OpenVPN, however, I REALLY love how WireGuard is much simpler and faster and am trying to migrate to using WireGuard instead.
Please help!
The Summary
Placing a port-forward entry on one site that is meant to cross the WireGuard VPN does not treat reply-to packets 100% correctly. See the attached image for an example flow of working traffic and failed traffic.
The setup:
Version: 2.6.0-RELEASE
Patches: All Recommended Applied.
- pfSense Site 1
- Assigned Interfaces
- WAN PubOne.domain
- LAN 10.1.1.1/24
- WireGuard 10.0.0.1/29
- WireGuard Config:
- Interface Group Membership: Only Unassigned Tunnels
- Public key: 8cky....
- Port 51820
- Peers
- Endpoint: PubTwo.domain:51820
- Public key: qX2z....
- Pre-shared key: uyfx....
- Allowed IP: 0.0.0.0/0
- Routing:
- Gateways:
- Interface: WireGuard
- Disable Gateway Monitoring
- Disable Gateway Monitoring Action
- Gateway: 10.0.0.2/29
- Static Routes:
- Destination Net: 10.2.2.0/24
- Gateway: WatchGuard - 10.0.0.2/29
- Gateways:
- NAT:
- Port Forwards:
- WAN / TCP / * / * / WAN address / 443 / 10.2.2.123 / Forward HTTPS to Server
- Port Forwards:
- Firewall:
- WAN:
- IPv4 UDP / * / * / WAN address / 51820 / * / none / _ / Allow WireGuard
- Port-Forward Auto Added Rule
- WAN:
- Assigned Interfaces
- pfSense Site 2
- Assigned Interfaces:
- WAN PubTwo.domain
- LAN 10.2.2.1/24
- WIREGUARD 10.0.0.2/29
- WireGuard Config:
- Interface Group Membership: Only Unassigned Tunnels
- Public key: qX2z....
- Port 51820
- Peers
- Endpoint: PubOne.domain:51820
- Public key: 8cky....
- Pre-shared key: uyfx....
- Allowed IP: 0.0.0.0/0
- Routing:
- Gateways:
- Interface: WIREGUARD
- Disable Gateway Monitoring
- Disable Gateway Monitoring Action
- Gateway: 10.0.0.1/29
- Static Routes:
- Destination Net: 10.1.1.0/24
- Gateway: WatchGuard - 10.0.0.1/29
- Gateways:
- Firewall:
- WAN:
- IPv4 UDP / * / * / WAN address / 51820 / * / none / _ / Allow WireGuard
- WAN:
- Assigned Interfaces:
The Fix
None that I know of currently.
The workaround
If you use the following config to amend the above config it will work as expected. However, this is undesired as it now filters out the source IP and the server in question would like to know that information to protect itself from brute-force attacks.
- pfSense Site 1
- NAT:
- Outbound:
- Hybrid Outbound NAT
- WIREGUARD / any / * / 10.2.2.123 / 443 / WIREGUARD address / * / Mix Port / Reply-to Work Around
- Outbound:
- NAT:
- pfSense Site 1
-
I am mostly submitting this here as a requirement by netgate. If I don't get any responses that push the issue towards a resolution in the next couple days I'm probably going to submit this to their Redmine bug tracking platform. Please, if anyone has ideas, or maybe they can test it themselves to either back me up or prove me wrong I would greatly appreciate it!
-
I have conducted a lot more testing and came to another discovery.
The reply-to ONLY works if the interface has a gateway assignment.
See below for the new edits.
Setting the interface with a gateway assignment is unnecessary in my case. As I understand it, setting the interface gateway only sets up outbound NAT translations. I suppose there might be another trigger happening in the background creating a reply-to trigger as well.
I additionally set my Outbound NAT to Manual and disabled the auto generated NAT entries that I did not need.
This work around is more desirable as the Source IP from incoming connections is now reserved and the connection is acting as intended now.
The workaround
Workaround 1 (Recommended)
This workaround is pretty much a fix, however since it is still a workaround I can not consider this a fix. Use the following config to amend the above config and it will work as intended.- pfSense Site 1
- Assigned Interfaces:
- WIREGUARD:
*IPv4 Upstream gateway: WIREGUARD_GATEWAY
- WIREGUARD:
- (Optional, Advanced) NAT:
- Outbound:
- Set NAT to Manual and Disable all automatioly created rules for the WIREGUARD interface. Doing this will allow the source IP from outbound connections to not be affected. In my case I need them, in your case you may not.
- Outbound:
- Assigned Interfaces:
Workaround 2 (NOT Recommended)
If you use the following config to amend the above config it will work as expected. However, this is undesired as it now filters out the source IP and the server in question would like to know that information to protect itself from brute-force attacks.- pfSense Site 1
- NAT:
- Outbound:
- Hybrid Outbound NAT
- WIREGUARD / any / * / 10.2.2.123 / 443 / WIREGUARD address / * / Mix Port / Reply-to Work Around
- Outbound:
- NAT:
- pfSense Site 1
-
Upon further reading on the documentation. When adding a gateway it changes the "type" of interface to WAN Type.
https://docs.netgate.com/pfsense/en/latest/interfaces/wanvslan.html
When reading the above, it seems that the WireGuard interface is not properly being set to a mixed VPN Interface.
Then also it states in the documentation that this is expected behavior when it comes to WireGuard. So, at the least some additional wording might be in order in the interface info. And at the most, more information on how to create the reply-to rules or a checkbox that does so would be in order. This all in the effort to make doing manual outbound NAT editing less of a need.
-
Posted on the Redmine:
https://redmine.pfsense.org/issues/14200 -
Since I cannot edit the original posts:
Adding corrections here:
Workaround 1 (Recommended)
- pfsense Site 2
pfsense Site 1- EDIT: (Optional, Easy) (Thank you _Giam on reddit.) NAT:
- Outbound:
- Set NAT to Hybrid and create one rule for the WIREGUARD interface with Do not NAT checked, source any, destination any.
- Outbound:
- EDIT: (Optional, Easy) (Thank you _Giam on reddit.) NAT:
- pfsense Site 2
-
@carrnelltech said in WAN to WireGuard to LAN reply-to bug:
I have been pulling my hair out for the better part of a week.
Me too, but I have to train my opportunities for searching in this forum... Smiley!
I have been build the same setup, posted here: Port forwarding through WG tunnel missing reply-to
Workaround 1 works for me! Thanks you (and _Giam)!
-
B Bronko referenced this topic on
-
Thank you, @carrnelltech , that was really really helpful. I think this is, at the end of the day, the fundamental problem behind my question a week or so ago: https://forum.netgate.com/topic/183150
This thread explains why I haven't been able to complete the red path in your diagram without the hybrid outbound NAT configuration that I currently have. I didn't understand ,though, why you consider that outbound NAT a security vulnerability? Surely, as far as any system located beyond the [1] WAN interface (as per your diagram) would only care that the traffic originates from your public IP address?
-
I was puzzled by the few seconds about outbound NAT specified here: https://youtu.be/8jQ5UE_7xds?si=0vdQagyJMyYJUOzJ&t=563 but this seems to be the same thing, right?
-
@dangersheep said in WAN to WireGuard to LAN reply-to bug:
I was puzzled by the few seconds about outbound NAT specified here: ... but this seems to be the same thing, right?
No, it isn't. It is what @carrnelltech described as Workaround 2 above which results in an address translation on pfsense Site 1 and so we loose the real source IP on pfsense Site 2, but it was the request to keep real source IP on pfsense Site 2 at server, because of brute-force protection or something like fail2ban, greylisting, etc...
Those security protections that you mislead here:
@dangersheep said in WAN to WireGuard to LAN reply-to bug:
I didn't understand ,though, why you consider that outbound NAT a security vulnerability?
@carrnelltech said in WAN to WireGuard to LAN reply-to bug:
EDIT: (Optional, Easy) (Thank you _Giam on reddit.) NAT:
I think worth to Link mentioned reddit post from above and it ends up by SgtKilgore406:
THANK YOU!! I was able to get port forwarding working with this documentation. It seems like such a stupid omission in hindsight but that was the issue. I had been pulling my hair out for months and leaving my VPS instances sitting around doing nothing.
Can finally move forward with migrating away from OpenVPN tunnels to WireGuard.Yo, bro!
-
Ah yes, I forgot to post a link to the reddit thread as well. Thank you!
