(IPsec outbound NAT to interface address) Reply traffic destination IP not being translated back to original source IP

kevindd992002

@stephenw10 said in (IPsec outbound NAT to interface address) Reply traffic destination IP not being translated back to original source IP:

Yeah, that looks good. I see nothing obviously wrong with that walk-through.

Steve

Great! I have a few concerns though:

In the Peer setup, shouldn't the "Peer WireGuard Address" be just a single IP (tunnel interface of the far side peer), instead of the IP in CIDR notation that he did on the video?
When he created the WAN rule for 51820, he noted that he made a mistake about putting 52180 but it still works. Any idea why? Also, why is this WAN rule not automatically defined (but hidden) just like in IPSec? I know in OpenVPN, the explicit WAN rule is also required.
In my use case where I have one side with a static public IP (Main Site) and another side that is behind a CGNAT (so a private IP is assigned on my WAN interace; Remote Site), would it make sense to keep the "Endpoint Address" BLANK on the Main Site peer settings so that it will accept any remote IP AND will not initiate the connection?

stephenw10

The peer IP address is not actually required for WireGuard it routes internally using the allowed IPs vs the peer public keys.
Adding that there adds routes in pfSense automatically. It's a single IP address inside that subnet, I would not expect that to make any difference.

You only actually need a firewall rule on one end to establish the tunnel. The reply traffic then uses that state so traffic goes both ways. It's better to have the rule correct at both ends though so the tunnels can establish from either end.
There are no hidden auto-rules like there are for IPSec. It's like OpenVPN in that regard. It would be better (IMO!) to have IPSec like that too but it would be a POLA violation to change that now.

Yes, with one end behind NAT you can only establish outbound from there and the public IP address may change so I would leave it blank on the opposite end.

Steve

kevindd992002

@stephenw10

Ok, so are you saying that if I put the far end tunnel interface address in the Peer IP Address field but not in the Allowed IP's list (like how it was done it that video), it won't add a route for that tunnel interface address? If so, what is the use the case of the Peer IP Address field if it won't be used anyway?

stephenw10

If you add the peer IP address pfSense will add it as a gateway and you can use it for policy routing or static routes etc. You can do that manually but the scripts try to make it easier. More like OpenVPN behaves.

Steve

kevindd992002

Ok, that makes sense. Then I would need that for my use case because I definitely will use the tunnel interface as gateways for routing.

Is it mandatory to put the the tunnel interface address in the allowed IP's though?

kevindd992002

I figured it out. My observations are:

Remote end tunnel interface IP in "Peer IP Address" field only (tunnel transit network not in "Allowed IP's" field)

Result: Tunnel works, route to far end subnets AND to transit network are automatically created, gateway automatically created, BUT gateway monitoring to remote tunnel interface IP does not work (it doesn't respond to ping).

Remote end tunnel interface IP in "Peer IP Address" field AND tunnel transet network in "Allowed IP's" field

Result: Everything works!

The most important thing for me that solved my return traffic issue is to create a rule in the WireGuard interface tab and NOT on the WireGuard group tab (I left this empty) for reply-to to work. This is documented in the Netgate documentation pages.

My remaining concern now is why do I get these failures in my WireGuard "Client":

They were also in the server side while I'm trying to figure out everything but now they stopped. They continued to show in the client side even though everything seems to be working. Any ideas?

kevindd992002

Also, I noticed that even though I get significant amount of hits in the WireGuard interface tabs on both sides, I only see 16KB for my 51820 allow rule on the server's WAN interface tab:

Server:

Client:

Shouldn't the WAN rule show approximately the same amount of traffic?

stephenw10

If the initial connection to the other side was outbound on that peer (there is no server/client in wg) then return traffic there will use the open state. That firewall rule would not be used to create a state that would that show traffic on it.

Steve

kevindd992002

@stephenw10 said in (IPsec outbound NAT to interface address) Reply traffic destination IP not being translated back to original source IP:

If the initial connection to the other side was outbound on that peer (there is no server/client in wg) then return traffic there will use the open state. That firewall rule would not be used to create a state that would that show traffic on it.

Steve

Yup, I'm aware that there is no server/client in WG but in my use case one side is "acting" like a server and the other side is acting like a client by initiating the outbound connection. No, the initial connection was outbound from the client side (the one behind an ISP NAT, naturally) so the WAN rule I'm showing is for inbound traffic (forward traffic, not return).

stephenw10

Hmm, well I would certainly expect to see traffic on that rule then.
Check the state table for a state open on WAN with that traffic on it.
Something else must be opening it.

kevindd992002

I do see an open state:

client dynamic public IP:random port -> server static public IP:51820

I then disabled the WAN rule and surprisingly the tunnel still worked. This is consistent with the behavior in the Youtube video above wherein he accidentally specified a wrong destination port in the WAN rule but it still worked. Although, the only difference in this case is the open state goes like the other way around:

server static public IP:51820 -> client dynamic public IP:random port

Why did the open state reverse?

I don't have any other rules for dst_port=51820. Here are all my WAN rules:

stephenw10

Run: pfctl -ss -vvvv

Find the state(s) open with port 51820. Note the rule number that opened them.

Now run: pfctl -sr -vvvv

Check what rule opened the state.

Steve

kevindd992002

With the WAN rule enabled, running "pfctl -sr -vvvv" shows the user rule I created and like I said, the open state is as expected:

pfctl -ss -vvvv | grep 51820
igb0 udp {server public IP}:51820 <- {client public IP}:19988       MULTIPLE:MULTIPLE

pfctl -sr -vvvv
@68(1613966106) pass in quick on igb0 reply-to (igb0 {server WAN gateway IP}) inet proto udp from any to {server public IP} port = 51820 keep state label "USER_RULE: pfSense WireGuard Server"
  [ Evaluations: 21757     Packets: 6025      Bytes: 4078532     States: 1     ]
  [ Inserted: pid 28901 State Creations: 1     ]

Right after disabling the rule and deleting the open state on the server, it opens a new state but this time in reverse and naturally no rules are triggered because it becomes an outbound call:

pfctl -ss -vvvv | grep 51820
igb0 udp {server public IP}:51820 -> {client public IP}:19988       MULTIPLE:MULTIPLE

pfctl -sr -vvvv
@68(1613966106) pass in quick on igb0 reply-to (igb0 {server WAN gateway IP}) inet proto udp from any to {server public IP} port = 51820 keep state label "USER_RULE: pfSense WireGuard Server"
  [ Evaluations: 24229     Packets: 6933      Bytes: 4638020     States: 0     ]
  [ Inserted: pid 28901 State Creations: 1     ]

So how do both sides have outbound connections and still achieve a tunnel? I'm just curious here. The open state on the client side never changes direction. It's always client to server and the source and destination port is always 51820.

stephenw10

You have client IP twice in the second state. I assume that's a typo?

Are you saying the state that opens without the WAN rule enabled shows it was created by the same rule number?

Steve

kevindd992002

@stephenw10 said in (IPsec outbound NAT to interface address) Reply traffic destination IP not being translated back to original source IP:

You have client IP twice in the second state. I assume that's a typo?

Are you saying the state that opens without the WAN rule enabled shows it was created by the same rule number?

Steve

Edited, sorry about that.

No. After I disable the WAN rule and delete the open state on the server side, a new state in the server side is recreated but seems to be outbound. So the same rule number does not show it was used because naturally it is outbound. But then after a while (like how I checked again now), the open state on the server seems to be as expected again BUT the rule state details under the WAN tab does not reflect the correct "bytes".

stephenw10

Ok, but the rule that is creating that state is not '@68' I assume?

I imagine the WireGuard process has a record of the client IP and source port and start sending traffic back to it. The 'allow everything out' rule will pass that and create a new state. Since that isn't the rule on WAN that doesn't show the traffic.

If you removed the rule on the server side WAN and then restarted WireGuard or rebooted I would expect it to fail to come back up.

Steve

kevindd992002

@stephenw10 said in (IPsec outbound NAT to interface address) Reply traffic destination IP not being translated back to original source IP:

Ok, but the rule that is creating that state is not '@68' I assume?

I imagine the WireGuard process has a record of the client IP and source port and start sending traffic back to it. The 'allow everything out' rule will pass that and create a new state. Since that isn't the rule on WAN that doesn't show the traffic.

If you removed the rule on the server side WAN and then restarted WireGuard or rebooted I would expect it to fail to come back up.

Steve

Correct, it's not '@68' for sure because the connection was outbound.

What you're saying makes sense. Since the client side has an open state, the server can reach it on that client IP and source port outbound. That's actually clever.

That will be a good test. I will expect it to fail too. I'll try later and report back.

However, do you have any ideas on the inaccurate "bytes" on the rule state details?

And why I see the "matchaddr failed" errors on both sides now?

stephenw10

@kevindd992002 said in (IPsec outbound NAT to interface address) Reply traffic destination IP not being translated back to original source IP:

matchaddr failed

Yeah that should not be there. Do you have more that one peer defined?
https://redmine.pfsense.org/issues/11502

Steve

kevindd992002

@stephenw10

Not at all. I only have one peer serup on both sides.

stephenw10

Hmm