Issue with MTU/MSS in Wireguard tunnel

[tiiash]

I recently configured two pfSense boxes to establish a Wireguard Site2Site VPN following the recipe in the official docs: https://docs.netgate.com/pfsense/en/latest/recipes/wireguard-s2s.html

After some initial issues with routing and firewall rules, I managed to establish a working connection.
I can access the pfSense web interface as well as most other services through the VPN.
However, I cannot establish a connection with the web interface of a Zyxel GS1200 switch.

I found a similiar issue described on the Zyxel forum: https://community.zyxel.com/en/discussion/26399/gs1200-8-webadmin-not-accessible-through-vpn
The user there states that they had to setup a normalization/MSS clamping rule in order to make it work. However they are using OPNsense.

I have the impression that I have a similar issue. Looking at the traffic via a packet capture in pfSense I see the line

10:58:19.813516 IP 10.9.106.249 > 10.9.106.12: ICMP 10.255.102.14 unreachable - need to frag (mtu 1420), length 576

In this setup:

• 10.9.106.249 is my remote pfSense
• 10.9.106.12 is the switch that I try to access
• 10.255.102.14 is my client IP address

How would one achieve a similar configuration as described in the Zyxel forum post in pfSense? I tried "Advanced settings -> Firewall & NAT" -> "activate MSS clamping for VPN connections" but that doesn't seem to make a difference. I also don't know if this would be the correct setting because the description states that the options under "VPN Packet Processing" are for IPSec, OpenVPN and PPPoE traffic (and not for Wireguard, likely because it is currently shipped as a separate package).

[TheNarc]

@tiiash I think you just want to set the MTU and MSS directly on the Wireguard interface:

https://docs.netgate.com/pfsense/en/latest/interfaces/configure.html

[tiiash]

@TheNarc You are right!
I configured the MTU on the Wireguard interface as described in the docs. However, it turns out that in order for getting the Zyxel switch to work I had to also specify a value for the MSS.

For future readers: I set the MSS value to exactly the same as the MTU (1420). The field description states that the actual MSS is calculated by subtracting 40 from this value. Which results in 1380, the same value that can be found in the referenced OPNsense documentation.

Thanks for the quick response :).

[McMurphy]

@tiiash

So you set both MTU & MSS to 1420 in the interface configuration settings?

Why did you make MSS the same?

[tiiash]

@McMurphy exactly.
I started by setting just the MTU (to 1420). This didn't work.
After the reply from @TheNarc I did a test and additionally set the MSS value as well.

Ultimately, you want the real MSS value to be smaller than the MTU (typically 20 bytes for IP header data and 20 bytes for TCP header, so 40 bytes in total).
However, when you read the description field of the MSS value in pfSense it says

If a value is entered in this field, then MSS clamping for TCP connections to the value entered above minus 40 for IPv4 (TCP/IPv4 header size) and minus 60 for IPv6 (TCP/IPv6 header size) will be in effect.

This is why I set the same value as MTU. I actually don't know why this changes things. I would think that implicitly, the MSS should be affected by changing the MTU value. After all, the amount of data that can fit in a TCP segment directly depends on the overall size of the packet minus all headers. I guess that it would probably also work if you only set the MSS (with reverse logic: How should a packet ever get bigger than its payload size plus all headers), but I haven't tested.

I am no network expert however and the finer details of packet delivery are a mystery to me. I am always happy if I can get things to work ;).