Wireguard poor throughput.

stephenw10

MSS only applies to TCP traffic so if the fragmentation is in the UDP Wireguard packets it might not help. Though if that was the case I might expect it to fail more spectacularly!

michmoor

@stephenw10 MSS does apply to TCP connections but i thought for tcp conversations going through the wireguard tunnel, that is where the firewall will step in and say "nope instead of 1460 lets do 1300" and send that new MSS signaled value to the other side.
MTU value is for the total size of the frame before going through the tunnel (additional encap) .

stephenw10

Exactly, it should. And that would reduce the size of the Wiregurad packets accordingly which you would expect to pass. But if you are passing any large udp packets it would do nothing.
It's worth running a packet capture to check. It's usually pretty obvious if it is fragmentation.

Steve

michmoor

@stephenw10
Am I running a pcap on the Wireguard interface?
What should i look for to find fragmentation?

stephenw10

I would run it on the wireguard interface to look at traffic inside the tunnel and also on the interface wireguard is running on to check for fragmented wg traffic.
Packet fragments are shown like:

16:06:11.757955 IP 172.21.16.206 > 172.21.16.226: ICMP echo request, id 64544, seq 2, length 1480
16:06:11.757959 IP 172.21.16.206 > 172.21.16.226: ip-proto-1
16:06:11.758319 IP 172.21.16.226 > 172.21.16.206: ICMP echo reply, id 64544, seq 2, length 1480
16:06:11.758325 IP 172.21.16.226 > 172.21.16.206: ip-proto-1

That's a 2000B ping over a 1500B link.

Steve

michmoor

@stephenw10 Update on this. Ran an iPerf test between two machines on two different vlans at the site with 500/500. I get excellent throughput ~940Mbps.
At the site with 200/35 bandwidth, i ran an iPerf test between a server and the firewall and im getting ~455Mbps...
Not sure what has changed between the IPsec and Wireguard change but clearly the issue seems to be local to the site. Will have to investigate.

stephenw10

Not necessarily. Running iperf to or from the firewall directly will always be a worse result than through it. Usually that's because the single threaded nature of iperf can use all of one CPU core on the firewall.
What hardware is the firewall at that end?

Steve

michmoor

@stephenw10
Intel(R) Celeron(R) CPU J3060 @ 1.60GHz (2 cores, 2 threads)

In theory that should be more than enough,
Monitoring shows very good cpu utilizaiton.

tquade

@michmoor The graph looks as if the CPU became less idle at the end of the test. Seems a bit odd.

Ted

michmoor

@tquade This wasnt during any test just over the span of 3hrs.

stephenw10

Run top -HaSP on it during the test and see what's actually happening. I'm betting one core will be pegged at 100%.