Is Wireguard 0.2.11 compatible with 0.2.9.5

cjbujold

We have two boxes (home and work offices), one running PFsense Plus with WireGuard 0.2.11 and the second running the community edition with WireGuard 0.2.9.5.

We are having issues in that the speed and connection between them are erratic and the connections from one side to the other is problematic. Sometimes we can connect, other times we cannot. The speed varies from 1Gbs to 0.5 Mbps between both locations.

Any help would be appreciated.

stephenw10

There should be no problems with compatibility there.

The first thing I would do here is test bandwidth between those sites outside the tunnel to be sure that's stable. A lot of times we see reports of VPN throughput issues it's actually a problem in the route.

However 0.5Mbps is extremely slow. I'd expect to see some sort of errors across the tunnel with that. Packet loss perhaps.

cjbujold

My PfSense setup is :

My PC Nic is a 2.5 GB and the pfSense Nic is a 10GB connected to a 10GB switch. Tested the speed to the Internet to see if it was some sort of communication issue, and I get a steady 2.5GB throughput with no packet lost and a 0.11 and 8 ms average latency. (Similar setup at the other end with similar results and 144 ms latency for down and 9ms for latency up)

The PfSense monitor shows :
Quality for the last 24 hrs to the tunnel:

Local:

Remote:

Packets for the last 24 hrs to the tunnel. Normally, I should see the tunnel do a backup of about 5+ TB during that time period. The packets are averaging about 14 kkps since 11:30pm on both sides of the tunnel.

Local:

Remote:

Here are the monitoring results for the last 24 hrs to the tunnel traffic. Normally, I should see the tunnel do a backup of about 5+ TB during that time period.

Local:

Remote:

stephenw10

I would definitely try a test between the two sites directly but outside the tunnel. You may well see perfect results at each end to speedtest.net but still have a fault in a route between the sites that never hits.

Just running a ping test to look for packet loss would be the first thing I'd try. Then I'd run some iperf3 tests in each direction.

cjbujold

You are correct, we must have some routing issue but not certain where to start. If you can point us in the right direction, we would appreciate it.

We installed IPerf3 and here are the results:

Thanks

tinfoilmatt

@cjbujold This is not a throughput test 'outside the tunnel.' It's not clear what you're testing here, nor what infrastructure might be implicated where.

stephenw10

Yup needs to be outside the tunnel really but inside the tunnel could also prove useful. That doesn't look too bad but check the other way too. By default iperf3 sends traffic from the client to the server. Use the -R switch to reverse that.
Since you're seeing 0.5Mbps on the backup job something must be a lot slower or lossier than 240Mbps. You could run a ping at the same time and see if that loses packets.

cjbujold

Thanks to everybody for your help. We found the solution/fix. Hope this can help others.

Did more testing and did like you suggested for testing outside the tunnel and found no packets lost.

Did some more research and found a performance optimization article for WireGuard which recommended setting the MTU to 1420. Once we changed the MTU setting to 1420 for the WireGuard interface, we are now averaging about 918 Mbps, with peaks at 1.09 Mbps.

It would seem that the default 1500 MTU on the Interface causes packet fragmentation, which slows the performance to a crawl.

tinfoilmatt

@cjbujold You might consider fine-tuning MTU for your infrastructure. There's nothing magical about any particular number of bytes.

cjbujold

@tinfoilmatt Thanks will do.

stephenw10

It should only need to be 1420 if it's carrying IPv6, otherwise 1440 is sufficient. See: https://docs.netgate.com/pfsense/en/latest/vpn/wireguard/assign.html#assignment-procedure

cjbujold

Thanks will adjust to 1440 , only using IPV4

stephenw10

That also assumes a 1500B MTU for the route. If you have PPPoE for example you'd need to allow for that.

cjbujold

Thanks, not using PPPoE, our Fiber connection uses DHCP and is set to 1500. We had to lower it to 1420 due to IPV6 usage.

tinfoilmatt

@cjbujold Is that the only value you tested with?

cjbujold

Yes we tried different MTU but because we have IP6 1420 seems to give us the best performance. We are getting about 1.2Gb on a 1.5Gb connection.

tinfoilmatt

@cjbujold Nice. Ostensible excessive fragmentation nail sledged at the edge!️