Improve Performance in WireGuard
-
BLUF: Slow transfer speeds using WireGuard. Max throughput of 10mbps.
I am currently set up to replicate TrueNAS between two locations. The remote TN is configured to connect to the host pfSense via WireGuard. I have confirmed that the remote TN can connect to and receive data from the host TN. The issue I am having is that WireGuard on the pfSense only allows a max transfer rate of 10mbps. CPU usage on the pfSense device never exceeds 20% and the RAM never exceeds 10% during data transfer. As you can imagine, transferring large amounts of data with these speeds is... less than ideal.
Testing I have done:
Connect another client to the same pfSense remotely, and monitored the speeds of ISO torrent downloads. I also performed multiple iperf tests with the remote TrueNAS and other remote clients. Still limited to 10mbps.
I have changed both MTU and MSS settings on the WireGuard interface on the pfSense device to see if any improvements to the transfer rate was observed. No change.
I have deleted then reinstalled a new WireGuard configuration on the remote TN. No change.
I added, then removed both “net.isr.dispatch deferred” and “ net.isr.maxthreads -1” to System Tunables. No change.
Multiple restarts of WireGuard and the pfSense device. No change.
All systems and software involved are up to date.My question is, are there any ways I might have missed to improve the performance of WireGuard on pfSense?
After a week of searching these forums, blogs, and YouTube videos, I am at a loss. I understand that many people have had issues with WireGuard speeds, but none of the provided solutions have worked for me. If anyone can provide some other useful information, it would be appreciated. Let me know if I need to provide more information, or logs. Thanks in advance.
Network set up:
Local:
pfSense w/ WireGuard “host”
TrueNAS – Replication task via SSH + RSA to offsite TN machine.Remote:
Asus Wireless Router
TrueNAS machine w/ WireGuard connection to Local pfSense firewallpfSense specs:
Netgate 7100
pfSense+ 22.05
CPU Type: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (inactive)
QAT Crypto: Yes (active)
RAM: 8GB -
@fawsewr3 Stuck at 10Mbps seems interesting to me. Almost as if there's a speed or duplex mismatch along the path.
Im running a 6100 and our cpu's are the same and i can achieve around 220Mbps connecting to one of my cloud instances so dont think the issue is on the PF side.
The Remote side, to me, seems suspicious. What sort of Asus hardware is that? All ports are at 1G? -
@michmoor almost forgot. What are the internet speeds local to each site?
-
@michmoor Thank you for the quick reply.
The Asus hardware in question is an RT-AX68U.
Local speeds: 300/300mbps
Remote: 50/50mbps
All devices have 1Gbps NICs.All LAN devices on the local network have no issue with large downloads. Same with all LAN devices at the remote location. The activity of other devices on either LAN have no affect to the WireGuard transfer rate of the two TrueNAS systems.
Please let me know if you have any other questions.
-
@fawsewr3 And youve done an iperf test between the two sites using two different systems?
-
@michmoor I have performed iperf test from local to remote, then remote to local, using various devices. Watching the pfSense dashboard, I can see the throughput of each interface for each test to verify the output of the commands. Each TrueNAS has no issue downloading or uploading data outside of the WireGuard tunnel. I hope this information helps.
-
@fawsewr3 Turns out, it was a limitation by my ISP. Looks like I decided to try ice skating uphill before verifying my network package. Hope people find my mistake useful. Thanks again for the help.
-
@fawsewr3 what was the limitation?
-
@michmoor When I initially set up the account, I falsely assumed the fiber network would be 300 down, 300 up since it is fiber. I had the same set up at a previous location. Turned out it is actually 300 down, 10 up. As usual, the issue was PEBKAC, as I did not do my due diligence researching the ISPs in the area.