Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Wireguard poor throughput.

    Scheduled Pinned Locked Moved General pfSense Questions
    20 Posts 3 Posters 2.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Ok so you should be able to see >100M from site2 to site1 if latency allows it and IPSec throughput shows it does. I assume you were seeing at least that in the test outside the tunnel?

      50Mbps sure looks like it could be packet fragmentation. Are you able to pcap for it to check?

      There's really not that much you can tweak in Wireguard.

      Steve

      M 1 Reply Last reply Reply Quote 0
      • M
        michmoor LAYER 8 Rebel Alliance @stephenw10
        last edited by

        @stephenw10 Fragmentation is my thought as well BUT if im already setting the MSS to 1300 and im not sure what more i can do. Should I adjust the interface MTU as well?
        Latency is unchanged 33ms.

        Firewall: NetGate,Palo Alto-VM,Juniper SRX
        Routing: Juniper, Arista, Cisco
        Switching: Juniper, Arista, Cisco
        Wireless: Unifi, Aruba IAP
        JNCIP,CCNP Enterprise

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          MSS only applies to TCP traffic so if the fragmentation is in the UDP Wireguard packets it might not help. Though if that was the case I might expect it to fail more spectacularly!

          M 1 Reply Last reply Reply Quote 0
          • M
            michmoor LAYER 8 Rebel Alliance @stephenw10
            last edited by

            @stephenw10 MSS does apply to TCP connections but i thought for tcp conversations going through the wireguard tunnel, that is where the firewall will step in and say "nope instead of 1460 lets do 1300" and send that new MSS signaled value to the other side.
            MTU value is for the total size of the frame before going through the tunnel (additional encap) .

            Firewall: NetGate,Palo Alto-VM,Juniper SRX
            Routing: Juniper, Arista, Cisco
            Switching: Juniper, Arista, Cisco
            Wireless: Unifi, Aruba IAP
            JNCIP,CCNP Enterprise

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Exactly, it should. And that would reduce the size of the Wiregurad packets accordingly which you would expect to pass. But if you are passing any large udp packets it would do nothing.
              It's worth running a packet capture to check. It's usually pretty obvious if it is fragmentation.

              Steve

              M 1 Reply Last reply Reply Quote 0
              • M
                michmoor LAYER 8 Rebel Alliance @stephenw10
                last edited by

                @stephenw10
                Am I running a pcap on the Wireguard interface?
                What should i look for to find fragmentation?

                Firewall: NetGate,Palo Alto-VM,Juniper SRX
                Routing: Juniper, Arista, Cisco
                Switching: Juniper, Arista, Cisco
                Wireless: Unifi, Aruba IAP
                JNCIP,CCNP Enterprise

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  I would run it on the wireguard interface to look at traffic inside the tunnel and also on the interface wireguard is running on to check for fragmented wg traffic.
                  Packet fragments are shown like:

                  16:06:11.757955 IP 172.21.16.206 > 172.21.16.226: ICMP echo request, id 64544, seq 2, length 1480
                  16:06:11.757959 IP 172.21.16.206 > 172.21.16.226: ip-proto-1
                  16:06:11.758319 IP 172.21.16.226 > 172.21.16.206: ICMP echo reply, id 64544, seq 2, length 1480
                  16:06:11.758325 IP 172.21.16.226 > 172.21.16.206: ip-proto-1
                  

                  That's a 2000B ping over a 1500B link.

                  Steve

                  M 1 Reply Last reply Reply Quote 1
                  • M
                    michmoor LAYER 8 Rebel Alliance @stephenw10
                    last edited by

                    @stephenw10 Update on this. Ran an iPerf test between two machines on two different vlans at the site with 500/500. I get excellent throughput ~940Mbps.
                    At the site with 200/35 bandwidth, i ran an iPerf test between a server and the firewall and im getting ~455Mbps...
                    Not sure what has changed between the IPsec and Wireguard change but clearly the issue seems to be local to the site. Will have to investigate.

                    Firewall: NetGate,Palo Alto-VM,Juniper SRX
                    Routing: Juniper, Arista, Cisco
                    Switching: Juniper, Arista, Cisco
                    Wireless: Unifi, Aruba IAP
                    JNCIP,CCNP Enterprise

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Not necessarily. Running iperf to or from the firewall directly will always be a worse result than through it. Usually that's because the single threaded nature of iperf can use all of one CPU core on the firewall.
                      What hardware is the firewall at that end?

                      Steve

                      M 1 Reply Last reply Reply Quote 0
                      • M
                        michmoor LAYER 8 Rebel Alliance @stephenw10
                        last edited by

                        @stephenw10
                        Intel(R) Celeron(R) CPU J3060 @ 1.60GHz (2 cores, 2 threads)

                        In theory that should be more than enough,
                        Monitoring shows very good cpu utilizaiton.

                        00c95b63-6303-4b26-b297-3dd1d91d8ccb-image.png

                        Firewall: NetGate,Palo Alto-VM,Juniper SRX
                        Routing: Juniper, Arista, Cisco
                        Switching: Juniper, Arista, Cisco
                        Wireless: Unifi, Aruba IAP
                        JNCIP,CCNP Enterprise

                        T 1 Reply Last reply Reply Quote 0
                        • T
                          tquade @michmoor
                          last edited by

                          @michmoor The graph looks as if the CPU became less idle at the end of the test. Seems a bit odd.

                          Ted

                          M 1 Reply Last reply Reply Quote 0
                          • M
                            michmoor LAYER 8 Rebel Alliance @tquade
                            last edited by

                            @tquade This wasnt during any test just over the span of 3hrs.

                            Firewall: NetGate,Palo Alto-VM,Juniper SRX
                            Routing: Juniper, Arista, Cisco
                            Switching: Juniper, Arista, Cisco
                            Wireless: Unifi, Aruba IAP
                            JNCIP,CCNP Enterprise

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Run top -HaSP on it during the test and see what's actually happening. I'm betting one core will be pegged at 100%.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post
                              Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.