Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    HP T730 Routing vs IPSEC vs OpenVPN vs Wireguard

    Hardware
    2
    6
    1.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mikeisfly
      last edited by

      So I built to identical systems to test the performance of Routing vs IPsec vs OpenSense vs Wireguard and are are some numbers that I got. But first the hardware:

      CPU - AMD RX-427BB with AMD Radeon(tm) R7 Graphics
RAM - 16GB DDR3 
Storage - M.2 SATA SSD (Crucial MX500 500GB 3D NAND SATA M.2 (2280SS) Internal)
NIC 0 - Built in Realtek
NIC 1 - Dell Broadcom 57810S Dual Port 10GBASE-T Converged Network Adapter HN10N Low Profile.

      The Setup
      WAN port I used the built in RealTek built in NIC
      P2P I used the first port of the dual 10Gb NIC and connected each machine together (Cable length 6')
      LAN I used the second port on each NIC connected to each laptop with SSD for the storage SATA interface.
      OS - pfSense 2.5.0-RELEASE (Because I wanted to test WireGuard)
      I will test 2.5.1 with the beta WireGuard Package coming soon!

      The bottle neck in this setup are the NICs on the laptops because they are 1Gb. I did buy 2.5Gb USB NICs for the laptops but the Dual 10GB Base-T NIC that I used will only negotiate at 1 or 10 Gb nothing in between. (I did order two Mikrotik switches with SFP+ modules that will Negotiate at 10, 5, 2.5, 1 Gb so I will update my findings once I get them)

      Here are my results:
      Directly connected P2P - 920Mbps
      OpenVPN 256 AES GCM - 336 Mbps
      IPSecVPN 256 AES GCM - 544 Mbps
      Wireguard ************** - 704 Mbps

      Some observations:
      The P2P was limited by the NICs of my laptops I got around 1.3Gbps Iper test pfSense to pfSense. However I want to test PC to PC because I think that number is low.

      All VPN boxes pretty much had consistent transfer speeds never dropping below 5% of their max speeds, I tested the speeds doing a windows SMB transfer across the link. The file size was 5.51 GB .iso of Video Game Roms (I wanted a real world test)

      At first I was getting around 20 Mbps with IPSec and OpenVPN until I set the encryption to use AES-GCM which is accelerated by the AES-NI instruction set on my CPU. Finally while the transfers where happening the CPU went from 1%-3% usage to 36% - 40% usage and the temps when from 36C to 45-47C. I have not measured the power draw of these systems but Server the Home says they draw about 35w underload. 10GBase-T NICs are not that loud barely audible to me at a normal distance in your rack in your garage lets say. You certainly will not here them over your switch if yours has active cooling. One of the systems has a fan that is off balance so the found is loud sometime but a little tap fixes that issue.

      Hope this helps anyone I will report back the routing speeds once I get the Mikrotik switches. I plan on pair these with my Brocard 6740T 48P (48 10Gb ports with PoE) switch. I have FiOS Gigabit Internet, just want to get ready if they offer 2Gb service someday. Currently I have a i5 based pfSense box but didn't want to lower my power draw where I can. I looked at the SG-5100 but that sucker is $700. I got the HP T730 for $99 on eBay and with the savings got a few 10/40GB switches. If I was installing this in a business I would definitely gotten the SG-5100 for warranty and tech support.

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Mmm, I would have expected at or very close to line rate in a routing test, 940Mbps between two local systems. I would suggest a possible issue there.
        Can you see that between the two laptops just with a switch in between?

        Did you try enabling asychronous crypto in the advanced ipsec settings?
        I have seen close to line rate over IPSec with that enabled between SG-5100s. When everything is local.

        Steve

        M 1 Reply Last reply Reply Quote 0
        • M
          mikeisfly @stephenw10
          last edited by

          @stephenw10 said in HP T730 Routing vs IPSEC vs OpenVPN vs Wireguard:

          Mmm, I would have expected at or very close to line rate in a routing test, 940Mbps between two local systems. I would suggest a possible issue there.
          Can you see that between the two laptops just with a switch in between?

          Did you try enabling asychronous crypto in the advanced ipsec settings?
          I have seen close to line rate over IPSec with that enabled between SG-5100s. When everything is local.

          Steve

          Wow Thanks! Turning on asynchronous crypto got my IPsec transfer speeds up to 100 MB/s (And it was locked at 100MB/s). As far as routing performance I'm going to chalk it up to SMB & TCP over head on why I didn't hit the 125 Mbps max. My Mikrotek switches should be arriving today and I got some SFP+ that should let me connect at 2.5Gbps and I will see what the new upper limit it.

          M 1 Reply Last reply Reply Quote 1
          • M
            mikeisfly @mikeisfly
            last edited by

            So I connected both laptops together via a 2.5Gbps usb dongle and Turned off Windows Firewall, this is was I got.

            Direct connection transferred a 22GB file seeing speeds of up to 250 MB/s the more test I did the slower it got. I'm guessing because the USB NICs (Plugable 2.5G USB C and USB to Ethernet Adapter) I was using was getting hot and had to throttle could be the SSDs where throttling too.

            Testing the IPsec tunnel with Windows defender firewall off I was locked at 105-6 MB/s
            WireGuard with Windows defender turned off was 106-110 MB/s
            OpenVPN with Windows defender turned off was on avg 42 MB/s
            Routing using the Gigabit NIC I'm locked at 112-113 MB/s

            Once I get the Mikrotek switches up I will report back, I think the biggest thing that I'm seeing here is that Wireguard is only 1% - 4% faster than IPsec and both of them are 2.5 times faster than OpenVPN.

            stephenw10S 1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator @mikeisfly
              last edited by

              @mikeisfly said in HP T730 Routing vs IPSEC vs OpenVPN vs Wireguard:

              both of them are 2.5 times faster than OpenVPN

              Yeah the mode switching kills OpenVPN performance. Though that's pretty respectable number.

              Wireguard is fast when you consider it's not, yet, accelerated like IPSec is.

              Steve

              1 Reply Last reply Reply Quote 0
              • M
                mikeisfly
                last edited by

                Routing I'm getting about 1712 Mbps avg (Highs in 280s MB/s) but I'm not sure if that is the upper limit or just the limitation of my 2.5Gbps USB NICS. I don't have any 10Gbps Gear other than this firewall and Brocade switches. My Windows 2019 Server is 10Gb as well but I'm thinking that this little box is a winner. I haven't tried NAT performance but I would expect similar results even though NATing I feel like is more resource intensive but this box never gets to 50% utilization. I think the bottle neck is the 2.5Gbps USB NICs or my SK hynix Gold S31 SATA Gen3 2.5 inch SSD or My Samsung 860 Evo SSD, 10 Gb USB NICs are too expensive right now in the neighborhood of $300.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.