Installing WireGuard VPN
juppin last edited by juppin
Hello pfSense community,
as i have many mips routers as OpenVPN client boxes and the best router with a dual core mips 1004k 880 MHz cpu does only reach a max throughput of 15MBit/s on aes256-gcm.
The limitation / bottleneck is the user space implementation of OpenVPN, WireGuard does run almost in kernel space and does not need that much context switches.
Some guy in the lede-project.org forum reported that with WireGuard VPN the throughput is about 80 MBit/s with such a low powered mips router.
I thought about trying wireguard on my pfSense (FreeBSD) gateway...
WireGurad was ported in may 2018 to FreeBSD, so it should be possible to get this running on pfSense...
Info from FreeBSD mailing list: https://lists.freebsd.org/pipermail/freebsd-ports/2018-May/113434.html
Is there anyone that has tried to install WireGurad from a non pfSense repo?
Is there anything that i should consider to install it from non pfSense package repos?
As i see there is only the dependency of bash and this is already included in pfSense...
Would be great to see a pfSense Package for WireGuard soon :-)
ekke last edited by
This is a high priority feature!
It is? Wow didn't know you were part of the development team and or steering committee of netgate/pfsense ;)
Pretty strange 1st post for someone so well connected.
Did you mean to say you would would like to see this as well ;)
ekke last edited by
Naturally I only speak for me self , aswell as everyone else except those with official mandate for a specific group :)
It will never be a "high priority feature" until they actually make a proven secure/stable release.
About The Project
Work in Progress
WireGuard is not yet complete. You should not rely on this code. It has not undergone proper degrees of security auditing and the protocol is still subject to change. We're working toward a stable 1.0 release, but that time has not yet come. There are experimental snapshots tagged with "0.0.YYYYMMDD", but these should not be considered real releases and they may contain security vulnerabilities (which would not be eligible for CVEs, since this is pre-release snapshot software). If you are packaging WireGuard, you must keep up to date with the snapshots.
However, if you're interested in helping out, we could really use your help and we readily welcome any form of feedback and review. There's currently quite a bit of work to do on the project todo list, and the more folks testing this out, the better.
It may be fast, but that means nothing with all of those disclaimers around it.
juppin last edited by
A member of the US Government endorsing it that enthusiastically could just as well mean they have backdoored the encryption or that it has a known flaw they can exploit.
It's just not ready for prime time yet. Give it time to mature, get audited properly.
Pushing this hard for it only makes it seem even more suspicious.
Wordo last edited by
This comes when starting wg-quick on FreeBSD:
[#] wg-quick tun
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
W This is alpha software. It will very likely not G
W do what it is supposed to do, and things may go G
W horribly wrong. You have been warned. Proceed G
W at your own risk. G
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
As OP pointed out, there is a Wireguard port for FreeBSD (but not pfSense, yet). Note that this port has dependencies on Go and bash and uses an implementation (in 'Go') over tun. There is a thread on the FreeBSD mailing lists with more info.
Further through that thread, there is a performance claim that Wireguard on FreeBSD 'seems to be faster than OpenVPN'. Note that the posting doesn't say how much faster.
There may be some flaw in their benchmarking, as the Wireguard site claims 1011mbps on a 1gbps Ethernet NIC using iperf3. As previously discussed on this forum, reddit and elsewhere, it's impossible to pass more than 949mbps over IPv4 and TCP on a 1gbps NIC.
Max payload on Ethernet is 1500 bytes (yes, I know these are actually octets).
Preamble + SFD = 8 bytes
MAC DST + SRC = 6 bytes each for 12 bytes.
Ethernet Type = 2 bytes
FCS = 4 bytes
Intra-Frame Gap = 12 bytes of (otherwise empty on the wire) time.
1500+8+12+2+4+12 = 1538
Sending TCP over IPv4 consumes another 20 bytes each for the IPv4 and TCP headers. Thus, we can send 1460 bytes of TCP for 1538 bytes 'on the wire'.
1460/1538 = .94928, so max throughput possible with iperf3 over TCP and IPv4 on a 1gbps NIC is 949.28mbps.
According to this post by the primary author of Wireguard:
The overhead of WireGuard breaks down as follows: 20-byte IPv4 header or 40 byte IPv6 header 8-byte UDP header 4-byte type 4-byte key index 8-byte nonce N-byte encrypted data 16-byte authentication tag So, if you assume 1500 byte ethernet frames, the worst case (IPv6) winds up being 1500-(40+8+4+4+8+16), leaving N=1420 bytes. However, if you know ahead of time that you're going to be using IPv4 exclusively, then you could get away with N=1440 bytes.
Modifying the math above, for a 1440 byte inner frame, and adding the 38 bytes of headers (and time) on Ethernet, we get to send 1440 bytes/1538 bytes. If the inner packet is TCP over IPv4, then an additional minimum of 40 bytes will be used for the inner IPv4 and TCP headers.
The result is that we send 1400 bytes of TCP payload and additional overhead totaling 1538 bytes on the wire at 1gbps, so the maximum bandwidth obtainable using Wireguard is 1400/1538 x 1gbps or 910.27mbps.
Moving on, the packet overheads for IPsec are actually ever so slightly lower than for Wireguard.
Additional overhead from ESP includes 20 bytes for an outer IP header, 8 bytes for an ESP header, 2 bytes for padding length & next header type, 16 (AES-CBC) or 8 (AES-GCM) bytes for an initialization vector, and 12 (HMAC-SHA1) or 16 (AES-GCM) bytes for an integrity check value. The total extra overhead is 58 bytes (AES-CBC HMAC-SHA1) or 54 bytes (AES-GCM).
Thus your 1460 becomes 1404. 1404 / 1538 = .9128, and we see that AES-GCM won’t flow more than 912.8mbps. For AES, CBC mode requires padding input to the block size, thus GCM mode produces smaller output if input is not multiple of block size. I may be flubbing something with required multiple of 16 bytes and padding for CBC:
87*16 = 1392 88*16 = 1408
so it might be (and probably is) 1392/1538 or 905.07mbps. It's far too late at night to dive much deeper and figure out which is true. In either case, the actual protocol overheads for both Wireguard and IPsec are incredibly similar, so there isn't a protocol advantage for either.
If you skipped past all that, the summary is:
Wireguard protocol overhead = 20+8+4+4+8+16 == 60 bytes for IPv4 IPsec protocol overhead = 58 bytes (AES-CBC HMAC-SHA1) or 54 bytes (AES-GCM) (both IPv4)
To be absolutely fair, Wireguard uses UDP encapsulation to get past NAT devices, and one would need to use UDP-Encapsulated ESP Headers (aka "NAT Traversal") as documented in RFC 3948 to provide an equivalent solution. pfSense has this, but it costs another 8 bytes. I don't know if it was used for the Wireguard performance testing though. In this case, AES-GCM overhead would be 62 bytes, .vs Wireguard's 60 bytes of framing overhead.
In addition to the per packet overheads due to framing, there are other overheads for traditional (policy-based) IPsec that will slow the packet processing down. Policy-based IPsec encrypts and encapsulates a subset of traffic flowing through an interface according to a defined policy. On a router running IOS or JunOS, this is often an access list. On FreeBSD and linux, it is implemented as something strongly resembling a list of firewall rules. There is additional overhead for traversing this list of 'policies'.
In contrast to a policy-based VPN, a route-based VPN employs routed tunnel interfaces as the endpoints of the virtual network. All traffic passing through a tunnel interface is placed into the VPN. Rather than relying on an explicit policy to dictate which traffic enters the VPN, static and/or dynamic IP routes are used to direct the desired traffic through the VPN tunnel interface. To be clear, both route-based and policy-based IPsec uses the same framing 'on the wire'.
Wireguard is a route-based VPN, and since there is less processing per packet with a route-based VPN such as Wireguard, I would expect a performance increase .vs a policy based VPN over the same link(s).
Fortunately, with pfSense 2.4.4, we have implemented route-based IPsec (based on the work in FreeBSD 11.2). This has occurred earlier than I promised in May of 2017. While we've not tested performance yet, I expect the route-based solution to be more performant given the explanation above.
TNSR also implements route-based IPsec, and we've measured 36.32 gbps (8 concurrent sessions) throughput on 40gbps NICs using AES-CBC + HMAC-SHA1 and QAT offload. We've also measured 13.70 gbps single-stream for AES-GCM using just AES-NI on an i7-6950x over those 40gbps NICs. (All benchmarking in this post is with max-sized packets.)
There are other issues. Wireguard tested AES-GCM-256 against 256-bit ChaCha20 with Poly1305 for MAC. AES-GCM-256 is roughly 20% slower than AES-GCM-128, yet Chacha20-Poly1305 is functionally comparable to AES128-GCM (giving confidentiality and integrity), but is easier to implement securely and efficiently, especially without AES support (such as AES-NI) in hardware. Still, pitting ChaCha20/Poly1305 against AES-GCM-256 in a benchmark seems a bit like stacking the deck to me.
Wireguard also currently has performance issues at moderate packet rates. See this post (and it's responses) on the Wireguard list. FYI, 2.5Mpps is less than 1.7gbps of tinygrams, and a bit less than 25gbps of throughput assuming 1400 byte frames, but you won't run the crypto at 25gbps. TNSR will do 3.3 - 3.9Mpps with AES-GCM-128 (AES_NI) or AES-CBC + HMAC-SHA1 (QAT) (faster QAT cards exist than what we tested with.)
Finally, the Wireguard kernel implementation for linux, and the userspace implementation are both GPLv2. This presents an additional hurdle to anyone implementing a kernel variant of Wireguard for FreeBSD (or any BSD), as GPL code can't be upstreamed to the project if it's part of base. Until someone has the time to implement a ground-up rewrite of Wireguard, (and security evaluation of same), this serves as an effective barrier to a high-performance implementation of Wireguard on FreeBSD and pfSense. (One could, perhaps, leverage the route-based IPsec interface code as a blueprint for implementing Wireguard's interfaces, and use the GPL userspace code, but I doubt the FreeBSD project would be in-favor of upstreaming just the kernel components of same.)
When you add all the above to the concerns that @jimp expressed, the result is, quite simply: "Not Yet".
THAT ALL SAID, one could make a pfSense package that included (or depended on a separate package that implemented) the requisite GUI changes.
Fantastic post Jim - thanks!!!
yon 0 last edited by
me too, i need wireguard. i have test it, faster than openvpn 2.4 now.