Installing WireGuard VPN
-
Hello pfSense community,
as i have many mips routers as OpenVPN client boxes and the best router with a dual core mips 1004k 880 MHz cpu does only reach a max throughput of 15MBit/s on aes256-gcm.
The limitation / bottleneck is the user space implementation of OpenVPN, WireGuard does run almost in kernel space and does not need that much context switches.Some guy in the lede-project.org forum reported that with WireGuard VPN the throughput is about 80 MBit/s with such a low powered mips router.
I thought about trying wireguard on my pfSense (FreeBSD) gateway...
WireGurad was ported in may 2018 to FreeBSD, so it should be possible to get this running on pfSense...
Info from FreeBSD mailing list: https://lists.freebsd.org/pipermail/freebsd-ports/2018-May/113434.html
Package: https://www.freshports.org/net/wireguard/Is there anyone that has tried to install WireGurad from a non pfSense repo?
Is there anything that i should consider to install it from non pfSense package repos?
As i see there is only the dependency of bash and this is already included in pfSense...Would be great to see a pfSense Package for WireGuard soon :-)
Greets
-
This is a high priority feature!
-
It is? Wow didn't know you were part of the development team and or steering committee of netgate/pfsense ;)
Pretty strange 1st post for someone so well connected.
Did you mean to say you would would like to see this as well ;)
-
Naturally I only speak for me self , aswell as everyone else except those with official mandate for a specific group :)
-
It will never be a "high priority feature" until they actually make a proven secure/stable release.
https://www.wireguard.com/#about-the-project
About The Project
Work in ProgressWireGuard is not yet complete. You should not rely on this code. It has not undergone proper degrees of security auditing and the protocol is still subject to change. We're working toward a stable 1.0 release, but that time has not yet come. There are experimental snapshots tagged with "0.0.YYYYMMDD", but these should not be considered real releases and they may contain security vulnerabilities (which would not be eligible for CVEs, since this is pre-release snapshot software). If you are packaging WireGuard, you must keep up to date with the snapshots.
However, if you're interested in helping out, we could really use your help and we readily welcome any form of feedback and review. There's currently quite a bit of work to do on the project todo list, and the more folks testing this out, the better.
It may be fast, but that means nothing with all of those disclaimers around it.
-
https://www.xda-developers.com/us-senator-pushes-government-use-wireguard-vpn/
-
@juppin said in Installing WireGuard VPN:
https://www.xda-developers.com/us-senator-pushes-government-use-wireguard-vpn/
A member of the US Government endorsing it that enthusiastically could just as well mean they have backdoored the encryption or that it has a known flaw they can exploit.
It's just not ready for prime time yet. Give it time to mature, get audited properly.
Pushing this hard for it only makes it seem even more suspicious.
-
This comes when starting wg-quick on FreeBSD:
[#] wg-quick tun
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
W G
W This is alpha software. It will very likely not G
W do what it is supposed to do, and things may go G
W horribly wrong. You have been warned. Proceed G
W at your own risk. G
W G
WARNING WARNING WARNING WARNING WARNING WARNING WARNING -
As OP pointed out, there is a Wireguard port for FreeBSD (but not pfSense, yet). Note that this port has dependencies on Go and bash and uses an implementation (in 'Go') over tun. There is a thread on the FreeBSD mailing lists with more info.
Further through that thread, there is a performance claim that Wireguard on FreeBSD 'seems to be faster than OpenVPN'. Note that the posting doesn't say how much faster.
There may be some flaw in their benchmarking, as the Wireguard site claims 1011mbps on a 1gbps Ethernet NIC using iperf3. As previously discussed on this forum, reddit and elsewhere, it's impossible to pass more than 949mbps over IPv4 and TCP on a 1gbps NIC.
Max payload on Ethernet is 1500 bytes (yes, I know these are actually octets).
Preamble + SFD = 8 bytes
MAC DST + SRC = 6 bytes each for 12 bytes.
Ethernet Type = 2 bytes
FCS = 4 bytes
Intra-Frame Gap = 12 bytes of (otherwise empty on the wire) time.1500+8+12+2+4+12 = 1538
Sending TCP over IPv4 consumes another 20 bytes each for the IPv4 and TCP headers. Thus, we can send 1460 bytes of TCP for 1538 bytes 'on the wire'.1460/1538 = .94928, so max throughput possible with iperf3 over TCP and IPv4 on a 1gbps NIC is 949.28mbps.
According to this post by the primary author of Wireguard:
The overhead of WireGuard breaks down as follows: 20-byte IPv4 header or 40 byte IPv6 header 8-byte UDP header 4-byte type 4-byte key index 8-byte nonce N-byte encrypted data 16-byte authentication tag So, if you assume 1500 byte ethernet frames, the worst case (IPv6) winds up being 1500-(40+8+4+4+8+16), leaving N=1420 bytes. However, if you know ahead of time that you're going to be using IPv4 exclusively, then you could get away with N=1440 bytes.Modifying the math above, for a 1440 byte inner frame, and adding the 38 bytes of headers (and time) on Ethernet, we get to send 1440 bytes/1538 bytes. If the inner packet is TCP over IPv4, then an additional minimum of 40 bytes will be used for the inner IPv4 and TCP headers.
The result is that we send 1400 bytes of TCP payload and additional overhead totaling 1538 bytes on the wire at 1gbps, so the maximum bandwidth obtainable using Wireguard is 1400/1538 x 1gbps or 910.27mbps.
Moving on, the packet overheads for IPsec are actually ever so slightly lower than for Wireguard.
Additional overhead from ESP includes 20 bytes for an outer IP header, 8 bytes for an ESP header, 2 bytes for padding length & next header type, 16 (AES-CBC) or 8 (AES-GCM) bytes for an initialization vector, and 12 (HMAC-SHA1) or 16 (AES-GCM) bytes for an integrity check value. The total extra overhead is 58 bytes (AES-CBC HMAC-SHA1) or 54 bytes (AES-GCM).
Thus your 1460 becomes 1404. 1404 / 1538 = .9128, and we see that AES-GCM won’t flow more than 912.8mbps. For AES, CBC mode requires padding input to the block size, thus GCM mode produces smaller output if input is not multiple of block size. I may be flubbing something with required multiple of 16 bytes and padding for CBC:
87*16 = 1392 88*16 = 1408so it might be (and probably is) 1392/1538 or 905.07mbps. It's far too late at night to dive much deeper and figure out which is true. In either case, the actual protocol overheads for both Wireguard and IPsec are incredibly similar, so there isn't a protocol advantage for either.
If you skipped past all that, the summary is:
Wireguard protocol overhead = 20+8+4+4+8+16 == 60 bytes for IPv4 IPsec protocol overhead = 58 bytes (AES-CBC HMAC-SHA1) or 54 bytes (AES-GCM) (both IPv4)To be absolutely fair, Wireguard uses UDP encapsulation to get past NAT devices, and one would need to use UDP-Encapsulated ESP Headers (aka "NAT Traversal") as documented in RFC 3948 to provide an equivalent solution. pfSense has this, but it costs another 8 bytes. I don't know if it was used for the Wireguard performance testing though. In this case, AES-GCM overhead would be 62 bytes, .vs Wireguard's 60 bytes of framing overhead.
In addition to the per packet overheads due to framing, there are other overheads for traditional (policy-based) IPsec that will slow the packet processing down. Policy-based IPsec encrypts and encapsulates a subset of traffic flowing through an interface according to a defined policy. On a router running IOS or JunOS, this is often an access list. On FreeBSD and linux, it is implemented as something strongly resembling a list of firewall rules. There is additional overhead for traversing this list of 'policies'.
In contrast to a policy-based VPN, a route-based VPN employs routed tunnel interfaces as the endpoints of the virtual network. All traffic passing through a tunnel interface is placed into the VPN. Rather than relying on an explicit policy to dictate which traffic enters the VPN, static and/or dynamic IP routes are used to direct the desired traffic through the VPN tunnel interface. To be clear, both route-based and policy-based IPsec uses the same framing 'on the wire'.
Wireguard is a route-based VPN, and since there is less processing per packet with a route-based VPN such as Wireguard, I would expect a performance increase .vs a policy based VPN over the same link(s).
Fortunately, with pfSense 2.4.4, we have implemented route-based IPsec (based on the work in FreeBSD 11.2). This has occurred earlier than I promised in May of 2017. While we've not tested performance yet, I expect the route-based solution to be more performant given the explanation above.
TNSR also implements route-based IPsec, and we've measured 36.32 gbps (8 concurrent sessions) throughput on 40gbps NICs using AES-CBC + HMAC-SHA1 and QAT offload. We've also measured 13.70 gbps single-stream for AES-GCM using just AES-NI on an i7-6950x over those 40gbps NICs. (All benchmarking in this post is with max-sized packets.)
There are other issues. Wireguard tested AES-GCM-256 against 256-bit ChaCha20 with Poly1305 for MAC. AES-GCM-256 is roughly 20% slower than AES-GCM-128, yet Chacha20-Poly1305 is functionally comparable to AES128-GCM (giving confidentiality and integrity), but is easier to implement securely and efficiently, especially without AES support (such as AES-NI) in hardware. Still, pitting ChaCha20/Poly1305 against AES-GCM-256 in a benchmark seems a bit like stacking the deck to me.
Further on still, the FreeBSD thread points out some additional issues that may give the readership here some pause.
Wireguard also currently has performance issues at moderate packet rates. See this post (and it's responses) on the Wireguard list. FYI, 2.5Mpps is less than 1.7gbps of tinygrams, and a bit less than 25gbps of throughput assuming 1400 byte frames, but you won't run the crypto at 25gbps. TNSR will do 3.3 - 3.9Mpps with AES-GCM-128 (AES_NI) or AES-CBC + HMAC-SHA1 (QAT) (faster QAT cards exist than what we tested with.)
Finally, the Wireguard kernel implementation for linux, and the userspace implementation are both GPLv2. This presents an additional hurdle to anyone implementing a kernel variant of Wireguard for FreeBSD (or any BSD), as GPL code can't be upstreamed to the project if it's part of base. Until someone has the time to implement a ground-up rewrite of Wireguard, (and security evaluation of same), this serves as an effective barrier to a high-performance implementation of Wireguard on FreeBSD and pfSense. (One could, perhaps, leverage the route-based IPsec interface code as a blueprint for implementing Wireguard's interfaces, and use the GPL userspace code, but I doubt the FreeBSD project would be in-favor of upstreaming just the kernel components of same.)
When you add all the above to the concerns that @jimp expressed, the result is, quite simply: "Not Yet".
THAT ALL SAID, one could make a pfSense package that included (or depended on a separate package that implemented) the requisite GUI changes.
-
Fantastic post Jim - thanks!!!
-
me too, i need wireguard. i have test it, faster than openvpn 2.4 now.
-
Has anyone created an integration for this yet in pfSense? Just curious.
-
Thee FreeBSD port itself is broken. There were some changes with 20181018 snapshot but you'll run into a stacktrace when restarting the service too often :)
-
My point of view should try to increase it now, although there is no official version, but everything has started, there is no absolute perfection, as long as it has no malicious security problems, you can try it, and now there are many people using it. To people More choices of the latest advanced technology.
-
Indeed, but the port itself crashes the kernel when installed on UFS, this has to be fixed first.
-
@yon-0 said in Installing WireGuard VPN:
there is no absolute perfection,
While true, there is a big difference between an unstable, unaudited, alpha software package and one that has been tested and found to be stable.
as long as it has no malicious security problems, you can try it,
Their own page says it might have those: https://www.wireguard.com/#about-the-project
WireGuard is not yet complete. You should not rely on this code. It has not undergone proper degrees of security auditing and the protocol is still subject to change.
That should scare the hell out of you and anyone wanting to use it in production. If it doesn't, then you shouldn't be in charge of a firewall.
and now there are many people using it. To people More choices of the latest advanced technology.
Just because people use it doesn't make it good, secure, or desirable. Lots of people fall for phishing e-mails and scams, that doesn't mean they are a good idea.
After it passes 1.0 and is stable and audited as secure, then maybe it could be considered for a package.
-
Every new product has a starting process, but if it's a good project, you can provide an option for everyone to try. Only use it to find the problem. Any formal product can't guarantee that there are no security issues, you need more People use to find problems. This is just a feature that you can choose to use, so let everyone decide if they need to use it.
-
Given that wireguard already have a very strong following and is up for mainstream linux kernel adoption, I am not sure how to take on the sceptics in this thread.
https://lists.openwall.net/netdev/2018/08/02/124
Why not look into how to integrate this already now, and release it when it has the full blessing of everyone?
-
@maglub said in Installing WireGuard VPN:
Given that wireguard already have a very strong following and is up for mainstream linux kernel adoption, I am not sure how to take on the sceptics in this thread.
Because FreeBSD is not Linux, and it does not have a proven track record of stability on FreeBSD.
Why not look into how to integrate this already now, and release it when it has the full blessing of everyone?
Because we don't want to introduce a potentially unstable and insecure new VPN into a security-focused project until it's ready.
-
@maglub Feel free to package it up yourself just like others have done with other unofficial pfSense packages like E2Guardian and Squidalyzer because no matter how much you complain, Netgate is not going to introduce some bleeding-edge, relatively-untested code that comes with a heap of not-ready-yet disclaimers into a project focused on security. Not going to happen.
