Installing WireGuard VPN
-
This comes when starting wg-quick on FreeBSD:
[#] wg-quick tun
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
W G
W This is alpha software. It will very likely not G
W do what it is supposed to do, and things may go G
W horribly wrong. You have been warned. Proceed G
W at your own risk. G
W G
WARNING WARNING WARNING WARNING WARNING WARNING WARNING -
As OP pointed out, there is a Wireguard port for FreeBSD (but not pfSense, yet). Note that this port has dependencies on Go and bash and uses an implementation (in 'Go') over tun. There is a thread on the FreeBSD mailing lists with more info.
Further through that thread, there is a performance claim that Wireguard on FreeBSD 'seems to be faster than OpenVPN'. Note that the posting doesn't say how much faster.
There may be some flaw in their benchmarking, as the Wireguard site claims 1011mbps on a 1gbps Ethernet NIC using iperf3. As previously discussed on this forum, reddit and elsewhere, it's impossible to pass more than 949mbps over IPv4 and TCP on a 1gbps NIC.
Max payload on Ethernet is 1500 bytes (yes, I know these are actually octets).
Preamble + SFD = 8 bytes
MAC DST + SRC = 6 bytes each for 12 bytes.
Ethernet Type = 2 bytes
FCS = 4 bytes
Intra-Frame Gap = 12 bytes of (otherwise empty on the wire) time.1500+8+12+2+4+12 = 1538
Sending TCP over IPv4 consumes another 20 bytes each for the IPv4 and TCP headers. Thus, we can send 1460 bytes of TCP for 1538 bytes 'on the wire'.1460/1538 = .94928, so max throughput possible with iperf3 over TCP and IPv4 on a 1gbps NIC is 949.28mbps.
According to this post by the primary author of Wireguard:
The overhead of WireGuard breaks down as follows: 20-byte IPv4 header or 40 byte IPv6 header 8-byte UDP header 4-byte type 4-byte key index 8-byte nonce N-byte encrypted data 16-byte authentication tag So, if you assume 1500 byte ethernet frames, the worst case (IPv6) winds up being 1500-(40+8+4+4+8+16), leaving N=1420 bytes. However, if you know ahead of time that you're going to be using IPv4 exclusively, then you could get away with N=1440 bytes.Modifying the math above, for a 1440 byte inner frame, and adding the 38 bytes of headers (and time) on Ethernet, we get to send 1440 bytes/1538 bytes. If the inner packet is TCP over IPv4, then an additional minimum of 40 bytes will be used for the inner IPv4 and TCP headers.
The result is that we send 1400 bytes of TCP payload and additional overhead totaling 1538 bytes on the wire at 1gbps, so the maximum bandwidth obtainable using Wireguard is 1400/1538 x 1gbps or 910.27mbps.
Moving on, the packet overheads for IPsec are actually ever so slightly lower than for Wireguard.
Additional overhead from ESP includes 20 bytes for an outer IP header, 8 bytes for an ESP header, 2 bytes for padding length & next header type, 16 (AES-CBC) or 8 (AES-GCM) bytes for an initialization vector, and 12 (HMAC-SHA1) or 16 (AES-GCM) bytes for an integrity check value. The total extra overhead is 58 bytes (AES-CBC HMAC-SHA1) or 54 bytes (AES-GCM).
Thus your 1460 becomes 1404. 1404 / 1538 = .9128, and we see that AES-GCM won’t flow more than 912.8mbps. For AES, CBC mode requires padding input to the block size, thus GCM mode produces smaller output if input is not multiple of block size. I may be flubbing something with required multiple of 16 bytes and padding for CBC:
87*16 = 1392 88*16 = 1408so it might be (and probably is) 1392/1538 or 905.07mbps. It's far too late at night to dive much deeper and figure out which is true. In either case, the actual protocol overheads for both Wireguard and IPsec are incredibly similar, so there isn't a protocol advantage for either.
If you skipped past all that, the summary is:
Wireguard protocol overhead = 20+8+4+4+8+16 == 60 bytes for IPv4 IPsec protocol overhead = 58 bytes (AES-CBC HMAC-SHA1) or 54 bytes (AES-GCM) (both IPv4)To be absolutely fair, Wireguard uses UDP encapsulation to get past NAT devices, and one would need to use UDP-Encapsulated ESP Headers (aka "NAT Traversal") as documented in RFC 3948 to provide an equivalent solution. pfSense has this, but it costs another 8 bytes. I don't know if it was used for the Wireguard performance testing though. In this case, AES-GCM overhead would be 62 bytes, .vs Wireguard's 60 bytes of framing overhead.
In addition to the per packet overheads due to framing, there are other overheads for traditional (policy-based) IPsec that will slow the packet processing down. Policy-based IPsec encrypts and encapsulates a subset of traffic flowing through an interface according to a defined policy. On a router running IOS or JunOS, this is often an access list. On FreeBSD and linux, it is implemented as something strongly resembling a list of firewall rules. There is additional overhead for traversing this list of 'policies'.
In contrast to a policy-based VPN, a route-based VPN employs routed tunnel interfaces as the endpoints of the virtual network. All traffic passing through a tunnel interface is placed into the VPN. Rather than relying on an explicit policy to dictate which traffic enters the VPN, static and/or dynamic IP routes are used to direct the desired traffic through the VPN tunnel interface. To be clear, both route-based and policy-based IPsec uses the same framing 'on the wire'.
Wireguard is a route-based VPN, and since there is less processing per packet with a route-based VPN such as Wireguard, I would expect a performance increase .vs a policy based VPN over the same link(s).
Fortunately, with pfSense 2.4.4, we have implemented route-based IPsec (based on the work in FreeBSD 11.2). This has occurred earlier than I promised in May of 2017. While we've not tested performance yet, I expect the route-based solution to be more performant given the explanation above.
TNSR also implements route-based IPsec, and we've measured 36.32 gbps (8 concurrent sessions) throughput on 40gbps NICs using AES-CBC + HMAC-SHA1 and QAT offload. We've also measured 13.70 gbps single-stream for AES-GCM using just AES-NI on an i7-6950x over those 40gbps NICs. (All benchmarking in this post is with max-sized packets.)
There are other issues. Wireguard tested AES-GCM-256 against 256-bit ChaCha20 with Poly1305 for MAC. AES-GCM-256 is roughly 20% slower than AES-GCM-128, yet Chacha20-Poly1305 is functionally comparable to AES128-GCM (giving confidentiality and integrity), but is easier to implement securely and efficiently, especially without AES support (such as AES-NI) in hardware. Still, pitting ChaCha20/Poly1305 against AES-GCM-256 in a benchmark seems a bit like stacking the deck to me.
Further on still, the FreeBSD thread points out some additional issues that may give the readership here some pause.
Wireguard also currently has performance issues at moderate packet rates. See this post (and it's responses) on the Wireguard list. FYI, 2.5Mpps is less than 1.7gbps of tinygrams, and a bit less than 25gbps of throughput assuming 1400 byte frames, but you won't run the crypto at 25gbps. TNSR will do 3.3 - 3.9Mpps with AES-GCM-128 (AES_NI) or AES-CBC + HMAC-SHA1 (QAT) (faster QAT cards exist than what we tested with.)
Finally, the Wireguard kernel implementation for linux, and the userspace implementation are both GPLv2. This presents an additional hurdle to anyone implementing a kernel variant of Wireguard for FreeBSD (or any BSD), as GPL code can't be upstreamed to the project if it's part of base. Until someone has the time to implement a ground-up rewrite of Wireguard, (and security evaluation of same), this serves as an effective barrier to a high-performance implementation of Wireguard on FreeBSD and pfSense. (One could, perhaps, leverage the route-based IPsec interface code as a blueprint for implementing Wireguard's interfaces, and use the GPL userspace code, but I doubt the FreeBSD project would be in-favor of upstreaming just the kernel components of same.)
When you add all the above to the concerns that @jimp expressed, the result is, quite simply: "Not Yet".
THAT ALL SAID, one could make a pfSense package that included (or depended on a separate package that implemented) the requisite GUI changes.
-
Fantastic post Jim - thanks!!!
-
me too, i need wireguard. i have test it, faster than openvpn 2.4 now.
-
Has anyone created an integration for this yet in pfSense? Just curious.
-
Thee FreeBSD port itself is broken. There were some changes with 20181018 snapshot but you'll run into a stacktrace when restarting the service too often :)
-
My point of view should try to increase it now, although there is no official version, but everything has started, there is no absolute perfection, as long as it has no malicious security problems, you can try it, and now there are many people using it. To people More choices of the latest advanced technology.
-
Indeed, but the port itself crashes the kernel when installed on UFS, this has to be fixed first.
-
@yon-0 said in Installing WireGuard VPN:
there is no absolute perfection,
While true, there is a big difference between an unstable, unaudited, alpha software package and one that has been tested and found to be stable.
as long as it has no malicious security problems, you can try it,
Their own page says it might have those: https://www.wireguard.com/#about-the-project
WireGuard is not yet complete. You should not rely on this code. It has not undergone proper degrees of security auditing and the protocol is still subject to change.
That should scare the hell out of you and anyone wanting to use it in production. If it doesn't, then you shouldn't be in charge of a firewall.
and now there are many people using it. To people More choices of the latest advanced technology.
Just because people use it doesn't make it good, secure, or desirable. Lots of people fall for phishing e-mails and scams, that doesn't mean they are a good idea.
After it passes 1.0 and is stable and audited as secure, then maybe it could be considered for a package.
-
Every new product has a starting process, but if it's a good project, you can provide an option for everyone to try. Only use it to find the problem. Any formal product can't guarantee that there are no security issues, you need more People use to find problems. This is just a feature that you can choose to use, so let everyone decide if they need to use it.
-
Given that wireguard already have a very strong following and is up for mainstream linux kernel adoption, I am not sure how to take on the sceptics in this thread.
https://lists.openwall.net/netdev/2018/08/02/124
Why not look into how to integrate this already now, and release it when it has the full blessing of everyone?
-
@maglub said in Installing WireGuard VPN:
Given that wireguard already have a very strong following and is up for mainstream linux kernel adoption, I am not sure how to take on the sceptics in this thread.
Because FreeBSD is not Linux, and it does not have a proven track record of stability on FreeBSD.
Why not look into how to integrate this already now, and release it when it has the full blessing of everyone?
Because we don't want to introduce a potentially unstable and insecure new VPN into a security-focused project until it's ready.
-
@maglub Feel free to package it up yourself just like others have done with other unofficial pfSense packages like E2Guardian and Squidalyzer because no matter how much you complain, Netgate is not going to introduce some bleeding-edge, relatively-untested code that comes with a heap of not-ready-yet disclaimers into a project focused on security. Not going to happen.
-
Wireguard is the future, it is 4k lines of code and Linus Torvalds declared it a piece of art compared to IPSEC/OpenVPN and recommended it be merged into the kernel ASAP.
-
@bobert Wireguard May well be the future, but not for pfsense while it’s only available as a GPLed package over tun/tap. When Wireguard is available as a kernel-native implementation on FreeBSD (as it is on Linux), without the external dependencies on Go, we’ll integrate it in pfsense.
Also, just to clarify Linus’ quote:
“Maybe the code isn't perfect, but I've skimmed it, and compared to the horrors that are OpenVPN and IPSec, it's a work of art.”
-
It seems like there are some work going on for this:
https://lists.freebsd.org/pipermail/freebsd-ports/2018-May/113434.html
The thread sort of dies in May last year, and I will have to research a bit more.
The go-dependency is only during build time, not during runtime, so I don't think that should be an issue.
-
It's already available and working in FreeBSD, but in some cases you get some crazy stack traces https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233955
-
I hope somebody sacrifices their time for an external wireguard package soon, there's a lot of talk about it and I'd like to try it. To my surprise it already has native macOS and iOS clients in the respective app stores.
-
Please read the whole thread .. your pfSense installation will crash with most of your service restarts. This is a problem of Wireguard implementation in FreeBSD (or FreeBSD kernel). Doesn't make any sense to build an "external package"
-
Time will tell but would be nice if it can get into PFsense in a official way.
