I made a WireGuard package for pfSense

dubatech

@Ashus said in I made a WireGuard package for pfSense:

@ubernupe I will try to compile it for FreeBSD 12, so it's compatible with pfSense 2.5.0, but only for amd64 architecture. If you have another one, you will have to make it on your own, set up FreeBSD 12 in virtual machine and install and compile until you get a package. It's rather lengthy process.
Watch the assets in releases on github, i'll try to get it done today. Unfortunately I have no way of testing it easily, so your input would be helpful.

I've just installed your pkg for FreeBSD 12 and finally also pfSense 2.5 works with WireGuard.
While trying to install it there is only an alert regarding mismatch between pkg and running kernel version but it works!

Thank you so much!

dubatech

@Talisker said in I made a WireGuard package for pfSense:

@dubatech I am using 0.0.0.0/0 as allowed IPs. I use Rules in order to prevent unwanted traffic to the tunnel.
I am not having lower latency and allthough it looks stable, it isn't really the performance boost I was hoping for.

I don't need the extra speed or lower latency, but I just had to implement it because it could be done...

As Ashus said, have you tried to set MTU and MSS on pfSense - Interfaces - tunwg0?
I've set the same MTU value under Package - VPN: WireGuard - Interface - Advanced Features - MTU and this fixed me some problems with some websites and also improve speed and low latency.

Talisker

@dubatech I have tried different things now and I seems to fail with most.
I first tried with adding a second interface, but I can't get my head around it. I am not that familiar with PFSense xml, but I think I would manage to configure the secondary wireguard interface. However, I don't understand how to do that and get it to show up as a tunwg1.

I also tried changing TCP MSS and MTU, but when I do that I get the following error:
The following input errors were detected:
Cannot add IPv4 Gateway Address because no IPv4 address could be found on the interface.

I can revert to an old configuration and get the wireguard up and running, but when I make changes the firewall doesn't accept saving because I set gateway address manually. If I have the gateway as dynamic I wont get the tunnel running.

big_bum

@Ashus the package for 2.4.5 is semi-broken. It installs but it doesn't contain all the files in the www folder so some functionality is incomplete (generating a new key for example).

ubernupe

@Ashus
Dear Ashus,
Hello and I hope that you are safe and well. I compiled my own pfsense 2.5.0 package for WireGuard. I also tested your pfSense-pkg-wireguard-1.0.1-freebsd12-amd64.txz package. I want to thank you for your efforts and development of this great software. I found some results which occurred both in my package and yours - I got a crash report and the contents are below:

DiagnosticsCrash Reporter

The firewall has enountered an error

Debugging output can be collected to share with pfSense 
developers or others providing support or assistance.

Inspect the contents to ensure this information 
is acceptable to disclose before distributing these files. 

Crash report begins.  Anonymous machine information:

amd64
12.1-STABLE
FreeBSD 12.1-STABLE f1de4082be8(devel-12) pfSense

Crash report details:

PHP Errors:
[08-Jun-2020 17:52:39 America/New_York] PHP Warning:  
Invalid argument supplied for foreach() in 
/usr/local/pkg/wireguard.inc on line 196
[08-Jun-2020 17:52:43 America/New_York] PHP Warning:  
Invalid argument supplied for foreach() in 
/usr/local/pkg/wireguard.inc on line 196

I had this crash report when I built the 2.4.5 package as well. If you or anyone here can assit me in rectifying the cause of this nagging crash message - I would appreciate that. Again, thanks for your work.
Peace

Ascrod

Boy, looks like I have some catching up to do, eh?

First off, apologies to all for the radio silence. Work has taken over my life and I've had almost no time for anything else in the past several weeks. I also recently migrated my pfSense dev VMs to a new computer and a new hypervisor, and have yet to get networking/file sharing to work properly on KVM/Qemu. Thus, I haven't been able to build or test with pfSense 2.5.0 yet. Again, my apologies.

Many thanks to everyone who has been providing patches and support in my absence. Unfortunately I won't be able to answer everyone's questions here, but I'm glad to see that people are making use of WireGuard on pfSense.

@lucas_nz said in I made a WireGuard package for pfSense:

Hi @alirz / @Ascrod ,

See the diff below for /usr/local/etc/rc.d/wireguard.sh (added below the wg-quick up command). This has resolved my issues no need to restart unbound. I also added my VPN subnet to the DNS resolver (allowed) access list.
diff ./wireguard.sh /usr/local/etc/rc.d/wireguard.sh
wireguard.sh
6a7
>         /etc/rc.filter_configure
Luke

Thanks for this, and the UI patch. If anyone is able to confirm that adding this line to wireguard.sh fixes the DNS issue, I can merge these changes in.

@ubernupe said in I made a WireGuard package for pfSense:

Dear Ascrod,
Hello and I hope that you are safe and well during these trying days and times. I set this up on a VM and have it working well using TORGUARD WIREGUARD service. I tried my hand at this earlier on see here : https://forum.netgate.com/topic/145099/pfsense-wireguard-client-working-with-catch-22 - with albeit limited success.

I should be thanking you. I based 1.0.0 on what you posted there! Glad you have been able to make use of this so far. :)

@Ashus Your changes look awesome. Unfortunately as I said before I'm unable do much dev/test work at the moment, but I'd love to merge this if you want to PR it.

Thanks everyone, and I hope to get back to this sometime soon.

Ashus

@Talisker As I noted in the readme, to use more interfaces, you need to add config files directly to /usr/local/etc/wireguard/. GUI only supports reloading the configuration. After adding interface config, you need to reinstall my package to allow them all to reload at the same time.

@ubernupe I'm safe, thank you for your concern. This error I have not seen, but I fixed it in the repository, maybe because I have already completed the peer configuration and that array wasn't empty. Most of the changes in code since @Ascrod released the module were made by lucasnz on GitHub. I liked his version more, so I used it as my base. My work there was only to add support for more interfaces by creating another config files directly. Adding complete GUI for more interfaces would require rewriting most of it (but it would be a bit simpler than OpenVPN config) and since I only needed to have it working automatically right after boot, I chose the easier way. I rebuilt the package for pfSense 2.4.5 and for pfSense 2.5.0 I changed manually only differences so it matches kernel version from error message in this thread as the packages built in both were the same except manifests in TGZ packages. So I'm not sure if the 2.5.0 version could be installed without warnings or not at all.

@Ascrod Thank you for your great work. I've created a pull request per your request. Most of the changes regarding key generation etc were made by lucasnz as I stated above. I think Wireguard deserves to have complete support for more interfaces as OpenVPN has only then could it be accepted as official module.

Talisker

@Ashus thank you. I saw that and configuring by xml-files is ok for me, but since I could not generate new keys easily I didn't manage to set up a second tunnel. Now I will give it another try.

I still have a problem with gateway/ip though.
I get an IP from mullvad. Let say it is 10.11.12.13/32
Where should I configure it besides the obvious wireguard peer section?

Should I configure it in the Interfaces section as IPv4 static?
Should I manually set it as gateway IP?

The only time I have gotten it to work I statically set it as gateway IP, but I think the intention is to only set it in the wireguard section.

dubatech

@Talisker Yes, by my side, I used Mullvad/32 ip both in the following fields:

Package -> VPN: WireGuard -> Interface -> Address;
System -> Routing -> Gateways -> Gateway IP;

Talisker

@dubatech but, you dont set it as static IPv4 address in Interfaces / Wireguard
Static IPv4
IPv4 address = Mullvad

I wonder if it is some kind of routing issue since I have OpenVPN. When I upgraded to version 2.5 I could not longer have two OpenVPN connections to mullvad, since the service died because of some kind of routing-problem. I will try to set up a static route to the IP I am pinging.

dubatech

@Talisker No, I didn't set it on Interfaces / tunwg0:

Nat Outbound rule is setup correctly?
I've OpenVPN and also WG with Mullvad and it works correctly with Manual Outbound NAT rule generation.

Talisker

@dubatech the problem is that the gateway is considered down. The NAT and Rules are correct. I had I working before, but not any more.

dubatech

@Talisker The same for me, everytime I reboot.
In that case I disable and then enable again gateway.
On monitor IP I've Mullvad DNS 10.64.0.1 because using Mullvad/32 ip as monitor IP doesn't work.

Talisker

I can see that work has been done, but no compiled package is available for download. Unfortunately I have not compiled any package before and I don't really have time to do read up upon it for the moment. (Though I could set up a freebsd in my VMware, so in theory it could be done).

I upgraded my hardware PFSense to 2.5, but got packet loss in OpenVPN which is not ok. Since I cannot easily reach the firewall physically I decided to boot up my virtual 2.4.5 and run them in parallell. (And I have a Clavister firewall as well, since I can get several IPs from my ISP). Anyway, this gives me plenty of opportunity to try things when I have more time:)

So as soon as someone could put up a new package made on the merged project I am happy to try.

Edit: I found it, but only for version 2.4.
pkg add https://github.com/lucasnz/pfSense-pkg-wireguard/releases/download/v1.0.1/pfSense-pkg-wireguard-1.0.1-freebsd11-amd64.txz

Lucasnz version is working. It has removed the bug with key generation and I am able to set up the tunnel as intended. Still with the "issue" that I have to set gateway IP manually in the xml-file since I otherwise get an error trying to configure a gateway IP on an interface with dynamic IP address.

xayide

Anyone made any benchmark comparing this package to openvpn on the same connection? ie throughput. Care to share?

papkee

I'm trying to get a basic wireguard link set up between pfSense and a remote server, and running into a strange issue. Both sides of the tunnel show it as active, and keepalive packets are flowing, but I can't ping one side from the other. I'm speculating this basic issue is what's preventing my source-based routing (explained in a different thread on here) from working properly.

The server's wireguard endpoint is 192.168.16.0/31, and the pfSense endpoint is set as 192.168.16.1/31. From pfSense, I can't ping 192.168.16.0, and additionally when I ping the local 192.168.16.1 address it actually goes through the wireguard tunnel and back again before replying which is even more confusing. That definitely doesn't seem right.

When I run an ifconfig from the console, this is what I get for the wireguard interface:

tunwg0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1420
        options=80000<LINKSTATE>
        inet 192.168.16.1 --> 192.168.16.1 netmask 0xfffffffe
        nd6 options=101<PERFORMNUD,NO_DAD>
        groups: tun
        Opened by PID 52527

Which seems to imply that the address does exist locally on the router. However, like I said above I can't ping it locally, or from the other side of the tunnel.

Talisker

@papkee don't know how it works in PFSense l, but it could be considered that. 1 is a broadcast address and .0 is the network address. Why not a slightly bigger network and not using the first and the last IP to rule that out? Like using. 1 and .2 in a /30 network.

xayide

OK, finally did a comparision....... I know WG is not in kernel on freebsd/pfsense yet!

Server Xeon E5-2667v2 (8x3.3Ghz turbo 4.0Ghz) pfsense running in VM on ESXi with 2 cores.

Pfsense openvpn 650Mbit (cpu 60%)
Pfsense Wireguard 650Mbit (cpu 80%)
Vyos Wireguard 900Mbit (cpu 65%)

Symmetrical 1Gbit FIber Internet.

lucas_nz

@papkee the interfaces do not need to be in the same subnet. I have mine both set to /32 addresses. As @Talisker suggests, it's probably advisable not to use zero in the last octet.

papkee

Thanks for the suggestions everyone, I'll give them a try and report back.

Edit

No dice.

I tried with the /30 and 192.168.16.1 & 16.2, and it got me the same result as the /31. Couldn't ping either endpoint from the other, and on the pfsense box pinging its own tunnel address went through the tunnel and back again.

Tried the /32s with the same addresses above and that didn't even allow the pfSense box to ping itself. Something is still strange with this setup.

Although as a sidenote, I just went and checked some of the other debian -> debian wireguard links I have set up the same way, and I actually can't ping their endpoint IPs directly either. But, those tunnels are still routing my subnet just fine so I'm thinking that might not be the root cause of my routing issues.