@viragomann Now it works and I have my Static IP, but another problem arises for which, however, I open a separate topic. In the meantime, thanks for the help.

I had this feeling and thanks for confirming it. I'll remove the direction.

doing more and more testing. two systems now. both 1151 based. both setups have the same memory, 32GB (16GB x2 of DDR4-2666Mhz ECC UDIMM) pfSense Hardware Supermicro 1019C-FHTN8 with Intel Xeon E-2278G (8c/16t, 3.4Ghz, 5.0Ghz turbo), idles at ~26w Supermicro 505-203B / X11SCL-IF with Intel Pentium Gold G5400 (2c/4t, 3.7Ghz, no turbo), idles at ~16w Both systems have Intel I210 NICs, but I also tested an Intel X710-DA2 10g dual port SFP+ NIC (on the LAN side only). The 1019C-FHTN8 is fun because it has 8 i210 NICs! [image: 1607481854295-aaaf76da-644d-4dc7-a768-d6f05bb91d92-image.png] OpenVPN Clients i9-9900KF running Ubuntu 20.04 i7-7800X running Ubuntu 20.04 Both clients are AIO water-cooled and slightly overclocked, so there should be no client-side bottlenecks with 1 Gbps links. Testing Matrix pfSense 2.4.5-p1 vs pfSense 2.5.0-nightly VM vs Bare metal installs PCIe pass-through of NICs vs VirtIO Again, is all cases, this OpenVPN test is totally bogus and is wildly off from real world numbers. openvpn --genkey --secret /tmp/secret time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-gcm Observations Proxmox KVM adds about 10-20% overhead VirtIO NIC perform nearly identical for 1 Gbps vs PCI pass-through (probably due to both CPUs being fairly powerful) pfSense 2.5 is about 4% faster than pfSense 2.4 in iperf3 tests OpenSSL could be used to compare openssl speed -elapsed -evp aes-256-gcm the results of this test matched nearly the differences in each iperf3 test, percentage-wise X710-DA2 NIC adds about 4-5 watts to each system's total idle power Bare Metal Results Intel Xeon E-2278G through using OpenVPN with AES-256-GCM was ~810-850 Mbps Intel Pentium Gold G5400 using OpenVPN with AES-256-GCM was ~760-800 Mbps Before I sent back my Supermicro A2SDi-4C-HLN4F, Intel Atom C3558, I managed to do some quick testing Observations Idles at 22w, but maxed out at 26w, whereas the 1151 systems maxed at at 40w and 110w when CPUs are loaded with stress-ng --matrix 0 Under Proxmox as a guest, OpenVPN performed at nearly 50% loss in total throughput using a simple iperf3 test C3558 was just not great under Hypervisor/Guest situation, even though pfSense was the only guest on an otherwise idle system. I have no explanation, other than it was repeatable and what I observed. Conclusions If you are using some embedded CPU like Intel Atom, than bare metal setup is the way to go. If you are using a fairly fast CPU, even the Pentium Gold series, it seems like for gigabit speeds on firewall, CPU is not the bottleneck. For OpenVPN itself, I was unable to achieve 1 Gbps AES-256-GCM even with the E-2278G @ 5Ghz. The convenience of VM, being able to easily snapshot VM before a major upgrade, etc, probably outweighs the OpenVPN performance hit, plus the power savings if you are already running a Proxmox setup. I would love for pfSense with ZFS to support taking a snapshot of itself before an upgrade so you can easily rollback if it goes south. If you needed real serious OpenVPN performance, you'd probably wouldn't be doing it on your router anyway and using a VPN appliance. I did not test any VLAN performance, which is all done on the CPU with pfSense, but I would imagine the VM overhead would exist there as well. I have CenturyLink Fiber, so it uses PPPoE and the FreeBSD bug (although pfSense won't call it bug for some odd reason, which it does not exist in Linux), basically only uses 1 of the WAN NIC's queues, so when testing outside of my lab and actually hooking this up to the internet, my overall speeds were even worst, given its basically singled thread now inside the kernel. Documented here, here, and here. Thoughts FreeBSD has become a toy compared to Linux over the past decade. The Linux device drivers, kernel, applications, etc all have eclipsed BSDs at this point and with nftables replacing iptables on Linux, I would love to see pfSense router based on Linux instead of FreeBSD :) I also tested Wireguard on Debian 10.6 and Ubuntu 20.04, behind pfSense, and in each case, Wireguard was easily able to achieve 1 Gbps. Wireguard is probably the future of VPNs at this point :)

@viragomann thank you for the suggestion, I am gonna give it a try, we should fix the issue by having the remote endpoint add a phase 2 for the openvpn subnet but in the meantime this should fix it as well.

The config in this article fixed my slow pfsense sg-3100 pia openvpn. The official documentation isn't accurate and I also had to piece together the setup, which matched this thread. I only got 30MiB out of 400MiB. I switched to AES 256 Strong Auth and the speed immediately jumped to 300. Thanks.

Glad you have it working now. -Rico

@freek_box If your onlay goal is to access the router itself for maintenance I would go with the existing VPN and the NAT. However, if you set up the router as OpenVPN client which is connecting to the cloud you can have a backup connection in case the main internet goes down. But you will need a route on the 4g router to the LAN behind pfSense. Another option is to set up a second VPN on pfSense itself using the 4g as gateway. This way it may be easier to configure a failover for both directions.

@calbha What VPN do you use? I've also experienced it from time to time, especially while traveling. It seems to be ok now as I'm staying at one place during lockdown ahah. Maybe try to read some reviews about VPNs for Torrenting and simply switch it. (I've inserted the link with my fav website). They're not vpnmentor yet but I like their approach and the way they test. As far as I know, in most cases, it's simply the issue with the VPN

@ekoo I am doing what your asking in your OP. But my setup is different. I have multiple Nics in my server and I can bind P2P to use one of those Nics. My router is pFSense with 8 ports to do whatever I want with. So I setup pFSense to use Opt3(port3) to route all traffic through expressvpn, then on my server, I bind my P2P client to use Nic3 only. I then set each nics index priority so that traffic is routed through nic1 first and so on. Only my P2P is traveling through VPN.

You can use packet capture to check whether the packets are going out the VPN interface. Also I'd recheck your firewall rules. Enable logging in all relevant rules and check the log to see which rule is applied. Keep in mind the rule order and that floating rules and such ones on gateway group have advantage over interface rules.

Wow that was easy! It worked straight away Thank you, i really appreciate it!

Thanks for all your help your comment about the windows firewall got me to look at it a different way. Turns out during one of my previous attempts to get internet to my VPN clients (a different issue not this one) I messed with some other firewall settings and pushed all of the VPN traffic out the WAN interface which worked fine for getting my clients internet access but caused issues when I tried to access the LAN. I removed that and now with the push route command my clients are able to access the Internet and my LAN

@Rico Those are respectable numbers. I'm curious to know how that would scale with 2 simultaneous connections? I'm thinking it's not entirely CPU bound, so maybe 2 by 60 or 70 Mbps?

easy-rsa together with easy-tls might be helpful: https://github.com/OpenVPN/easy-rsa https://github.com/TinCanTech/easy-tls