@McMurphy said in Where to set MTU:

The maximum packet size for the internet link before fragmentation is 1472 (+28 = 1500)

in your case 1440 is fine for IPv4 only Tunnel. If the Tunnel also shall transport IPv6-Trafic you shall not use a MT bigger 1420. The reason is the slightly bigger overhead of IPv6 compared to IPv4.

Using tracepath you can check out pmtu and packet transfer, to find optimal results

See: https://schroederdennis.de/vpn/wireguard-mtu-size-1420-1412-best-practices-ipv4-ipv6-mtu-berechnen/ (german language)

@Bob-Dig
Awww. Suggesting that is like taking a xmas present back from a child. ☹️

Good idea - I'll try pruning the clients back to maybe 2 or 3 and experiment from there. If I have no luck with that, I'll check out opendwt (I used to run ddwrt - i didn't realise openwrt was unlocked/unlicensed for x86).

@Bob-Dig This is an added layer of security, if the device/machine is stolen for example they would have the private key. So by blocking by public IP we can stop the WG connect being used elsewhere. At least to certain networks using a VLAN firewall rule.

I have just install and is up and runing pf sense CE in my infrastrature. But regrading WireGuard with all the user I have is impossible to generate peer for every user on manual base.

So I create a new Ubuntu Server with pivpn with wireguard, then a port foward to that server.

At this point I have 700mb download and 650mb upload using wireguard and this configuration.

I know this is stupid, but creating peer manual for many users is also stupid. If someone convert the pivpn files to pfsense would be great.

@gabacho4 Did you ever figure it out?

@gunnyp

Self-inflicted!
Forgot to change the IP address of the managed switch that sits between the servers and the pfSense vm.
IVPN is up and running.

Good news. I've managed to figure out my major bottleneck. My main Win10 machine at home has a stu*** Intel I225-V 2.5 Gbit on board ethernet NIC that I was using. It has caused weird issues in the past with random disconnects and is globally known for its issues.. I have plugged in a ASUS USB 2.5 Gbit adapter and sure enough I am able to max out my wire speed now using the wireguard tunnel.

OpenVPN also works equally well now as on the other machine, and I figured out that enabling DCO in the OpenVPN client side can bring me close to peak speeds of 400Mbit wire speed. I haven't enabled DCO on the OpenVPN server side yet. I am still using OpenVPN via TCP, and I read for DCO thats not supported/recommended - even though interestingly enough it already gives me a performance boost already now just enabling it on the client side. Switching OpenVPN server to using UDP and enabling DCO on the server side as well might further improve things I imagine - something I might try in the next days.

Hope this thread will help somebody down the road troubleshooting the same.

@ahking19 Thanks for your reply. Both machines are on wired ethernet on the same router at home. I think I am not near maxing out the Netgate CPU, but I've attached a screenshot below to be sure (not sure how to interpret that plot exactly):
e7b8a2a8-aea9-4b8a-a82e-09a91455386a-image.png

Latency wise: I have about 9-15 ms ping through the OpenVPN tunnel from client to my netgate.
Using wireguard is about 9-11ms.

@compsmith said in Road Warrior need access all spokes in hub/spoke multisite:

Anyone out there have any insight to get this to work?

Okay, i got it fixed up myself.

It was the "Advanced Outbound NAT Entry" (Firewall / NAT / Outbound).
Rules defined there does not work if not "Outbound NAT Mode" is set at least to "Hybrid Outbound NAT rule generation". I did not marked that and therefor the defined rules below was ignored.
And without an appropiate Outbound NAT rule it is impossible to route via Wireguard all internet traffic from clients via the tunnel.
OpenVPN does not need such a rule or (most likely) generate the needed rules automatically (similar to IPsec). The standard "Outbound NAT Mode" is set to the entry "Automatic outbound NAT rule generation.
(IPsec passthrough included)". So I suspect the outbound NAT rules will be generated for both IPsec and OpenVPN automaticaly, but not automaticaly for Wireguard.

[Troubleshooting/Fix]

Pinging from Client -> Youtube.com (142.250.180.14).

PCAP on em0:WAN

#Tunnel Disabled: 18:48:21.808185 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 990, offset 0, flags [none], proto ICMP (1), length 84) WAN_address > 142.250.180.14: ICMP echo request, id 61165, seq 7, length 64 18:48:22.516482 xx:xx:xx:xx:28:27 > xx:xx:xx:xx:66:8a, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 110, id 0, offset 0, flags [none], proto ICMP (1), length 84) 142.250.180.14 > WAN_address: ICMP echo reply, id 61165, seq 7, length 64 18:48:22.817329 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 63388, offset 0, flags [none], proto ICMP (1), length 84) WAN_address > 142.250.180.14: ICMP echo request, id 61165, seq 8, length 64 18:48:23.518344 xx:xx:xx:xx:28:27 > xx:xx:xx:xx:66:8a, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 110, id 0, offset 0, flags [none], proto ICMP (1), length 84) 142.250.180.14 > WAN_address: ICMP echo reply, id 61165, seq 8, length 64 18:48:23.822630 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 5156, offset 0, flags [none], proto ICMP (1), length 84) WAN_address > 142.250.180.14: ICMP echo request, id 61165, seq 9, length 64 18:48:24.526677 xx:xx:xx:xx:28:27 > xx:xx:xx:xx:66:8a, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 110, id 0, offset 0, flags [none], proto ICMP (1), length 84) 142.250.180.14 > WAN_address: ICMP echo reply, id 61165, seq 9, length 64 #Tunnel Enabled: 18:48:24.824198 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 30834, offset 0, flags [none], proto ICMP (1), length 84) 10.100.0.101 > 142.250.180.14: ICMP echo request, id 30738, seq 10, length 64 18:48:25.833285 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 23547, offset 0, flags [none], proto ICMP (1), length 84) 10.100.0.101 > 142.250.180.14: ICMP echo request, id 30738, seq 11, length 64 18:48:26.834520 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 64833, offset 0, flags [none], proto ICMP (1), length 84) 10.100.0.101 > 142.250.180.14: ICMP echo request, id 30738, seq 12, length 64 18:48:27.840448 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 50822, offset 0, flags [none], proto ICMP (1), length 84) 10.100.0.101 > 142.250.180.14: ICMP echo request, id 30738, seq 13, length 64 18:48:28.843054 xx:xx:xx:xx:66:8a > xx:xx:xx:xx:28:27, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 62192, offset 0, flags [none], proto ICMP (1), length 84) 10.100.0.101 > 142.250.180.14: ICMP echo request, id 30738, seq 14, length 64

So it appears there's no NAT for the 10.100.0.100/31 traffic. I tried creating the Hybrid NAT rule below:

Interface: Wireguard Source: Wireguard networks Source Port: any Static Port: unchecked Destination: any Destination Port: any NAT Address: WAN_address

However I was still seeing traffic not being NAT'ed. So I tried adjusting it:

Interface: WAN Source: Wireguard networks Source Port: any Static Port: unchecked Destination: any Destination Port: any NAT Address: WAN_address

Alas, this STILL did not work. On a hunch, I tried using the subnet specifically instead of "Wireguard networks" and suddenly it worked:

Interface: WAN Source: 10.100.0.100/31 Source Port: any Static Port: unchecked Destination: any Destination Port: any NAT Address: WAN_address

Is this behavior desired? It seems incongruent with the rest of the UI. The alias "Wireguard networks" does not appear to include the peer network. It seems like a rather simple addition to the Wireguard package.

Now I have encrypted traffic that cannot be easily cracked over the air :)

How is this still not a feature???
This should be the feature to be included in year 2024!!

I want this feature so bad that I might visit forum once a week to keep this post on the top page...

@Jarhead I do it like that. It might be less secure, but how much?
I wish we could get rid of the Resolver ACL too. 😁

@Jarhead said in Why use Allowed IP's?:

why would we ever set specific Allowed IP's if they really aren't doing anything needed? (like creating routes for example)

If you have more than one other peer, you can do 0.0.0.0/0 only on one.

@The-Party-of-Hell-No

No, it's not that.
In simple worlds when using WG client and you set pfSense as DNS server which is also WG client(peer) along with pfBlocker in some casses issues starts to happen like slow loading certain website's.
Probably some problem with traffic from/to pfBlocker virtual server if you not allowed it in adding ip addres of blocker server as allowed in wg0.conf file.