[UnSolved] Possible BUG : Wireguard routing weirdly
-
If you can stop all traffic that would be crossing WireGuard so that only your test pings would show, that would help narrow down what you see on WAN. Otherwise it would be impossible to know for sure if there is a lot of traffic.
-
So I tried pinging with a 1000 byte packet and for destinations that I can ping I see a 1072 byte packet on the WAN capture. For destinations that I cannot Ping these packets are missing in the WAN capture. As a conclusion I think, the packets are disappearing inside pfsense. BTW did you see the interface speed of WG0 in my screenshot ? It’s defaulting to a very odd looking one - perhaps that may be the cause ?
-
That speed is fine/normal for WireGuard. It's irrelevant.
If the packets are on the wg0 interface but don't leave WAN the only explanation I can think of is that WireGuard can't find the peer to route them. Not sure what might be happening there or why it's only certain addresses, though.
I'm not aware of anyone else having anything like this happen.
-
@jimp
I guess the only option left is factory reset with manual config. I have tried a factory reset and reinstated the config file with the same results. Will keep you posted. -
@jimp
A clean reinstall after a factory reset solved the issue. I did not have to specify any MTU on the interface. This closes the issue. Thank you for the time and the attention. -
@jimp I have hit this again. After the clean install and built everything was working fine till I noticed today that some apps that use the WG tunnel do not work - so I started checking and the same issue is back. Can't reach certain IP's - the packets are leaving the WG tunnel but not leaving the WAN interface.
Is there something you suggest or can I provide more logs to see if this is a bug ?
P.S - You were right, it does look like WireGuard cannot find peer to route the packet. As soon as I add the offending IP's in the remote peers allowed IP's - the traffic starts to flow. So that eliminates everything else as the root cause.
peer: qvsssssxxxxxxxxxxxxxxx=
endpoint: 4.xx.xx.xx:58451
allowed ips: 8.8.8.8/32, 192.168.29.0/24, 10.100.100.50/32, 0.0.0.0/0 -
Guys anyone for this ? Should I raise a BUG report ?
-
Still not clear it's a bug here, and not a config issue, since nobody else can seem to reproduce it but you.
-
Ok I'll wait on it. I don't have a spare/lab setup to reproduce the bug - but if someone is keen enough to test this here are the steps to trigger this (at least in my setup).
- Setup a new wireguard tunnel wg0 to remote destination - In allowed IP's allow - REMOTE_LAN, Wireguard tunnel IP and 0.0.0.0/0. Everything else is default.
- Setup a Wireguard Interface [WG] - Policy to allow source WG net to any any proto any
- No rules for WIREGUARD group - Leave as is
- Gateway set to WAN_DHCP
- Create a LAN rule to route a PFSENSE_LAN_IP and set gateway to WG_WGv4 GW (Note: all LAN traffic is not being routed via the tunnel, only selected traffic)
- Now trace / ping from the machine (PFSENSE_LAN_IP) to a few websites and see if you can reach. I particularly had issues with 8.8.8.8 , LinkedIn.com and a few others.
-
-
@arrmo Quite possible - different symptoms with a common underlying cause. Will have to wait and see. I asked for help on reddit and until now no one else seems to come across this issue. A few days ago I did try setting up rules under WIREGUARD group to see if that'd make a difference to the lost packets and it did not :(.
-
@ab5g OK, NP - let's see how it goes. As long as that group rule is in place, most traffic gets through (still some odd sites). But with it off, much more trouble. And I have tried adding all sorts of pass rules in LAN and WG (interface), none of them seem to be working. Dang it!
Thanks!
-
@arrmo ping jimp here with your issue details. One more voice will help maybe.
-
@ab5g said in [UnSolved] Possible BUG : Wireguard routing weirdly:
ping jimp here with your issue details
ping? Meaning IM? Thinking the comments above are a ping of sorts, no?
Thanks!
-
@AB5G BTW, are you finding that a pass-all rule on the WireGuard group does help any, or not at all? I find it helps, but it's not a fix-all. Still some issues.
I checked the firewall logs, nothing there noted as blocked, so fun to debug. Any suggestions? Enabled logging on default rules? Or try tcpdump? To try to help resolve this.
Thanks!
-
@arrmo No it doesn't work for me. The packet passes the WG filter, get Natted to the WG Tunnel IP and then gets lost - I don't see it on the WAN.
-
@ab5g Dang it! And you have WireGuard set like this, right?
This is matching to what you recommended to me, so assuming you do - but just in case. Once I do this, and set up Hybrid Outbound NAT, then things are better (not 100%, but a lot better). Using the WG Interface causes me all sorts of grief
Thanks!
-
@AB5G Please let me know if you have any luck. Had to tear down WireGuard, go back to OpenVPN. Just not finding it up / consistent enough. Dang it! I see the (good) potential though
Thanks!
BTW, this isn't perhaps the split routing that OpenVPN uses (on the "client" side), is it? Or are you not redirecting all traffic?
-
Nope no luck and no one else is reporting this issue so I'm holding still.
-
@ab5g Understood, here as well. Just yell if there is anything I can do to help!
. But if not, perhaps another example?