Crash due to CGNAT conflict with Wireguard

Laxarus

Hi,
My bloody ISP assigns me a CGNAT ip in the 10.0.0.0/8 range with gateway IP of 10.0.0.1.

My wireguard settings I got from my privacy VPN provider is also in that range.
Wireguard gateway 10.11.0.1
with static ipv4 10.11.3.37/16

Though, I am not an expert, looking at the crash dump, I suspect this is crashing my pfsense instance.
Am I correct to assume as such? How can I prevent this?
Crash dump attached.

textdump.tar.0

Bob.Dig

@Laxarus What ISP, what VPN Provider and show Status/Interfaces for WAN.

Laxarus

@Bob-Dig It is an obscure ISP called STC

VPN provider: VPN.AC

Bob.Dig

@Laxarus said in Crash due to CGNAT conflict with Wireguard:

ip in the 10.0.0.0/8 range

That is awful. Is this IP static, then you could also make a static configuration in pfSense. Otherwise I would use another router in front of pfSense but then, IPv6 might be problematic.

If you have a Homeserver, you could run OpenWRT-VMs as VPN-Clients...

stephenw10

Umm, your ISP gives you a /8 subnet on WAN?! That's.... a new level of fail. Even for an ISP!

There's a specific range for CGN and it isn't in 10/8.

But it still shouldn't cause a crash. Routing problems perhaps.

The actual panic is in pimd:

Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 06
fault virtual address	= 0x0
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80ef1f7a
stack pointer	        = 0x28:0xfffffe00fa98bb40
frame pointer	        = 0x28:0xfffffe00fa98bb70
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 95567 (pimd)
rdi: fffffe0020621f38 rsi: 0000000000000004 rdx: 0000000000000001
rcx: 0000000000000000  r8: 0000000000000000  r9: fffff80006f60000
rax: 0000000000000100 rbx: fffffe00fc423900 rbp: fffffe00fa98bb70
r10: 0000000000000000 r11: fffffe0020511950 r12: 0000000000000000
r13: 0000000000000000 r14: fffff8040819fe00 r15: 0000000000000000
trap number		= 12
panic: page fault
cpuid = 1
time = 1745672721
KDB: enter: panic

db:0:kdb.enter.default>  bt
Tracing pid 95567 tid 100268 td 0xfffffe00fc423900
kdb_enter() at kdb_enter+0x32/frame 0xfffffe00fa98b820
vpanic() at vpanic+0x163/frame 0xfffffe00fa98b950
panic() at panic+0x43/frame 0xfffffe00fa98b9b0
trap_fatal() at trap_fatal+0x40c/frame 0xfffffe00fa98ba10
trap_pfault() at trap_pfault+0x4f/frame 0xfffffe00fa98ba70
calltrap() at calltrap+0x8/frame 0xfffffe00fa98ba70
--- trap 0xc, rip = 0xffffffff80ef1f7a, rsp = 0xfffffe00fa98bb40, rbp = 0xfffffe00fa98bb70 ---
X_ip_mrouter_done() at X_ip_mrouter_done+0x32a/frame 0xfffffe00fa98bb70
rip_detach() at rip_detach+0x3f/frame 0xfffffe00fa98bba0
sorele_locked() at sorele_locked+0x89/frame 0xfffffe00fa98bbc0
soclose() at soclose+0x14a/frame 0xfffffe00fa98bc20
_fdrop() at _fdrop+0x11/frame 0xfffffe00fa98bc40
closef() at closef+0x24a/frame 0xfffffe00fa98bcd0
fdescfree() at fdescfree+0x4c6/frame 0xfffffe00fa98bd90
exit1() at exit1+0x49e/frame 0xfffffe00fa98bdf0
sys_exit() at sys_exit+0xd/frame 0xfffffe00fa98be00
amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe00fa98bf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00fa98bf30
--- syscall (1, FreeBSD ELF64, exit), rip = 0x8224fd40a, rsp = 0x820662108, rbp = 0x820662120 ---

And both times it's when ovpnc comes up. So I'd try disabling either that interface or pimd entirely at least as a test.

Then get a new ISP!

Laxarus

Believe me, this is the best ISP I can get so I am stuck with it.
I have disabled pimd but it is still weird. ovpnc is not configured in pimd. It is disabled. PIMD is set to bind to none.

stephenw10

It's this for reference: https://redmine.pfsense.org/issues/14917

If you need pimd I would test 2.8-beta if you can.

Laxarus

@stephenw10 Ahh, I see then no wonder. Thanks for the info.

stephenw10

It would be useful if you can test 2.8-B because that bug was originally though fixed in 23.09 and 2.7.2 but clearly isn't.

JKnott

@Laxarus

I see you have IPv6. Any chance your VPN can work over it? I run OpenVPN and have it configured to work over IPv4 or IPv6.

JKnott

@Laxarus said in Crash due to CGNAT conflict with Wireguard:

It is an obscure ISP called STC

Perhaps you could remind them they're supposed to be using 100.64.0.0 to 100.127.255.255 for CGNAT.

Laxarus

@JKnott I do have a ovpn server on another site connected to this site with site2site. If I absolutely need remote access, I use that. Theoretically, I can also try the ipv6 but they give me a /64 so apparently based on my research, I can only give ipv6 to one of my subnets.

JKnott

@Laxarus You show an IPv6 address on your WAN. You'd use that for the VPN. Also, it doesn't have to a OpenVPN. I assume Wireguard will work over IPv6. The question is whether VPN.AC supports it at the other end.