XG-7100 - no internet connectivity on latest 2.5.0 builds?

victorhooi

Hi,

I have a Netgate XG-7100 running pfSense 2.5.0 - great little unit.

However, around 2 days ago I upgraded to the latest build, and something seems to have broken internet connectivity for all LAN clients.

The router itself has internet connectivity (verified by pinging 8.8.8.8, and checking that DNS resolution works) - however, nothing on the LAN interfaces does.

I upgraded to 2.5.0.a.20200910.0050 (previous build was likely from sometime in July 2020) which is when the issue started. I've since upgraded to 2.5.0.a.20200910.0650, and the issue persists.

In /var/log/system.log, I see it filled with lines like:

Sep 11 05:01:15 grandstandparade-router kernel: cannot forward src fe80:15::204:4bff:fee4:6dcf, dst 2404:6800:4006:80b::2003, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:01:23 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:01:28 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:01:35 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:01:42 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:01:49 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:01:56 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:02:03 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:02:10 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:02:16 grandstandparade-router kernel: cannot forward src fe80:15::204:4bff:fee4:6dcf, dst 2404:6800:4006:808::200a, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:02:24 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:02:32 grandstandparade-router kernel: cannot forward src fe80:15::204:4bff:fee4:6dcf, dst 2404:6800:4006:80b::2003, nxt 6, rcvif lagg0.4091, outif lagg0.4090
Sep 11 05:02:38 grandstandparade-router kernel: cannot forward src fe80:15::46a:9c20:b626:5bad, dst 2001:19f0:5801:10b7:5400:2ff:feaa:284c, nxt 6, rcvif lagg0.4091, outif lagg0.4090

Does anybody have any idea what the issue is, or how to fix it please?

Thanks,
Victor

kiokoman

I don't think that's the problem of your internet connectivity
https://forum.netgate.com/post/933504

there must be something else
like a firewall rule / floating rule or a routing problem or a gateway / interface problem
I'm actually using

Version 2.5.0-DEVELOPMENT (amd64)
built on Wed Sep 09 01:01:28 EDT 2020
FreeBSD 12.2-PRERELEASE

but I don't own a xg-7100

victorhooi

The thing is - the configuration was the same before/after the upgrade ...perplexed

jimp

I updated my XG-7100 to the latest 2.5.0 snapshot today and connected a LAN client, it pulled an IP address, could resolve DNS, and browse without problems on IPv4 and IPv6. I don't see a general problem here.

Do you see any other messages in the system log or other logs that might point to other causes?

Do LAN clients get an IP address? Resolve DNS? Can they ping pfSense? The upstream gateway? A host on the Internet?

Run through https://docs.netgate.com/pfsense/en/latest/routing/connectivity-troubleshooting.html and try to narrow things down

victorhooi

Hmm, that is super strange - I just did the upgrade on a different XG-7100 unit we own, and it exhibits similar symptoms.

Previously version was around July, current version is now 2.5.0.a.20200916.1850.

LAN clients do get an IP address.

However, they cannot resolve DNS:

# dig www.google.com
; <<>> DiG 9.11.5-P4-5.1+deb10u2-Debian <<>> www.google.com
;; global options: +cmd
;; connection timed out; no servers could be reached

They are able to ping the pfSense device:

# ping 10.5.10.1
PING 10.5.10.1 (10.5.10.1) 56(84) bytes of data.
64 bytes from 10.5.10.1: icmp_seq=1 ttl=64 time=0.224 ms
64 bytes from 10.5.10.1: icmp_seq=2 ttl=64 time=0.108 ms
64 bytes from 10.5.10.1: icmp_seq=3 ttl=64 time=0.259 ms
^C
--- 10.5.10.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 51ms
rtt min/avg/max/mdev = 0.108/0.197/0.259/0.064 ms

From a LAN client, I am also able to ping the WAN IP address:

# ping 149.97.163.250
PING 149.97.163.250 (149.97.163.250) 56(84) bytes of data.
64 bytes from 149.97.163.250: icmp_seq=1 ttl=64 time=0.103 ms

64 bytes from 149.97.163.250: icmp_seq=2 ttl=64 time=0.101 ms
64 bytes from 149.97.163.250: icmp_seq=3 ttl=64 time=0.106 ms
64 bytes from 149.97.163.250: icmp_seq=4 ttl=64 time=0.103 ms
^C
--- 149.97.163.250 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 72ms
rtt min/avg/max/mdev = 0.101/0.103/0.106/0.007 ms

But they cannot ping 8.8.8.8 - it just hangs indefinitely:

# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

Back on the pfSense box itself - this does appear to have internet connectivity:

[2.5.0-DEVELOPMENT][root@angusmtv-mgmt.localdomain]/var/log: ping www.google.com
PING www.google.com (216.58.194.196): 56 data bytes
64 bytes from 216.58.194.196: icmp_seq=0 ttl=121 time=1.417 ms
64 bytes from 216.58.194.196: icmp_seq=1 ttl=121 time=1.419 ms
^C
--- www.google.com ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.417/1.418/1.419/0.001 ms

Here are my system.log files:

https://gist.github.com/victorhooi/969e3a6f2d2a19d2036eba2cf65a68c1 (Most recent)
https://gist.github.com/victorhooi/a2ffea7d1b2edf2b26e3f694c261275e (Older)

I can't spot anything obvious there.

jimp

Maybe your rules aren't loading?

https://redmine.pfsense.org/issues/10861