Not able to access websites/network connection issues

bmeeks

The XG 7100 is an older appliance first introduced in 2018 from what I found in a search. So, I am assuming you obtained this applicance used or did you purchase it new directly from Netgate?

I am tagging @stephenw10 in this thread. He is the Netgate hardware guru.

The XG 7100 is a special animal in that it has an integrated Ethernet switch. Not all of the ports on the device are fully independent ports. Some are part of the built-in switch.

Have you read all through the section in the XG 7100 docs on the switch configuration? Here is that link: https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/.

I do not believe that lagg0 configuration is valid with both LAN and WAN in the same lagg. That makes no sense. I can perhaps see those particular switch ports being grouped as a lagg, but that lagg should not contain both LAN and WAN ports. So, in other words, all the lagg0 ports are LAN or all are WAN, but never both LAN and WAN.

Here is an example switch reconfiguration from the docs to separate out the WAN interface from the default lagg0 virtual interface: https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/configuring-the-switch-ports.html#switch-configuration-examples. You could try following this example to isolate at least the WAN port from the lagg0.

SteveITS

https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/switch-overview.html#switch-lagg

“In the default configuration, two VLANs are used to create the ETH1 WAN interface and ETH2-8 LAN interface:

WAN
VLAN 4090

LAN
VLAN 4091”

Looks like they are set up as VLANs on an 8 port switch, similar to the 1100. The VLAN isolates the port used for WAN and leaves the rest on LAN.

So while it looks a bit odd, seems to be normal.

If 192.168.0.1 is the ISP router, seems like it has a problem, bad patch cable, something causing packet loss.

bmeeks

@SteveITS said in Not able to access websites/network connection issues:

https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/switch-overview.html#switch-lagg

“In the default configuration, two VLANs are used to create the ETH1 WAN interface and ETH2-8 LAN interface:

WAN
VLAN 4090

LAN
VLAN 4091”

Looks like they are set up as VLANs on an 8 port switch, similar to the 1100. The VLAN isolates the port used for WAN and leaves the rest on LAN.

So while it looks a bit odd, seems to be normal.

If 192.168.0.1 is the ISP router, seems like it has a problem, bad patch cable, something causing packet loss.

Yeah, that is definitely weird to me with wrapping LAN and WAN in the lagg0 interface but with VLANs. There is a big red DANGER warning in the docs that says "do not delete the lagg0 interface". So, it must be some special requirement for the XG 7100 hardware to function.

I agree the box is having problems getting out reliably (or at least getting replies back in). At first I thought it would be all confused with the lagg0 setup, but apparently that's something unique to the XG 7100.

LMorefield

This is what happens when it starts malfunctioning

Then it says the web address could've been mistyped. I tried screenshotting that as well but the website loaded before I could switch tabs back.

Any idea where to look or what to troubleshoot to fix this?

bmeeks

@LMorefield said in Not able to access websites/network connection issues:

This is what happens when it starts malfunctioning

Then it says the web address could've been mistyped. I tried screenshotting that as well but the website loaded before I could switch tabs back.

Any idea where to look or what to troubleshoot to fix this?

This error from a web browser means it was unable to connect to the requested website. There can be two fundamental reasons for that.

The first thing that must work correctly is the domain name must be looked up and translated into an actual IP address for a connection to happen. So, your web browsers asks its configured DNS server to find the IP address for the domain name "www.thechunkychef.com". That domain name resolves to three IP addresses:

173.239.8.164
173.239.5.6
74.206.228.78

Once it receives the IP address (or addresses in this case), the browser attempts a connection to the IP.

So, the first place for failure is the DNS lookup fails and so the browser has no IP address to use. The second place for failure is the ISP connection is faulty and the connecction to the IP address fails (or, back to the first problem, the connection is so bad that the DNS lookup attempt failed because it could not connect to the Internet to query the domain name servers).

You need to determine which issue you are having: DNS failures or network connectivity failures -- or both (network failures will naturally lead to DNS lookup failures).

Examine the pfSense system log under STATUS > SYSTEM LOGS. Do you see messages about WAN interface alarms and packet loss? If you do, then your ISP connection is at fault. That could be a bad port on your firewall, it could be a bad RJ45 connecting cable, or it could be a problem with your ISP equipment or connection.

Looking at the log entries you supplied earlier, it certainly looks to me that your ISP connection is sporadic. That might be a bad cable, or their support folks might be sandbagging you when they claim there is no problem on their side.

There is a process in the default pfSense setup called dpinger. This is a program that constantly sends an icmp ping request to the configured default gateway. In your case that gateway IP appears to be 192.168.0.1. So long as that gateway IP replies to the ping request in a timely manner, dpinger assumes everything is fine. But if the gateway does not respond to the ping request in the timeout window configured, then dpinger assumes the interface is down and it will attempt to restart it. This is all configured under the SYSTEM > ROUTING menu for gateway monitoring. If your ISP connection is flaky, or if the configured gateway gets busy and does not reply to the pings in a timely manner, dpinger can be triggered to restart things. During that restart your WAN connection will go away and then come back. That disrupts things on your firewall including the unbound DNS Resolver that looks up IP addresses for domain names.

At first I was shocked by the lagg0 setup, but upon further reading in the docs it appears that's a normal but unique thing for the XG 7100.

LMorefield

@bmeeks
Recent logs

It's weird that it worked fine for 1 week, then yesterday it started malfunctioning.

bmeeks

@LMorefield said in Not able to access websites/network connection issues:

It's weird that it worked fine for 1 week, then yesterday it started malfunctioning.

Then that points a finger directly at either your ISP or a bad cable.

First thing to do is swap out the cable connecting the WAN port of your firewall to the ISP's equipment. Next thing I would do, if that does not correct the problem, is get back on the phone with the ISP support and try to convince them it's their problem.

You have actual packet loss there. It's not enough to trigger dpinger to restart the interface, but it is significant loss. That loss is enough to cause issues with regular network traffic such as DNS name resolutions.

Gertjan

@bmeeks said in Not able to access websites/network connection issues:

At first I was shocked by the lagg0 setup, but upon further reading in the docs it appears that's a normal but unique thing for the XG 7100.

Thanks for the info.

LMorefield

I changed a few things today and seem to have fixed the issue. Now the issue will be trying to figure out which one actually fixed it (at least for the time being).

I was still having the same intermittent issues after making a brand new cat-6 ethernet cable that passed when tested. There are multiple lines of internet entering my org. The ISP is the same, however, the accounts are different. I used the ethernet from a different account. The speeds are much slower, but it's a different account, different modem, etc.

I disabled the DHCP6 Server

I changed the IPv6 Configuration type from "Tracking" to "none"

Here are my logs:

System General

Gateways

Firewall

I'll leave the current ethernet (not the original) connected through Tuesday 11/28, as that's the next day we'll have everyone in and the system will be loaded. If it works through then, I'll revert to the old ethernet and get on the horn with the ISP ensuring they fix the issue.

Any feedback on the changes I've made regarding IPv6 and DHCP6? Thanks in advance! Happy Thanksgiving!

bmeeks

If your ISP does not provide IPv6 service, then certainly disable those settings. But if your ISP provides an IPv6 connection, enabling that in pfSense is fine. However, if you are not skilled in the networking art, it may be better to not attempt to configure IPv6 because it seems each ISP has their own unique "quirks" in their implemention of that protocol.

The other thing I notice in your logs is that you seem to have the "Block Private Networks" setting enabled under INTERFACES > WAN. Your default gateway looks to be in RFC 1918 space (192.168.0.1), so you definitely would want to uncheck that option as shown below: