WAN-Link goes down once in a while

Alphaphi

Hello everyone,

on a pfSense 2.0.2 the WAN link goes down every once in a while, which cuts off internet access. The logs say:

Feb 20 09:21:50
php: : The command '/sbin/route change -inet default '192.168.1.1'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable route: writing to routing socket: Network is unreachable change net default: gateway 192.168.1.1: Network is unreachable'

Feb 20 09:21:51 apinger: Starting Alarm Pinger, apinger(3459)
Feb 20 09:22:01 apinger: ALARM: GW_WAN(217.5.98.14) *** down ***
Feb 20 09:22:01 apinger: ALARM: WANGW(192.168.1.1) *** down ***

(complete log here: http://pastebin.com/ekTvmGFK)

I don't know where these two IPs come from. 217.5.98.14 is not the WAN IP of the pfSense, 192.168.1.1 is not the LAN IP, nor is it in the LAN IP scope, which is 192.168.4.0.
The WAN is of type PPPoE with a fixed IP, using username and password to login to the ISP (see XML snippet below).

It does not help to unplug/plug the WAN link. I need to reboot the pfSense, and everything is fine for a few weeks.

Anybody an idea what goes wrong here?

Thanks,
Andre


	 <ppps><ppp><ptpid>0</ptpid>
			<type>pppoe</type>
			<if>pppoe0</if>
			<ports>nfe0</ports>
			<username>TheUserNameHere</username>
			<password>ThePasswordHere=</password></ppp></ppps> 
	 <gateways><gateway_item><interface>wan</interface>
			<gateway>192.168.1.1</gateway>
			<name>WANGW</name>
			 <weight><interval><defaultgw></defaultgw></interval></weight></gateway_item> 
		 <gateway_item><gateway>dynamic</gateway>

			<name>GW_WAN</name>
			<interface>wan</interface></gateway_item> 
		 <gateway_item><gateway>dynamic</gateway>

			<name>GW_OPT1</name>
			<interface>opt1</interface></gateway_item></gateways>

Klaws

By default, apinger pings the WAN's gateway. You can change the ping target via System: Gateways: Edit gateway, by entering some other IP address into Monitor IP.

If the WAN gateway responds unreliably to pings, you'll receive "interface down" alerts, even though it's in reality up all the time. You might try the next hop's IP address (which you might out via a traceroute) as an alternative monitor IP. Perhaps it's more reliable. Or perhaps it doesn't respond ti pings at all…some ISPs are paranoid and think that it's harful if their modems/routers/whatever responds to pings.

Edit: Sorry, seems I skipped a line when reading your post. Your problem is not an unrealibly pingable gateway.

I experienced a similar situation which was caused by the modem (or rather the modem's connection to the "outside world"). Whenever the modem experienced a loss of internet connectivity, it entered some sort of "local admnistration mode", where it activated it's own DHCP server, handing out IP addresses if the 192.168.* range and advertizing itself as default gateway in the same subnet. Obviously, the thing which was reachable in this case was the modem's WebGUI. However, in my case, my ISP doesn't use PPPoE (just simple DHCP), so pfSense would resume normal operation after the connection issue got resolved.

In your case, it might be that the modem enters some sort of "local administration mode" as well, and pfSense gets...irritated by this - perhaps enough that it cannot resume normal operation automatically once the modem recovers.

Well, just some sort of wild guess, I guess. You might check if you can trigger the problem by simply disconnecting your modem from the "outside world". It might not happen all the time, so you might have to try it a few times.

wallabybob

@Klaws:

In your case, it might be that the modem enters some sort of "local administration mode" as well, and pfSense gets…irritated by this - perhaps enough that it cannot resume normal operation automatically once the modem recovers.

This scheme might work "tolerably" well if the modem gave a short DHCP lease time and shutdown its DHCP server when its WAN link is up. Maube such modems have configuration options to control the DHCP server, in which case for opertion with pfSense it would probably be better to disable the modem's DHCP server.

Alphaphi

Hi Klaws,

thanks for your detailed answer.

@Klaws:

I experienced a similar situation which was caused by the modem (or rather the modem's connection to the "outside world"). Whenever the modem experienced a loss of internet connectivity, it entered some sort of "local admnistration mode", where it activated it's own DHCP server, handing out IP addresses if the 192.168.* range and advertizing itself as default gateway in the same subnet. Obviously, the thing which was reachable in this case was the modem's WebGUI. However, in my case, my ISP doesn't use PPPoE (just simple DHCP), so pfSense would resume normal operation after the connection issue got resolved.

It is unknown to me that the modem has some sort of IP address or something like a webGUI. But maybe it has, and it will activate these upon connectivity loss.

I will do two things: first, disonnect/reconnect the DSL line and see, if I can make the pfs's WAN-link go down this way. Second, next time the WAN link goes down by itself, I will ask the ISP if they can see any connectivity interruptions on their port.

Another thing: Clicking "System" > "Routing", the pfs shows the gateways. This is what it looks like here:

WANGW (default) | WAN | 192.168.1.1 | 192.168.1.1 | DLink
GW_WAN | WAN | 217.5.98.14 | 217.5.98.14 | Interfacewandynamic gateway
GW_OPT1 | OPT1 | dynamic | dynamic | Interfaceopt1dynamic gateway

Where does the 192.168.1.1 come from - is this something my ISP provides? Is there supposed to be any IP address at all when using pppoe? Is it supposed to be a private IP? Seeing this, is it a good idea that "Interfaces" > "WAN" > "Block private networks" is ticked?

What about the GW_WAN IP - where does it come from, and what does it stand for?

Again, thanks for reading and thinking,

Klaws

I only have experience with the Cisco EPC3208. In "local mode" (WAN link down), the DHCP lease time is very short (don't exactly remember, maybe a minute or even less). The DHCP server cannot be disabled, though. This leads to an interesting effect: if the modem's WAN link goes, apringer will detect this condition and the RRD quality graph will turn red. Then, after a short time, the modem will go into "local mode" and activate it#s own DHCP server. pfSense will get a 192.168.100.* IP address, and apinger will begin to the modem instead of the "real" ISP gateway. Result: The RRD quality graph will not only turn white again, it will also show that my "internet connection" has just become ultra-fast! Of course, the only thing which I can access with this ultra-fast connection is the modem's WebGUI :)

Alphaphi

@Klaws:

Of course, the only thing which I can access with this ultra-fast connection is the modem's WebGUI :)

At least you cannot catch viruses with this ultra-fast connection ;)

OK, seriously…

1. Do you see the 192.168.100.x address in System > Routing in the WANGW (default) line while your ultra-fast connection is established? Which IP do you see there during normal operation?

2. What did you do to work around this effect? Is there a way to make pfs not request/receive this unwanted dhcp traffic?

wallabybob

Please provide the output of pfSense shell commands```
/etc/rc.banner ; ifconfig


I wonder if:
1\. Your WAN interface used to be DHCP but you forgot to change the interface type to None when you changed to run PPPoE over that physical interface; or
2\. The physical interface supporting the WAN PPPoE interface has a 192.168.1.x IP address so you can talk to the modem (e.g. to a web server on the modem for configuration).

Klaws

@Alphaphi:

1. Do you see the 192.168.100.x address in System > Routing in the WANGW (default) line while your ultra-fast connection is established? Which IP do you see there during normal operation?

When the modem's WAN connection gies down, Default Gateway and Monitor IP get set to the modem's admin IP (192.168.100.1). When everything works as it should, both entries are set to my ISP's gateway (62.143.x.x).

@Alphaphi:

2. What did you do to work around this effect? Is there a way to make pfs not request/receive this unwanted dhcp traffic?

I had my ISP fix the cabling (the cable fault was outside the area of my responsibility). ;)

As DHCP communication is based un UDP, it should be possible to create a firewall rule which blocks UDP traffic from the moden's management IP address. Source port would be 67, destination port 68, source address the management IP address of the modem (192.168.1.1 or 192.168.100.1 or whatever, depending on the model). I haven't tried this out, but I know of people who accidently blocked their WAN DHCP server, so it should work ;)

Of course, all of this assumes that the root cause of your issue is somehow related to the effect described by me. Which I'm actually not totally convinced of.

Best regards, Klaus