sendto: 65 with UK ISP and PFsense

[stephenw10]

The system log would at least show the WAN going down at that point, the gateway monitoring failing etc. There must be something shown?

[F022Y]

@stephenw10 So under Gateways i have this at time of fail.

From PPP

None of the other logs show anything at that time.

[stephenw10]

Nothing in the main system log? I expect to at least some duplicated entries there.

[F022Y]

@stephenw10 Where should I be looking in case im being stupid?

[stephenw10]

In the main system logs in Status > System Logs > System Tab you wiulkd usually see most of the ppp entries as well as gateway entries and, importantly, other things that may have triggered the connection to close.
"connection closed" is not normally the first log entry like that. If it was caused by something like the parent interface loosing link, that's where it would appear.
Trying to find an example but my own PPPoE has been up so long it's scrolled out of the logs.

Steve

[F022Y]

@stephenw10 In which case thats the logs i've given just dies off. Could it be the Vigor166 then? I have it in modem/bridge mode

[stephenw10]

This is what it looks like if the parent NIC loses link for example:

Jul 21 22:42:05 	kernel 		e6000sw0port3: link state changed to DOWN
Jul 21 22:42:05 	check_reload_status 	453 	Linkup starting $e6000sw0port3
Jul 21 22:42:06 	check_reload_status 	453 	Reloading filter
Jul 21 22:42:19 	rc.gateway_alarm 	99048 	>>> Gateway alarm: LAN3_PPPOE (Addr:10.0.10.254 Alarm:1 RTT:.855ms RTTsd:.123ms Loss:22%)
Jul 21 22:42:19 	check_reload_status 	453 	updating dyndns LAN3_PPPOE
Jul 21 22:42:19 	check_reload_status 	453 	Restarting ipsec tunnels
Jul 21 22:42:19 	check_reload_status 	453 	Restarting OpenVPN tunnels/interfaces
Jul 21 22:42:19 	check_reload_status 	453 	Reloading filter
Jul 21 22:42:21 	php-fpm 	63657 	/rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. ''
Jul 21 22:42:22 	php-fpm 	63657 	/rc.openvpn: Static Routes: Gateway IP could not be found for 192.168.140.0/24
Jul 21 22:42:22 	php-fpm 	63657 	/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use LAN3_PPPOE.
Jul 21 22:42:25 	ppp 	72767 	[opt3_link0] LCP: no reply to 1 echo request(s)
Jul 21 22:42:35 	php-fpm 	25413 	/rc.newipsecdns: IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
Jul 21 22:42:35 	check_reload_status 	453 	Reloading filter
Jul 21 22:42:35 	ppp 	72767 	[opt3_link0] LCP: no reply to 2 echo request(s)
Jul 21 22:42:45 	ppp 	72767 	[opt3_link0] LCP: no reply to 3 echo request(s)
Jul 21 22:42:55 	ppp 	72767 	[opt3_link0] LCP: no reply to 4 echo request(s)
Jul 21 22:43:05 	ppp 	72767 	[opt3_link0] LCP: no reply to 5 echo request(s)
Jul 21 22:43:05 	ppp 	72767 	[opt3_link0] LCP: peer not responding to echo requests
Jul 21 22:43:05 	ppp 	72767 	[opt3_link0] LCP: state change Opened --> Stopping
Jul 21 22:43:05 	ppp 	72767 	[opt3_link0] Link: Leave bundle "opt3"
Jul 21 22:43:05 	ppp 	72767 	[opt3] Bundle: Status update: up 0 links, total bandwidth 9600 bps
Jul 21 22:43:05 	ppp 	72767 	[opt3] IPCP: Close event
Jul 21 22:43:05 	ppp 	72767 	[opt3] IPCP: state change Opened --> Closing
Jul 21 22:43:05 	ppp 	72767 	[opt3] IPCP: SendTerminateReq #4
Jul 21 22:43:05 	ppp 	72767 	[opt3] IPCP: LayerDown
Jul 21 22:43:06 	check_reload_status 	453 	Rewriting resolv.conf
Jul 21 22:43:06 	ppp 	72767 	[opt3] IFACE: Down event
Jul 21 22:43:06 	ppp 	72767 	[opt3] IFACE: Rename interface pppoe0 to pppoe0
Jul 21 22:43:06 	ppp 	72767 	[opt3] IPCP: Down event
Jul 21 22:43:06 	ppp 	72767 	[opt3] IPCP: LayerFinish
Jul 21 22:43:06 	ppp 	72767 	[opt3] Bundle: No NCPs left. Closing links...
Jul 21 22:43:06 	ppp 	72767 	[opt3] IPCP: state change Closing --> Initial
Jul 21 22:43:06 	ppp 	72767 	[opt3_link0] LCP: SendTerminateReq #3
Jul 21 22:43:06 	ppp 	72767 	[opt3_link0] LCP: LayerDown
Jul 21 22:43:09 	ppp 	72767 	[opt3_link0] LCP: SendTerminateReq #4
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] LCP: state change Stopping --> Stopped
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] LCP: LayerFinish
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] PPPoE: connection closed
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] Link: DOWN event
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] LCP: Down event
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] LCP: state change Stopped --> Starting
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] LCP: LayerStart
Jul 21 22:43:11 	ppp 	72767 	[opt3_link0] Link: reconnection attempt 1 in 3 seconds 

And this if you just disconnect the PPPoE manually:

Jul 21 22:53:06 	ppp 	72767 	caught fatal signal TERM
Jul 21 22:53:06 	ppp 	72767 	[opt3] IFACE: Close event
Jul 21 22:53:06 	ppp 	72767 	[opt3] IPCP: Close event
Jul 21 22:53:06 	ppp 	72767 	[opt3] IPCP: state change Opened --> Closing
Jul 21 22:53:06 	ppp 	72767 	[opt3] IPCP: SendTerminateReq #8
Jul 21 22:53:06 	ppp 	72767 	[opt3] IPCP: LayerDown
Jul 21 22:53:07 	check_reload_status 	453 	Rewriting resolv.conf
Jul 21 22:53:07 	ppp 	72767 	[opt3] IFACE: Down event
Jul 21 22:53:07 	ppp 	72767 	[opt3] IFACE: Rename interface pppoe0 to pppoe0
Jul 21 22:53:07 	ppp 	72767 	[opt3] IPCP: rec'd Terminate Ack #2 (Closing)
Jul 21 22:53:07 	ppp 	72767 	[opt3] IPCP: state change Closing --> Closed
Jul 21 22:53:07 	ppp 	72767 	[opt3] IPCP: LayerFinish
Jul 21 22:53:07 	ppp 	72767 	[opt3] Bundle: No NCPs left. Closing links...
Jul 21 22:53:07 	ppp 	72767 	[opt3] Bundle: closing link "opt3_link0"...
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] Link: CLOSE event
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: Close event
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: state change Opened --> Closing
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] Link: Leave bundle "opt3"
Jul 21 22:53:07 	ppp 	72767 	[opt3] Bundle: Status update: up 0 links, total bandwidth 9600 bps
Jul 21 22:53:07 	ppp 	72767 	[opt3] IPCP: Close event
Jul 21 22:53:07 	ppp 	72767 	[opt3] IPCP: Down event
Jul 21 22:53:07 	ppp 	72767 	[opt3] IPCP: state change Closed --> Initial
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: SendTerminateReq #94
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: LayerDown
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: rec'd Terminate Ack #3 (Closing)
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: state change Closing --> Closed
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: LayerFinish
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] Link: DOWN event
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: Down event
Jul 21 22:53:07 	ppp 	72767 	[opt3_link0] LCP: state change Closed --> Initial
Jul 21 22:53:09 	ppp 	72767 	[opt3] Bundle: Shutdown
Jul 21 22:53:09 	ppp 	72767 	[opt3_link0] Link: Shutdown
Jul 21 22:53:09 	ppp 	72767 	process 72767 terminated

Your logs do not match either.

[F022Y]

@stephenw10 Is there an export function of all the logs? Just thinking when it next happens i just export everything.

[stephenw10]

You can just download the full log files. The main system log for example is:
/var/log/system.log

You can download that from Diag > Command Prompt.

Steve

[F022Y]

@stephenw10 Thank you for the help, i've cleared the logs and will wait to see if it dies.

[F022Y]

So it all went again today I've grabbed the logs for someone to see if i've missed anything but these are the screenshots along with one from the draytek that shows a link (shows wrong date as in modem mode).

I'm really struggling now.

[F022Y]

Just noticed that even though i can access the internet the gateway in PFsense is showing down, could it be if the gateway stops responding the connection drops?

I'm clutching at straws now lol

[F022Y]

@stephenw10 Is a gateway needed in pfsense or dpinger?

[stephenw10]

You need a gateway of some sort for pfSense to send traffic to but dpinger does not need to use that for the monitoring address. You can disable the monitoring completely if you only have one WAN.

In that last set of logs it look like the first thing shown in the general system log is 'PPPoE: Connection closed' but there is more shown before that in the PPP log.
Is what you showed there the first thing at that time in that log?

Really we want to see what is triggering the connection to close. If it's something on the pfSense end we may be able to address that. If it;s the server end closing the connection the options are limited.

Steve

[F022Y]

@stephenw10 Happy to send the full logs over if you want to take a look. ISP is being...... difficult. They have advised the gateway address they gave me was actually another customers IP who has now left facepalm.

I'm waiting on them to give me all the details of my connection and i'm going to start again.

[stephenw10]

@f022y said in sendto: 65 with UK ISP and PFsense:

Cloudscape Connect

I'm not familiar with them but the fact they gave you any information at all is a good sign. Sounds like they at least looked which is more than you'd get from most ISPs!

If they are saying that they sent you a bad gateway then it's on them to not do that really. I wouldn't expend too much more effort until they are at least claiming to have rectified that issue.

Steve

[F022Y]

@stephenw10 So just got an email saying the IP i had isn't a customer IP but it is the gateway address.

I have asked for all the settings they would provide someone to configure a router just for my sanity.

[Thondwe]

So I had a very similar scenario last week - PFsense would nicely close the PPPoE connection (3 or 4 times over a couple of days) and then take ages to reconnect. My ISP (Aquiss) got OpenReach out to check the Fibre and everything seemed to check out fine, though they reported they could see rather more drops than just the 3 or 4 I could see - 18 or so - no further issues since though.

So I think the connection was dropping, but only a few times it was long enough for PFsense to give up and shut up shop? However, the other end of PPPoE link (run by OpenReach?) would stay up longer and we had to wait until it timed out before Pfsense would reconnect?

I've since added the Gateways widget to my Dashboard and replaced the monitor IPs with something further down the line - as ISP provided gateway didn't respond to ping -

Can't seem to get the IPv6 address to respond (though IPv6 is running fine). Noting that I assume I've got a reused IPv4 address as my firewall is repeating blocking an attempt to connect on port 500 (IPSEC VPN?) from some other address!

Bottom line - my problem has disappeared for the moment, - so either fixed by re-seating of cables during OpenReach Test or some other hidden change - but if it's some random issue...