PPPoE reconenction fix - mpd fix ($100)

w0w

I can't see any other software fix, instead on initiating reboot. This looks like more complex problem then driver or mpd only. So we need to write the if function to the ppp-linkdown, that checks for running shutdown process, and if it does not exists, starts one like 'shutdown -r +10', that means if script is running for the first time it always wants to restart pfsense after 10 minutes. And we need to add one line only to the ppp-linkup that looks like
'pkill shutdown'.

I don't have knowledge to write the if function correctly, so I just added to the end of ppp-linkdown
shutdown -r +1

and

pkill shutdown
to the ppp-linkup file.
Every reconnect starts own shutdown process but you don't have them much in one minute (yes +1 is a minute) before first of them restarts the pfsense. And if the PPPoE is coming up in less then 1 minute then shutdown process will be killed.
It would be good if somebody helps to write if function described above.

w0w

My friend suggested me to modify his old SQL check script, so it would be

ppp-linkdown


#!/bin/sh
if [ -f /tmp/$1up ] && [ -f /conf/$1.log ]; then
        seconds=$((`date -j +%s` - `/usr/bin/stat -f %m /tmp/$1up`))
        /usr/local/sbin/ppp-log-uptime.sh $seconds $1 &
fi
if [ "$3" != "" ]; then
	echo "Removing states from $3" | logger -t ppp-linkdown
	/sbin/pfctl -k 0.0.0.0/0 -k $3/32
	/sbin/pfctl -k $3/32
	pfctl -K $3/32
fi
if [ "$4" != "" ]; then
	echo "Removing states to $4" | logger -t ppp-linkdown
	/sbin/pfctl -b 0.0.0.0/32 -b $4/32

	if [ -f "/tmp/${interface}_defaultgw" ]; then
		route delete default $4
	fi

fi
# delete the node just in case mpd cannot do that
/usr/sbin/ngctl shutdown $1:
if [ -f "/var/etc/nameserver_$1" ]; then
	# Remove old entries
	for nameserver in `cat /var/etc/nameserver_$1`; do
		/sbin/route delete $nameserver >/dev/null 2>&1
	done
	/bin/rm -f /var/etc/nameserver_$1
fi
# Do not remove gateway used during filter reload.
/bin/rm -f /tmp/$1_router
/bin/rm -f /tmp/$1up
/bin/rm -f /tmp/$1_ip
/usr/local/sbin/pfSctl -c 'service reload dns'
ifconfig em0 down
ifconfig em0 up

#initiate reboot if PPPoE reconnection fails.

PID=`ps auxww | grep shutdown | grep -v grep | awk '{ print $2 }'`
if  ps auxww | grep -e shutdown | grep -v grep > /dev/null ; then
	echo "shutdown running on PID $PID"
	else
	echo "starting shutdown/reboot of pfsense"
	shutdown -r +10
fi

Tested OK!

ppp-linkup

#!/bin/sh

# let the configuration system know that the ip has changed.
/bin/echo $4 > /tmp/$1_router
/bin/echo $3 > /tmp/$1_ip
/usr/bin/touch /tmp/$1up

ALLOWOVERRIDE=`/usr/bin/grep dnsallowoverride /conf/config.xml | /usr/bin/wc -l`
if [ $ALLOWOVERRIDE -gt 0 ]; then
	# write nameservers to file
	if [ $6 = "dns1" ]; then
		echo $7 > /var/etc/nameserver_$1
		/sbin/route change $7 $4
	fi

	if [ $8 = "dns2" ]; then
		echo $9 >> /var/etc/nameserver_$1
		/sbin/route change $9 $4
	fi
	/usr/local/sbin/pfSctl -c 'service reload dns'
	/bin/sleep 1
fi

/usr/local/sbin/pfSctl -c "interface newip $1"
pkill shutdown
exit 0

w0w

Please use 'shutdown -r +5' instead of 'shutdown -r +10'
I've found that it is impossible to kill the shutdown process if using +10 (10 minutes).
When tested I've used less then 5 minutes delay.


[2.1-DEVELOPMENT][root@pfhacom.xxxnetxxx]/root(326):                                                            
*** System shutdown message from root@pfhacom.xxxnetxxx ***
System going down in 10 minutes

[2.1-DEVELOPMENT][root@pfhacom.xxxnetxxx]/root(327): ps auxww | grep shutdown | grep -v grep  
 root    2293  0.0  0.1  3316  1124  ??  I
``` ~~after "pkill shutdown"

ps auxww | grep shutdown | grep -v grep
root 25185 0.0 0.1 3316 1124 ?? S


*** System shutdown message from root@pfhacom.xxxnetxxx ***
System going down in 5 minutes

I can kill the process by pkill shutdown. WTF? What am I doing wrong?

If I start shutdown -r +10 or any number in console, I can always kill it.~~~~

xbipin

i havent tried this as yet, will do so soon

TimAtHome

After suffering from very similar logs and lots of googling I found a cure (well a cause) for me anyway.

If the router came up after pfSense (if it was powered off then on again), pfSense would not connect to it no matter what unless you power cycled it again or yanked the cable. What I found was happening was when the link first came up pfSense would grab the ip address from the router using DHCP and start sending its pppoe requests to it (producing the same logs about connecting posted earlier), it would never log in. Pulling the cable, waiting a bit then plugging it back in would then log in straight away.

What was happening was the first ip address it obtained from the router was a local 'admin' address (in my case 192.168.2.10), the router would then connect to whatever destination and sync, it would then change it's ip address to the correct WAN IP (In my case 217.155..), but as the link status stayed as 'up' pfSense would not grab the new ip address and keep sending it's pppoe junk to the original 192.168.2.10 address. No amount of ifconfig up/down would shift it.

The solution to my problem was just to disable the DHCP server in the router as it isn't required anyway (admin access can be got using a static ip). Now I can power on/off router or pull/reconnect cable and pfSense picks up the status and the correct ip address straight away with no rebooting or anything.

I hope this is of use to someone as googling this issue picks this post as the top one.

w0w

I'd just like to say that I have no problem with reconnection on the latest 2.1 versions based on freebsd 8.3 and new MPD version

xbipin

i also got sick and tired of this issue but currently im using w0w's hack but the strange thing is after i implemented it, the rotuer is supposed to restart in a specific time if pppoe reconnection fails and i have set this to 3 mins so almost about 2:50 mins pppoe doesnt reconnect and when its about to reboot, it just reconencts magically.
i tried removing that hack but then it would reconnect in hours also.

i hope some of the developers read this and posted some code change such that u can use DHCP as well as pppoe reconnects also if dhcp was the main issue

w0w

xbipin
looks like
"ifconfig em0 down
ifconfig em0 up"
works but delayed.

Actually I've written another one startup script


#!/bin/sh
IP=`/usr/bin/netstat -rn | /usr/bin/grep 'default' | /usr/bin/awk '{ print $2}'`
/sbin/ping -c 2 $IP > /dev/null
if [ $? != 0 ] ; then
/sbin/ifconfig em0 down
/sbin/ifconfig em0 up
    if  /bin/ps auxww | /usr/bin/grep -e shutdown | /usr/bin/grep -v grep > /dev/null ; then
	echo "shutdown running"
	else
	echo "starting shutdown/reboot of pfsense"
	/sbin/shutdown -r +5
fi
    fi

I've named it "ppp_force_linkup.sh" in the /usr/bin/ (Don't forget to change access to the file via WinSCP - properties - Octal: 0755 aka rwxr-xr-x )

Next I've downloaded config and edited:

/usr/local/sbin/expiretable -v -t 3600 sshlockout

-------------------------------------inserted code begin-----------------------------------------------------------
		 <minute>*/30</minute>
			<hour>*</hour>
			<mday>*</mday>
			<month>*</month>
			<wday>*</wday>
			<who>root</who>
			<command></command>/usr/bin/nice -n20 /usr/bin/ppp_force_linkup.sh 
-------------------------------------inserted code end-----------------------------------------------------------
		 <minute>1</minute>
			<hour>1</hour>
			<mday>*</mday>
			<month>*</month>
			<wday>*</wday>
			<who>root</who>
			<command></command>/usr/bin/nice -n20 /etc/rc.dyndns.update 
		 <minute>*/60</minute>
			<hour>*</hour>
			<mday>*</mday>
			<month>*</month>
			<wday>*</wday>

Every 30 min script looks for current getaway IP, pings it and if no success then it restarts the link and reboot is sheduled. if you are using previously modded ppp-linkup and ppp connection is established in that time, then sheduled reboot is canceled as it should be.
All those mods are needed only for the 2.0x branch, I think, this script was made for those situations, when PC is rebooted but no PPPoE link established for some reason or mpd runned into "stuck" mode or somethin else…
Even having this script enabled, I've got situation when getaway ping was OK but pinging any internet server
was failed (ISP problem, actually)

So may be it's better to ping some www.google.com instead of pinging ISP getaway
something like that:
delete
IP=/usr/bin/netstat -rn | /usr/bin/grep 'default' | /usr/bin/awk '{ print $2}'
/sbin/ping -c 2 $IP > /dev/null
and
write only
/sbin/ping -c 2 www.google.com > /dev/null

But if it is ISP problem, then reboot does not help, so I've preffered to ping ISP getaway

xbipin
Have you tried new 2.1 builds already? My reconnection bug is gone completely.
[2.1-BETA0][root@pfhacom.xxxnetxxx]/root(1): mpd5 -v
Version 5.6 (root@base-8_3-i386.builders.pfsense.org 21:22 6-Aug-2012)
works great for me.
I am not sure about other things as it is BETA0 :)
At least you can try, don't forget to backup your config or do full backup.
Before update don't forget to select the update source in "System: Firmware: Updater Settings"
http://snapshots.pfsense.org/FreeBSD_RELENG_8_3/i386/pfSense_HEAD/.updaters
and check also
"Allow auto-update firmware images with a missing or invalid digital signature to be used." and
"After updating, sync with the following repository/branch before reboot." (branch: master and repository url: git://github.com/bsdperimeter/pfsense.git )

Good luck :)

xbipin

thanks w0w, actually the first script works which u gave earlier so ill stick to that for now, i was planning to move to 2.1 but problem is its too early for that as it seems there r still many bugs in the core system functionality so ill wait a bit its its more stable

things i need to work properly r
upnp
dhcp
static arp
traffic shaper
limiter
pppoe
openvpn client
accesspoint
aliases
NAT
firewall rules

w0w

I hope stable release coming mid-October.
Currently
upnp - OK for me
dhcp - OK on LAN
static arp - OK
traffic shaper - fixed some time ago, OK
limiter - may be :)
pppoe - OK
openvpn client - don't know
accesspoint - don't know
aliases - OK
NAT - OK (IPV4)
firewall rules - OK

xbipin

im on 2.1 and the problem still persists, can u provide some patch like u did for 2.0.1

w0w

That's strange, xbipin, that the problem still persists.. try to add only
ifconfig em0 down
ifconfig em0 up
those to the ppp-linkdown and look what is happening
or just do the full modification similar to old version

xbipin

ill try the old patch u gave on this and let u know.

i dont think the mpd etc has changed from 2.0.1 to 2.1 thats y the problem persists

xbipin

the old hack still works except some errors popup during boot saying itnerface vr1 not available or something like that but it connects eventually and even if i reset my isp modem or unplug cable etc still manages to connect within 3 mins

w0w

from 2.0.x to 2.1 there is changed freebsd version from 8.1 to 8.3 and mpd is also 5.6 compiled in august.

xbipin

but still doesnt solve my issue

flipmoo

im running into the same issue here.
i just switched to pfSense to get WANFailover.
i have 2 WANs, 16MBit DSL and 50MBit VDSL

after initioal install everything is running right. For testing i powered off the VDSL Modem, Gateway is recognized as down, WAN Failover goes to DSL connection.
BUT after repowering the VDSL connection (Telekom) no automatic reconnect. Not even a manual reconnect. The Interface does not get any IP information, though the status is connected…
now i just run with my slow conenction.

i tryed 386 and amd64 version and also 2.0.1 and 2.1
seeing that this issue exists for a long time now, i doubt that pfSense is the product of my choice. It looks very prommising, but essentials like the PPPoE connect just need to work properly!

I hope that the Developers give it a fast try of fixing, so i can stay with this firewall/router distribution.

thx in advance

xbipin

actually its not a pfsense issue but some incompatibility with mpd and the pppoe server by the isp which has some security feature enabled and it seems many isp run the same hardware thats y more ppl will have the same issue sooner or later.

u can try the first patch posted by wow as i use that and works as of now to fix it temporarily.

i wonder how the local routers in my region reconnect fine without issues, actually all of them, only pfsense has this issue

flipmoo

yeah… even the cheapest ones are doin it right... kinda disappointing.
allright.. i gonna try that, though i am very unexperanced with bsd and coding in genaral :(

thx for the quick advice!

flipmoo

thx xbipin, the reconnect works now.
but my failover is not working anymore.. do i need to reconfigure that competely?