Freeradius 2 doesn't start after upgrade to PFS 2.2



  • All counter files on my system are 128 KiB. Can you upload your good file, I will compare the two. I have been running FreeRADIUS package since 2.0 RC days and never experienced this issue.



  • With me the problem is mysql56, freeradius2 can´t start after boot but works ok manually starting service after boot.

    
    radiusd[84593]: rlm_sql_mysql: Couldn't connect socket to MySQL server radius@localhost:radius
    
    

    I´m looking for for a way to delay freeradius2 service to start, I think in this way mysql will be ready for new connections before freeradius starts.



  • New version of FreeRADIUS package 1.6.11 fixes the issue of multiple radiusd instances being launched (see: https://redmine.pfsense.org/issues/4337). It is a long shot that that will resolve your problem, but please upgrade and report back. You do need to remove/install package rather than reinstall for the changes to appear on your system. If the update is installed correctly you should see below lines in /usr/local/etc/rc.d/radiusd.sh

    
    ...
    rc_start() {
            SERVICENAME="radiusd"
            LOCKFILE="/tmp/${SERVICENAME}_start.lock"
            PIDFILE="/var/run/${SERVICENAME}.pid"
    ...
    
    


  • FreeRadius 2.2.6_3 pkg v1.6.11 Pfsense 2.2.1.  remove/install package. The problem is not solved. How to remove a second copy of the program (The FreeRADIUS daemon)? After removal pkg it remains in line "service"
    Log:
    rlm_counter: Failed to open file /var/log/radacct/timecounter/db.daily: No such file or directory
    radiusd[88428]: /usr/local/etc/raddb/modules/counter[71]: Instantiation failed for module "daily"

    /usr/local/etc/rc.d/radiusd.sh –>
    ....
    rc_start() {
    SERVICENAME="radiusd"
    LOCKFILE="/tmp/${SERVICENAME}_start.lock"
    PIDFILE="/var/run/${SERVICENAME}.pid"
    ....






  • Removing the package does not remove "/var/log/radacct/timecounter/db.daily" so delete the file manually and try to start FreeRADIUS.



  • Here also installed the update and rebooted and after that powered off the box and restarted but in both restarts the problem is not solved.



  • Can't reproduce this issue, so unless you guys can post additional info that might help in identifying this problem, this is not going get solved.

    As a work around you can probably modify startup script '/usr/local/etc/rc.d/radiusd.sh' to remove corrupt file before every start.



  • Dears,
    The problem appeared only when I do restore a full backup & the box reboot & re-installing the installed packages…

    I have tried all work around here with no luck !
    in the console it's saying

    Starting package freeradius2...done.
    Starting /usr/local/etc/rc.d/radius.sh...done.
    

    but the service is not started & in the log I can find this:

    
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: Restarting/Starting all packages.
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    Apr 20 02:58:32 	php-fpm[245]: /rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'
    
    

    Seems like the box try to stop the service instead of starting it !!!



  • I 've same problem and I can't resolve it  :-\  :(



  • I have a fresh pfsense install… with only freeradius & got the error only when restoring the all backup file & the system reboot & reinstalling the packages again...

    SOLVED after installing any other package (i.e. squid3 or dansguardian or any other),

    if I remove those packages… the problem retains !!! it must have another package installed to start automatically :D :D



  • For my system with regular install (not NanoBSD) and memory file system for /tmp and /var I found a workaround that worked for me in https://forum.pfsense.org/index.php?topic=92636.0.

    Still I believe that the problem described is a bug in pfsense/freeradius, which in my case was first introduced when upgrading from 2.2 to 2.2.1 (occasionally no service start at (re)boot). It became even worse (never start at (re)boot) when upgrading to 2.2.2. Note that the in the original clean install of 2.2 freeradius worked flawlessly.



  • Hi;

    I experience the exact same issue with a fresh install of pfSense 2.2.2 and freeradius2 1.6.12. Well, not exactly fresh install: I did have to upgrade from pfSense 2.1 as 2.2 does not install correctly on soekris board 6501.The setup is basic with only a connection from RADIUS to a LDAP Active Directory, nothing else installed or configured behond your typically home firewall setup. Service will only start if I save a file on the GUI (no changes necessary) or start manually from command line as described in this post. I notice this now so I have nothing yet to provide in addition to what was already posted.

    I'll use the mentioned work around at https://forum.pfsense.org/index.php?topic=92636.0 for now as I need to move on to other priorities but willing to help if some more knowledgeable people can provide pointers or things to try. As mentioned, starting in debug mode provides no additional clue as start-up manually works flawlessly. Errors in system logs are already provided I those are the ones I see.



  • We were struggling with the same problem, but on a fresh, pristine 2.2.4 on a standar PC

    2.2.4-RELEASE (amd64)
    built on Sat Jul 25 19:57:37 CDT 2015
    FreeBSD 10.1-RELEASE-p15

    So far we have found this strange behavior on logs as shown on snapshot1
    The server is stopping the radius servce instead of starting it up at the boot process.

    The logs shown was taken on a clean boot so we're sure there is some error on php-fm rc.start process for FreeRadius

    By now the workaround was to install Watchdog package and mark FreeRadius for monitor and start up when found stopped or crashed. It's usefull for other purposes :P




  • After looking at the code, it seems that it is the way the package is handling a resync.  During a resync it is restarting the service about 9 times as bg calls and confusing the service.

    I fixed mine by modifying the /usr/local/etc/rc.d/radius.sh so that it did not do a restart unless the service was already running.

    #!/bin/sh
    # This file was automatically generated
    # by the pfSense service handler.
    
    SERVICENAME="radiusd"
    PIDFILE="/var/run/${SERVICENAME}.pid"
    
    rc_start() {
    	LOCKFILE="/tmp/${SERVICENAME}_start.lock"
    
    	# prevent this part of script from running in parallel
    	if ( set -o noclobber; echo "$$" > "$LOCKFILE") 2> /dev/null; then
    		# make sure lock file is removed even if script is terminated
    		trap 'rm -f "$LOCKFILE"; exit $?' INT TERM EXIT
    
    		/usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestart
    
    		# try to wait until the service starts
    		if [ ! -f "$PIDFILE" ]; then
    			echo "$SERVICENAME.sh: PID file was not found"
    
    			for i in 1 2 3 4 5; do
    				if [ -f "$PIDFILE" ]; then
    					echo "$SERVICENAME.sh: Service started PID: `cat $PIDFILE`"
    					break
    				else
    					echo "$SERVICENAME.sh: Waiting 0.5 seconds"
    					sleep 0.5
    				fi
    			done
    		else
    			echo "$SERVICENAME.sh: Service running PID: `cat $PIDFILE`"
    		fi
    
    		rm -f "$LOCKFILE"
    		trap - INT TERM EXIT
    	else
    		echo "$SERVICENAME.sh: Cannot continue at this moment, this script is already trying to start service PID: $(cat $LOCKFILE)"
    	fi
    }
    
    rc_stop() {
    	if [ ! -f "$PIDFILE" ]; then
    		echo "$SERVICENAME.sh: PID file was not found"
    	else
    		/usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestop
    	fi
    }
    
    rc_restart() {
    	if [ ! -f "$PIDFILE" ]; then
    		echo "$SERVICENAME.sh: PID file was not found"
    	else
    		/usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestop
    
    		# try to wait until the service starts
    		for i in 1 2 3 4 5; do
    			if [ ! -f "$PIDFILE" ]; then
    				echo "$SERVICENAME.sh: Service Stopped"
    				rc_start
    				break
    			else
    				echo "$SERVICENAME.sh: Waiting 0.5 seconds to stop"
    				sleep 0.5
    			fi
    		done
    	fi
    }
    
    case $1 in
    	start)
    		rc_start
    		;;
    	stop)
    		rc_stop
    		;;
    	restart)
    		rc_restart
    		;;
    esac
    
    

    And then to clean up the boot process, I then added an internal_name to the package in the /cf/conf/config.xml

    	 <installedpackages><package><name>freeradius2</name>
    			<internal_name>radiusd</internal_name>
    			<website>http://www.freeradius.org/</website></package></installedpackages> 
    

    Now the service is running after every boot.



  • To prevent radiusd being startet multiple times the code in rc_start was added by https://redmine.pfsense.org/issues/4337. This was then merged  in https://github.com/pfsense/pfsense-packages/pull/839/files

    Like SmileyAU suggestet /usr/local/etc/rc.d/radiusd.sh ist started with the Argument start and stop multiple times. This is done by pfSctl -c 'service reload packages'.
    This leads to a RaceCondition where rc_start is still running and rc_stop is called. For me this leads to a always stopped radiusd.

    I fixed this by checking the LOCKFILE and the PIDFILE in rc_stop.

    
    #!/bin/sh
    # This file was automatically generated
    # by the pfSense service handler.
    
    SERVICENAME="radiusd"
    LOCKFILE="/tmp/${SERVICENAME}_start.lock"
    PIDFILE="/var/run/${SERVICENAME}.pid"
    
    rc_start() {
    
            # prevent this part of script from running in parallel
            if ( set -o noclobber; echo "$$" > "$LOCKFILE") 2> /dev/null; then
                    # make sure lock file is removed even if script is terminated
                    trap 'rm -f "$LOCKFILE"; exit $?' INT TERM EXIT
    
                    /usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestart
    
                    # try to wait until the service starts
                    if [ ! -f "$PIDFILE" ]; then
                            echo "$SERVICENAME.sh: PID file was not found"
    
                            for i in 1 2 3 4 5; do
                                    if [ -f "$PIDFILE" ]; then
                                            echo "$SERVICENAME.sh: Service started PID: `cat $PIDFILE`"
                                            break
                                    else
                                            echo "$SERVICENAME.sh: Waiting 0.5 seconds"
                                            sleep 0.5
                                    fi
                            done
                    else
                            echo "$SERVICENAME.sh: Service running PID: `cat $PIDFILE`"
                    fi
    
                    rm -f "$LOCKFILE"
                    trap - INT TERM EXIT
            else
                    echo "$SERVICENAME.sh: Cannot continue at this moment, this script is already trying to start service PID: $(cat $LOCKFILE)"
            fi
    }
    
    rc_stop() {
            # Don't stop if service start is in progress
            # pfSctl -c 'service reload packages' call start and stop multiple times
    
            if [ ! -f "$LOCKFILE" -a -f "$PIDFILE" ]; then
                /usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestop
            fi
    }
    
    case $1 in
            start)
                    rc_start
                    ;;
            stop)
                    rc_stop
                    ;;
            restart)
                    rc_stop
                    rc_start
                    ;;
    esac
    
    


  • All the technical details are way beyond my knowledge. Since the last upgrade the Freeradius problems are gone and Freeradius is (re)starting as it should after a reboot of pfSense. Great I don't have to start it manually anymore.



  • I'm on pfSense 2.2.6 and freeradius2 Package 1.6.19, which both are the newest Versions.
    Since this is a Race Condition the Problem will hit only some users.  It depends on the Hardware you are running pfSense on.

    I like to see this Problem solved in the freeradius2 Package, because now I have to manually patch this every time a new pfSense Version or freeradius2 Package is installed.

    I can create a Pull Request against freeradius.inc on https://github.com/pfsense/pfsense-packages, if someone give me a hint howto get something out of the rc_ Functions.



  • @sherbeeny:

    I got it fixed when I went to "Packages" tab and reinstalled Freeradius2 by pressing on its [pkg] button.

    I run PFS 2.2.6 and had the same problem and reinstalling the packadge solved the problem.



  • The correct solution may be to make pfsense not try to start and stop service at the same time.
    Checking PIDs and lockfiles seems like a workaround.
    Another workaround i found is to add "sleep 10" at the beginning of rc_start so all rc_stops are finished before rc_starts. (but this may fail if rc_stops take more than 10s)
    @bbaumer:

    if someone give me a hint howto get something out of the rc_ Functions.

    What do you mean?



  • @bbaumer:

    To prevent radiusd being startet multiple times the code in rc_start was added by https://redmine.pfsense.org/issues/4337. This was then merged  in https://github.com/pfsense/pfsense-packages/pull/839/files

    Like SmileyAU suggestet /usr/local/etc/rc.d/radiusd.sh ist started with the Argument start and stop multiple times. This is done by pfSctl -c 'service reload packages'.
    This leads to a RaceCondition where rc_start is still running and rc_stop is called. For me this leads to a always stopped radiusd.

    I fixed this by checking the LOCKFILE and the PIDFILE in rc_stop.

    
    #!/bin/sh
    # This file was automatically generated
    # by the pfSense service handler.
    
    SERVICENAME="radiusd"
    LOCKFILE="/tmp/${SERVICENAME}_start.lock"
    PIDFILE="/var/run/${SERVICENAME}.pid"
    
    rc_start() {
    
            # prevent this part of script from running in parallel
            if ( set -o noclobber; echo "$$" > "$LOCKFILE") 2> /dev/null; then
                    # make sure lock file is removed even if script is terminated
                    trap 'rm -f "$LOCKFILE"; exit $?' INT TERM EXIT
    
                    /usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestart
    
                    # try to wait until the service starts
                    if [ ! -f "$PIDFILE" ]; then
                            echo "$SERVICENAME.sh: PID file was not found"
    
                            for i in 1 2 3 4 5; do
                                    if [ -f "$PIDFILE" ]; then
                                            echo "$SERVICENAME.sh: Service started PID: `cat $PIDFILE`"
                                            break
                                    else
                                            echo "$SERVICENAME.sh: Waiting 0.5 seconds"
                                            sleep 0.5
                                    fi
                            done
                    else
                            echo "$SERVICENAME.sh: Service running PID: `cat $PIDFILE`"
                    fi
    
                    rm -f "$LOCKFILE"
                    trap - INT TERM EXIT
            else
                    echo "$SERVICENAME.sh: Cannot continue at this moment, this script is already trying to start service PID: $(cat $LOCKFILE)"
            fi
    }
    
    rc_stop() {
            # Don't stop if service start is in progress
            # pfSctl -c 'service reload packages' call start and stop multiple times
           
            if [ ! -f "$LOCKFILE" -a -f "$PIDFILE" ]; then
                /usr/pbi/freeradius-amd64/local/etc/rc.d/radiusd onestop
            fi
    }
    
    case $1 in
            start)
                    rc_start
                    ;;
            stop)
                    rc_stop
                    ;;
            restart)
                    rc_stop
                    rc_start
                    ;;
    esac
    
    

    Thanks, this works, please someone commit this code in so others won't have this issue.  Whenever the:

    "check_reload_status: Starting packages"

    initiates, it'll restart the radius and the race condition occurs, the checking for PID on rc_stop makes this problem go away.



  • @malvank:

    @sherbeeny:

    I got it fixed when I went to "Packages" tab and reinstalled Freeradius2 by pressing on its [pkg] button.

    I run PFS 2.2.6 and had the same problem and reinstalling the packadge solved the problem.

    This works as well, unsure why…

    Also the previous post on the fix, it works but then you can never stop it, since the PID file is there when it's running.  Not sure how to 100% prevent this issue.



  • Same problem here on 2.3 release, service doesn't start in 90% of the cases on reboot….

    As suggested by bbaumer just edit /usr/local/etc/rc.d/radiusd.sh by this way to solve the situation  :) :

    #!/bin/sh
    # This file was automatically generated
    # by the pfSense service handler.
    
    SERVICENAME="radiusd"
    LOCKFILE="/tmp/${SERVICENAME}_start.lock"
    PIDFILE="/var/run/${SERVICENAME}.pid"
    
    rc_start() {
    
            # prevent this part of script from running in parallel
            if ( set -o noclobber; echo "$$" > "$LOCKFILE") 2> /dev/null; then
                    # make sure lock file is removed even if script is terminated
                    trap 'rm -f "$LOCKFILE"; exit $?' INT TERM EXIT
    
                    /usr/local/etc/rc.d/radiusd onestart
    
                    # try to wait until the service starts
                    if [ ! -f "$PIDFILE" ]; then
                            echo "$SERVICENAME.sh: PID file was not found"
    
                            for i in 1 2 3 4 5; do
                                    if [ -f "$PIDFILE" ]; then
                                            echo "$SERVICENAME.sh: Service started PID: `cat $PIDFILE`"
                                            break
                                    else
                                            echo "$SERVICENAME.sh: Waiting 0.5 seconds"
                                            sleep 0.5
                                    fi
                            done
                    else
                            echo "$SERVICENAME.sh: Service running PID: `cat $PIDFILE`"
                    fi
    
                    rm -f "$LOCKFILE"
                    trap - INT TERM EXIT
            else
                    echo "$SERVICENAME.sh: Cannot continue at this moment, this script is already trying to start service PID: $(cat $LOCKFILE)"
            fi
    }
    
    rc_stop() {
            # Don't stop if service start is in progress
            # pfSctl -c 'service reload packages' call start and stop multiple times
    
            if [ ! -f "$LOCKFILE" -a -f "$PIDFILE" ]; then
                /usr/local/etc/rc.d/radiusd onestop
            fi
    }
    
    case $1 in
            start)
                    rc_start
                    ;;
            stop)
                    rc_stop
                    ;;
            restart)
                    rc_stop
                    rc_start
                    ;;
    esac
    
    

  • Banned


Log in to reply