Executing Commands at Boot

vwaniel · Jan 4, 2024, 11:52 PM

I'm not sure if this belongs in the Documentation or Development area, so I'm starting here...

According to https://docs.netgate.com/pfsense/en/latest/development/boot-commands.html, one of the ways to have pfSense run commands at boot time is to put an executable script with a .sh file extension in /usr/local/etc/rc.d. What the documentation doesn't say is that the script(s) will also run any time an OpenVPN interface comes up. I think this is because (to the best of my very limited understanding of pfSense internals):

-pfSense includes "up /usr/local/sbin/ovpn-linkup" in any system-generated OpenVPN configuration file
-/usr/local/sbin/ovpn-linkup includes the line '/usr/local/sbin/pfSctl -c "interface newip ${1}"'
-Based on my read of what I think is the source code (https://github.com/marcelloc/pfsense-tools/blob/master/pfPorts/check_reload_status/files/server.h), 'pfSctl -c "interface newip"' calls /etc/rc.newwanip.
-/etc/rc.newwanip calls a function, "restart_packages()", which calls "send_event("service reload packages")". Based on some Googling, this ultimately triggers /etc/rc.start_packages
-/etc/rc.start_packages includes code that effectively executes any scripts in /usr/local/etc/rc.d

Normally this wouldn't be a problem, unless one needs to run certain commands at boot time and only at boot time. I'm toying with getting HA/CARP failover working with DHCP WAN interfaces using cloned MAC addresses and in order for everything to work it is necessary to shut down the WAN interface(s) on the secondary node when it boots. In order to do so I put a script in /usr/local/etc/rc.d. What I didn't realize (and spent an embarrassingly long time figuring out) was that when my OpenVPN site-to-site tunnel interface would come up on the secondary node during failover it was ultimately triggering my script which brought my WAN interfaces down, effectively defeating the purpose of HA. :-)

In order to save anyone else the headache, I think it would be a good idea to update the documentation to indicate that shell scripts in /usr/local/etc/rc.d will get executed at boot time, but will also get executed any time packages are reloaded, which includes OpenVPN interface link up events and potentially other interface link up/down events. I'm not sure if the shellcmd and/or earlyshellcmd methods can also get triggered at times other than boot, but it's probably worth investigating and annotating those accordingly as well.

Oh, and expect more to come regarding DHCP WAN and HA/failover. I have a couple more kinks to work out but I'm close to having everything working and plan to post what I've learned in the forum.

SteveITS · Jan 5, 2024, 2:27 AM

@vwaniel There should be a feedback link on the doc page to suggest that.

If memory serves the cron package can be used with @reboot…?

vwaniel · Jan 5, 2024, 2:28 AM

@SteveITS Thanks for the suggestion regarding documentation feedback. I've created https://redmine.pfsense.org/issues/15141.

I have the cron package installed and it does appear that I can use @reboot with it. However I ended up solving my specific issue by having my script check the CARP status before executing so that interfaces get shut down only if the node is not master. This way, if the CARP master node reboots while the CARP backup node happens to be down, nothing effectively happens (assuming that CARP election has happened at boot prior to scripts in /usr/local/etc/rc.d being executed).

#!/bin/csh
set carp_status=`/sbin/ifconfig $interface_to_monitor | grep 'carp:' | awk '{print $2}'`
if ( "$carp_status" != "MASTER" ) then
     ifconfig $wan_interface down
     # ...etc
endif

Gertjan · Jan 5, 2024, 8:13 AM

@vwaniel said in Executing Commands at Boot:

According to https://docs.netgate.com/pfsense/en/latest/development/boot-commands.html, one of the ways to have pfSense run commands at boot time is to put an executable script with a .sh file extension in /usr/local/etc/rc.d. What the documentation doesn't say is that the script(s) will also run any time an OpenVPN interface comes up. I think this is because (to the best of my very limited understanding of pfSense internals):

That 'pfSense help page' shouldn't even exist anymore
Years ago, the pfSense Shellcmd package was created that does all this for you.

See here : System > Package Manager > Available Packages

Install it - and now have a new menu entry called Shellcmd under the Services menu :

vwaniel · Jan 5, 2024, 3:47 PM

@Gertjan Yessir, I have that package installed since I'm using shellcmd to work around another issue. Much easier than manually editing config.xml, and also much easier to see if/how earlyshellcmd/shellcmd is being used on an existing install.