Fails to finish booting, hangs after configuring firewall



  • Not sure what is going on here but at some point my box has stopped fully booting. It stops showing any boot messages on the console after the second filter reload, right after starting DHCP.

    Strange thing is, it appears to work fine otherwise (except miniupnpd needing a save before it will start) but due to not finishing the boot process pfblockerng refuses to update its IP blocklists which is a big problem.

    At first I thought it was my custom script which starts using shellcmd but I tried removing that and it made no difference. I also tried disabling my AirVPN clients and gateways as they muddy the boot by needing to restart a few times before routing works (no clue why) but again that made no difference.

    I made sure all software is updated. I have no swap partition, not sure if that makes any difference as I have 4GB RAM so really don't need one.


  • Netgate Administrator

    Can we see a boot log?

    Steve



  • How do I get a boot log?


  • Netgate Administrator

    If you have a serial console you can copy/paste it from the terminal client. Otherwise the system log immediately after booting should contain the boot output or you can check /var/log/dmesg.boot.
    If you are not using a serial console a photo of the cons0ole showing where it stops would be useful.

    Steve



  • I haven't updated as there is nothing in the boot logs to indicate a problem at all and strangely I suddenly found the boot script had finished after being sat for days.

    I will take another look the next time I need to reboot.


  • Netgate Administrator

    Hmm, I don't think I've seen that before. The console output should give us some clues though.

    Steve



  • It does seem to be related to my traffic_monitor.php script, if I kill that then booting resumed. What I can't figure out is WHERE my startup script is being started from.

    I removed it from crontab and shellcmd but its STILL starting up, which makes no sense as this was a fresh install and it was only added to the startup using the shellcmd option as how I previously started it did not seem to be restored from the backup. :/ Now I'm wondering if it DID restore it, but I can't see an /etc/rc.local which is how it previously started up.

    Where on earth could it be starting up from?


  • Netgate Administrator

    /usr/local/etc/rc.d is where I would check.

    Steve



  • It seems somehow the shellcmd was in the config.xml twice in different places. Removed them both and just left the crontab entry so it starts up a minute after booting. Not ideal, but it may be the only way as it seems there is a conflict with pfSense executing the shellcmd too early before everything else has finished starting up.

    I'm guessing one option may be to have the shellcmd script look for the booted file to appear, like pfBlockerNG does, before actually starting the script.

    Its all very odd as it was working perfectly months ago, I can only assume either pfSense made some tweak in when the shellcmd is executed causing it to hang like this due to starting too early, or I was using some alternative way of starting the script on boot (I lost that old config so not sure).

    Its baffling as surely shellcmd should be the vast last thing to execute once everything else has finished, not half-way through the boot process. Seems a very questionable choice if your startup script can hang the boot process.

    They certainly don't seem to document all changes, possibly because some things are happening upstream in FreeBSD itself. I noticed after the upgrade to 2.4.4 I'm using a lot more RAM, I think its related to zfs as before it was detecting I had less than 4GB RAM so disabling some functionality, perhaps now its ignoring the RAM allocated to the integrated graphics so acting like I have 4GB RAM (which I do minus iGPU). But I digress.