Squid upgrade to 2.2.3 full crash report



  • Crash report begins.  Anonymous machine information:

    i386
    10.1-RELEASE-p13
    FreeBSD 10.1-RELEASE-p13 #0 c77d1b2(releng/10.1)-dirty: Tue Jun 23 17:02:27 CDT 2015    root@pfs22-i386-builder:/usr/obj.i386/usr/pfSensesrc/src/sys/pfSense_SMP.10

    Crash report details:

    PHP Errors:
    [25-Jun-2015 14:14:27 Asia/Kolkata] PHP Fatal error:  Call to a member function read() on a non-object in /etc/inc/pfsense-utils.inc on line 468
    [25-Jun-2015 14:14:35 Asia/Kolkata] PHP Fatal error:  Call to a member function read() on a non-object in /etc/inc/pfsense-utils.inc on line 468

    Filename: /var/crash/minfree
    2048



  • seems to be related to squid



  • Warning: dir(/usr/pbi/squid-i386/local/etc/squid/errors/): failed to open dir: No such file or directory in /etc/inc/pfsense-utils.inc on line 467 Fatal error: Call to a member function read() on a non-object in /etc/inc/pfsense-utils.inc on line 468



  • seems that errors directory dont exist, after creating that directory i can access proxy server gui but one more issue there, under languages i see this

    Warning: Invalid argument supplied for foreach() in /usr/local/www/pkg_edit.php on line 595



  • All those problems are in squid, not 2.2.3, moved to relevant board.



  • yes squid related because i upgraded a few alix and those went fine, only the full installs with squid have this issue


  • Netgate Administrator

    Which Squid version are you running?



  • also, do you know if it's only 32 bit, or have you tried 64?



  • squid3 0.2.8
    that was there on 2.2.2 and after upgrade it reinstalled it fine but all those issues so didnt start.
    its a 32bit machine, dont have any 64bit to try


  • Banned

    64bit is affected as well.


  • Netgate Administrator

    Hmm, I just updated my test box here from 2.2.2 to 2.2.3 and it updated OK. It's running 32bit, Squid 0.2.8 and Squidguard 1.9.14. The only thing I had to do was re-download the Blacklists to get Squidguard going.

    Steve



  • i went to packages and reinstalled and rebooted a couple of times but still same, my box has both lightsquid and squid3

    once i created errors directory then i was able to open squid config page and hit save there also just to update the config but that too didnt help


  • Netgate Administrator

    What did you do to trigger those errors? Where are you seeing them?



  • after upgrade, it rebooted, installs the packages and then i started getting crash reports and squid wasnt running, so i reinstalled it and rebooted but still same


  • Banned

    Jun 25 12:21:26 php-fpm[99958]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -k reconfigure' returned exit code '1', the output was 'squid: ERROR: No running copy'
    Jun 25 12:21:26 php-fpm[99958]: /pkg_edit.php: Reloading Squid for configuration sync
    Jun 25 12:21:26 check_reload_status: Reloading filter
    Jun 25 12:21:26 check_reload_status: Syncing firewall
    Jun 25 12:21:26 check_reload_status: Reloading filter
    Jun 25 12:21:26 php-fpm[99958]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -k reconfigure' returned exit code '1', the output was 'squid: ERROR: No running copy'
    Jun 25 12:21:26 php-fpm[99958]: /pkg_edit.php: Reloading Squid for configuration sync

    This is in the logs when re-installing squid.

    Everything is running in the GUI as package seems to be installed fine.



  • Jun 25 15:49:52 check_reload_status: Syncing firewall
    Jun 25 15:49:52 check_reload_status: Reloading filter
    Jun 25 15:49:52 php-fpm[59436]: /pkg_edit.php: [Squid] - Squid_resync function call pr: bp: rpc:no
    Jun 25 15:49:52 php-fpm[87300]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "nat" rules.
    Jun 25 15:49:53 php-fpm[87300]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "pfearly" rules.
    Jun 25 15:49:53 php-fpm[59436]: /pkg_edit.php: Starting Squid
    Jun 25 15:49:53 php-fpm[59436]: /pkg_edit.php: The command '/usr/pbi/squid-i386/sbin/squid -f /usr/pbi/squid-i386/local/etc/squid/squid.conf' returned exit code '255', the output was ''
    Jun 25 15:49:53 php-fpm[87300]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "filter" rules.


  • Netgate Administrator

    Hmm, I see many of those things also but Squid seems to be running fine.
    The re-install log from the upgrade:

    Jun 25 10:38:21 testbox1 php: rc.bootup: Uninstalling package squid3
    Jun 25 10:38:26 testbox1 php: rc.bootup: The command '/usr/local/etc/rc.d/squid.sh stop' returned exit code '1', the output was 'squid: No running copy' 
    Jun 25 10:38:40 testbox1 php-fpm[247]: /index.php: Successful login for user 'admin' from: 192.168.114.5
    Jun 25 10:38:40 testbox1 php-fpm[247]: /index.php: Successful login for user 'admin' from: 192.168.114.5
    Jun 25 10:39:35 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:39:35 testbox1 kernel: Removing cache ... One moment please...Finishing package cleanup.
    Jun 25 10:39:35 testbox1 kernel: Reloading filter...
    Jun 25 10:39:36 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:39:36 testbox1 kernel: 
    Jun 25 10:39:36 testbox1 php: rc.bootup: Finished uninstalling package squid3
    Jun 25 10:39:36 testbox1 php: rc.bootup: Reinstalling package squid3
    Jun 25 10:39:36 testbox1 php: rc.bootup: Beginning package installation for squid3 .
    Jun 25 10:39:38 testbox1 kernel: 100%
    Jun 25 10:39:38 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:39:38 testbox1 kernel: 
    Jun 25 10:39:39 testbox1 kernel: 0%
    Jun 25 10:39:39 testbox1 kernel:    1%
    Jun 25 10:39:40 testbox1 kernel:    2%
    Jun 25 10:39:40 testbox1 kernel:    3%
    Jun 25 10:39:40 testbox1 kernel:    4%
    Jun 25 10:39:41 testbox1 kernel:    5%
    Jun 25 10:39:41 testbox1 kernel:    6%
    Jun 25 10:39:41 testbox1 kernel:    7%
    Jun 25 10:39:42 testbox1 kernel:    8%
    Jun 25 10:39:42 testbox1 kernel:    9%
    Jun 25 10:39:43 testbox1 kernel:   10%
    Jun 25 10:39:46 testbox1 kernel:   20%
    Jun 25 10:39:53 testbox1 kernel:   30%
    Jun 25 10:39:57 testbox1 kernel:   40%
    Jun 25 10:40:00 testbox1 kernel:   50%
    Jun 25 10:40:04 testbox1 kernel:   60%
    Jun 25 10:40:07 testbox1 kernel:   70%
    Jun 25 10:40:14 testbox1 kernel:   80%
    Jun 25 10:40:20 testbox1 kernel:   90%
    Jun 25 10:40:25 testbox1 kernel:  100%
    Jun 25 10:43:25 testbox1 kernel: 70% 100%
    Jun 25 10:43:31 testbox1 kernel: 100%
    Jun 25 10:43:39 testbox1 kernel: Checking Squid cache... One moment please...Checking if there is configuration to migrate... One moment please...Writing configuration... One moment please...
    Jun 25 10:43:39 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:43:39 testbox1 kernel: Creating squid cache pools... One moment please...
    Jun 25 10:43:45 testbox1 php: rc.bootup: Stopping any running proxy monitors
    Jun 25 10:43:46 testbox1 kernel: Checking for running processes... One moment please...
    Jun 25 10:43:47 testbox1 php: rc.bootup: Starting Squid
    Jun 25 10:43:47 testbox1 kernel: Checking cache... One moment please...Starting... One moment please...Reconfiguring filter... One moment please...
    Jun 25 10:43:47 testbox1 php: rc.bootup: Starting a proxy monitor script
    Jun 25 10:43:47 testbox1 php: rc.bootup: [Squid] - Squid_resync function call pr: bp:1 rpc:no
    Jun 25 10:43:47 testbox1 squid[36169]: Squid Parent: will start 1 kids
    Jun 25 10:43:47 testbox1 squid[36169]: Squid Parent: (squid-1) process 36632 started
    Jun 25 10:43:48 testbox1 (squid-1): I don't handle this error well!
    Jun 25 10:43:48 testbox1 squid[36169]: Squid Parent: (squid-1) process 36632 exited with status 1
    Jun 25 10:43:51 testbox1 squid[36169]: Squid Parent: (squid-1) process 45648 started
    Jun 25 10:43:52 testbox1 (squid-1): I don't handle this error well!
    Jun 25 10:43:52 testbox1 squid[36169]: Squid Parent: (squid-1) process 45648 exited with status 1
    Jun 25 10:43:54 testbox1 php: rc.bootup: [Squid] - Squid_resync function call pr:1 bp:1 rpc:no
    Jun 25 10:43:55 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:43:55 testbox1 php: rc.bootup: Successfully installed package: squid3.
    Jun 25 10:43:55 testbox1 php: rc.bootup: Finished installing package squid3
    Jun 25 10:43:55 testbox1 php: rc.bootup: Uninstalling package squidGuard
    Jun 25 10:43:55 testbox1 squid[36169]: Squid Parent: (squid-1) process 67255 started
    Jun 25 10:43:56 testbox1 (squid-1): Cannot open '/var/squid/logs/access.log' for writing. 	The parent directory must be writeable by the 	user 'proxy', which is the cache_effective_user 	set in squid.conf.
    Jun 25 10:43:56 testbox1 squid[36169]: Squid Parent: (squid-1) process 67255 exited with status 1
    Jun 25 10:43:59 testbox1 squid[36169]: Squid Parent: (squid-1) process 10385 started
    Jun 25 10:43:59 testbox1 (squid-1): Cannot open '/var/squid/logs/access.log' for writing. 	The parent directory must be writeable by the 	user 'proxy', which is the cache_effective_user 	set in squid.conf.
    Jun 25 10:43:59 testbox1 squid[36169]: Squid Parent: (squid-1) process 10385 exited with status 1
    Jun 25 10:44:02 testbox1 squid[36169]: Squid Parent: (squid-1) process 39015 started
    Jun 25 10:44:02 testbox1 (squid-1): Cannot open '/var/squid/logs/access.log' for writing. 	The parent directory must be writeable by the 	user 'proxy', which is the cache_effective_user 	set in squid.conf.
    Jun 25 10:44:02 testbox1 squid[36169]: Squid Parent: (squid-1) process 39015 exited with status 1
    Jun 25 10:44:02 testbox1 squid[36169]: Squid Parent: (squid-1) process 39015 will not be restarted due to repeated, frequent failures
    Jun 25 10:44:02 testbox1 squid[36169]: Exiting due to repeated, frequent failures
    Jun 25 10:45:18 testbox1 Squid_Alarm[96933]: Squid has exited.  Reconfiguring filter.
    Jun 25 10:45:18 testbox1 Squid_Alarm[97023]: Attempting restart...
    Jun 25 10:45:24 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:45:28 testbox1 squid: redirect_program /usr/pbi/squidguard-i386/bin/squidGuard: (2) No such file or directory
    Jun 25 10:45:28 testbox1 kernel: 
    Jun 25 10:45:29 testbox1 check_reload_status: Syncing firewall
    Jun 25 10:45:29 testbox1 php: rc.bootup: [Squid] - Squid_resync function call pr: bp:1 rpc:no
    Jun 25 10:45:40 testbox1 php: rc.bootup: Starting Squid
    Jun 25 10:45:41 testbox1 squid[76028]: Squid Parent: will start 1 kids
    Jun 25 10:45:41 testbox1 squid[76028]: Squid Parent: (squid-1) process 76578 started
    

  • Netgate Administrator

    I'll put some traffic through it to check the logging etc.


  • Banned

    Can you browse HTTP sites?



  • i now tried to remove squid3 completely, then reboot and then try a fresh install of it but it never goes past extracting

    Beginning package installation for squid3 .
    Downloading package configuration file… done.
    Saving updated package information... done.
    Downloading squid3 and its dependencies...
    Checking for package installation...
    Downloading https://files.pfsense.org/packages/10/All/squid-3.4.10_2-i386.pbi ...  (extracting)


  • Netgate Administrator

    Ok, I put some traffic through it, seems to be logging fine. I had to stop-start Squid to get the Squidguard blacklists blocking correctly.
    It's not fast but my test box is 'hardware limited'! Also it's mostly all cache misses at this point.

    Steve



  • is there anyway to clean complete squid directory etc to get it to install properly from scratch


  • Netgate Administrator



  • i reapplied the 2.2.3 update file, it installed cron package on reboot but squid is stuck for ever



  • finally fixed it, applied the 2.2.3 update file once again and then gave it a while and finally squid started and all those errors and warnings disappeared


  • Netgate Administrator

    Ah, nice.  :)
    What hardware were you installing on?

    Steve



  • on the full install machine


  • Netgate Administrator

    Ah, yes I assumed it was a full install I just wondered about your box specs.
    My test box is pretty low end, 1.3GHz Celeron, 512MB ram.

    Steve



  • well this is those asus eeepc with intel atom and 2GB ram



  • squid sorted out and on one another machine now squidguard wont install and this is in a remote location so lost complete access to webgui as well, i believe its something to do with package installation in general because smalelr packages install just fine but bigger ones r the issue



  • as soon as i reinstall squidguard the system goes dead and inaccessible, any fix for it?


  • Netgate Administrator

    Hmm, not able to replicate it yet. My box upgraded without issue including Squid3 and Squidguard with ClamAV. All running smoothely once I'd reloaded the Blacklists.
    The only issue I saw was speed which is slow on my relatively ancient firebox.

    Steve



  • https://redmine.pfsense.org/issues/4793

    after playing around for a while it seems squidguard needs to have something in target category then only it seems to avoid the crash when i tried in virtualbox but during upgrade thats already there

    in general i still suffer package reinstalls on upgrade which go for ever or stuck at extracting which i guess some1 else also mentioned
    https://forum.pfsense.org/index.php?topic=95674.0


  • Netgate Administrator

    That error looks exactly like the well discussed no-Blacklists problem. https://forum.pfsense.org/index.php?topic=90961.0
    Just re-load the blacklists and Squidguard will then start. Add and enable at least one custom target to prevent it happening again.

    Steve



  • I don't use any blacklists and the majority of my issues is reinstall of packages after upgrade because custom targets I already have before the upgrade


  • Netgate Administrator

    Hmm, Squidguard not starting for some other reason then.
    I would try completely removing Squidguard and reinstalling it clean with a minimal config. Make sure it runs like that before adding anything complex.
    My config here on my test box is pretty basic, Squidguard with blacklists and 1 custom target.

    Steve



  • not starting is secondary, to get it to install is the main issue in boxes in remote locations and that gets stuck on extracting for both squid and squidguard, the smaller packages like cron install just fine. Imagine the tar program running for like more than 5mins on a intel atom and core2duo processors


  • Netgate Administrator

    Hmm, yes. Didn't take anything like that even on my ancient Celeron. Running from an SSD though.
    Can you see what it's waiting for at the console? Try hitting Ctrl+T a few times to see what process is actually running.

    Steve



  • Firstly, you need to stop squid if it is running then go to squidGuard and download the blacklist again, check the enable squidGuard; save and apply. Once the your squidGuard is up the Squid will be up too if not go to status>service and click run for squid.

    Good Luck



  • @stephenw10:

    Hmm, yes. Didn't take anything like that even on my ancient Celeron. Running from an SSD though.
    Can you see what it's waiting for at the console? Try hitting Ctrl+T a few times to see what process is actually running.

    Steve

    the tar program


Log in to reply