Can't login to WebGUI after a couple of days



  • Hello,

    When accessing WebGUI following error occured:

    504 Gateway Time-out

    nginx

    Restarting Webconfigurator via SSH didn't help, only a reboot seems to solve the problem.
    Problem returns every couple of days.

    Running latest stable pfSense via vmware.

    Anyone a idea what's wrong here?
    There's nothing in the logs.



  • I also had this problem 1 or 2 times while experimenting…

    next time it happens go to console or SSH and try "16 - Restart PHP-FPM" to narrow it done...


  • LAYER 8 Netgate

    I think that's been fixed in 2.3.1 here:

    https://redmine.pfsense.org/issues/6177



  • Switched to 2.3.1 Development now, and testing…

    Thanks!



  • Problem returned after 1 day, restarting php-fpm from ssh does work.

    I'm on:

    2.3.1-DEVELOPMENT (amd64)
    built on Mon May 16 06:51:39 CDT 2016
    FreeBSD 10.3-RELEASE-p2

    The system is on the latest version.



  • You have the IPsec widget on your dashboard? If so, if you remove that, does it stop happening? Seems a couple have seen that, haven't run into it myself, but would be good to narrow it down.



  • Yes, having that one als widget, turning it off now.



  • I started having the exact same problem with the update to 2.3. I updated to 2.3.1 hopeful that this problem was resolved, but it still happens with the IPSEC widget active



  • Same problem here on 2.3.1.  I do not have the IPSec widget.

    Menu item 16 fixes issue for a time.  Also, it seem the GUI slows down significantly over a short time after doing the reset.



  • @cmb:

    You have the IPsec widget on your dashboard? If so, if you remove that, does it stop happening? Seems a couple have seen that, haven't run into it myself, but would be good to narrow it down.

    No IPSec on my dashboard.  While the OP mentioned the 504 returning after "days", I see it return within an hour or two.

    thanks,
    Dave



  • Most if not all remaining instances of 504 gateway timeouts were fixed in 2.3.1_1.



  • @cmb:

    Most if not all remaining instances of 504 gateway timeouts were fixed in 2.3.1_1.

    I'm still experiencing the problem with 2.3.1_1

    Dave


  • Banned

    +1

    2.3.1_1 64bit full, especially when re-starting a snort interface.

    Intel(R) Atom(TM) CPU D525 @ 1.80GHz 4 CPUs, with 4 GB RAM and 120 GB SSD.



  • I just had this happen at our remote site, not using the ipsec widget.

    Netgate SG-2440

    2.3.1-RELEASE-p1 (amd64)
    built on Wed May 25 14:56:42 CDT 2016



  • @afreaken:

    I just had this happen at our remote site, not using the ipsec widget.

    Netgate SG-2440

    2.3.1-RELEASE-p1 (amd64)
    built on Wed May 25 14:56:42 CDT 2016

    Just happened to our local pfsense box, same as above (2.3.1_1). Will try to restart the web interface from console.



  • https://redmine.pfsense.org/issues/6396 is marked closed.  Is that a different bug, or should it be re-opened? I know there are 502's and 504's going on.

    In any case, what is the correct redmine ticket for the remaining issue being discussed here?

    edit: I found https://redmine.pfsense.org/issues/6406  which seems to be "this"



  • @luckman212:

    https://redmine.pfsense.org/issues/6396 is marked closed.  Is that a different bug, or should it be re-opened? I know there are 502's and 504's going on.

    In any case, what is the correct redmine ticket for the remaining issue being discussed here?

    edit: I found https://redmine.pfsense.org/issues/6406  which seems to be "this"

    I'm getting
    "504 Gateway Time-out

    nginx"

    It just seems to happen randomly. Doing the reset from the console fixes it for some time.

    My one box was up for ~12 days before it happened, the other was ~9 days. So…

    EDIT: The widgets I'm using on both are: System Information, Interfaces, Gateways, Traffic Graphs

    Not sure if that helps. Also running ipsec tunnel between locations.



  • Just happened to me. 2.3.1_1. No IPSec or OpenVPN widgets. 20 days uptime. 504 error. I have an IPSec site-to-site running.

    :(

    OpenVPN server also stops working

    Doing an option 16) Restart PHP-FPM  did allow me to access the web gui but did not fix OpenVPN. Had to restart OpenVPN server to fix OpenVPN access.



  • Happened again a couple days ago… I want to say this problem has persisted through the "fixes" in 2.3.1_1


  • LAYER 8 Netgate

    Do people seeing this with any regularity have anything interesting in the system log? Try filtering on frag.



  • @Derelict:

    Do people seeing this with any regularity have anything interesting in the system log? Try filtering on frag.

    well… could I make a suggestion, could the "Reset Log Files" button not be the default accept button on the logs settings page, rather than the save button... I was trying to increase the number of logs per page, hit enter and instinctively clicked OK as I read the message, and instantly regretted my decision.

    Not sure if this is the same error, but I saw a lot of these in the logs before the above happened.

    nginx: 2016/06/20 08:37:47 [error] 18895#0: *1464191 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "POST /status_ipsec.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.10.2:445", referrer: "https://192.168.10.2:445/status_ipsec.php"

    I see there is an update 5, running update 5 and will report back if it happens again, though it took 9 days of uptime to happen last time.



  • @afreaken:

    @Derelict:

    Do people seeing this with any regularity have anything interesting in the system log? Try filtering on frag.

    well… could I make a suggestion, could the "Reset Log Files" button not be the default accept button on the logs settings page, rather than the save button... I was trying to increase the number of logs per page, hit enter and instinctively clicked OK as I read the message, and instantly regretted my decision.

    Not sure if this is the same error, but I saw a lot of these in the logs before the above happened.

    nginx: 2016/06/20 08:37:47 [error] 18895#0: *1464191 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "POST /status_ipsec.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.10.2:445", referrer: "https://192.168.10.2:445/status_ipsec.php"

    I see there is an update 5, running update 5 and will report back if it happens again, though it took 9 days of uptime to happen last time.

    UPDATE: 10days 8Hours uptime, hasn't happened yet, so maybe the issue has been resolved between release 1 and release 5?

    UPDATE: 22 Days uptime without issue. I'd say whatever caused my specific issue has been resolved.



  • @afreaken:

    UPDATE: 10days 8Hours uptime, hasn't happened yet, so maybe the issue has been resolved between release 1 and release 5?

    Several possibilities for that to happen were fixed between update 1 and 5.



  • @cmb:

    Several possibilities for that to happen were fixed between update 1 and 5.

    Just happened to me in 2.3.1_p5. Same error logged about the thermal sensors and OpenVPN required a restart to allow connections again. I left the WebGUI open during the night on the status screen (my mistake)

    nginx: 2016/07/01 06:07:42 [error] 49003#0: *57704 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.17.18, server: , request: "GET /widgets/widgets/thermal_sensors.widget.php?getThermalSensorsData=11467344975628 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.17.4", referrer: "https://192.168.17.4/" 
    

    The first message logged is about the IPSec widget:

    2016/07/01 05:11:08 [error] 49003#0: *57391 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.17.18, server: , request: "POST /widgets/widgets/ipsec.widget.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "192.168.17.4", referrer: "https://192.168.17.4/"
    
    

    Luckily I am able to login using SSH to an internal host and can then go back to the pfsense box and restart php-fpm (option 16) to get the WebGUI restarted and then restart OpenVPN. With vacations coming up I see a lot of calls from colleagues not being able to connect…  :'(



  • Hi I have the same problem. A random day the webgui cant access and OpenVPN Client not connect. (by Ldap valitation, localy i dont test)

    I have temporally solution by restart PHP-FPM by ssh local connection (option 16 of menu).

    I have some error logs like that:

    nginx: 2016/07/05 10:20:50 [error] 29511#0
    nginx: 2016/07/05 10:20:49 [error] 29543#0

    See attach fille.
    It's happens from update 2.2.6 > 2.3.1 and untill 2.3.1-RELEASE-p5 (amd64)

    Thanks for your helps
    I hope that fix the problem becase if not, i will donwgrade.

    pfsenselog2016-07.txt



  • I have the same problem with the latest version 2.3.1-p5 and the older one
    It happen randomly.

    Hope will be fixed as soon as possible , can't work this way  :'(



  • p5 still results in:

    504 Gateway Time-Out



  • Ugh, please tell me I'm not the only one still getting this.

    I'm getting 502 and 504 errors.  I do the menu choice 16 reset but the problem returns within about 2 minutes or less.

    It seems the 502 is tied to the "initial login" but then I get 504 errors afterward.

    FreeBSD 10.3-RELEASE-p5



  • Hello,
    I have same problem, cant access 14.14.14.14
    I need log file in pfsense server. how can i get log file.






  • Web configuration error, while pc is opening.










  • I have been having 504 timeout issue since last once week now. No updates and no additional package installed lately. I am currently using pfSense version 2.3.2 with standard settings, pfBlockerNG & Snort is configured and the services are executing as expected.

    I am able to access the shell and the firewall is running smooth without any issues. Can someone please shed some light, thanks in advance.

    In the meantime, i have changed below values;

    max_execution_time = 900 -> /etc/php.ini
    request_terminate_timeout = 300 -> /usr/local/etc/php-fpm.conf

    
    2016/08/16 20:42:58 [error] 20217#0: *1 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1", referrer: "http://10.10.10.1/"
    2016/08/16 21:26:02 [error] 18966#0: *1 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1", referrer: "http://10.10.10.1/"
    2016/08/16 21:33:05 [error] 18966#0: *5 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1", referrer: "http://10.10.10.1/"
    2016/08/17 09:51:20 [error] 20268#0: *1 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1", referrer: "http://10.10.10.1/"
    2016/08/17 16:41:59 [error] 18589#0: *9 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1", referrer: "http://10.10.10.1/"
    2016/08/17 16:49:59 [error] 18589#0: *20 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1", referrer: "http://10.10.10.1/"
    2016/08/17 17:13:56 [error] 18589#0: *23 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 10.10.10.101, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "10.10.10.1"
    
    


  • Finally, after weeks, I determined that it was pfblockerng that was causing the 504 errors.

    I disabled pfblockerng and have not had a problem with the GUI since.



  • How can I disable pfblockerng


  • Moderator

    @vronp:

    Finally, after weeks, I determined that it was pfblockerng that was causing the 504 errors.

    I disabled pfblockerng and have not had a problem with the GUI since.

    Can you provide any more details about this issue?

    If you have a large DNSBL database, it can take some time to reload the Resolver… and if you are logging into pfSense with the Domain name, then you can have some issues at that time... best to use the pfSense firewall literal IP address to login...

    Also there are some issues with the Resolver and DHCP....  (Hoping the devs provide some feedback...)
    https://redmine.pfsense.org/issues/5413



  • Happens on the latest 2.3.2 also. In fact, just happened to me.

    I suspect it's this elusive bug: https://redmine.pfsense.org/issues/6406

    I tried restarting the webconfigurator but it did nothing, never tried the other option as the box is not yet live, just rebooted.

    Apparently it's more likely to happen if you leave the web page up for a period of time.

    Thankfully it doesn't affect SSH, and it doesn't affect functionality, so it's an annoyance rather than a bona fide forest fire emergency.


Log in to reply