New 502 Bad Gateway
-
Very interesting. I did not have this 502 Bad Gateway Issue after upgrading to 2.4.0-RELEASE. (Uptime since Upgrade 1 Day 18 Hours).
I just installed now the new Update (Upgrading pfSense-pkg-pfBlockerNG from 2.1.1_11 to 2.1.2)My Setup:
Kettop Mi19N (32GB SSD UFS Filesystem + 8GB RAM)
PFBlockerNG + DNSBL enabled
MultiWAN (Failover)
IPSEC + OpenVPN -
To anyone experiencing the DNSBL issue in this thread: Update to the latest pfBlocker (2.1.2 or later) as it contains a fix for the locking issue leading to these conditions. After updating, ideally you will want to reboot the firewall to make sure it's starting with a clean slate. At the very least, stop and restart the DNSBL daemon.
I'll edit this into the first post of the thread for visibility.
-
To anyone experiencing the DNSBL issue in this thread: Update to the latest pfBlocker (2.1.2 or later) as it contains a fix for the locking issue leading to these conditions. After updating, ideally you will want to reboot the firewall to make sure it's starting with a clean slate. At the very least, stop and restart the DNSBL daemon.
Ok, I did install the latest update to pfBlocker (2.1.2) and seemed to install just fine. When I went to restart the box, it is now in a constant restart state. I am able to SSH into it to see what is going on. I can see the commands as the system tries to load, but it will stop at…
"cpu_reset_proxy: Stopped CPU 1" (see attached)
Then it just restarts. It will keep doing this. I am not sure if the update caused this or not so am posting it here. Anyways, I am just trying to factory reset now and load my current configuration, but can't get it to work. I've tried the button method and is still the same. It doesn't make it to the "main" command menu to run option #4, but I can pause it at the boot menu (see attached). Any suggestions on what to look for? Is a Factory Reset suggested? Any help would be appreciated. Thanks in advance.
 -
Ok, I did install the latest update to pfBlocker (2.1.2) and seemed to install just fine. When I went to restart the box, it is now in a constant restart state. I am able to SSH into it to see what is going on. I can see the commands as the system tries to load, but it will stop at…
I was able to fix this. I found searching in the forums a command to use…
https://forum.pfsense.org/index.php?topic=135466.0
I performed the steps that @Derelict (Thank You) mentioned and it worked. I was able to reboot pfSense and also able to log into the Web GUI. It immediately came up with a crash reporter and wanted me to submit it the developers, which I did. I hope they can find what happened in my case.
As of right now, I am back up and running and didn't have to reset back to defaults or reload my config. I hope this post will help someone else.
-
I've had to go back to 2.3 as 2.4 has been way to unstable for me. Even a fresh install with ZFS is having issues.
-
I updated the pfBlockerNG package this AM and rebooted pfSense but I just got the 504 Gateway Time-out again. Not sure if there is something else going on?
-
I successfully applied the 2.1.2 pfBlockerNG update. I did a reboot when the update completed. I am able to access the web GUI and SSH with no issues. I'll report back later today to make sure I can still access the web gui and SSH after it has been running for awhile.
-
I successfully update 2.1.2 pfBlockerNG . I did NOT reboot my unit jet. I am able to access the web GUI and SSH with no issues for 2 days so far. Before it was hang after 2-3 hours.
It seems it works fine for me.
Thanks
-
i dont use pfblocker and have 502 bad gateway errors on all most all my upgraded pfsense's
what should i do. -
I updated the pfBlockerNG package this AM and rebooted pfSense but I just got the 504 Gateway Time-out again. Not sure if there is something else going on?
i dont use pfblocker and have 502 bad gateway errors on all most all my upgraded pfsense's
what should i do.Gather the information requested in https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994 and post it as a text file attachment.
-
I've been working with BBcan177 to help test some possible fixes in the pfBlockerNG package and just got the 502 error on one of the two pfSense machines that I maintain. I've attached a text file containing the output requested in https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994. The only modification I made to the output from the commands was to replace instances of my WAN IP with <wan_ip>.
I don't know whether this is consistent with others' experiences with this error, but from all external appearances the machine still seems to be operating. By that I mean that Internet access is still operational. I just can't get to the web GUI. I can SSH in normally using a non-root user, but when I try to execute any command requiring root privileges (e.g. sudo pftop), the console hangs with no output, and does not respond to CTRL-C or CTRL-Z. I performed an upgrade from 2.3.4 so both of my machines are still UFS. I'm glad to provide any other information that may be useful.
pfSense_info_dump.txt</wan_ip>
-
It seems like this 502 issue has been around for a along time. Here's a thread on it starting from April 2016: https://forum.pfsense.org/index.php?topic=110515.0. And here is a non-pfSense-specific article about it from May 2014: http://jvdc.me/fix-502-bad-gateway-error-on-nginx-server-after-upgrading-php/. Another from 2012 https://wildlyinaccurate.com/solving-502-bad-gateway-with-nginx-php-fpm/. Presently I'm unable to attempt any of the suggestions from these pages because my machine exhibiting the problem is remote, and I cannot execute any commands requiring root privileges via SSH without the console freezing.
I have never seen the 502 error myself before 2.4.0 and I've run pfSense for many years. I only started running pfBlockerNG about 6 months ago or so, but the supposed link to pfBlockerNG specifically seems like it may be a red herring. The only common threads seem to be that it involves bad blood between PHP and nginx, and within the context of pfSense specifically users seem to often report it being linked to widgets on the dashboard.
-
I've been working with BBcan177 to help test some possible fixes in the pfBlockerNG package and just got the 502 error on one of the two pfSense machines that I maintain. I've attached a text file containing the output requested in https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994. The only modification I made to the output from the commands was to replace instances of my WAN IP with <wan_ip>.
I don't know whether this is consistent with others' experiences with this error, but from all external appearances the machine still seems to be operating. By that I mean that Internet access is still operational. I just can't get to the web GUI. I can SSH in normally using a non-root user, but when I try to execute any command requiring root privileges (e.g. sudo pftop), the console hangs with no output, and does not respond to CTRL-C or CTRL-Z. I performed an upgrade from 2.3.4 so both of my machines are still UFS. I'm glad to provide any other information that may be useful.</wan_ip>
Yours is definitely still DNSBL. Make sure you are on the latest version of pfBlocker there (2.1.2) and that you have restarted the box after updating pfBlocker.
-
It seems like this 502 issue has been around for a along time. Here's a thread on it starting from April 2016: https://forum.pfsense.org/index.php?topic=110515.0. And here is a non-pfSense-specific article about it from May 2014: http://jvdc.me/fix-502-bad-gateway-error-on-nginx-server-after-upgrading-php/. Another from 2012 https://wildlyinaccurate.com/solving-502-bad-gateway-with-nginx-php-fpm/. Presently I'm unable to attempt any of the suggestions from these pages because my machine exhibiting the problem is remote, and I cannot execute any commands requiring root privileges via SSH without the console freezing.
I have never seen the 502 error myself before 2.4.0 and I've run pfSense for many years. I only started running pfBlockerNG about 6 months ago or so, but the supposed link to pfBlockerNG specifically seems like it may be a red herring. The only common threads seem to be that it involves bad blood between PHP and nginx, and within the context of pfSense specifically users seem to often report it being linked to widgets on the dashboard.
There are a large number of things that could cause a 502 error. This specific case is from DNSBL in pfBlocker. There are likely other issues out there that have not yet been fully identified, including one that can happen from the IPsec status/widget in some cases that are hard to reproduce. pfBlocker is not a red herring for the case discussed in this thread, but others who may have an unrelated 502 are assuming their issue is related and chime in. Having a 502 doesn't mean this thread is relevant, having a 502 with supporting detail matching the other symptoms exactly does. It's currently the most common cause, but not the only cause.
Without some supporting detail we can't track down what happened. Gather the information requested in https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994 and put it in a text file attached to a response. If you can't login as root directly, install and configure the sudo pkg so you can login as another user and switch to root that way. For nearly everyone else here, hitting CTRL-Z and then running /bin/tcsh allowed them to use the console or ssh as root.
-
Thanks for the additional info Jim. I didn't intend to muddy the waters; this just seems like a tough one to nail down. I'm not sure why CTRL-Z isn't working in my case, but I've tried numerous times. I also already have sudo installed, but attempting to execute anything with sudo makes the console unresponsive as well. I think I'm dead in the water until I can get physical access to the machine later today. When I can though, I'll reboot it and go from there. I had updated to the latest pfBlockerNG, but had only restarted the service instead of the machine. So perhaps my issue is that a full reboot is required. Also I did gather the information requested from https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994 and attached it as a TXT file to my earlier post. But I wasn't sure if that request was directed at me specifically or just anyone still experiencing the issue.
-
I'm not sure why CTRL-Z isn't working in my case, but I've tried numerous times. I also already have sudo installed, but attempting to execute anything with sudo makes the console unresponsive as well.
Try leaving an ssh session and the console sitting at a root shell so you don't need to get past that step later (just be sure to secure your computer if you leave those open). And rather than running programs through sudo, try launching a shell with sudo: "sudo -s"
I had updated to the latest pfBlockerNG, but had only restarted the service instead of the machine. So perhaps my issue is that a full reboot is required.
A reboot is best because there will still be stale locks out there otherwise.
Also I did gather the information requested from https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994 and attached it as a TXT file to my earlier post. But I wasn't sure if that request was directed at me specifically or just anyone still experiencing the issue.
It's directed at everyone. Even if you took it before, when you are on the latest pfBlocker it might be different. We can't assume the outputs are identical when other factors have changed.
-
Unfortunately (in this case, at least) I have root login disabled for SSH. So my only recourse is to SSH in non-root and then try sudo. But I don't even receive a password prompt when I try to run things with sudo (I did just try "sudo -s" too). It just hangs as soon as I press Enter. However, not a big deal, I'll be able to reset the machine later this evening and I'll see how things go from there. If I get a 502 again following that, I will re-gather and re-post the requested information. If it might be helpful to have that information following a reboot but before the 502 error occurs, let me know and I'd be happy to grab that too. Thanks again.
-
Updated pfBlocker and am getting a 504 Gateway Time-out, after letting the page load for 3-4 minutes. Uptime was roughly 4 hours. Rules are also set up auto update every 4 hours I believe.
Output attached.
-
Updated pfBlocker and am getting a 504 Gateway Time-out, after letting the page load for 3-4 minutes. Uptime was roughly 4 hours. Rules are also set up auto update every 4 hours I believe.
Output attached.
The output still shows it's hung up on the DNSBL daemon. You might manually check the index.php page for dnsbl (linked earlier in this thread) and see if it's the updated code or not.
-
Updated pfBlocker and am getting a 504 Gateway Time-out, after letting the page load for 3-4 minutes. Uptime was roughly 4 hours. Rules are also set up auto update every 4 hours I believe.
Output attached.
The output still shows it's hung up on the DNSBL daemon. You might manually check the index.php page for dnsbl (linked earlier in this thread) and see if it's the updated code or not.
The code had not been removed from the index.php file after installing the update. I manually edited it and will report back with any issues.
