New 502 Bad Gateway
-
I'm still learning. Can I just enter the two files from reply 165 into the command box via Diagnostics, Command? I am a GUI user.
I have not had any issues yet but following along to prevent any issues. I do use DNSBL and I am on 2.4.1. Was on 2.4 and the related RC's. -
I'm still learning. Can I just enter the two files from reply 165 into the command box via Diagnostics, Command? I am a GUI user.
I have not had any issues yet but following along to prevent any issues. I do use DNSBL and I am on 2.4.1. Was on 2.4 and the related RC's.Use putty as stated here, and when you log in choose shell and copy paste one row after another. Reboot unit and thats it. And use login "root" not "admin" as stated in video.
https://www.youtube.com/watch?v=krNuKDGEjvQ
Cheers!
-
https://pastebin.com/Ek9R0qkh
it starts to become a major issue as well for us. Im about to restore backups..
All virtual firewall's that we upgraded have the same problem…
we have to reboot them multiple times per day to get it working, otherwise ipsec's and openvpn stop working.If that is the output when you have the problem, then it's NOT this problem. No sign of pfBlocker or anything blocking PHP. Start a new thread, it's probably something already solved on 2.4.1 if it's a VM issue. Check the release notes.
since upgrade tot 2.4.1 no problems yet!
-
So far so good with the updated files.
-
I'm still learning. Can I just enter the two files from reply 165 into the command box via Diagnostics, Command? I am a GUI user.
I have not had any issues yet but following along to prevent any issues. I do use DNSBL and I am on 2.4.1. Was on 2.4 and the related RC's.Use putty as stated here, and when you log in choose shell and copy paste one row after another. Reboot unit and thats it. And use login "root" not "admin" as stated in video.
https://www.youtube.com/watch?v=krNuKDGEjvQ
Cheers!
MAC user so I used terminal. Thank you for the point in the right direction, patched this morning after waking up to 502 Bad Gateway.
-
As per jimp's suggestion, please try these two patched files which use a pfSense function called try_lock() as opposed to flock().
Run the following commands to download the patched version of the two files from my Github Gist:
fetch -o /usr/local/www/pfblockerng/www/index.php "https://gist.githubusercontent.com/BBcan177/9f9c8e62b166cee07ad16cd4ff59103c/raw" fetch -o /usr/local/pkg/pfblockerng/pfblockerng.inc "https://gist.githubusercontent.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/raw"
Recommend a reboot after downloading the patches.
You can review the Gist revisions here:
index.php
https://gist.github.com/BBcan177/9f9c8e62b166cee07ad16cd4ff59103c/revisionspfblockerng.inc
https://gist.github.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/revisionsNote: The try_lock() function calls might require increasing the timeout setting from the default setting of "5" (seconds)
So Far running for more than 24 hrs without problems on pfsense 2.4.0 - I would say it looks pretty good.
I will upgrade to pfsense 2.4.1 tonight.
-
I haven't posted here but have been following the thread as I've had similar issues. After so many hours(less than half a day) pfsense gui and shell would become completely unresponsive even though clients still had internet access.
Replacing the files "index.php" and "pfblockerng.inc" with the ones BBcan177 posted seems to have fixed the issue for me. I've been up for 41hrs now without a problem.
-
Guess I spoke too soon. Just got the 502. Also this seems to stop the firewall schedules from working as a schedule that was allowing access should have been stopped but access was still available. Hopefully it's not affecting other aspects of the firewall security.
-
Is 2.4.1 supposed to fix this without the updated files here?
I saw the redmine ticket was closed due to a new version of pfbng fixing this. Is that automatically installed in 2.4.1? I'm not seeing a package update on my 2.4.1 RELEASE box.
-
If you ran the 2.4.1 update after the new package was uploaded (2.1.2_1) then it would pick up the new files automatically.
-
I still have 502 issue when use pfSense 2.4.1 and pfblockerng 2.1.2_1. :(
-
Still having the issue after 2.4.1, had to disable DNSBL again.
-
Hey all,
Try disabling the Dashboard auto-update check. It seems unrelated, but I made this change last weekend and my system has been stable ever since.
-
I am also experiencing this issue. Error 502 Bad Gateway roughly every 15 hours. Latest version of pfSense and pfBlocker as of the date of this post. I'd post the actual versions but I need to reboot the router to get into the GUI and I can't afford to take down the internet here at this very moment.
As a side note, I've used pfSense for years now and this '502 Bad Gateway' seems to be a reoccurring theme..
-
Running 2.4.1 and 2.1.2_1 and still getting 502 but also getting a few crash reports to send in each day as well.
PHP ERROR: Type: 1, File: /usr/local/pkg/pfblockerng/pfblockerng.inc, Line: 2496, Message: Allowed memory size of 536870912 bytes exhausted (tried to allocate 8192 bytes) @ 2017-10-28 07:02:39
-
Hi,
same here - updates onto last version of pfsene & pfblocker - after about a day it becomes unresponsive.
-
Disabling Snort and it's updates has kept me up and running for 2 days now. I'm going to wait another day and then re-enable snort and see what happens. Perhaps Snort needs the same changes that pfblocker has gotten.
-
I made some additional mods to the code. Run the following command to download the patched version from my Github Gist:
fetch -o /usr/local/pkg/pfblockerng/pfblockerng.inc "https://gist.githubusercontent.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/raw"
Recommend a reboot after downloading the patch.
Please let me know your feedback!
-
For me after 2.4.1-RELEASE (amd64) and pfBlockerNG 2.1.2_1 Finally no more err 502 or 504. Open VPN keep connections.
Sistem running for 2 Days 09 Hours 02 Minutes 23 Seconds. Before i had issues after 6-9 hrs.
Many thnx fo all.
-
Hey all,
Try disabling the Dashboard auto-update check. It seems unrelated, but I made this change last weekend and my system has been stable ever since.
Well it finally failed after a week of being stable…
-
I made some additional mods to the code. Run the following command to download the patched version from my Github Gist:
fetch -o /usr/local/pkg/pfblockerng/pfblockerng.inc "https://gist.githubusercontent.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/raw"
Recommend a reboot after downloading the patch.
Please let me know your feedback!
I installed this today and after 6 hours of running my pFsense VM increased disk usage of over 20gb and crashed the VM and needed to be rebuilt.
-
I installed this today and after 6 hours of running my pFsense VM increased disk usage of over 20gb and crashed the VM and needed to be rebuilt.
I don't think the patch would have done that… Confirmed with a few other users. Check your PM for details and we can go from there... Thanks!
-
I made some additional mods to the code. Run the following command to download the patched version from my Github Gist:
fetch -o /usr/local/pkg/pfblockerng/pfblockerng.inc "https://gist.githubusercontent.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/raw"
Recommend a reboot after downloading the patch.
Please let me know your feedback!
I installed this today and after 6 hours of running my pFsense VM increased disk usage of over 20gb and crashed the VM and needed to be rebuilt.
Works on my machines since 4 days without a hassle and without filling up the disks.
What was filled up, did you have had a look on the files? -
I made some additional mods to the code. Run the following command to download the patched version from my Github Gist:
fetch -o /usr/local/pkg/pfblockerng/pfblockerng.inc "https://gist.githubusercontent.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/raw"
Recommend a reboot after downloading the patch.
Please let me know your feedback!
I installed this today and after 6 hours of running my pFsense VM increased disk usage of over 20gb and crashed the VM and needed to be rebuilt.
Works on my machines since 4 days without a hassle and without filling up the disks.
What was filled up, did you have had a look on the files?I checked BBcan's pm and started the process again and keeping check on a few things this time. I couldn't check when it happened though as it had reached 109% storage and the VM crashed and when I rebooted it, it took everything with it. Configs for all services where 95% lost (like 3 DHCP mappings remained of a list of 200+, lost all NAT/Rules). So I ended up just factory resetting and am now attempting to replicate the findings. I will keep everyone updated as the day progresses.
-
2.4.1-RELEASE (amd64) and pfBlockerNG 2.1.2_1 which has been just fine until this morning. I now get the 502 bad gateway again. I installed the two patch files as well prior to the update to 2.1.2_1. I don't know when I lost access since the system will continue to run but I think I looked at the GUI yesterday.
I just installed the most recent change from the previous page so ignore this until I have more time to keep an eye on it. I read past the 3rd file thinking it was one of the previous files.
-
I've been struggling with this same issue on two devices (SG-2240 and SG-4860) since this thread was created as well. I'm currently on the 2.4.1 release with the latest pfblockerng, snort and squid/squidguard packages. In all cases, it seems like this issue is encountered at the stroke of midnight. I've been ready to capture the console text but like other users have reported, the console stops responding completely so I can't capture the data requested.
Last night, I looked at all the cron jobs that were kicking off around midnight and there were several that were happening at exactly midnight. I modified them to have at least 5 minutes separation to see if that might help. I've also tried pausing pfblockerng and snort and still get the 502.
Just thought I would also report this is still an on going issue.
-
I've already posted it once, but a ZFS install cured all my issues. Even on 2.4.0 and pbng 2.1.2
-
Hi, All
Is it just the pfBlockerNG DNSBL issue? Can I turn on the IPv4 blocks only? I have upgraded to 2.4.1 (ufs) already, pfBlockerNG is disabled due to the issue, but I really want to turn it on. Thanks.
-
Really dumb question I am sure, but if the WEB GUI is giving a Bad Gateway, is the only way to correct this with a clean install? I am not sure if SSH access was enabled but ssh to 192.168.1.1 and .254 times out.
If I restart the router with via the power cord, I still can't get past the bad gateway even for a second.
Am I doomed?
-
I've already posted it once, but a ZFS install cured all my issues. Even on 2.4.0 and pbng 2.1.2
I did a clean install with ZFS and am still getting this issue.
-
For me after 2.4.1-RELEASE (amd64) and pfBlockerNG 2.1.2_1 Finally no more err 502 or 504. Open VPN keep connections.
Sistem running for 2 Days 09 Hours 02 Minutes 23 Seconds. Before i had issues after 6-9 hrs.
Many thnx fo all.
Just to add that my system still works without issues for 6 Days 23 Hours 15 Minutes. If anyone needs any outputs i will provide, but just tell me what to do to create logs.
Cheers!
-
I disabled pfBlockerNG but I'm also getting this 502 Bad Gateway message.
When I log in via SSH it just shows:
pfSense -
And you can't do anything. However if I hit ctrl-z it drops to a working shell allowing me to reboot so I can get in via the GUI.
Hope this helps others who don't have easy access to their routers.
Steve
-
Hello,
still getting this issue after a few hours of uptime.
2.4.1-RELEASE (amd64) running on ufs
pfBlockerNG - 2.1.2_1
snort - 3.2.9.5_2 (newer one available)Just disabling the DNSBL helps to keep everything working. But thats not the sense to deactivate it.
pfblockerNG with disabled DNSBL runs fine. -
This has happened twice today so this is still an issue.
pfSense Netgate SG-4860
2.4.1-RELEASE (amd64)pfBlockerNG 2.1.2_1
I have attached the recommended Output File but was wondering if there is anything else that needs to be supplied to help?
[pfSense Output File_11-2-17.txt](/public/imported_attachments/1/pfSense Output File_11-2-17.txt)
-
Just to add to this thread, I can confirm that the above fix worked for me. I had this issue after pushing out the upgrade to 2.4 and followed the post above (I commented the lines out rather than deleting them). Since then it has been stable and all pfSense routers in my environment have stopped giving the bad gateway error.
After I commented out that block of code, I've been stable although I know it's just a bandaid for now. On one of my 8 devices, I've been pushing out the updates for pfblockerng and am still getting the Bad Gateway 502 nginx error. In turn, with all packages up to date, I've simply commented out the updated block of code and again it seems to be stable. I know this is not the fix, but at least I'm not having to reboot the gateway router 1-2x a day.
Here is what I commented out:
File: /usr/local/www/pfblockerng/www/index.php
// Increment DNSBL Alias counter /*if (!empty($pfb_query)) { * $pfb_found = FALSE; * * $dnsbl_info = '/var/db/pfblockerng/dnsbl_info'; * if (($handle = @fopen("{$dnsbl_info}", 'r')) !== FALSE) { * $lock_handle = @try_lock($handle, 5); * if ($lock_handle) { * if (($pfb_output = @fopen("{$dnsbl_info}.bk", 'w')) !== FALSE) { * $lock_pfb_output = @try_lock($pfb_output, 5); * if ($lock_pfb_output) { * $pfb_found = TRUE; * * // Find line with corresponding DNSBL Aliasname * while (($line = @fgetcsv($handle)) !== FALSE) { * if ($line[0] == $pfb_query) { * $line[3] += 1; * } * @fputcsv($pfb_output, $line); * } * @unlock($lock_pfb_output); * } * @unlock_force($pfb_output); * @fclose($pfb_output); * } * @unlock($lock_handle); * } * @unlock_force($handle); * @fclose($handle); * } * * if ($pfb_found) { * @rename("{$dnsbl_info}.bk", "{$dnsbl_info}"); * } *} */
I'll check back Monday to see if there are any updates! Have a nice weekend everyone!
-
I made some additional mods to the code. Run the following command to download the patched version from my Github Gist:
fetch -o /usr/local/pkg/pfblockerng/pfblockerng.inc "https://gist.githubusercontent.com/BBcan177/7ff15715be0f02afdbe0a00c676aedce/raw"
Recommend a reboot after downloading the patch.
Please let me know your feedback!
I installed this today and after 6 hours of running my pFsense VM increased disk usage of over 20gb and crashed the VM and needed to be rebuilt.
Works on my machines since 4 days without a hassle and without filling up the disks.
I did this too. Everything is on the latest release. pfBlockerNG seems to be working fine. Ok it's just an hour ago however it is -up to now- one hour without issues. It looks a bit like stable. I'll give you a feedback if there is any change.
-
This is still happening to me on 2.4.1 and the latest PfBlocker. Took 8 days from reboot for the 502's to start and all SSH connections to fail, and approx 1 more day after that for all traffic to be dropped. Needed to get it back asap so don't have logs.
-
This is still happening to me on 2.4.1 and the latest PfBlocker. Took 8 days from reboot for the 502's to start and all SSH connections to fail, and approx 1 more day after that for all traffic to be dropped. Needed to get it back asap so don't have logs.
I can confirm too. Exactly the same happens here :-(
-
Ok… don't know if this is luck and I'll be jinxing it with this post but after battling this for weeks (on both UFS and ZFS) I decided to alter my CRON jobs such that all recurring tasks would be assured to have a minimum of 5 minutes. Since doing that, I've gone over 7 days without a hitch for the first time in over a month.
-
This is more of an info post to help try and sort out the issue.
I also had the Bad Gateway error after the 2.4.0 and 2.4.1 updates. pfBlockerNG is installed and running GeoIP and DNSBL parts only, with some periodic updates (essentially Pi-Hole). The pfsense system runs in a VM on XenServer (7.1, I believe).
What I found interesting was that I'm monitoring the firewall with Observium and the graphs are attached. (All of the same unit, same timeline, I just had to take 2 screenshots as the page is long.) Noting the graphs are 1 day / 7 days / 4 weeks / 1 year.
You can clearly see the 'spike' to crash/reboot time on the graphs, in both the running processes and the memory usage (etc)… the first spike is after the 2.4.0 install, with the 2.4.1 install coming immediately after the 'crash' of the 2.4.0 install. Then over a week running fine on 2.4.1... then processes ramp up again to crash point.
I could get to the console on the 2.4.1 box today but selecting 'reboot' from the console menu basically just hung the box... after 15mins it needed a 'force reboot' power cycle.
I'll be keeping a close eye on the firewall's health.. as well as this forum thread.
Happy to try and help debug this issue. It seems to me that something is 'triggering' the process madness and that doesn't seem to be a change (in my case) as the system ran for over a week without any involvement from me.
![Screen Shot 2017-11-07 at 8.31.55 pm.png](/public/imported_attachments/1/Screen Shot 2017-11-07 at 8.31.55 pm.png)
![Screen Shot 2017-11-07 at 8.31.55 pm.png_thumb](/public/imported_attachments/1/Screen Shot 2017-11-07 at 8.31.55 pm.png_thumb)
![Screen Shot 2017-11-07 at 8.32.15 pm.png](/public/imported_attachments/1/Screen Shot 2017-11-07 at 8.32.15 pm.png)
![Screen Shot 2017-11-07 at 8.32.15 pm.png_thumb](/public/imported_attachments/1/Screen Shot 2017-11-07 at 8.32.15 pm.png_thumb)