New 502 Bad Gateway

jimp

@jimp:

@ak:

Reading through the thread and I get a similar issue but with 504 Gateway Timeout not a 502. Just wanted to confirm this is the same problem.

PfSense locks up, 504 timeout on web admin. When I go to the machine and use the console via a keyboard, i try and initiate a shell via option 8 and get

Password for admin@pfSense.rice-bucket.com:
*** Welcome to pfSense 2.4.0-RELEASE (amd64) on pfSense ***

And nothing further. The same if I ssh in. Via the shell I can use CTRL-Z and type reboot to reboot the server.

I am using pfBlocker, but did turn it off. However did not restart and its now locked again. I will reinstall pfsense fresh (this is an upgrade from 2.3 to latest 2.4) and restore from backup for now.

If it happens again, hit CTRL-Z, type /bin/tcsh, and then gather the info mentioned in https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994

I just did:

https://pastebin.com/ryQSY9y8

That still appears to be pfBlocker DNSBL getting backed up

PiBa

Can someone with the DNSBL active and as the 'probable cause' of 502 issues perhaps try and see if adding code below to the top of the index.php would help?

ignore_user_abort(true); 
set_time_limit(0);

I could reproduce xmlrpc locks sometimes hanging php due to 'lost' locks and made a PR for that:
(https://github.com/pfsense/pfsense/commit/ee0e2507b819c142b7ad6c331d401002188f1ab8), i dont have/use dnsbl myself so cant properly check.. Did try and hit index.php with AB and putting sleeps inside the index.php the code, but couldnt make it 'fail' in a way that it would just completely hang as described in the thread..

Not sure if it will help for pfB.. but it is 'somewhat similar' and might be an easy fix.. or not..

Jailer

@jimp:

That still appears to be pfBlocker DNSBL getting backed up

It is, I had a 502 today but unfortunately was not able to get the requested information. If and when it does again I will post it here.

steky9

Have upgraded to Coreboot 0.11 and Pfblocker 2.1.2 through the console connection. Have no access to the GUI via the normal LAN interface even immediately after a reboot, but can access the GUI login on the DMZ interface but it gives a 504 error timeout when I try to login. Problem only showed up after upgrading from 2.3.4 to 2.4.0

Logs at https://pastebin.com/xn7WMF0W

Xentrk

@Xentrk:

I successfully applied the 2.1.2 pfBlockerNG update. I did a reboot when the update completed. I am able to access the web GUI and SSH with no issues. I'll report back later today to make sure I can still access the web gui and SSH after it has been running for awhile.

It has been several days since I applied the pfBlockerNG 2.1.2 update. Everything is running normal. The upgrade fixed my 502 Bad Gateway and SSH logon issues.

D-Kun

Hi,

getting the same issue after upgrade from 2.3.x to 2.4.0 even with the latest pfB update. Uptime varies between 15-30 hours till its happening again.

https://pastebin.com/2L9WhTb0

Jailer

Well now I'm getting a 504 Gateway timeout and no console. It was up for about 10 hours before the GUI and console went out. See attached text file for the requested info.

pfSense 2.40
pfBlockerNG 2.1.2
Snort - 3.2.9.5_2

pfserror.txt

rightnow

Getting alot of 502 bad gateway without any package installed at all.

PiBa

@rightnow, and anyone else having 502 issues.
Please do gather the requested information from:
https://forum.pfsense.org/index.php?topic=137103.msg753994#msg753994
(if console doesn't respond Ctrl+Z and start /bin/tcsh then get the information.)

ntct

I also have 502 Bad Gateway on pfSense 2.4 and pfblockerng 2.1.2…... :-\

output.txt

Bordi

I also have. Work with v.2.4.1 ..

hdejongh

hee Jim,

https://pastebin.com/JVMQTWbY

bad gateway 502 issue

akong

I have get 502 Bad Gateway and use pfsense 2.4.0 pfblockerng 2.1.2
How to fix it?

jimp

@hdejongh:

hee Jim,

https://pastebin.com/JVMQTWbY

bad gateway 502 issue

That looks like it was taken just after a reboot, not when the problem was happening. No sign of anything getting backed up in there.

jimp

@ntct:

I also have 502 Bad Gateway on pfSense 2.4 and pfblockerng 2.1.2…... :-\

And judging by the output, it's still DNSBL getting backed up.

PiBa

@jimp:

@ntct:

I also have 502 Bad Gateway on pfSense 2.4 and pfblockerng 2.1.2…... :-\

And judging by the output, it's still DNSBL getting backed up.

That why the report of @rightnow would be interesting. As he has this error without any packages installed. Makes me wonder if some other component causes (part of) the problem, and maybe pfBlocker is just the most visible to suffer from that 'defect'.

jimp

@PiBa:

That why the report of @rightnow would be interesting. As he has this error without any packages installed. Makes me wonder if some other component causes (part of) the problem, and maybe pfBlocker is just the most visible to suffer from that 'defect'.

If that were the case there should be some other sign of stress on the PHP-FPM socket, and there isn't. Most likely it's a completely unrelated problem that happens to present with one common symptom.

Hans_P

@jimp:

@PiBa:

That why the report of @rightnow would be interesting. As he has this error without any packages installed. Makes me wonder if some other component causes (part of) the problem, and maybe pfBlocker is just the most visible to suffer from that 'defect'.

If that were the case there should be some other sign of stress on the PHP-FPM socket, and there isn't. Most likely it's a completely unrelated problem that happens to present with one common symptom.

It has started becoming an issue for me again. I have attached my outputs as requested.

Netgate Sg-4860
Ver. 2.4.0
pfBlocker
Ver. 2.1.2

[pfSense Freeze.txt](/public/imported_attachments/1/pfSense Freeze.txt)

Jailer

@jimp:

@PiBa:

That why the report of @rightnow would be interesting. As he has this error without any packages installed. Makes me wonder if some other component causes (part of) the problem, and maybe pfBlocker is just the most visible to suffer from that 'defect'.

If that were the case there should be some other sign of stress on the PHP-FPM socket, and there isn't. Most likely it's a completely unrelated problem that happens to present with one common symptom.

If thats the case I’m curious as to why a ZFS install mitigates it?

ak

@PiBa:

@jimp:

@ntct:

I also have 502 Bad Gateway on pfSense 2.4 and pfblockerng 2.1.2…... :-\

And judging by the output, it's still DNSBL getting backed up.

That why the report of @rightnow would be interesting. As he has this error without any packages installed. Makes me wonder if some other component causes (part of) the problem, and maybe pfBlocker is just the most visible to suffer from that 'defect'.

I did a fresh install of 2.4 and had pfblockerng with the issue. I didn't come on the forum so didn't know of the instructions to get log details.

However, I disabled pfblockerng and have not had a lock up for over 24 hours. Let me run without pfblockerng for a bit more then switch it back on and see.