502 Bad Gateway, nginx

[Aseknet]

Hi Experts,

Really need your help now.
Running latest pfSense on an Intel NUC and this issue has been escalating.
The latest year I started to get “502 Bad Gateway, nginx”.
Often this can be solved by using SSH and select option “16. Restart PHP-FFM”.
After 15 to 60 minutes, I am back with the same problem, often even WAN connection drops.

Tried guides on Internet that suggests to disable different widgets but with no success.

Any ideas what to look for and what logs to check to solve this?

I have many years of configurations and would not like to start from scratch.
Tried a new installation and imported my configurations and still had these issues.

Thanks,
Asenet

[stephenw10]

Check the system logs. Check the nginx logs in /var/log.

You are running 2.5.2?

Have you been running it for some time and this just started?

Steve

[Aseknet]

@stephenw10,
Running 2.5.2
I think this behaviour started with this version but happened quite seldom.
It is in the last month it has escalated to a real problem for me.

Need some guidance to read the logs (not an linux guy).
I manager to goto cd /var/log
Then i cannot use dir to see the files.
How to see the files and then read the logs?

Thanks,
Aseknet

[fireodo]

@aseknet said in 502 Bad Gateway, nginx:

How to see the files and then read the logs?

ls (to see the files in the directory)

cat <filename> (to see the content of the file)

[stephenw10]

Yes, the nginx logs need to be read manually like that. The main system log can be accessed in the gui though via Status > System Logs.

[Aseknet]

Thanks Guys for the help,

I can now open the nginx.log fine.
I restarted the PHP-FPM service so I could narrow down the time when this happens and find an error message that matches that timeframe, of course it has now been working for 2 hours when trying to chase it...
Will come back when I catch the problem in the log.

Thanks,
Aseknet

[Aseknet]

Hi Guys,

Everything seemed to work until today when I logged on pfSense.
After some minutes I got a freeze, 502 Bad Gateway, this is what the logs say:
Seems like the widgets are involved.

Replaced the actual IPs with ClientIP and pfSenseIP.

Nov 18 17:06:30 pfSense nginx: 2021/11/18 17:06:30 [error] 83498#100152: *2302 connect() to unix:/var/run/php-fpm.socket failed (61: Connection refused) while connecting to upstream, client: ClientIP, server: , request: "GET /widgets/widgets/snort_alerts.widget.php?getNewAlerts=1637251590358 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "pfSenseIP", referrer: "http://pfSenseIP/"

Nov 18 17:06:30 pfSense nginx: ClientIP - - [18/Nov/2021:17:06:30 +0100] "GET /widgets/widgets/snort_alerts.widget.php?getNewAlerts=1637251590358 HTTP/1.1" 502 552 "http://pfSenseIP/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36"

Nov 18 17:06:35 pfSense nginx: 2021/11/18 17:06:35 [error] 83498#100152: *2302 connect() to unix:/var/run/php-fpm.socket failed (61: Connection refused) while connecting to upstream, client: ClientIP, server: , request: "GET /widgets/widgets/pfblockerng.widget.php?getNewWidget=1637251595359 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "pfSenseIP", referrer: "http://pfSenseIP/"

Nov 18 17:06:35 pfSense nginx: ClientIP - - [18/Nov/2021:17:06:35 +0100] "GET /widgets/widgets/pfblockerng.widget.php?getNewWidget=1637251595359 HTTP/1.1" 502 552 "http://pfSenseIP/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36"

And to resolve this the solution was:
Using SSH and select option “16. Restart PHP-FFM”
Usually this works but sometimes a reboot of pfSense is needed.

Thanks,
Asknet

[Bambos]

Hello everyone,

i run to the same problem with version 2.6.0.
502 bad gateway nginx.

Do we have some updates about that ? what is recommended ?

[stephenw10]

Upgrade to the current version, 2.7.2.

[Aseknet]

Sadly I never got around this after a huge amount of time for a fix, not even in 2.7.2 so I turned to OPNsense instead of waiting for 2.8.0.
Missing some features from pfSense, but now I don’t have issues anymore to access the interface.
Loved pfSense and the community but this was a big issue for me :(

[stephenw10]

Hmm, odd. We don't really see that in 2.7.2 any longer. I used to be a much bigger issue. It pretty much always implies some php process running away though.

[vector88]

Hello @stephenw10 and everyone.

I've recently installed Pfsense 2.7.2 and kept on getting bad gateway 502.

while troubleshooting I tried reverting back to 2.7.0 and after doing so, my pfsense IP no longer gave me 502, but after updating again to 2.7.2 i kept getting 502.

I tried restarting php-fpm and it loads webgui before it says 502 again.

I reverted to 2.7.0 and webgui is now working again.

The only problem I have is I'm unable to use pkg manager due to version mismatch :(

Can anyone help?

Thanks everyone,
Vector

[stephenw10]

If you see 502 errors it's almost always because php failed for some reason. You should see something logged.

[vector88]

@stephenw10
I do see things get logged after accessing: tail -f /var/log/system_general.log

It says: "[ERROR] 556088#100299 *88 connect () to unix: var/run/php-fpm.socket failed (61: connection refused) while connecting to upstream, cleint: [PFSENSE IP] server:

And while running tail netstat -an | grep LISTEN
It listens on 127.0.0.1.953.

Would you please help?

[stephenw10]

But nothing in the main system log?

How is it configured? What are you running on?

[vector88]

@stephenw10
I'm sorry, I'm a beginner.

Would you please explain further?

[stephenw10]

I mean a basic 2.7.2 install will run without php issues. So what configuration have to done to yours. Something there is causing it a problem. Maybe some package you added is exhausting the RAM for example.

[vector88]

@stephenw10
I get your point; however, this issue occurs as soon as I do a clean install of 2.7.2
Without changing anything in the configuration at all.

I just set the IPs and the interfaces. pfsense WAN: 192.168.1.10 ISP router: 192.168.1.1 Pfsense LAN : 10.27.27.1

**Edit

I didn't add any packages and I have 16GB of RAM

[stephenw10]

Hmm, odd. What hardware is it installed on?

You are connecting to it from a client on the LAN?

[vector88]

@vector88

It's on Dell Latitude 7490 i7-8650U CPU
And the installation was USB memstick.

Again, this issue only occurred on 2.7.2 and 2.7.1.
2.7.0 worked fine, and I had to update it because the package manager didn't work :(

Yes, the client is on the LAN but even when I try to access WebGui from the ISP's side it returns the same 502.