502 Bad Gateway on 2.4.5p1

mogarchy

I am seeing 502 Bad Gateway in the GUI on multiple brand new devices running 2.4.5p1 but I have also seen this issuing going back several versions, just seems to have gotten much worse. I've seen this on multiple SG-3100 and also an SG-5100 with 16GB RAM and 128GB SSD

Pretty basic setups, nmap is the only package added from stock. Sometimes can SSH in and restart PHP-FPM - but often not - sometimes I just get

ssh_exchange_identification: Connection closed by remote host

I can still reach devices that I have port forwarded - so there is still some routing/firewalling happening, but with no way to access the firewall this is a pretty major bug - especially considering that PF/FreeBSD does not handle dirty shutdowns well - which is the only option to recover.

I'm happy to try to provide logs, though they are a bit difficult to obtain with constant crashing - so if there's something specific I should try to pull please let me know.

Thanks!

mogarchy

Some logs:

/var/log/nginx/error.log has several

2020/08/15 18:50:23 [error] 22886#100492: *53180 connect() to unix:/var/run/php-fpm.socket failed (61: Connection refused) while connecting to upstream, client: xx.xx.xx.xx, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm.socket:", host: "xx:xx:xx:xx:xxxx"

/var/log/system.log has

kernel: sonewconn: pcb 0xfffff800545f1960: Listen queue overflow: 193 already in queue awaiting acceptance (26 occurrences)

And a hundreds of these repeated:

rtsold: Received RA specifying route xxxx::xxxx:xxxx:xxxx:xxxx for interface wan(igb0)

chrcoluk

you have possibly hit some kind of resource limitation.

try these sysctl commands (they just report values wont change anything) and report back the values here, you will need cli access, either via ssh, web console mode or locally.

sysctl kern.ipc.somaxconn
sysctl kern.sigqueue.max_pending_per_proc
sysctl -a | grep net.inet.ip.portrange

the nginx error is reporting it was unable to connect to the fpm backend.

plualked

The ssh_exchange_identification: read: connection reset by peer is very rare, but you can run into it if you are trying to ssh into any Unix server. It won’t matter if you are using Windows with Cygwin to gain access to macOS or Ubuntu with the terminal to the ssh into the arch, CentOS, or fedora.

You should "Check the Hosts.deny File".

tstockman

I'd love a solution to this - see it constantly on my lab SG-3100 - have even pruned it back in terms of packages and still does it :(
Same scenario - usually I can SSH in and restart PHP-FPM but other times i have to hard reboot the device. Not the result i was hoping for testing an SG3100 for use at clients :/