pfsense 2.6.0 sshguard @ web gui bug/crash

[stephenw10]

I would think they are. You always see an entry for each pool you have configured like that.

185.83.169.27 is in 2.uk.pool.ntp.org

[22.05-DEVELOPMENT][admin@plusdev-2.stevew.lan]/root: host 2.uk.pool.ntp.org
2.uk.pool.ntp.org has address 185.103.117.60
2.uk.pool.ntp.org has address 185.83.169.27
2.uk.pool.ntp.org has address 85.199.214.101
2.uk.pool.ntp.org has address 103.214.44.30
2.uk.pool.ntp.org has IPv6 address 2001:8b0:df52:914d::123
2.uk.pool.ntp.org has IPv6 address 2a00:da00:1800:7f::1
2.uk.pool.ntp.org has IPv6 address 2a00:2381:19c6::100
2.uk.pool.ntp.org has IPv6 address 2a0b:9b00:463::123

Steve

[VioletDragon]

@stephenw10 urmm interesting. when i do host host 2.uk.pool.ntp.org it shows different here,

2.uk.pool.ntp.org has address 103.214.44.30
2.uk.pool.ntp.org has address 45.63.100.187
2.uk.pool.ntp.org has address 139.143.5.30
2.uk.pool.ntp.org has address 134.0.16.1
2.uk.pool.ntp.org has IPv6 address 2606:4700:f1::1
2.uk.pool.ntp.org has IPv6 address 2a0b:9b00:463::123
2.uk.pool.ntp.org has IPv6 address 2a03:b980:123:2::a
2.uk.pool.ntp.org has IPv6 address 2a01:7e00::f03c:91ff:fe73:fd27

[stephenw10]

Probably more things in the pool. I've never really looked into it that deeply. Local caching involved in the resolving.

Steve

[VioletDragon]

@stephenw10 Hi i'm back, the problem has started again Web Gui is not working and comes up with ERR_CONNECTION_CLOSED, top is not showing much any ideas? I believe either php or nginx is crashing and not restarting.

[VioletDragon]

@violetdragon I have noticed that PHP is using some RAM and some CPU when the Gui works.

23165 www           1  20    0    27M    16M kqread   1   0:59   0.11% haproxy
71979 root          1  20    0    13M  3644K CPU1     1   0:00   0.10% top
85706 root          1  27    0    60M    43M nanslp   1   0:31   0.09% php
23709 root          1  20    0    30M  9612K kqread   0   0:00   0.06% nginx
62037 root          3  41   20   538M   471M bpf      2   0:07   0.03% snort
68274 root          1  20    0    20M  9396K select   3   0:00   0.02% sshd
54051 dhcpd         1  20    0    25M    13M select   1   0:29   0.02% dhcpd
39229 root          5  52    0    11M  2644K uwait    0   0:36   0.01% dpinger
29141 root          1  20    0    21M  8408K select   1   0:04   0.01% mpd5
65272 root          1  20    0    11M  2200K select   3   0:53   0.01% powerd
95019 avahi         1  20    0    12M  3568K select   1   0:34   0.01% avahi-daemon
38649 root          5  52    0    11M  2644K uwait    1   0:42   0.01% dpinger
 2502 root          1  20    0    11M  2700K select   2   0:29   0.01% syslogd
31450 root          1  20    0    19M  7216K select   1   0:14   0.01% ntpd
77306 root          1  20    0    11M  2212K kqread   0   0:13   0.01% tail_pfb
85029 root          1  20    0    17M  7900K kqread   2   1:35   0.00% lighttpd_pfb
  372 root          1  20    0   101M    27M kqread   1   0:07   0.00% php-fpm
60982 root          1  20    0    12M  3024K bpf      3   0:48   0.00% filterlog
77510 root          1  21    0    77M    59M piperd   0   4:36   0.00% php_pfb
 5480 root          2  20    0    19M  7788K select   2   1:54   0.00% openvpn
12965 uucp          1  20    0    12M  2864K select   1   0:37   0.00% usbhid-ups
57324 root          1  52    0   134M    52M accept   3   0:19   0.00% php-fpm
58571 root          1  52    0   134M    52M accept   1   0:19   0.00% php-fpm
47138 root          1  20    0   132M    50M piperd   2   0:17   0.00% php-fpm
85596 root          1  52    0   132M    50M accept   0   0:16   0.00% php-fpm
 8628 root          1  52    0   134M    51M accept   0   0:15   0.00% php-fpm

[VioletDragon]

@violetdragon Just thought i'd mention this, after having another look it's something on the home page that is causing the issue, when the home page does not load if i go to any of the tab it loads them but not the home page.

(Edit)

The problem is the Disks Widget, when the Disk widget is on the home page the problem appears home page does not load when removed off the home page problem disappears

[stephenw10]

Huh, that's interesting. The disks widget is there by default on on 2.6 installs so I would have expected many more reports of similar behaviour.
Do you have an unusual disk setup?
Is there anything logged in the nginx or system logs when this happens?

Steve

[VioletDragon]

@stephenw10 Hi, only disk setup I have are 2x 60GB Solid State Drives in a mirror, nope nothing in the logs, would it be possible to post a video so you can see it? It's strange ain't it.

[stephenw10]

Sure post a video, or link to it. I'd like to see it.

I have systems with dual ZFS disks in a mirror but they are smaller.

Steve

[jimp]

@violetdragon said in pfsense 2.6.0 sshgaurd @ web gui bug/crash:

@stephenw10 Hi, only disk setup I have are 2x 60GB Solid State Drives in a mirror, nope nothing in the logs, would it be possible to post a video so you can see it? It's strange ain't it.

Is this a gmirror setup that's been upgraded over time or a ZFS mirror?

I have several ZFS mirrors and the disk widget works fine there but I don't think I have any gmirror setups on 2.6 currently.

[stephenw10]

No problems on the test box I use for this:

[Gertjan]

Probably not related, but :
@violetdragon said in pfsense 2.6.0 sshguard @ web gui bug/crash:

2020/09/08 04:19:59 [error] 4127#100429: *20842 upstream timed out (60: Operation timed out) while reading response header from upstream, client: 192.168.1.9, server: , request: "POST /acme/acme_certificates.php HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.socket", host: "violetdragon.ddns.net:10443", referrer: "https://violetdragon.ddns.net:10443/acme/acme_certificates.php"

Who is accessing what from where ?
Why is a LAN based client using "violetdragon.ddns.net" (the WAN IP ?? )- why not using the LAN IP of pfSense host name, which is 192.168.1.1 ?
Or is your pfsense really called "violetdragon" and your domain set to "ddns.net" ? So "violetdragon.ddns.net" is 192.168.1.1 (looks very wrong to me).

[stephenw10]

It's unusual but it should work fine that way. The disks widget shouldn't care.

[Gertjan]

@stephenw10

Sure thing.
It looked to me as if the request came from the 'outside' which means he opened up the GUI to the outside world. And that opens up a can of worms.

[VioletDragon]

@gertjan If you look at the logs carefully, you will see that the 1.9 IP is my workstation, violetdragon.ddns.net was the DDNS Hostname of the firewall and I was internally wrapping it inside meaning, I was using the DDNS Hostname with DNS Resolver it is not unusual to do, I moved to two Static IPs for Ha on my WAN so now i am using a proper FQDN with DNS Resolver & Haproxy with SSL Offloading for Lets Encrypts for both Internal Services and External Services, I guess your not familiar with this kind of setup, and yes I have moved the IP of the Firewall from 1.1 this is what you do in the CCNA world. Web Gui is not publicly exposed I am not that dumb to publicly expose the Web Gui same with SSH on everything, for External use I use my FQDN and OpenVPN/IPsec for offsite Servers.

[VioletDragon]

@jimp Hi, it is a ZFS Mirror.

[stephenw10]

Mmm, not seeing any issues on systems with ZFS mirrors here.
Hopefully the video should clarify things.

Steve

[VioletDragon]

@stephenw10 I will get the video to you in a few hours, I have had a busy weekend with it being bank holiday. Sorry for the delays.

[stephenw10]

No worries, I'm glad you were able to narrow down the cause this far already.

[VioletDragon]

@stephenw10 Hi, Just to report back. Even after removing the Disk Widget the problem is still there cannot access the web gui at all then it starts working but can access different tabs. This is weird.