2.60 GUI causes services to fail?
-
Hi there,
Recently I updated all my boxes from 2.5.2 to 2.60 and I found out some interesting facts:
If I set everything up and save, then close the web GUI, the whole system works like a charm. There are no errors in the log.
However, If I keep the web GUI showing to moniter system status, Dynamic DNS configuration will not be saved. It shows 504 error after a very long load time. And the IP address can not be updated after PPPOE redial. Also, OpenVPN service will automatically stop running after PPPOE redial. Under these circumstances, there are still no errors in the log. I guess the log service also hangs.
Anyway, as long as the web GUI is not opened, the whole system runs very smoothly without any problems.
-
Do you see any logging changes when you have the GUI open?
For example if you have the dashboard open the 'GUI Service' log will be filled with entries from the auto updating widgets. That can cause the logs to rotate for more frequently.
What is different between the logs covering a PPP reconnection with and without the GUI open?
Steve
-
Hi Steve,
I just did more detailed tests.
@stephenw10 said in 2.60 GUI causes services to fail?:
Do you see any logging changes when you have the GUI open?
Yes. As long as I don't save any configuration, the logs look fine.
@stephenw10 said in 2.60 GUI causes services to fail?:
What is different between the logs covering a PPP reconnection with and without the GUI open?
With GUI opened, after manually PPPOE redialing, the GUI got stuck for about ten seconds.
-
No warnings or errors in PPP logs. PPP logs looked normal.
-
OpenVPN logs showed "TCP/UDP: Socket bind failed on local address [AF_INET]xxx.xxx.xxx.xxx:xxx: Can't assign requested address (errno=49)" and OpenVPN service stoped running immediately (no more log entries ).
-
Four http 403 errors appeared in GUI Service logs which are "/widgets/widgets/disks.widget.php", "/widgets/widgets/interfaces.widget.php", "/widgets/widgets/gateways.widget.php" and "/widgets/widgets/dyn_dns_status.widget.php".
-
Saving Dynamic DNS configuration still led to a "/services_dyndns_edit.php?id=0" HTTP 504 error.
Without GUI opened, by redialing PPPOE using "Periodic Reset", there were no warnings or errors in any logs.
Deck
-
-
Hmm, I guess you could be hitting this if you have a lot of rules:
https://redmine.pfsense.org/issues/12827
And the errors you see in the gui are just a by-product of that.Try adding the work-around patch for that from the system patches package.
Steve
-
@upper-deck re: log rotation, if you have a slower CPU Netgate recommends turning off log compression in the system log settings. See if Diagnostics/System Activity is showing several bzip processes. I've seen cases where something is generating a LOT of logs where a device is using all its CPU to rotate+zip log files. (alt: fix whatever is generating the logs)
If you have the web GUI open to a page that isn't the dashboard is it still an issue?
-
@stephenw10 said in 2.60 GUI causes services to fail?:
Hmm, I guess you could be hitting this if you have a lot of rules:
https://redmine.pfsense.org/issues/12827
And the errors you see in the gui are just a by-product of that.Try adding the work-around patch for that from the system patches package.
Steve
Hi Steve,
I just tried the work around patch. No luck, OpenVPN still stops running after PPPOE redial.
Deck
-
@steveits said in 2.60 GUI causes services to fail?:
@upper-deck re: log rotation, if you have a slower CPU Netgate recommends turning off log compression in the system log settings. See if Diagnostics/System Activity is showing several bzip processes. I've seen cases where something is generating a LOT of logs where a device is using all its CPU to rotate+zip log files. (alt: fix whatever is generating the logs)
If you have the web GUI open to a page that isn't the dashboard is it still an issue?
Hi SteveITS,
You are totally right. Turning off log compression does work even though the CPU usage is still running at a very low percentage. All my boxes are installed Intel Celeron J1900.
Deck.
-
@upper-deck Interesting, usually that maxes the CPU at least when I've seen it. So maybe due to slower storage?
Your next task is to see if something is generating a lot of log entries. The dashboard by itself generates log entries but it shouldn't be anywhere near enough to overwhelm the system. Or maybe the max log (rotation) size is small so it rotates frequently?
-
I have been running these three pfsense servers with the similar configuration for many years. All devices were initially installed with version 2.2, and updated with each stable release all the way up to 2.6. Over the years, I have rarely modified the configuration, and it has been running smoothly without any problems until 2.5.2. What I‘ve posted earlier was not for one specific device, but all three.
Same as the past few years, only site-2-site OpenVPN keeps generating warning logs like this “WARNING: 'ifconfig' is present in local config but missing in remote config, local='ifconfig xxx.xxx.xxx.xxx'”. This warning log has been around for years, and I've had a ton of attempts to fix it before. Unfortunately, it still can't be solved.
I just double checked the logs and found no abnormal entries. Probably I should perform a clean installation of pfsense 2.6 with ZFS.
These pfsense servers (with Unifi UAP-LR & FIOS) are used for providing free WIFI access to the community's playground. More than 200 devices are connected at every baseball game. pfSense has been very stable over the years.
-
Are you seeing this?: https://redmine.pfsense.org/issues/12747
That's typical in situations where the logs are rotating rapidly. -
@stephenw10 said in 2.60 GUI causes services to fail?:
Are you seeing this?: https://redmine.pfsense.org/issues/12747
That's typical in situations where the logs are rotating rapidly.No, I don't have this log entries. SSH is disabled in my config.
Deck
-
Hmm, maybe it's failing to rotate the logs at all then. sshguard is enabled for webgui logins whether or not SSH is enabled.
Steve
