Pfsense/Netgate unable to login, Fatal error
-
I have a Pfsense / Netgate that's been running in a datacenter/rack for over a year that is only accessed occasially via VPN and serving as a local router for private IP and very basic firewall.
I went to VPN this weekend and it gives me Auth Failure. I went to the Dashboard (public facing IP, I know bad security) I get long string below, nothing else. I have made no changes on this device in many months.
Was it hacked or did the system just crash? It seems to still be running on some level as basic routing on the LAN is working using the Pfsense as default router.
Code on web browser when accessing dashboard:
Fatal error: Uncaught TypeError: Return value of g_get() must be an instance of mixed, string returned in /etc/inc/globals.inc:411 Stack trace: #0 /etc/inc/globals.inc(129): g_get('product_version') #1 /etc/inc/config.gui.inc(28): require_once('/etc/inc/global...') #2 /etc/inc/auth.inc(34): require_once('/etc/inc/config...') #3 /etc/inc/authgui.inc(27): include_once('/etc/inc/auth.i...') #4 /usr/local/www/guiconfig.inc(62): require_once('/etc/inc/authgu...') #5 /usr/local/www/index.php(46): require_once('/usr/local/www/...') #6 {main} thrown in /etc/inc/globals.inc on line 411 Fatal error: Uncaught TypeError: Return value of g_get() must be an instance of mixed, string returned in /etc/inc/globals.inc:411 Stack trace: #0 /etc/inc/config.lib.inc(1157): g_get('tmp_path_user_c...') #1 [internal function]: pfSense_clear_globals() #2 {main} thrown in /etc/inc/globals.inc on line 411 -
Thats a typical PHP error message.
Did you update to latest 23.01 recently?
In some configs it seems that code is not matching PHP 8.0 requirements.Regards
-
@fsc830 I haven't logged into the unit in months. So I haven't upgraded it. Perhaps I had selected to auto upgrade? I won't be able to check it until I get to the data center in a few weeks as I can't log into it remotely.
-
@stormchaser5 can you ssh?
There is no auto upgrade ability, though I’ve read posts where people insist that happened. I’ve never seen that at any of our clients or routers.
-
@steveits No I tried ssh intoing it. I probably disabled it for security. I set it up about a year ago I can't remember exactly. It just odd that it stopped working. Its a 7100 model. I am the only user.
-
After you've logged in can you visit a page other than the dash directly?
Like the /system_advanced_admin.php for example?
If you can reach that you can enable ssh there.
If you can connect the USB console to some other local machine you can enable ssh from the menu there also.
Steve
-
@stephenw10 I can't log in at all, ssh, web anything. I won't be physically in front of the device for a couple of weeks at least as it in in a datacenter in another state. I will have to console into it I guess.
It is troubling the device just threw this error and can't be logged in. Unless it was hacked, there is a issue somewhere in the code base. As it worked fine for months, then all the sudden this, with no login, except some occasional low volume OpenVPN sessions. It's in a rack, in a datacenter so power/ac is not an issue.
-
So you don't even see the login screen?
If there was no change made on the firewall then I would suspect a drive issue. If the boot drive fails services will fail over time as and when they need to read/write data. Without some sort of access it's hard to say though.
Steve
-
This is what the web page looks like when you go to the external web page (dashboard) to login. Just this string of characters.
A hardware failure, such as a disk sound plausable. But this XG-7100 is like a year and half old. So seems unlikely and is in a pristine enviroment, power/cooling and very low workload. In a few weeks will probably get to the bottom of it when I get console access. But it's very strange. I am glad it's still working on some level as a gateway.
-
@stormchaser5 If I was traveling on site I'd bring a USB stick with the install image on it just in case.
https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/reinstall-pfsense.html
And maybe some type of spare/replacement. :-/There is also https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html#emmc
-
@steveits Thanks. It's been over year, since I was imersed in pfsense/netgate when I set this up. So I am going to have to sort through this. Thanks for your help. Working on these things in a loud, cramped data center is not fun.
-
@stormchaser5 oh gosh yeah. Bring earplugs. Or noise canceling headphones. As with airplanes! Makes a big difference.
-
@steveits Do you happen to know what type of disk is in the XG-7100 I should bring as a replacement? I am going to go on the assumption the disk is bad and a reinstall is required. It looks like they don't sell a XG-7100 anymore. So 1:1 swap out is not possible, if it is not the disk.
-
@steveits I found the docs.
https://docs.netgate.com/pfsense/en/latest/solutions/xg-7100-1u/m-2-sata-installation.html
-
@stormchaser5 m.2 mSATA. Not NVMe.
-
The 7100 also has on-board eMMC, it may not have shipped with an m.2 SATA drive.
But if the eMMC has failed you can always fit an SSD and boot from that.Steve
-
@stephenw10 I purchased a Western Digital 500GB WD Red SA500 NAS SSD - SATA III 6 Gb/s, M.2 2280, - WDS500G1R0B and hope to get installed this week.
Do you know what the typical filename /extension is for the configuration backup is for pfsense? I am sure I made a backup of it. Just need to search for it on my NAS.
-
Typical filename looks like this: config-pfsense.mydomain.biz-20220222183909.xml, where pfsense.mydomain.biz is to replace with the pfSense name followed by FQDN.
Regards