Undefined symbol "__libc_start1@FBSD_1.7" on pfSense Plus 23.01

stephenw10

Hmm, I assume reinstalling this would be difficult?

pkg-static -d update runs successfully against the 23.05 repo?

sgw

reinstalling would include sending the appliance from one country to the other .. I am a few hundred kms away from that box. I have a 2nd SG1100 there (already on 23.05, no such issues there!), so it's possible, but it would be great to avoid that.

Yes, pkg-static -d update succeeds while:

# /usr/local/etc/pkg/repos: ls -l
total 1
-rw-r--r--  1 root  wheel  26 Feb 10 21:04 FreeBSD.conf
lrwxr-xr-x  1 root  wheel  57 Jun 22 15:18 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-stable.conf

# cat pfSense.conf 
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_aarch64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_05_aarch64-pfSense_plus_v23_05",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

stephenw10

Ah, it's an 1100?

Is it installed as ZFS? You can probably roll back to a snapshot before you tried to upgrade if so.

sgw

@stephenw10 yeah, zfs.

tried to reinstall php stuff and gettext etc ... a mess

gui broken, can I roll back from cli?

stephenw10

Yes, you can using bectl. Or from the loader if you have console access.

First check what there is:

bectl list -s

sgw

@stephenw10

 bectl list -s
BE/Dataset/Snapshot    Active Mountpoint Space Created

default
  pfSense/ROOT/default NR     /          897M  2023-02-10 22:00

nothing else

sgw

Could I get some list of php-packages to reinstall maybe? That might fix something ...

Rolling back is not possible as far as I understand. Looks as if everything has been written into this first boot env.

thanks

stephenw10

Yup no BE snaps other than default, nothing to roll back to.

What pkg versions have you ended up with:

pkg-static info -x pfsense

sgw

@stephenw10 said in Undefined symbol "__libc_start1@FBSD_1.7" on pfSense Plus 23.01:

pkg-static info -x pfsense

# pkg-static info -x pfsense
pfSense-base-23.01
pfSense-boot-23.01
pfSense-default-config-serial-23.01
pfSense-kernel-pfSense-23.01
pfSense-pkg-aws-wizard-0.10
pfSense-pkg-ipsec-profile-wizard-1.1_1
pfSense-pkg-squid-0.4.46
pfSense-rc-23.01
pfSense-repo-23.01
pfSense-u-boot-1100-20220428
pfSense-u-boot-2100-20210930_1
pfSense-u-boot-env-20230123
pfSense-upgrade-1.0_62

thanks for your patience!

stephenw10

Ok those are old versions of pfSense-upgrade.

Try running: pkg-static upgrade pfSense-upgrade

Then try to run that again once it's updated.

sgw

@stephenw10

done:

pkg-static info -x pfsense
pfSense-base-23.01
pfSense-boot-23.01
pfSense-default-config-serial-23.01
pfSense-kernel-pfSense-23.01
pfSense-pkg-aws-wizard-0.10
pfSense-pkg-ipsec-profile-wizard-1.1_1
pfSense-pkg-squid-0.4.46
pfSense-rc-23.01
pfSense-repo-23.01
pfSense-u-boot-1100-20220428
pfSense-u-boot-2100-20210930_1
pfSense-u-boot-env-20230123
pfSense-upgrade-1.0_62
php81-pfSense-module-0.91

sgw

@sgw said in Undefined symbol "__libc_start1@FBSD_1.7" on pfSense Plus 23.01:

php81-pfSense-module-0.91

... comes from my attempts to "fix things".

I suggested the customer to send the appliance, will get his reply tmrw.

I think we can't trust it anymore, right now this:

*** Welcome to Netgate pfSense Plus 23.01-RELEASE (arm64) on xxx ***

Fatal error: Uncaught Error: Call to undefined function ctype_digit() in /etc/inc/util.inc:257
Stack trace:
#0 /etc/inc/interfaces.inc(162): is_numericint('4090')
#1 /etc/inc/interfaces.inc(247): vlan_valid_tag('4090')
#2 /etc/inc/interfaces.inc(5932): interface_is_vlan('mvneta0.4090')
#3 /etc/inc/interfaces.inc(6375): get_real_interface('mvneta0.4090')
#4 /etc/inc/interfaces.inc(6618): find_interface_ipv6('mvneta0.4090', false)
#5 /etc/rc.banner(147): get_interface_ipv6('wan')
#6 {main}
  thrown in /etc/inc/util.inc on line 257
PHP ERROR: Type: 1, File: /etc/inc/util.inc, Line: 257, Message: Uncaught Error: Call to undefined function ctype_digit() in /etc/inc/util.inc:257
Stack trace:
#0 /etc/inc/interfaces.inc(162): is_numericint('4090')
#1 /etc/inc/interfaces.inc(247): vlan_valid_tag('4090')
#2 /etc/inc/interfaces.inc(5932): interface_is_vlan('mvneta0.4090')
#3 /etc/inc/interfaces.inc(6375): get_real_interface('mvneta0.4090')
#4 /etc/inc/interfaces.inc(6618): find_interface_ipv6('mvneta0.4090', false)
#5 /etc/rc.banner(147): get_interface_ipv6('wan')
#6 {main}
  thrown
 0) Logout (SSH only)                  9) pfTop
 1) Assign Interfaces                 10) Filter Logs
 2) Set interface(s) IP address       11) Restart webConfigurator
 3) Reset webConfigurator password    12) PHP shell + Netgate pfSense Plus tools
 4) Reset to factory defaults         13) Update from console
 5) Reboot system                     14) Disable Secure Shell (sshd)
 6) Halt system                       15) Restore recent configuration
 7) Ping host                         16) Restart PHP-FPM
 8) Shell

stephenw10

Hmm, did it not offer an upgrade for pfSense-upgrade?

It's still looking at the 23.05 repo branch?

sgw

@stephenw10 it's looking at 23.01 (/usr/local/etc/pfSense/pkg/repos/pfSense-repo-prev.conf)

sgw

 pkg-static upgrade pfSense-upgrade
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
Your packages are up to date.

stephenw10

Ah, Ok if it's still looking at 23.01 it might be time to do a forced reinstall of all pkgs from that repo. It's not something I'd normally recommend but there may not be much else we can do here:
https://docs.netgate.com/pfsense/en/latest/troubleshooting/upgrades.html#forced-pkg-reinstall

sgw

@stephenw10

 pkg-static install -fy pkg pfSense-repo pfSense-upgrade
Updating pfSense-core repository catalogue...
pkg-static: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_aarch64-core/meta.txz: Bad Request
repository pfSense-core has no meta file, using default settings
pkg-static: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_aarch64-core/packagesite.pkg: Bad Request
pkg-static: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_aarch64-core/packagesite.txz: Bad Request
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/meta.txz: Bad Request
repository pfSense has no meta file, using default settings
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/packagesite.pkg: Bad Request
pkg-static: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/packagesite.txz: Bad Request
Unable to update repository pfSense
Error updating repositories!

even that fails now

sgw

Couldn't I "simply" scp/rsync some directories from a working appliance into the buggy one to make php work for now? I know it's ugly.

stephenw10

The user cert has expired. Updating that could be a problem if you can't run either pfSense-repoc or pfSense-upgrade -c

Where is this actually installed? Is there no way you can get remote hands on it to help with a re-install?

sgw

@stephenw10 We plugged in the other appliance yesterday and the box will be sent to me for reinstallation. Thanks to you for all your help.