Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Undefined symbol "__libc_start1@FBSD_1.7" on pfSense Plus 23.01

    Scheduled Pinned Locked Moved
    General pfSense Questions
    9
    52
    8.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • O
      opoplawski
      last edited by

      So, I know installing packages from the FreeBSD repos is not supported, but this still seems a bit surprising to me.

      I installed sssd-1.16.5_8 on pfSense Plus 23.01 by enabling the FreeBSD repo. But sssd fails to start with:

      ld-elf.so.1: /usr/local/sbin/sssd: Undefined symbol "__libc_start1@FBSD_1.7"

      which seems to indicate some kind of mismatch between the libc version expected by the sssd package and pfSense's BSD base. Could anyone else shed some more light?

      1 Reply Last reply Reply Quote 0
      • bmeeksB
        bmeeks
        last edited by bmeeks

        Now you know why mixing package repos is strongly discouraged ... ☺.

        The error is caused by slight differences in the kernel version. pfSense Plus is based on FreeBSD 14-CURRENT, but just a little bit behind in the specific version number.

        You are lucky you only broke one executable instead of your entire pfSense installation. Stick to using only the packages available from the official pfSense Plus repo (under SYSTEM > PACKAGE MANAGER). The only other option is to create your own package builder based on the current FreeBSD source code version used in pfSense Plus and compile the packages from ports. But that is a lot of trouble and still likely to not work as pfSense Plus is different from the pfSense CE code that is available to the public. The specific pfSense Plus FreeBSD fork is not available to the public.

        Edit: because FreeBSD CURRENT is pretty much constantly getting updated, there are small build version number changes as new commits are added. pfSense is not automatically following all of those small version number updates (meaning not every commit that goes into FreeBSD CURRENT is going to immediately go into pfSense). Thus you can expect that the official FreeBSD ports repo for CURRENT will slowly creep ahead of pfSense until the next pfSense release "catches up" with CURRENT again. That means packages from the FreeBSD CURRENT ports repo will not necessarily work on pfSense due to the small version difference.

        For example, FreeBSD CURRENT will update the ports tree quarterly with packages compiled for the latest kernel update. But pfSense, once a release is out, does NOT follow these quarterly ports updates for its release branch. A given pfSense release will generally not get "new" quarterly ports updates until the next pfSense release. Development snapshots, on the other hand, will get quarterly ports updates from FreeBSD upstream. And that just recently happened over in the pfSense 2.7 CE snapshots. They pulled in the latest March quarterly updates for the ports and the latest kernel from FreeBSD CURRENT. But that will not happen in the RELEASE branch of pfSense.

        The above does not mean no package ever gets updated in a pfSense release, though. If some package has a vulnerability or otherwise needs to be updated, then it will be updated and recompiled against the flavor of FreeBSD CURRENT that pfSense RELEASE is built on. That way shared library versions stay the same. Your import from the stock FreeBSD CURRENT package repo pulled in a version of your port that was compiled against the newer shared libraries in the more recent FreeBSD CURRENT source tree. That newer library version does not exist in the FreeBSD CURRENT from last quarter that the present pfSense Plus release is built upon, thus you see the library load error.

        O 1 Reply Last reply Reply Quote 2
        • jimpJ jimp moved this topic from Problems Installing or Upgrading pfSense Software on
        • O
          opoplawski @bmeeks
          last edited by

          @bmeeks Thanks for the reply. I did think to make sure that the pkg install command only added packages and didn't replace any existing ones.

          I'm still a bit surprised to learn that pfSense Plus 23.01 is based on a pre-release version of FreeBSD 14.0 which isn't considered stable yet. But I guess the push for new stuff is inexorable.

          Perhaps even more crazy on my part - I ended up working around the issue by installing the latest sssd package from FreeBSD 13.

          D 1 Reply Last reply Reply Quote 0
          • D
            djgans @opoplawski
            last edited by

            @opoplawski
            I did a similar fub, because I wanted to install inxi to get extended system info. Well, I get the same result. What I found that fixed the problem was change the repo options for Freebsd back to "no" and used the following command:
            pkg-static install -f pkg
            This reverted my pkg back to pfSense repo version.

            M 1 Reply Last reply Reply Quote 0
            • F FSC830 referenced this topic on
            • CatSpecial202C CatSpecial202 referenced this topic on
            • M
              martys @djgans
              last edited by

              @djgans said in Undefined symbol "__libc_start1@FBSD_1.7" on pfSense Plus 23.01:

              @opoplawski
              I did a similar fub, because I wanted to install inxi to get extended system info. Well, I get the same result. What I found that fixed the problem was change the repo options for Freebsd back to "no" and used the following command:
              pkg-static install -f pkg
              This reverted my pkg back to pfSense repo version.

              Can you give specifics on what 'change the repo options for Freebsd back to "no"' entails on my 23.01 installation?

              I haven't done anything since the fresh installation on this SSD that I can think of that would have triggered this failure of the pkg utility.

              /Marty

              D 1 Reply Last reply Reply Quote 0
              • D
                djgans @martys
                last edited by

                @martys
                If you look under /usr/local/etc/pkg/repos directory, you'll find 2 .conf files that control whether pfsense uses it's sources and/or whether to include Freebsd's sources. I you do a manual update for some packages when using Freebsd sources, it'll ask to update pkg to the latest Freebsd version, which will mess up your source caching I believe. Setting the repos back to no Freebsd sources and then using pkg-static install -f pkg will revert pkg back to the pfsense version and reupdate the source caching. Keep in mind, this is my best guess as to what's happening since I'm not one of the developers. Even after this, I've had issues trying to upgrade from 23.01 to 23.05

                M 1 Reply Last reply Reply Quote 0
                • M
                  martys @djgans
                  last edited by

                  @djgans Thanks for the additional info!

                  I checked both .conf files and each say:

                  FreeBSD: { enabled: no }

                  I then did pkg-static clean -ay ; pkg-static install -f pkg with good looking (to me) output:

                  [23.01-RELEASE][admin@gateway.home]/root: pkg-static clean -ay ; pkg-static install -f pkg
Nothing to do.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be REINSTALLED:
	pkg-1.19.1_1 [pfSense]

Number of packages to be reinstalled: 1

9 MiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching pkg-1.19.1_1.pkg: 100%    9 MiB   9.5MB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Reinstalling pkg-1.19.1_1...
[1/1] Extracting pkg-1.19.1_1: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

                  but still get the original issue:

                  [23.01-RELEASE][admin@gateway.home]/root: pkg info pkg
ld-elf.so.1: /usr/local/sbin/pkg: Undefined symbol "__libc_start1@FBSD_1.7"

                  No harm, but no progress either. I just hope that no part of pfSense's own upgrading processes expects to use pkg and not pkg-static at this point.

                  L 1 Reply Last reply Reply Quote 0
                  • L
                    lucasbender @martys
                    last edited by

                    @martys
                    I've the same issue here, using:
                    Version 23.01-RELEASE (amd64)
                    built on Fri Feb 10 20:06:33 UTC 2023
                    FreeBSD 14.0-CURRENT

                    I did the same mistake on mixing package repositories, replacing pkg instead of installing my desired package:

                    [23.01-RELEASE][root@router.domain]/root: pkg install -y autossh
Updating FreeBSD repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
Fetching packagesite.pkg: 100%    7 MiB   6.9MB/s    00:01
Processing entries:   0%
Processing entries: 100%
FreeBSD repository update completed. 32883 packages processed.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
New version of pkg detected; it needs to be installed first.
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        pkg: 1.18.4_4 -> 1.19.1_1 [FreeBSD]

Number of packages to be upgraded: 1

9 MiB to be downloaded.
[1/1] Fetching pkg-1.19.1_1.pkg: 100%    9 MiB   9.6MB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Upgrading pkg from 1.18.4_4 to 1.19.1_1...
[1/1] Extracting pkg-1.19.1_1: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.
ld-elf.so.1: /usr/local/sbin/pkg: Undefined symbol "__libc_start1@FBSD_1.7"

                    but in order to reverse the mistake, i did:

                    [23.01-RELEASE][root@router.domain]/root: pkg-static install -f pkg-1.18.4_4
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Checking integrity... done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be DOWNGRADED:
        pkg: 1.19.1_1 -> 1.18.4_4 [pfSense]

Number of packages to be downgraded: 1

Proceed with this action? [y/N]: y
[1/1] Downgrading pkg from 1.19.1_1 to 1.18.4_4...
[1/1] Extracting pkg-1.18.4_4: 100%
You may need to manually remove /usr/local/etc/pkg.conf if it is no longer needed.

                    Now pkg is working again...😌

                    [23.01-RELEASE][root@router.domain]/root: pkg update
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
                    L M 2 Replies Last reply Reply Quote 0
                    • L
                      lucasbender @lucasbender
                      last edited by

                      If others need to install a package without upgrading/crashing pkg, i had success by lock pkg, install needed package, unlock pkg and then disable the FreeBSD repository to avoid any future issues on pfSense:

                      [23.01-RELEASE][root@router.domain]/root: pkg lock pkg
pkg-1.18.4_4: lock this package? [y/N]: y
Locking pkg-1.18.4_4
[23.01-RELEASE][root@router.domain]/root: pkg install autossh
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        autossh: 1.4g [FreeBSD]

Number of packages to be installed: 1

24 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching autossh-1.4g.pkg: 100%   24 KiB  24.3kB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Installing autossh-1.4g...
[1/1] Extracting autossh-1.4g: 100%
[23.01-RELEASE][root@router.domain]/root: pkg unlock pkg
pkg-1.18.4_4: unlock this package? [y/N]: y
Unlocking pkg-1.18.4_4
                      1 Reply Last reply Reply Quote 0
                      • M
                        martys @lucasbender
                        last edited by

                        @lucasbender I do appreciate the success you had, and wish it had worked for me… but it did not. Here's what I got:

                        [23.01-RELEASE][admin@gateway.home]/root: pkg-static install -f pkg-1.18.4_4
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
pkg-static: No packages available to install matching 'pkg-1.18.4_4' have been found in the repositories

[23.01-RELEASE][admin@gateway.home]/root: pkg-static info pkg
pkg-1.19.1_1
Name           : pkg
Version        : 1.19.1_1
Installed on   : Tue May 30 22:31:25 2023 EDT
Origin         : ports-mgmt/pkg
Architecture   : FreeBSD:14:amd64
Prefix         : /usr/local
Categories     : ports-mgmt
Licenses       : BSD2CLAUSE
Maintainer     : pkg@FreeBSD.org
WWW            : https://github.com/freebsd/pkg
Comment        : Package manager
Options        :
	DOCS           : off
Shared Libs provided:
	libpkg.so.4
Annotations    :
	FreeBSD_version: 1400085
	build_timestamp: 2023-05-23T17:56:06+0000
	built_by       : poudriere-git-3.3.99.20220831
	port_checkout_unclean: no
	port_git_hash  : 38f48ce64acc
	ports_top_checkout_unclean: yes
	ports_top_git_hash: 5ceffeaef96e
	repo_type      : binary
	repository     : pfSense
Flat size      : 34.1MiB
Description    :
Package management tool

WWW: https://github.com/freebsd/pkg

                        Notice how there is no mention about FreeBSD repository (2 lines in your output). And pkg-static seems to think this 'bad' version of pkg is in the PfSense repository.

                        I will admit that while I know lots of other package managers very well, I know next to nothing about pkg. So I'm probably missing something obvious or reading more into the output I'm getting than I should.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Did you set the repo to 23.01 before doing that?

                          Since 23.05 was released it will have become the default and pulled in pkg from that. It's not usually an issue, the upgrade process uses pkg-static for everything for just that reason.

                          M 1 Reply Last reply Reply Quote 1
                          • M
                            martys @stephenw10
                            last edited by

                            @stephenw10 said in Undefined symbol "__libc_start1@FBSD_1.7" on pfSense Plus 23.01:

                            Did you set the repo to 23.01 before doing that?

                            Since 23.05 was released it will have become the default and pulled in pkg from that. It's not usually an issue, the upgrade process uses pkg-static for everything for just that reason.

                            Now how was I to know to do that? I'm just minding my business running 23.01 and without using the System > Update GUI out comes a newer version for me to eventually choose… and something in the background seems to have chosen to spring forward the CLI's repository view to the newer version that I'm not ready for, and upgraded pkg to match.

                            You're saying to visit the System > Update GUI to 'drop back' was the final key to getting pkg back. This is probably one of the unwritten things about the upgrade notes warning about not upgrading packages before an upgrade.

                            So after dropping back the setting to 23.01 and reinstalling from pkg-1.18.4_4, all is well on my installation.

                            I do thank you all for responding as quickly, and completely, as you did. Here's to a healthy community of users!

                            /Marty

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              It shouldn't matter that pkg has been updated. It's only an issue if you're using pkg at the CLI in which case you will see that error if you don't use pkg-static.

                              S 1 Reply Last reply Reply Quote 0
                              • S
                                sgw @stephenw10
                                last edited by

                                I somehow lost my active repos. Toggling back and forth between 23.01 and 23.05 doesn't help.
                                I wonder if I could scp some files from a working machine into the system. Or somehow fix a symlink or so.

                                [23.01-RELEASE][root@some.host]/root: pkg-static clean -ay ; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
pkg-static: No active remote repositories configured
pkg-static: No packages available to install matching 'pfSense-repo' have been found in the repositories
pkg-static: No packages available to install matching 'pfSense-upgrade' have been found in the repositories

                                Unfortunately this box is in a protected environment, I can't let Netgate support in right now.

                                help appreciated, thanks

                                S 1 Reply Last reply Reply Quote 0
                                • S
                                  sgw @sgw
                                  last edited by

                                  Some more infos:

                                  # pkg --version
ld-elf.so.1: /usr/local/sbin/pkg: Undefined symbol "__libc_start1@FBSD_1.7"

# pkg-static install -f pkg-1.18.4_4
pkg-static: No packages available to install matching 'pkg-1.18.4_4' have been found in the repositories

                                  According to the GUI I am on 23.01 branch ... but some repo-information seems to be lost right now.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Try running: pfSense-repoc-static

                                    That should pull in repo data. Or show an error.

                                    S 1 Reply Last reply Reply Quote 0
                                    • S
                                      sgw @stephenw10
                                      last edited by

                                      @stephenw10 "Command not found"

                                      I managed to downgrade "pkg" as far as I see. And I reinstalled the repos, and somehow fixed that symlink:

                                      ln -s /usr/local/etc/pfSense/pkg/repos/pfSense-repo-prev.conf pfSense.conf

                                      I assume I should get things right on 23.01 at first?

                                      Currently:

                                      # pkg update
Updating pfSense-core repository catalogue...
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
2427236352:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
2427236352:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com
2427236352:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/var/jenkins/workspace/pfSense-Plus-snapshots-23_01-main/sources/FreeBSD-src-plus-RELENG_23_01/crypto/openssl/ssl/statem/statem_clnt.c:1921:
Certificate verification failed for /C=US/ST=Texas/L=Austin/O=Rubicon Communications, LLC (Netgate)/OU=pfSense Plus/CN=pfsense-plus-pkg01.atx.netgate.com

[..]

                                      thanks ...

                                      1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        If you re-installed the repo pkg and it repopulated the symlink then it looks like it might not have the correct cert for the repo.

                                        Try running pkg -d update to see the full error.

                                        Check what cert it's using:

                                        cat /usr/local/etc/pkg.conf
                                        S 1 Reply Last reply Reply Quote 1
                                        • stephenw10S
                                          stephenw10 Netgate Administrator
                                          last edited by

                                          Oh, yeah if you added the symlink manually it probably also needs the pkg.conf updating manually. In 23.01 when set to 'Prev' it should look like:

                                          ABI=FreeBSD:14:amd64
ALTABI=freebsd:14:x86:64
PKG_ENV {
	SSL_CA_CERT_FILE=/etc/ssl/netgate-ca.pem
	SSL_CLIENT_CERT_FILE=/usr/local/etc/pfSense/pkg/repos/pfSense-repo-prev-cert.pem
	SSL_CLIENT_KEY_FILE=/usr/local/etc/pfSense/pkg/repos/pfSense-repo-prev-key.pem
}
                                          S 1 Reply Last reply Reply Quote 0
                                          • S
                                            sgw @stephenw10
                                            last edited by

                                            @stephenw10

                                            Compared /usr/local/etc/pkg.conf with another 23.01 box.

                                            The whole block "PKG_ENV" was missing, added it.

                                            Now:

                                            pkg -d update
DBG(1)[32839]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[32839]> PkgRepo: verifying update for pfSense-core
DBG(1)[32839]> PkgRepo: need forced update of pfSense-core
DBG(1)[32839]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[32839]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_01_aarch64-core/meta.conf
DBG(1)[32839]> opening libfetch fetcher
DBG(1)[32839]> Fetch > libfetch: connecting
DBG(1)[32839]> Fetch: fetching from: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_aarch64-core/meta.conf with opts "i"
DBG(1)[32839]> Fetch: fetcher chosen: https
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
DBG(1)[32839]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_01_aarch64-core/packagesite.pkg
DBG(1)[32839]> opening libfetch fetcher
DBG(1)[32839]> Fetch > libfetch: connecting
DBG(1)[32839]> Fetch: fetching from: https://pfsense-plus-pkg01.atx.netgate.com/pfSense_plus-v23_01_aarch64-core/packagesite.pkg with opts "i"
DBG(1)[32839]> Fetch: fetcher chosen: https
Fetching packagesite.pkg: 100%    2 KiB   1.8kB/s    00:01    
DBG(1)[32839]> PkgRepo: extracting packagesite.yaml of repo pfSense-core
DBG(1)[32947]> PkgRepo: extracting signature of repo in a sandbox
DBG(1)[32839]> Pkgrepo, reading new packagesite.yaml for '/var/db/pkg/repo-pfSense-core.sqlite'
Processing entries: 100%
pfSense-core repository update completed. 7 packages processed.
Updating pfSense repository catalogue...
DBG(1)[32839]> PkgRepo: verifying update for pfSense
DBG(1)[32839]> PkgRepo: need forced update of pfSense
DBG(1)[32839]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[32839]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/meta.conf
DBG(1)[32839]> opening libfetch fetcher
DBG(1)[32839]> Fetch > libfetch: connecting
DBG(1)[32839]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/meta.conf with opts "i"
DBG(1)[32839]> Fetch: fetcher chosen: https
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01    
DBG(1)[32839]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/packagesite.pkg
DBG(1)[32839]> opening libfetch fetcher
DBG(1)[32839]> Fetch > libfetch: connecting
DBG(1)[32839]> Fetch: fetching from: https://pfsense-plus-pkg00.atx.netgate.com/pfSense_plus-v23_01_aarch64-pfSense_plus_v23_01/packagesite.pkg with opts "i"
DBG(1)[32839]> Fetch: fetcher chosen: https
Fetching packagesite.pkg: 100%  143 KiB 146.3kB/s    00:01    
DBG(1)[32839]> PkgRepo: extracting packagesite.yaml of repo pfSense
DBG(1)[33279]> PkgRepo: extracting signature of repo in a sandbox
DBG(1)[32839]> Pkgrepo, reading new packagesite.yaml for '/var/db/pkg/repo-pfSense.sqlite'
Processing entries: 100%
pfSense repository update completed. 503 packages processed.
All repositories are up to date.

                                            Looks better, thanks ;-)

                                            How to proceed now? Switch to 23.05 in the GUI?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post