Unable to upgrate 23.09.1 to 24.11

nanda

"The currently recommended process for upgrades is to backup the pfSense Plus software configuration from the existing instance and restore it on a new instance when an upgrade is available."

Indeed, we deployed pfSense in Microsoft Azure through the official Azure marketplace. The above statement on replacing the old instance with the new instance may lead to problems with the operational costs. pfSense firewall instance was deployed through Azure reservation. If we forgo the current instance, it will, perhaps, take away the reservation.

Please provide a technical solution to solve the upgrade issue.

stephenw10

Hmm, OK try running: pfSense-repoc

Make sure that returns cleanly. Then try saving the branch again.

You should be able to upgrade in Azure.

nanda

@stephenw10

I executed the command pfSense-repoc and it did not return any message. So I assume that it was a clean execution (see Figure 1), then saved the branch again (Figure 2). The result was the same (see Figure 3).

Figure 1: pfSense-repoc execution

Figure 2: Save the branch again

Figure 3: Trying after the above steps

Verified again via pkg-static -d update command, but the result still points to 23.09.1 repo.

$ pkg-static -d update
DBG(1)[18723]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[18723]> PkgRepo: verifying update for pfSense-core
DBG(1)[18723]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[18723]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-core/meta.conf
DBG(1)[18723]> curl_open
DBG(1)[18723]> Fetch: fetcher used: pkg+https
DBG(1)[18723]> curl> fetching https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-core/meta.conf

DBG(1)[18723]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pfsense-plus-pkg01.atx.netgate.com in the .netrc file; using defaults
*   Trying 208.123.73.209:443...
* Connected to pfsense-plus-pkg01.atx.netgate.com (208.123.73.209) port 443
* ALPN: curl offers http/1.1
*  CAfile: /etc/ssl/netgate-ca.pem
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: C=US; ST=Texas; L=Austin; O=Rubicon Communications, LLC (Netgate); OU=pfSense Plus; CN=pfsense-plus-pkg01.atx.netgate.com
*  start date: Mar 15 20:23:37 2022 GMT
*  expire date: Feb 19 20:23:37 2122 GMT
*  common name: pfsense-plus-pkg01.atx.netgate.com (matched)
*  issuer: C=US; ST=Texas; L=Austin; O=Rubicon Communications, LLC (Netgate); OU=Netgate CA; CN=Netgate CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_plus-v23_09_1_amd64-core/meta.conf HTTP/1.1
Host: pfsense-plus-pkg01.atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Wed, 06 Dec 2023 23:08:55 GMT

< HTTP/1.1 200 OK
Fetching meta.conf: < Server: nginx
< Date: Mon, 17 Mar 2025 18:50:51 GMT
< Content-Type: application/octet-stream
< Content-Length: 163
< Last-Modified: Wed, 06 Dec 2023 23:08:54 GMT
< Connection: keep-alive
< ETag: "6570ff06-a3"
< Accept-Ranges: bytes
<
* The requested document is not new enough
* Simulate an HTTP 304 response
* Closing connection

DBG(1)[18723]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-core/packagesite.pkg
DBG(1)[18723]> curl_open
DBG(1)[18723]> Fetch: fetcher used: pkg+https
DBG(1)[18723]> curl> fetching https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-core/packagesite.pkg

DBG(1)[18723]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pfsense-plus-pkg01.atx.netgate.com in the .netrc file; using defaults
* Hostname pfsense-plus-pkg01.atx.netgate.com was found in DNS cache
*   Trying 208.123.73.209:443...
* Connected to pfsense-plus-pkg01.atx.netgate.com (208.123.73.209) port 443
* ALPN: curl offers http/1.1
*  CAfile: /etc/ssl/netgate-ca.pem
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: C=US; ST=Texas; L=Austin; O=Rubicon Communications, LLC (Netgate); OU=pfSense Plus; CN=pfsense-plus-pkg01.atx.netgate.com
*  start date: Mar 15 20:23:37 2022 GMT
*  expire date: Feb 19 20:23:37 2122 GMT
*  common name: pfsense-plus-pkg01.atx.netgate.com (matched)
*  issuer: C=US; ST=Texas; L=Austin; O=Rubicon Communications, LLC (Netgate); OU=Netgate CA; CN=Netgate CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_plus-v23_09_1_amd64-core/packagesite.pkg HTTP/1.1
Host: pfsense-plus-pkg01.atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Wed, 06 Dec 2023 23:08:55 GMT

< HTTP/1.1 304 Not Modified
< Server: nginx
< Date: Mon, 17 Mar 2025 18:50:51 GMT
< Last-Modified: Wed, 06 Dec 2023 23:08:55 GMT
< Connection: keep-alive
< ETag: "6570ff07-628"
<
* Connection #1 to host pfsense-plus-pkg01.atx.netgate.com left intact
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[18723]> PkgRepo: verifying update for pfSense
DBG(1)[18723]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[18723]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1/meta.conf
DBG(1)[18723]> curl_open
DBG(1)[18723]> Fetch: fetcher used: pkg+https
DBG(1)[18723]> curl> fetching https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1/meta.conf

DBG(1)[18723]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pfsense-plus-pkg01.atx.netgate.com in the .netrc file; using defaults
*   Trying 208.123.73.209:443...
* Connected to pfsense-plus-pkg01.atx.netgate.com (208.123.73.209) port 443
* ALPN: curl offers http/1.1
*  CAfile: /etc/ssl/netgate-ca.pem
*  CApath: /etc/ssl/certs/
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: C=US; ST=Texas; L=Austin; O=Rubicon Communications, LLC (Netgate); OU=pfSense Plus; CN=pfsense-plus-pkg01.atx.netgate.com
*  start date: Mar 15 20:23:37 2022 GMT
*  expire date: Feb 19 20:23:37 2122 GMT
*  common name: pfsense-plus-pkg01.atx.netgate.com (matched)
*  issuer: C=US; ST=Texas; L=Austin; O=Rubicon Communications, LLC (Netgate); OU=Netgate CA; CN=Netgate CA
*  SSL certificate verify ok.
* using HTTP/1.1
> GET /pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1/meta.conf HTTP/1.1
Host: pfsense-plus-pkg01.atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Fri, 10 Jan 2025 15:23:45 GMT

< HTTP/1.1 304 Not Modified
< Server: nginx
< Date: Mon, 17 Mar 2025 18:50:52 GMT
< Last-Modified: Fri, 10 Jan 2025 15:23:45 GMT
< Connection: keep-alive
< ETag: "67813b81-b2"
<
* Connection #0 to host pfsense-plus-pkg01.atx.netgate.com left intact
DBG(1)[18723]> Request to fetch pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1/packagesite.pkg
DBG(1)[18723]> curl_open
DBG(1)[18723]> Fetch: fetcher used: pkg+https
DBG(1)[18723]> curl> fetching https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1/packagesite.pkg

DBG(1)[18723]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pfsense-plus-pkg01.atx.netgate.com in the .netrc file; using defaults
* Found bundle for host: 0x398516efec20 [serially]
* Re-using existing connection with host pfsense-plus-pkg01.atx.netgate.com
> GET /pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1/packagesite.pkg HTTP/1.1
Host: pfsense-plus-pkg01.atx.netgate.com
User-Agent: pkg/1.20.8
Accept: */*
If-Modified-Since: Fri, 10 Jan 2025 15:23:45 GMT

< HTTP/1.1 304 Not Modified
< Server: nginx
< Date: Mon, 17 Mar 2025 18:50:52 GMT
< Last-Modified: Fri, 10 Jan 2025 15:23:45 GMT
< Connection: keep-alive
< ETag: "67813b81-30264"
<
* Connection #0 to host pfsense-plus-pkg01.atx.netgate.com left intact
pfSense repository is up to date.
All repositories are up to date.

Tried rebooting and then the system update, but the webpage still returning "Up to date" message.

stephenw10

Hmm, odd. Try running: pfSense-repoc -DJ

That should show you what info is being sent and which branches are being pushed back. Obviously don't post that output here.

nanda

@stephenw10

In the post data repo section, I can see three repos: 24.11, 24.03, 23.09.1. The 23.09.1 has the default field set to yes; other repo sections do not have this field.

Would you tell me, how to deduce/solve the problem with the command response?

stephenw10

Ok try running: ls -ls /usr/local/etc/pkg/repos

That should show a symlink for pfSense.conf that points to the selected repo like:

24.03-RELEASE][admin@plusdev-4.stevew.lan]/root: ls -ls /usr/local/etc/pkg/repos
total 1
1 -rw-r--r--  1 root wheel 25 Apr 19  2024 FreeBSD.conf
1 lrwxr-xr-x  1 root wheel 55 Mar 18 12:49 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0003.conf

When you change the update branch in the webgui that symlink should be updated.

Then if you check that repo it should be 24.11 like:

[24.03-RELEASE][admin@plusdev-4.stevew.lan]/root: cat /usr/local/etc/pfSense/pkg/repos/pfSense-repo-0003.conf
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

I'm guessing your install is not showing that for some reason.

nanda

@stephenw10

Your guess is true, symlink did not change to the selected repo when the update branch changed in the GUI. It stays with 23.09.1.

$ ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 62 Mar 18 16:45 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-23_09_1_rel.conf

pfSense-repo-23_09_1_rel.conf has urls for 23.09.1 in pfsense-core and pfsense section.

"pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-core"
"pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1"

Is there any command that can be executed via the terminal to change the repo?

stephenw10

Yes you can try manually creating the symlink. What conf files do you have in /usr/local/etc/pfSense/pkg/repos ?

nanda

@stephenw10

The repos directory has three config files for 23.09.1, 24.03 and 24.11

-rw-r--r--  1 root wheel 512 Mar 18 16:45 /usr/local/etc/pfSense/pkg/repos/pfSense-repo-23_09_1_rel.conf
-rw-r--r--  1 root wheel 506 Mar 18 16:45 /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_03-rel.conf
-rw-r--r--  1 root wheel 506 Mar 18 16:45 /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf

stephenw10

OK first try running: pfSense-repo-setup then re-check.

nanda

@stephenw10

pfSense-repo-setup command did not return any message, so it should be a clean execution.

[23.09.1-RELEASE][...]/root: pfSense-repo-setup
[23.09.1-RELEASE][...]/root: ls -l /usr/local/etc/pfSense/pkg/repos/*.conf
-rw-r--r--  1 root wheel 512 Mar 18 19:22 /usr/local/etc/pfSense/pkg/repos/pfSense-repo-23_09_1_rel.conf
-rw-r--r--  1 root wheel 506 Mar 18 19:22 /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_03-rel.conf
-rw-r--r--  1 root wheel 506 Mar 18 19:22 /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf

After saving the 24.11 update branch in the system update via GUI, symlink still stays with 23.09.1

[23.09.1-RELEASE][...]/root: ls -ls/usr/local/etc/pfkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 62 Mar 18 19:24 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-23_09_1_rel.conf
[23.09.1-RELEASE][...]/root:

stephenw10

Hmm, weird. Is it not updating the line in the config perhaps?

Anyway you should be able to run:
rm /usr/local/etc/pkg/repos/pfSense.conf

Then:
ln -s /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf /usr/local/etc/pkg/repos/pfSense.conf

That should allow it to see the 24.11 upgrade.

nanda

@stephenw10

ln -s ... command created the symlink pointing 24.11 conf and it is verified too.

[23.09.1-RELEASE][...]/root: ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 56 Mar 18 20:06 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf

When I tried to update via Sys => Update => System Update, the page returned 'Up to date' status with 23.09.1 as the base and latest versions. Symlink verification for pfSense.conf shows that the config reverted to 23.09.1. It seems the problem lies with the GUI.

[23.09.1-RELEASE][...]/root: ls -ls/usr/local/etc/pfkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 62 Mar 18 20:14 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-23_09_1_rel.conf

Should I retry the update via option 13 after recreating the symlink?
Will it cause problems?

stephenw10

Try running pfSense-upgrade directly after creating the symlink.

nanda

@stephenw10

Sorry for my belated response. The firewall cannot be taken offline immediately.

I executed the pfSense-upgrade command, but it did not upgrade the firewall. I verified the pfSense.conf symlink before and after command execution. It looks like the pfSense.conf again reverted to 23.09.1.

[23.09.1-RELEASE][...]/root: ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 56 Mar 19 21:03 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf
[23.09.1-RELEASE][...]/root: pfSense-upgrade
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ....... done
Processing entries: .......... done
pfSense repository update completed. 739 packages processed.
All repositories are up to date.
Your packages are up to date
[23.09.1-RELEASE][...]/root: ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 62 Mar 19 21:03 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-23_09_1_rel.conf

stephenw10

OK check the config file. What branch is it saving there?

Or change the branch then check the config history. You should see it switch from 23.09.1 to 24.11.

nanda

@stephenw10

Update branch change to 24.11 after symlink creation.

[23.09.1-RELEASE][...]/root: ls -ls /usr/local/etc/pkg/repos
total 4
4 -rw-r--r--  1 root wheel 26 Dec  6  2023 FreeBSD.conf
0 lrwxr-xr-x  1 root wheel 56 Mar 19 23:01 pfSense.conf -> /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf
[23.09.1-RELEASE]...]/root: cat /usr/local/etc/pkg/repos/pfSense.conf
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

Updated branch rollback to 23.09.1 after pfSense-upgrade execution

[23.09.1-RELEASE][...]/root: pfSense-upgrade
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: . done
Processing entries: . done
pfSense-core repository update completed. 5 packages processed.
Updating pfSense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ..... done
Processing entries: .......... done
pfSense repository update completed. 739 packages processed.
All repositories are up to date.
Your packages are up to date
[23.09.1-RELEASE][...]/root: cat /usr/local/etc/pkg/repos/pfSense.conf
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v23_09_1_amd64-pfSense_plus_v23_09_1",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

[23.09.1-RELEASE][...]/root:

stephenw10

But what does it show in the main pfSense config.xml file? That's where pfSense-upgrade it triggering it be pulled from.

I can't imagine why it would not be getting updated but....

nanda

@stephenw10

I assume the main pfSense config.xml you are referring to is the file in /cf/conf/config.xml. <system/> section is the point of interest, I believe, as it contains information related to host, users, groups and webui.

	<system>
		<optimization>normal</optimization>
		...
		<already_run_config_upgrade></already_run_config_upgrade>
		<pkg_repo_conf_path>/usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf</pkg_repo_conf_path>
		<firmware>
			<disableautobe></disableautobe>
		</firmware>
		<gitsync>
			<repositoryurl></repositoryurl>
			<branch></branch>
		</gitsync>
		...
	</system>

The <pkg_repo_conf_path> directs to the 24.11 configuration file and the file has links to the 24.11 repo.

[23.09.1-RELEASE][...]/root: cat /usr/local/etc/pfSense/pkg/repos/pfSense-repo-24_11.conf
FreeBSD: { enabled: no }

pfSense-core: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-core",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

pfSense: {
    url: "pkg+https://pfsense-plus-pkg.netgate.com/pfSense_plus-v24_11_amd64-pfSense_plus_v24_11",
    mirror_type: "srv",
    signature_type: "fingerprints",
    fingerprints: "/usr/local/share/pfSense/keys/pkg",
    enabled: yes
}

[23.09.1-RELEASE][...]/root:

Should <firmware/>and <gitsync/> sections have values pointing to 24.11?
Or am I looking at the wrong place?

stephenw10

No that's all correct. Weird.

What happens if you try upgrading to 24.03 first?