upgrade from 21.05 to latest on sg-3100

corrupteddb

Hello there!

Lost my forum login so had to make a new account, but i am no stranger to pfsense. We have 13 of these Sg-3100 devices. I have tried this week to update them. about 9 updated without issues. 4 i have had problems with, but was able to recover 3 of them with a combination of researched commands. So thanks previous people for those.

The last one, i believe has some kind of disk corruption. The files in /usr/local/share/pfsense/pkg/repos/ are all 0 bytes as you can see. The SQL lite database complains of being corrupt as well. Do i have any options other than reinstall? This is at a remote site so not impossible to service but i would prefer not to. I tried copying back the .conf file from the temp file in this directory but it still fails. gets a bit farther than before though.

2026-03-18 10_15_12-.png

Shell Output - pfSense-upgrade -d -c
>>> Updating repositories metadata...
pkg-static: Unable to determine ABI
pkg-static: Cannot parse configuration file!
ERROR: Unable to compare version of pfSense-repo

DBG(1)[26589]> pkg initialized
pkg-static: Invalid ABI
pkg-static: Warning: Major OS version upgrade detected.  Running "pkg bootstrap -f" recommended
Updating pfSense-core repository catalogue...
DBG(1)[26589]> PkgRepo: verifying update for pfSense-core
DBG(1)[26589]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[26589]> Request to fetch pkg+https://firmware.netgate.com/pkg/pfSense_plus-v22_05_armv7-core/meta.conf
DBG(1)[26589]> opening libfetch fetcher
DBG(1)[26589]> Fetch > libfetch: connecting
DBG(1)[26589]> Fetch: fetching from: https://pkg01-atx.netgate.com/pkg/pfSense_plus-v22_05_armv7-core/meta.conf with opts "i"
DBG(1)[26589]> Fetch: fetcher chosen: https
DBG(1)[26589]> Request to fetch pkg+https://firmware.netgate.com/pkg/pfSense_plus-v22_05_armv7-core/packagesite.pkg
DBG(1)[26589]> opening libfetch fetcher
DBG(1)[26589]> Fetch > libfetch: connecting
DBG(1)[26589]> Fetch: fetching from: https://pkg01-atx.netgate.com/pkg/pfSense_plus-v22_05_armv7-core/packagesite.pkg with opts "i"
DBG(1)[26589]> Fetch: fetcher chosen: https
DBG(1)[26589]> Request to fetch pkg+https://firmware.netgate.com/pkg/pfSense_plus-v22_05_armv7-core/packagesite.txz
DBG(1)[26589]> opening libfetch fetcher
DBG(1)[26589]> Fetch > libfetch: connecting
DBG(1)[26589]> Fetch: fetching from: https://pkg01-atx.netgate.com/pkg/pfSense_plus-v22_05_armv7-core/packagesite.txz with opts "i"
DBG(1)[26589]> Fetch: fetcher chosen: https
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[26589]> PkgRepo: verifying update for pfSense
pkg-static: sqlite error while executing iterator in file pkgdb_iterator.c:1086: database disk image is malformed
DBG(1)[26589]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[26589]> Request to fetch pkg+https://firmware.netgate.com/pkg/pfSense_plus-v22_05_armv7-pfSense_plus_v22_05/meta.conf
DBG(1)[26589]> opening libfetch fetcher
DBG(1)[26589]> Fetch > libfetch: connecting
DBG(1)[26589]> Fetch: fetching from: https://pkg01-atx.netgate.com/pkg/pfSense_plus-v22_05_armv7-pfSense_plus_v22_05/meta.conf with opts "i"
DBG(1)[26589]> Fetch: fetcher chosen: https
Fetching meta.conf: . done
DBG(1)[26589]> Request to fetch pkg+https://firmware.netgate.com/pkg/pfSense_plus-v22_05_armv7-pfSense_plus_v22_05/packagesite.pkg
DBG(1)[26589]> opening libfetch fetcher
DBG(1)[26589]> Fetch > libfetch: connecting
DBG(1)[26589]> Fetch: fetching from: https://pkg01-atx.netgate.com/pkg/pfSense_plus-v22_05_armv7-pfSense_plus_v22_05/packagesite.pkg with opts "i"
DBG(1)[26589]> Fetch: fetcher chosen: https
Fetching packagesite.pkg: .......... done
DBG(1)[26589]> PkgRepo: extracting packagesite.yaml of repo pfSense
DBG(1)[26902]> PkgRepo: extracting signature of repo in a sandbox
DBG(1)[26589]> Pkgrepo, reading new packagesite.yaml for '/var/db/pkg/repo-pfSense.sqlite'
Processing entries:
pkg-static: wrong architecture: FreeBSD:12:armv7 instead of ALTABI=
pkg-static: repository pfSense contains packages with wrong ABI: FreeBSD:12:armv7
Processing entries... done
Unable to update repository pfSense
Error updating repositories!

I have ran the following commands to no effect (however they did fix other firewalls, just not this one)

pkg-static clean -ay; pkg-static install -fy pkg pfSense-repo pfSense-upgrade
sed -i "" 's/url: "/url: "pkg+https:\/\/firmware.netgate.com\/pkg/g' /usr/local/share/pfSense/pkg/repos/pfSense-repo.conf
pkg-static -d update
env ABI=FreeBSD:12:armv7 pkg bootstrap
env ASSUME_ALWAYS_YES=yes pkg-static bootstrap -f

If i do have to reload, do i need a system image downloaded to usb or something? or can i just go on the console of the device and somehow factory reset it? please let me know! thanks

The firewall works fine for routing traffic. just the update list is blank.

SteveITS

@corrupteddb it’s not out of disk space is it?

corrupteddb

@SteveITS

negative.

i think it was likely powercycled hard by staff too many times, knowing the history of the site :(

i also tried to run fsck on the disk but it either didnt work, or the touch command didnt work that i had used. if someone knows how to chkdsk the disk and possibly heal the corruption that would be good too.

SteveITS

@corrupteddb There are a few ways here: https://docs.netgate.com/pfsense/en/latest/troubleshooting/filesystem-check.html

FWIW if people power it off a lot, a RAM disk will probably help with corruption, for next time.

A "factory reset" on pfSense can be done via the GUI or console menu but it just overwrites the config file. (I think it uninstalls packages too?)

stephenw10

Coming from a version that old I would probably reinstall anyway. Yes you need to get a recovery image and write it to USB to do that. Open a TAC ticket to request that.

https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/reinstall-pfsense.html

corrupteddb

@stephenw10

ok so i first tried this on my own. I had a file from years ago which was 2.4.5. i am able to usb boot the device, and recover to this image! awesome.

I can lease IP addresses, i can get out to the internet.

I then try and use the console to upgrade the unit. It tells me it wants to go to version 21 i think it is, however during the upgrade process it starts complaining about missing files and other errors. Ok, sometimes BSD /linux spits out errors that are not real errors so i proceed.

However after the upgrade, i cannot lease an ip address. i can ping out from the console, but it appears dhcp doesnt work. i also cannot manually ping the gateway. I havent even tried loading the config xml onto it yet...

So they i try and upgrade from 21 to latest, and it tells me "no space left on device".... So i have now reached out to TAC team to get an updated firmware. But i feel like maybe the mmc disk inside it is bad or something... i wonder if its socketed or soldered on. Could of course be something else, so i will post back when i am able to talk with TAC

UPDATE. TAC got right back to me with the latest firmware. I did the reinstall with the latest firmware and the appliance now works. I dare not try upgrading it again! still think the disk is bad, but at least the device works now and is upgraded. firmware file was pfSense-plus-Netgate-3100-recovery-25.11.1-RELEASE-armv7.img.gz

stephenw10

Nice. Upgrades on the 3100 should be fine. I've been testing 25.11.1 to 26.03 here and haven't seen any issues in RC.

Mission-Ghost

@corrupteddb the evidence suggests that eMMCs go bad, and they go bad faster if one is using ZFS or logging pretty much anything.

It's not if, it's when. In my opinion, these kinds of devices should never run on eMMCs; it's the wrong storage device for the job.

Given your machine is working now, you have a reprieve to move forward and resolve this issue before it just dies on you. Which it will. It won't get better by itself.

SteveITS

@corrupteddb If the upgrade worked then the drive must be writeable, at least.

I don't recall if https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html#emmc works on a 3100 but you could try.

In related news, 2.5 years ago Netgate said they could provide no more updates for the 3100 because FreeBSD dropped support for 32 bit ARM. Netgate has continued to provide updates to pfSense since then, with a few packages dropping off along the way. But longer term it will presumably stop updating at some point.

https://docs.netgate.com/pfsense/en/latest/releases/24-03.html#netgate-3100-32-bit-arm-limitations

stephenw10

Yup, it looked like it 26.03 wouldn't be possible at one point but someone upstream resurrected it.

At some point it will become unbuildable for us.