SG1100 Appears bricked after webUI upgrade

ansel

New image acquired and reinstall intiated through support process at
Reinstall Link

Backup of exsisting configuration over console
cat /cf/conf/config.xml -> Paste to new file (config.xml)

Write image for fresh install to new USB drive.

I suspect everything will be good. Thanks for the support @netgate

Edit: I forgot to mention, I could only access this device from another netgate FW I had on hand. Using TIP. I added the following to /etc/remote

pf:dv=/dev/cuaU0:br#115200

then used

tip pf

to access the console over usb on the bricked FW.

ansel

Some type of mount error occured. Will retry the process.

(da0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
(da0:umass-sim0:0:0:0): Error 5, Retries exhausted
g_vfs_done(): ufsid/646b91cb17a620c2 converting all errors to ENXIO
g_vfs_done():ufsid/646b91cb17a620c2[WRITE(offset=2033745920, length=20480)]error = 6 supressing further ENXIO
UFS: forcibly unmounting /dev/ufsid/646b91cb17a620c2 from /
failed to read stdin: Device not configured
gzcat: /usr/local/share/pfSense/pfSense-plus-23.05-RELEASE-aarch64.img.gz: uncompress failed
pid 1072 (sh), jid 0, uid 0: exited on signal 4
pid 1069 (sh), jid 0, uid 0: exited on signal 4
pid 1068 (login), jid 0, uid 0: exited on signal 4
pid 1051 (cron), jid 0, uid 0: exited on signal 4
(da0:umass-sim0:0:0:0): got CAM status 0x444
(da0:umass-sim0:0:0:0): fatal error, failed to attach to device
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <SMI USB DISK 1100>  s/n AA00000000000489 detached
(da0:umass-sim0:0:0:0): Periph destroyed

jimp

Those errors are suggestive of a USB thumb drive failing to read.

I would try a different USB drive before anything else.

SteveITS

@ansel If a second USB stick doesn't work there is a note on step 6 of https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html:
"If the device does not boot after issuing this command, enter usb reset and then try run usbrecovery again. Some USB drives require an extra reset to fully initialize."

I believe one recent poster said it took several tries.

jimp

If it were that, it wouldn't have made it so far into the process. It would have failed in uboot. The errors here look like it was partway through the recovery process when it failed to read the USB drive.

ansel

Second retry (after removal of power) the install from USB worked as expected.

ansel

Reload of 23.05 seemed to work. Placed the device back in service for it to not allow any reload of the firewall rules. The device is complaining about a invalid rule for

rdr on $WAN proto ipv6 from any to any ->

I proceeded to remove all instantiations of IPv6 interfaces, rules, routes and provided a reboot. The device will not load any rules due to this error. I even removed the check box for "Allow IPv6" in Advanced/Networking. Which I assumed would prevent any IPv6 rule from being generated.

What is more annoying is that trying to downgrade to 23.01 seems to not be allowed via the UI.

Edit:
Removing the functionality of IPv6 over IPv4 tunneling seems to remove the miss generated rule. I can reproduce a incorrect/incomplete IPv6 rule as defined above by enabling v6 over v4 tunneling.

This debug session brought to you by deleting all rules on my FW and starting from scratch to find out IPv6 over IPv4 tunneling in 23.05 appears to generate bad rules that will not allow the FW set to be generated.

SteveITS

@ansel Glad you got it working, hopefully Jim can duplicate if it's a bug.

re: downgrade, there is no native downgrade as it would usually mean downgrading the OS. If you had upgraded successfully, you could boot to the old boot environment, delete the new, and downgrade that way. Or you can ask support for a 23.01 image and reinstall with that. I don't know how far back they'll go but I assume at least recent ones. Also, not a bad idea to ask for one before upgrading.

@jimp yeah I was drive by posting my bad

jimp

That's a fun little buglet there for the tunneling thing. I was able to reproduce that here and pushed a fix. A variable name was missing a single letter.

https://redmine.pfsense.org/issues/14415

You can install the System Patches package and then create an entry for ea79a4fe5707898fff89e80d7252e5c84fca7dd4 to apply the fix.

Or use this diff:

diff --git a/src/etc/inc/filter.inc b/src/etc/inc/filter.inc
index 7063798529..255af0903c 100644
--- a/src/etc/inc/filter.inc
+++ b/src/etc/inc/filter.inc
@@ -2490,7 +2490,7 @@ function filter_nat_rules_generate() {
            (is_ipaddr($ipv6nataddr)) &&
            (is_array($FilterIflist['wan']))) {
                /* XXX: FIX ME!  IPV6 */
-               $natrules .= "rdr on \${$FilterIflist['wan']['descr']} proto ipv6 from any to any -> {$pv6nataddr}\n";
+               $natrules .= "rdr on \${$FilterIflist['wan']['descr']} proto ipv6 from any to any -> {$ipv6nataddr}\n";
        }

        unlink_if_exists("{$g['varetc_path']}/xinetd.conf");

ansel

Applied, tested and working. Thank you

SteveITS

/* XXX: FIX ME! IPV6 */

somebody missed that one ...would have been easier to type the "i" ;)