504 Gateway error after update

jvamos

@stephenw10
Hi Stephen,
Yeah the update did not go smoothly.
Maybe it's something I have misconfigured or maybe it's a bug.

pfSense-22.05
pfSense-Status_Monitoring-1.7.11_4
pfSense-base-22.05
pfSense-default-config-serial-22.05
pfSense-kernel-pfSense-22.05
pfSense-pkg-Avahi-2.2_1
pfSense-pkg-Status_Traffic_Totals-2.3.2_2
pfSense-pkg-acme-0.7.1_1
pfSense-pkg-bandwidthd-0.7.4_5
pfSense-pkg-darkstat-3.1.3_5
pfSense-pkg-iperf-3.0.2_5
pfSense-pkg-nmap-1.4.4_6
pfSense-pkg-pfBlockerNG-devel-3.1.0_4
pfSense-rc-22.05
pfSense-repo-22.05_2
pfSense-u-boot-1100-20220428
pfSense-u-boot-2100-20210930_1
pfSense-u-boot-env-20220429
pfSense-upgrade-1.0_26
php74-pfSense-module-0.81

jvamos

@jvamos
none of these package show updates available
when trying to access https://192.168.1.1/firewall_rules.php there is a 502 bad gateway error.

stephenw10

Hmm, do you have any 'unusual' rules?

jvamos

@stephenw10
Maybe some poorly configured outdated vlans?

stephenw10

No it would need to be some combinations of odd settings making it unique. Like maybe a gateway, schedule, IP options and one TCP flag or something equally weird.

Try running at the command line: pfctl -vvf /tmp/rules.debug

If it's a rules issue directly that will fail but should show what it's choking on.

Steve

stephenw10

You should also check the running kernel with uname -a in case it's booted the 22.01 kernel somehow. For example:

[22.05-RELEASE][admin@2100-2.stevew.lan]/root: uname -a
FreeBSD 2100-2.stevew.lan 12.3-STABLE FreeBSD 12.3-STABLE plus-RELENG_22_05-n202700-3ddaea61055 pfSense  arm64

Steve

jvamos

@stephenw10

Loaded 762 passive OS fingerprints
pfctl: pfctl_rules
pfctl: DIOCXROLLBACK: Invalid argument

I think you were right. The upgrade only half finished.

uname -a
FreeBSD pfsense.johnst 12.3-STABLE FreeBSD 12.3-STABLE plus-RELENG_22_01-n202637-97cc5c23e13 pfSense  arm64

jvamos

@stephenw10

pfSense-upgrade -d -c
>>> Updating repositories metadata...
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Your system is up to date

stephenw10

Another possibility here is that the 22.01 kernel has loaded from another boot device.

Do you have an SSD in that 2100? Is it booting ZFS from the SSD?

jvamos

@stephenw10
No it’s just stock. I think it’s just emmc.
Is there a good way to check?

Josh

stephenw10

You can run zpool status to see what it booted from.

You can run geom disk list to see the drives in the system.

jvamos

@stephenw10

zpool status
KLD zfs.ko: depends on kernel - not available or version mismatch
internlize ZFS library

This is the output of the first command

Geom name: flash/spi0
Providers:
1. Name: flash/spi0
   Mediasize: 4194304 (4.0M)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: w25q32
   ident: (null)
   rotationrate: unknown
   fwsectors: 0
   fwheads: 0

Geom name: mmcsd0
Providers:
1. Name: mmcsd0
   Mediasize: 7820083200 (7.3G)
   Sectorsize: 512
   Stripesize: 512
   Stripeoffset: 0
   Mode: r1w1e3
   descr: MMCHC DG4008 0.1 SN A603B710 MFG 12/2019 by 69 0x0000
   ident: A603B710
   rotationrate: 0
   fwsectors: 0
   fwheads: 0

Geom name: mmcsd0boot0
Providers:
1. Name: mmcsd0boot0
   Mediasize: 4194304 (4.0M)
   Sectorsize: 512
   Stripesize: 512
   Stripeoffset: 0
   Mode: r0w0e0
   descr: MMCHC DG4008 0.1 SN A603B710 MFG 12/2019 by 69 0x0000
   ident: A603B710
   rotationrate: 0
   fwsectors: 0
   fwheads: 0

Geom name: mmcsd0boot1
Providers:
1. Name: mmcsd0boot1
   Mediasize: 4194304 (4.0M)
   Sectorsize: 512
   Stripesize: 512
   Stripeoffset: 0
   Mode: r0w0e0
   descr: MMCHC DG4008 0.1 SN A603B710 MFG 12/2019 by 69 0x0000
   ident: A603B710
   rotationrate: 0
   fwsectors: 0
   fwheads: 0

It's just the stock option.

stephenw10

Hmm, running ufs then and only one drive. Not loading a kernel from the wrong device then.

You can always reinstall 22.05 clean and restore the config. That is usually the quickest way to recover. Open a ticket with us to hey the firmware image if you need it:
https://www.netgate.com/tac-support-request

Steve

jvamos

@stephenw10 thanks Stephen

jvamos

@stephenw10
Form submitted.

jvamos

@jvamos

Now that things are working again all I can think of as the problematic settings are pfblockerNG-devel with a lengthy set of lists or the Firewall Log widget GUI being set to 500 entries while default rule logs was not turned off.

stephenw10

Hmm, that shouldn't be an issue. Glad you're back and running though.