Upgrade from 2.7.2 to 2.8.0 Failed and now /boot/efi/ empty

Sinfonia97

Yet Another UPDATE:
I think I may have made some progress. It appears that on my system instead of /dev/msdosfs/EFISYS I have /dev/msdosfs/EFI.

I was able to follow @jimp instructions partially from another post and was able to mount /boot/efi to /dev/msdosfs/EFI and copy files to a temp directory /tmp/efitmp.

Here are the modified commands I performed. They appear to have worked and I can browse the directories and files now in the /tmp/efitmp directory.

# mkdir -p /boot/efi
# mount_msdosfs /dev/msdosfs/EFI /boot/efi
# mkdir -p /tmp/efitmp
# cp -Rp /boot/efi/* /tmp/efitmp

What I am trying to figure out is if this is correct for my setup before I proceed and create more problems doing the following:

# umount /boot/efi
# newfs_msdos -F 32 -c 1 -L EFI /dev/msdosfs/EFI
# mount_msdosfs /dev/msdosfs/EFI /boot/efi
# cp -Rp /tmp/efitmp/* /boot/efi/

Again, thanks in advance for the help.

EDIT: I took a look at the contents of the file in the /tmp/efitmp/EFI/BOOT called STARTUP.NSH and it has one entry BOOTx64.efi. When I try to search for this file I do not find it on the system.

EDIT 2: Per another forum post that I found with @jimp involved I ran find /boot/efi and got the following reults that are different from what was seen in the posts from 2023:

/boot/efi
/boot/efi/EFI
/boot/efi/EFI/BOOT
/boot/efi/EFI/BOOT/STARTUP.NSH

SteveITS

@Sinfonia97 I don’t have a direct answer for you but at some point it may be easier to just reinstall.

Sinfonia97

@SteveITS Thanks, unfortunately I am one of those types that wants to fix the problem before I break down and reinstall. Thankfully I took a backup before I attempted the update while everyone was taking naps. Now I just want to make sure if we have a power outage the system will come back up.

Sinfonia97

UPDATE: Found something that I am hoping can move things forward some more. It looks like potentially the BOOTx64.efi file can be replaced by copying the /boot/loader.efi to the /boot/efi/EFI/BOOT folder and rename it to BOOTx64.efi. If this is the case, I would like to hear from others.

Sinfonia97

UPDATE: Found the following in my /etc/fstab file and now I am even less sure where /boot/efi should be mounted . Per this post in Unable to upgrade from 2.7.1 to 2.7.2 I should see something like:

/dev/msdosfs/EFISYS       /boot/efi       msdosfs rw              2       2

but I see this:

/dev/gptid/ac11fbb1-5651-11e8-b5a2-00907fd0950c	/	ufs	rw	1	1
/dev/gptid/ac128803-5651-11e8-b5a2-00907fd0950c	none	swap	sw	0	0

Just trying to provide as much info as possible in case anyone can help.

Patch

@Sinfonia97 Sounds like a corrupted install to me.
I would

Save pfsense configuration
Do a clean install from a recent installer to ensure partitions are consistent with more recent requirements (v2.7.2 or 2.8 or 2.81)
Document what low level file structure is used with a good current install so you maybe able to guess what went wrong with the prior corruption (partitions sizes, hardware failure, edge case configuration, edge case update series)

Sinfonia97

@Patch Thanks, I am kind of hoping if I can figure out exactly where it needs to be mounted I can use the steps that were provided in other threads to fix the EFI. The steps used before clearly work and even help with the size of the EFI partition. My problem is figuring out where exactly /boot/efi should be mounted to create the backup of the files and the perform the rest of the steps. In what I have been finding, there possibly could be a few places to mount /boot/efi. I ran a geom -t just a bit ago and it pointed me to likely needing to mount it at /dev/ada0p1. The output of geom -t was:

Geom                                                 Class      Provider
ada0                                                 DISK       ada0
  ada0                                               DEV
  ada0                                               PART       ada0p1
    ada0p1                                           DEV
    msdosfs.ada0p1                                   VFS
  ada0                                               PART       ada0p2
    ada0p2                                           DEV
    ada0p2                                           LABEL      gptid/ac11fbb1-5651-11e8-b5a2-00907fd0950c
      gptid/ac11fbb1-5651-11e8-b5a2-00907fd0950c     DEV
      ffs.gptid/ac11fbb1-5651-11e8-b5a2-00907fd0950c VFS
  ada0                                               PART       ada0p3
    ada0p3                                           DEV
    ada0p3                                           LABEL      gptid/ac128803-5651-11e8-b5a2-00907fd0950c
      gptid/ac128803-5651-11e8-b5a2-00907fd0950c     DEV
      swap                                           SWAP

I see in that output in shows msdosfs.ada0p1, and the rest lines up with what is in /dev/gptid and /etc/fstab. It also lines up with the output of gpart list showing ada0p1 as type efi:

Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 488397127
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 209715200 (200M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 20480
   Mode: r1w1e2
   efimedia: HD(1,GPT,ac1172b7-5651-11e8-b5a2-00907fd0950c,0x28,0x64000)
   rawuuid: ac1172b7-5651-11e8-b5a2-00907fd0950c
   rawtype: c12a7328-f81f-11d2-ba4b-00a0c93ec93b
   label: (null)
   length: 209715200
   offset: 20480
   type: efi
   index: 1
   end: 409639
   start: 40
2. Name: ada0p2
   Mediasize: 245677162496 (229G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 209735680
   Mode: r1w1e2
   efimedia: HD(2,GPT,ac11fbb1-5651-11e8-b5a2-00907fd0950c,0x64028,0x1c99c000)
   rawuuid: ac11fbb1-5651-11e8-b5a2-00907fd0950c
   rawtype: 516e7cb6-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 245677162496
   offset: 209735680
   type: freebsd-ufs
   index: 2
   end: 480247847
   start: 409640
3. Name: ada0p3
   Mediasize: 4172430848 (3.9G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 245886898176
   Mode: r1w1e1
   efimedia: HD(3,GPT,ac128803-5651-11e8-b5a2-00907fd0950c,0x1ca00028,0x7c591f)
   rawuuid: ac128803-5651-11e8-b5a2-00907fd0950c
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 4172430848
   offset: 245886898176
   type: freebsd-swap
   index: 3
   end: 488397126
   start: 480247848
Consumers:
1. Name: ada0
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r3w3e8

stephenw10

What actually failed during the upgrade here? Do you have the failure output?

stephenw10

Try: df -hi /boot/efi

You have a 200MB EFI partition but the filesystem on it may still be much smaller. That is what jimp's fix works past.

Sinfonia97

@stephenw10 The upgrade failed with the following error: 'failed insufficient space remaining for /boot/loader.efi', which is exactly why I was looking at jimp's fix. Only problem was, the install did a umount on /boot/efi just before I got that error. When it did that, it did not revert the umount so I have been trying to figure out exactly where mine should be mounted. Most posts refer to /boot/efi being mounted to /dev/msdosfs/EFISYS, which I don't have. Based on everything I have found, it looks like it should be mounted to /dev/ada0p1. Unfortunately when I do that I also notice that BOOTx64.efi is missing from that directory. I did find where I can replace that file with loader.efi and rename it.

Before I proceed, I just want to make sure I have mounted the correct location to /boot/efi. When this problem first happened, I did run the df -hi /boot/efi and got the following reults:

Filesystem                                         Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/gptid/ac11fbb1-5651-11e8-b5a2-00907fd0950c    222G     18G    186G     9%     69k   30M    0%   /

I have since temporarily performed a mount_msdosfs /dev/ada0p1 /boot/efi and when I run the same command I get

Filesystem     Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/ada0p1    766K    1.5K    765K     0%       2   510    0%   /boot/efi

Which according to information I have been able to find online in other forum posts would be correct for some installation, even potentially mine.

Patch

My understanding is early pfsense installs created a smaller efi partition than is required for current versions of pfsense.

@Sinfonia97 You may be able to patch your system to fix that but after which you are still going to have an edge case system.
What else broke during the many updates and repair attempts?

A clean install offers a more reliable solution.

stephenw10

Ah that's the issue. The partition is 200MB but the filesystem is only 766K! So that is the problem that jimps instructions should address. You should be able to copy out the EFI data, expand the filesystem to fill the partion then copy the it back.

Sinfonia97

@Patch I only attempted the upgrade once so nothing else broke. It was just the issue that jimp provided the fix for. Unfortunately his instructions were using /dev/msdosfs/EFISYS which my system does not have.

I probably should have mentioned early on I am running bare metal on an old watchguard M500 with an upgraded CPU that was required for the future releases of PFSense.

Here are my CPU stats in case anyone was wondering:

CPU Type	Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz
                4 CPUs: 1 package(s) x 2 core(s) x 2 hardware threads
                AES-NI CPU Crypto: Yes (inactive)
                QAT Crypto: No

Sinfonia97

@stephenw10 That is exactly what I've been trying to do is the jimp fix. Unfortunately I am needing help that hopefully will confirm what I think is the directory that /boot/efi should be mounted to. Once I figure that out, I should be able to finish with jimp's instructions and correct the size problem and upgrade to 2.8.