Upgrade from 2.2.6 to 2.3 Release fails - can't find kernel, can't mount filesys
-
I'm seeing something similar when attempting to upgrade from 2.2.6 amd64 running under Hyper-V 2012 R2. I removed all packages from pfSense prior to the upgrade. I had 4.9 GB of free HDD space before performing the upgrade.
Nothing was output to the console during the upgrade process (other than the standard "firmware upgrade in process") until the end of the process, just before the machine reboots. The first two screenshots reflect the errors. The third screenshot is after the machine reboots and can no longer mount the filesystem.
Hope it helps.
-
I'm seeing something similar when attempting to upgrade from 2.2.6 amd64 running under Hyper-V 2012 R2. I removed all packages from pfSense prior to the upgrade. I had 4.9 GB of free HDD space before performing the upgrade.
Nothing was output to the console during the upgrade process (other than the standard "firmware upgrade in process") until the end of the process, just before the machine reboots. The first two screenshots reflect the errors. The third screenshot is after the machine reboots and can no longer mount the filesystem.
First two errors are normal, on 2.2 as the VM shuts down it (briefly) tries to load 2.3's PHP files on 2.2, which won't work, but after reboot it would be fine)
Last one is different. That may be an issue with HyperV's disk controller setting. There's another thread for that already ( can't find the link at the moment ), likely related to https://forum.pfsense.org/index.php?topic=109952.0 – at least one report in another thread said they had to recreate the VM, but if that's the case, it's possible just fixing the disk controller settings in HyperV may be sufficient. I don't have any Hyper-V that I can test on myself, so I can't offer any more concrete suggestions for that situation.
-
Without more info about why it happened, we can't address the issue. Every other time we've seen this happen it was because the disk was filled during the update (so it was installed on a too-small disk or allocated too little VM disk space, etc). That will be better from 2.3 forward because of how the updates work.
If someone keeps installing to the same too-small disk, it's quite possible they could keep shooting their foot in the same way every time…
Have about 5GiB free space before performing the upgrade (10GiB on VM, no additional packages).
What information do you need?
And how can I get it ? -
In general, "Mounting from ufs:/dev/ <mumble>failed with error 19" means the OS tried to mount the disk from that specific device, but the device wasn't there.
Why wasn't it there? Two main potential answers:
1. The disk controller driver didn't attach to the hardware. Rare that it would happen on a new version and not an old, but if for some reason a driver was dropped, it could happen. Unlikely here, since it's bare metal and was using AHCI, but still possible.
2. The disk changed locations and is no longer at "/dev/<mumble>". This isn't a factor on modern installs since we go by ufs ID and not a specific device name, but if you upgraded over the years and the driver name changed, it could hit that. If you can find the right device name and enter it at the prompt, it can then be made to work by manually editing /etc/fstab. A reinstall is probably quicker if you don't know your way around FreeBSD though.</mumble></mumble>
-
Without more info about why it happened, we can't address the issue. Every other time we've seen this happen it was because the disk was filled during the update (so it was installed on a too-small disk or allocated too little VM disk space, etc). That will be better from 2.3 forward because of how the updates work.
If someone keeps installing to the same too-small disk, it's quite possible they could keep shooting their foot in the same way every time…
Have about 5GiB free space before performing the upgrade (10GiB on VM, no additional packages).
What information do you need?
And how can I get it ?If it's a VM, probably not the same problem as the OP in this thread, there are other threads for VM issues (especially Hyper-V), you're better off moving there.
-
If it's a VM, probably not the same problem as the OP in this thread, there are other threads for VM issues (especially Hyper-V), you're better off moving there.
Actually, your second point in the above post (https://forum.pfsense.org/index.php?topic=109802.msg612250#msg612250) solves the issue. pfSense v2.3 was trying to load the root from /dev/ada0s1a, when it should have been using /dev/da0s1a. Entering "ufs:/dev/da0s1a" at the mountroot> prompt boots v2.3 correctly. After the booting, /etc/fstab shows it's trying to mount the device directly, not by ufs id. Editing the fstab and rebooting solves the issue and allows the machine to reboot in the future without issue.
Also, the two posts linked in the Hyper-V thread both say the issue is with Hyper-V 2008 R2 and is fixed in Hyper-V 2012. I'm running Hyper-V 2012 R2, so don't believe they're related.
-
If it's a VM, probably not the same problem as the OP in this thread, there are other threads for VM issues (especially Hyper-V), you're better off moving there.
Actually, your second point in the above post (https://forum.pfsense.org/index.php?topic=109802.msg612250#msg612250) solves the issue. pfSense v2.3 was trying to load the root from /dev/ada0s1a, when it should have been using /dev/da0s1a. Entering "ufs:/dev/da0s1a" at the mountroot> prompt boots v2.3 correctly. After the booting, /etc/fstab shows it's trying to mount the device directly, not by ufs id. Editing the fstab and rebooting solves the issue and allows the machine to reboot in the future without issue.
Also, the two posts linked in the Hyper-V thread both say the issue is with Hyper-V 2008 R2 and is fixed in Hyper-V 2012. I'm running Hyper-V 2012 R2, so don't believe they're related.
Great! Once you have a correct /etc/fstab that works after rebooting, you can run /usr/local/sbin/ufslabels.sh to permanently adjust fstab to use UFS labels rather than the device name
-
Great! Once you have a correct /etc/fstab that works after rebooting, you can run /usr/local/sbin/ufslabels.sh to permanently adjust fstab to use UFS labels rather than the device name
Out of the dozen or so pfSense v2.2 instances I manage (most are physical, not virtual), all but one are using device references in their fstab, rather than UFS labels. Running /usr/local/sbin/ufslabels.sh and rebooting prior to upgrading to v2.3 seems to obviate this issue entirely.
Note: I've only tried this on one VM that was failing to upgrade correctly, but upgraded without issue by running /usr/local/sbin/ufslabels.sh and rebooting prior to attempting to upgrade to v2.3. Perhaps after someone else can confirm this, the pfSense v2.3 Upgrade Guide (https://doc.pfsense.org/index.php/UpgradeGuide#pfSense_2.3_Upgrade_Guide) could be updated to reflect this?
-
It's already mentioned on that page, under 2.2, but perhaps needs repeated ( https://doc.pfsense.org/index.php/UpgradeGuide#Disk_Driver_Changes )
-
Great! Once you have a correct /etc/fstab that works after rebooting, you can run /usr/local/sbin/ufslabels.sh to permanently adjust fstab to use UFS labels rather than the device name
How can I edit fstab?
Now I have in /etc/fstab
/dev/vtbd0s1a for /
/dev/vtbd0s1b for swap
and in /dev/
ad0s1a
ad0s1
and no da0s1aAnd how to run /usr/local/sbin/ufslabels.sh?
-
Actually, your second point in the above post (https://forum.pfsense.org/index.php?topic=109802.msg612250#msg612250) solves the issue. pfSense v2.3 was trying to load the root from /dev/ada0s1a, when it should have been using /dev/da0s1a. Entering "ufs:/dev/da0s1a" at the mountroot> prompt boots v2.3 correctly. After the booting, /etc/fstab shows it's trying to mount the device directly, not by ufs id. Editing the fstab and rebooting solves the issue and allows the machine to reboot in the future without issue.
Also, the two posts linked in the Hyper-V thread both say the issue is with Hyper-V 2008 R2 and is fixed in Hyper-V 2012. I'm running Hyper-V 2012 R2, so don't believe they're related.
Just to confirm I had exactly the same problem on my HyperV 2012R2 pfSense VM. It's also a pretty old install so was using device names rather than the UFS label.
After a brief panic I got round it in exactly the same way, will take a look at ufslabels.sh later. -
-
vi and ee both exist in the shell, and both can be used.
If they won't run, try using the full path: /usr/bin/vi or /usr/bin/ee
-
Mounting from ufs:/dev/ada0xxx failed with error 19
mountroot>IS this a 1 GB CF Card? Then you should go with a 2 GB or 4 GB since the version 2.3 is no more for 1 GB
CF Cards!a ? did only show the cd rom…
And on the other side with 1 GB RAM you could be easily run out of RAM space or in other words
1 GB RAM is perhaps to small! Insert as a minimum 2 GB or better 4 GB if needed and pending on the
the installed packets or used services.Now I am trying to reinstall 2.2.6 again - seems to work fine, now…
Could be a short work around until the version 2.3.1 is out, but then perhaps read once more the lines above
with 1 GB CF Card and 1 GB RAM it would be perhaps underpowered or not right sorted.No Idea what 2.3 does there…?
Is this a 32Bit or 64Bit install?
After dozens of flawless updates, now this..:-(
There will be each version one day where the user or customer should be thinking on to do something
by its own intense because his hardware could be outdated at one day. So if you are using a 1 GB CF card
change it against a 2 GB or 4 GB model, the 1 GB CF Cards are to small, and if then all other things must be
running in the RAM als the 1 GB of RAM is to small too. -
vi and ee both exist in the shell, and both can be used.
If they won't run, try using the full path: /usr/bin/vi or /usr/bin/ee
How do I use the editor to edit the file /etc/fstab?
I can't open vi or ee, see attachment!
-
Sounds almost like you don't have vi installed on your box.
It should come in from a standard install.What happens if you type:
ls -al /usr/bin/vi
At a command prompt?
Are you running a bare metal or VM install?
How did you load pfSense? -
Sounds almost like you don't have vi installed on your box.
It should come in from a standard install.What happens if you type:
ls -al /usr/bin/vi
At a command prompt?
Are you running a bare metal or VM install?
How did you load pfSense?illegal option – a
My pfSense broken after upgrading to 2.3 via gui.
System can't find kernel (see attachment).
PfSense installed on Proxmox (kvm).
-
That's the boot loader prompt, not the single user mode shell.
-
Sorry, I looked back through the thread and realized your not getting a properly mounted system drive, but you're getting farther than the OP in this thread.
Normally you need to mount the required disk and prove that the pfSense install is up and running before making any of those fstab changes.
As far as the "ls" command, try it without the "-al" -> "ls /usr/bin/vi".
You may be able to recover your install, but it's probably far easier just to reinstall and reload a config backup.
-
