Upgrade to 2.1.2: Stuck on 2.1

jimp

All of mine are in remote locations and in production. Can't risk taking them down.

I'll be swapping them out with upgraded (Software) replacements in the next week or so.

Is there value in trying these fixes after that?

It would still help to know if any of the above methods would correct the faulty partition table, so that others can benefit from the knowledge.

mkomar

I should have a couple of those units on hand next week. I'll give it a shot and report back once I've done so.

mkomar

JimP - Should have my hands on one or two of the malfunctioning units in the next few days. I'd be happy to try the various fixes you have proposed, and/or if it would be of more value, I'd be happy to either get you serial access to one or both of them and/or get either CF card(s) and/or dd img dumps out to you.

Would any of the options work better than others as far as getting a 'known good' fix out there?

jimp

From my post a few entries up ( https://forum.pfsense.org/index.php?topic=75069.msg413219#msg413219 ) – I listed them in order of preference (and likely destructive potential!)

So try them in that order, method #1, then #2, then #3 only if both 1 and 2 fail.

mkomar

[2.1-RELEASE][root@pfsense]/root(1): sysctl kern.geom.debugflags=16
kern.geom.debugflags: 0 -> 16
[2.1-RELEASE][root@pfsense]/root(2): d if=/dev/ad0 of=/tmp/mbr_part_bkup.img bs=512 count=1
dd of=/dev/ad0 if=/tmp/mbr_part_bkup.img bs=512 count=1dd if=/dev/ad0 of=/tmp/mbr_part_bkup.img bs=512 count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000771 secs (664033 bytes/sec)
[2.1-RELEASE][root@pfsense]/root(3): dd of=/dev/ad0 if=/tmp/mbr_part_bkup.img bs=512 count=1
1+0 records in
GEOM_PART: integrity check failed (ad0, MBR)

512 bytes transferred in 0.024720 secs (20712 bGEOM: ad0s1: media size does not match label.
ytes/sec)
GEOM: ad0s2: media size does not match label.
[2.1-RELEASE][root@pfsense]/root(4):

pmiller

I also have the problem in this post and have been following it closely. I am particularly interested in an in-place fix since I don't have an extra CF card or reader to re-flash.

mkomar - in the third line of your post shouldn't there be 2 d's instead of one for the 'dd' command? Maybe a copy/paste error?

MattMeyer

Adding my experience to the thread. I am also having the same issue. I am using a SanDisk Extreme 4 GB CF. This is the output for fdisk -p /dev/ad0:

/dev/ad0

g c7751 h16 s63
p 1 0xa5 63 3861585
a 1
p 2 0xa5 3861711 3861585
p 3 0xa5 7723296 102816

i have tried recreating the MBR using both the dd method and the fdisk method. Both did not help with a successful upgrade. I do not have a working device that I can copy the MBR so option #3 is out.

Additionally, trying to change the boot slice does not "stick".

mkomar

@pmiller:

mkomar - in the third line of your post shouldn't there be 2 d's instead of one for the 'dd' command? Maybe a copy/paste error?

Must have been dropped somehow. If it wasn't 'dd' the we would see an error instead of the command output.

I've still got a 'broken' device standing by if someone is interested in finding a reliable on-line fix. In the mean time, I've just done a config backup/restore and replaced the production units.

gazoo

Since 2.1.3 came out, has this solved any of these problems for anyone? I was going to go for it, but I don't know if that's a good idea - wanted to see what others have seen for 2.1.3

I'm on nano alix 4g netgate

trunix

jimp, I've got the same problem on a 4gb CF. Output of fdisk -p /dev/ad0:

/dev/ad0

g c7745 h16 s63
p 1 0xa5 63 3854529
p 2 0xa5 3854655 3854529
a 2
p 3 0xa5 7709184 102816

I've tried method #1 and #2, but neither worked. The output of fdisk -if /tmp/fdisk_bkup.txt /dev/ad0 from method #2 is below in case it's notable. I didn't get any errors from method #1, the system just booted back into 2.1 on the same slice. The same thing happened after method #2. I'm also not able to switch the bootup slice for whatever reason.

fdisk: WARNING line 2: number of cylinders (7745) may be out-of-range
(must be within 1-1024 for normal BIOS operation, unless the entire disk
is dedicated to FreeBSD)
******* Working on device /dev/ad0 *******

This system and CF card have been in stable operation for awhile now and I've successfully installed all the updates from 2.0.1 to 2.1. I never got a chance to install 2.1.1, I've had similar problems attempting to install 2.1.2.

cmb

@gazoo:

Since 2.1.3 came out, has this solved any of these problems for anyone? I was going to go for it, but I don't know if that's a good idea - wanted to see what others have seen for 2.1.3

Nothing related to this in particular has changed. The worst that'll happen is it'll reboot back on the same version so there isn't any harm in trying. The vast majority can upgrade just fine, so there's no reason to not try, as it'll more than likely work for you.

For those who have an ALIX (or anything else with 256 MB RAM and nano), if you're running more than you reasonably should be on a box with 256 MB RAM, disabling some services (especially OpenVPN if you're running multiple instances) before upgrading will allow you to upgrade successfully in an unrelated circumstance to this thread, where it fails because you run out of RAM when trying to upgrade.

dchicks

I'm seeing the exact same reported issues that I've read in this thread…


gpart set -a active -i 1 ad0
gpart: table 'ad0' is corrupt: Operation not permitted

None of the prescribed fixes have done anything to help. I'm running a NetGate with M0n0wall on 256M RAM, of which only 54% is currently in use. I tried turning off a couple of services to see if it would help, but it is no use.

As with others, it looks like it's upgrading and says it will reboot when finished, but it appears to just log my out after a time, and no upgrade occurs. Do we have any clue why this isn't working? Do I need a new CF card?

Thanks,
Dave

pmiller

I ordered a new flash card and installed the latest version (2.1.3) without incident, following the guide posted at https://doc.pfsense.org/index.php/Installing_pfSense. I was able to backup & restore my configuration from the old card to the new card. I will probably re-flash the old card in a few weeks just to make it serviceable again, but right now it is my fallback. Everything transferred cleanly. I didn't need a serial modem cable since I already had the latest bios, but it's worth checking the bios version to make sure. By default the new image will install configure the gateway to 192.168.1.1 with admin and pfsense for login/password. I had to manually configure my laptop's IP address to be in that subnet before I could start the restore, but that might be an issue of the laptop I was using since it should work with DHCP. After the restore the pfsense IP changed to match my old setup, so I set the laptop back to DHCP and all is back to normal. Finally I can feel good about using the VPN again after heart bleed.

For those still stuck on 2.1 I would suggest doing the same, since I wasn't willing to risk a production firewall with the commands in this post, nor did others seem to have luck with them. A new flash card is only $10 and card readers are also cheap if you don't have one already. It's unfortunate that some of us got stuck, but at least it can be fixed.

RCS-Michael

Pmiller,

That's nice that you were able to do that, but it's not an option for some of us. I have pfSense firewalls distributed around the world. Am I supposed to hop on my jet and get this all done? Some sites don't have people onsite to ship it to me.

I, for one, am still waiting on a workable solution.

Michael

VampBoy

Finally found the right googleFU to find this thread ;)

I'm in the same boat with ad0 corrupt during the upgrade whether or not auto or manual, going from 2.1 to 2.1.3 (tried 2.1.2 for giggles with the same result)

Running a netgate fw-7535 with a 4gb CF card that has been upgraded successfully in the past.

fdisk output looks the same as others have posted.


$ fdisk -p /dev/ad0
# /dev/ad0
g c7745 h16 s63
p 1 0xa5 63 3854529
p 2 0xa5 3854655 3854529
a 2
p 3 0xa5 7709184 102816

Hours later
Ended up flashing to a new card which was a massive PITA as I had forgotten how to do it and the new card kept hanging on boot.

In case it helps here's a fdisk dump of the newly working 4g card I have in there now. (running 4g nano amd64 2.1.3)


$ fdisk -p /dev/ad0
# /dev/ad0
g c7773 h16 s63
p 1 0xa5 63 3844449
a 1
p 2 0xa5 3844575 3844449
p 3 0xa5 7689024 102816

jimp

Can someone try setting this tunable:

sysctl kern.geom.part.check_integrity=0

And then perform an upgrade, see if that lets it get by.

njlinuxmike

@jimp:

Can someone try setting this tunable:
sysctl kern.geom.part.check_integrity=0
And then perform an upgrade, see if that lets it get by.

I tried the above. No joy.

This is a Soekris Net 5501-60 running pfSense 2.1-RELEASE NanoBSD 4G trying to up to 2.1.3

Additionally, after the first or second attempt pfBlocker was not re-installed.

/dev/ad0

g c7745 h16 s63
p 1 0xa5 63 3861585
a 1
p 2 0xa5 3861711 3861585
p 3 0xa5 7723296 102816

Upgrade log available here:
http://pastebin.com/tq6Y3gPq

Multiple attempts to upgrade using manual and auto.

eweri

Hello!

Do not know it this helps but I could not update my APU with SanDisk 4GB card and I started testing the sd-card.

And this is what I got:
dd if=/dev/random of=/dev/rdisk5 bs=64k
dd: /dev/rdisk5: Input/output error
60505+0 records in
60504+0 records out
3965190144 bytes transferred in 596.342498 secs (6649183 bytes/sec)

This means that the 4GB card seams to have a bad block there, so I tested the next card of the same type and was very surprised that the second card had a bad block at the exact same block number. :o

Maybe it is a fake 4GB card?
It is printed with "SanDisk Ultra 30MB/s SDHC I 4GB"

At the moment I am testing the third card …. ups ... same result - I am going to call my Supplier >:(

Bye,
eweri

P.S. Okay - spoke to my supplier - this is expected - a 4GB SanDisk SD-Card has a capacity of 3.965 billion bytes and not 4 billion bytes as printed on the label :-\

kpa

I/O error means that either there is a real bad block on the card or there's some odd compatibility issue with the card/controller combination with FreeBSD 8.3. I would test the cards on a different machine to see if the issue can be repeated there.

luckman212

I've got a few Netgate FW-7535's that are "stuck" with 2.1 or 2.1.3 and trying to upgrade them to 2.1.4 seems to go OK but after the reboot they are back on their original version. Both stuck units that I happened to be banging my head against tonight are running a nanobsd 2g (i386) image burned to a SanDisk Extreme 2GB CF P/N# SDCFH-002G

Here's some fdisk output (both units give the identical output)

fdisk -p /dev/ad0
# /dev/ad0
g c3875 h16 s63
p 1 0xa5 63 1902033
p 2 0xa5 1902159 1902033
a 2
p 3 0xa5 3804192 102816

I'm reluctant to roll out there to swap CF cards as it would be both time consuming and cause noticeable downtime. If anyone knows of a way to fix this conundrum remotely I would be so grateful! ::)