Is it safe to upgrade to 2.3.3?
-
New CF cards should be fine. I meant that a CF card that has been in use for some time (i.e. had quite a few cycles of writes…) gets slower and eventually writes start to fail. So many Alix systems that have been in use for some years and seen a few upgrades and had a "not so great CF card" in the first place will be getting slow and possibly failing to upgrade.
I have no peer-reviewed study to quantify "for some time", "seen a few upgrades" or "not so great CF card".
I would guess that the fail in your upgrade log is something to do with the CF card struggling. Unfortunately it ran for a long time, duplicating the slice, downloading all the upgrade packages and so on before an error - that's enhanced Murphy's law at work.
Old configs upgrade fine from any old version. So you can write 2.3.3 release to a CF card, boot it in your system and restore your previous config. When the system reboots it will upgrade anything it needs to in the config.
-
As a side note here - the "duplicate slice" feature is completely irrelevant here. The slice is always duplicated before upgrade and the upgrade happens there.
-
I would guess that the fail in your upgrade log is something to do with the CF card struggling. Unfortunately it ran for a long time, duplicating the slice, downloading all the upgrade packages and so on before an error - that's enhanced Murphy's law at work.
Thanks Phil,
I certainly don't want to get into an argument about CompactFlash, and I admit I'm not a subject matter expert in that domain, but I am a bit skeptical that all my pfSense problems are due to "you must just have a bad CompactFlash".
I was one of the people affected by pfSense bug 4814 / FreeBSD bug 176169 (slow remounts on CompactFlash.) At the time, the pfSense team didn't really seem to be interested in addressing that issue, even when a patch was offered in 176169. They insisted that the root cause was just slow/bad CompactFlash cards, and said that they would "attempt to develop a solution. But this patch is not that solution", with no further explanation of why that patch was not a valid solution other than "it's dangerous". (I'm sure there was a valid reason, but it wasn't disclosed in more detail than that in the bug report.)
Eventually 4814 was closed as "normal behavior", and at one point I was told by someone (I forget who now) that I should stop being so cheap and just go buy new media. (I admit, I am a cheap-o.) :-) Well, despite being on a limited budget, I did just that, and bought a brand new 4GB CompactFlash card. While this did reduce the delay described in 4814 noticeably, it was by no means a fix (if memory serves, the delay went from about a minute to about 30 seconds), and again, this was with a brand new, brand name, fairly fast, well-rated card. Over the past two years I've run pfSense on at least 3 different CF cards, and all exhibited this behavior, some worse than others, but they've all been significant. The suggested workaround in the bug report was to leave the card mounted read-write at all times, and indeed, per my comment earlier in this thread, it appears that that was what was eventually implemented as a "fix" for this bug, write-wear be damned. :-(
Chris Buechle closed the bug, with the comment that he had only observed it in "one make and model each of CF and SD card made in the last 7 years". I have no idea how big his sample of cards was (he only described it as "a stack"), but finding an outlier in each card type that easily still sounds statistically significant to me, especially combined with my own personal experience (I realize that my three attempts represent a statistically small sample, but still… do I really just have phenomenally bad luck?)
Do I just keep buying random cards until I get one that works? At what point does it become a legitimate problem? When I have 5 cards in a row that don't work? 10? I realize CF isn't horribly expensive, but they're not free either.
Anyway, since pfSense is dropping support for ALIX, it's probably just time for me to move on to another platform. OpnSense looks interesting, and there seem to still be a number of other solutions that support ALIX (according to the ALIX web site at least, I haven't had a chance to personally investigate them.)
It's a shame, because I really like pfSense, and the fact that it runs on BSD. However, it is what it is.
Thanks again for your help with this. I'm going to try reflashing 2.3.3 directly and restoring my config. I probably won't get a chance to until this evening though. I'll post an update with my results though!
-
I'm going to try reflashing 2.3.3 directly and restoring my config. I probably won't get a chance to until this evening though. I'll post an update with my results though!
Well, I shut down the firewall, moved the flash to a Linux box, used DD to write the new image, reinstalled it, restored my config, and after quite a bit of churning (downloading packages, regenerating SSH keys, and such), things finally got back to normal. Things seem to be stable and so far all of the packages I am using seem to be working properly. Quite the cumbersome update process for something that's supposed to be "click here to update", but at least in the end it works.
One oddity is that on my dashboard under version it now shows this:
2.3.3-RELEASE (i386) built on Thu Feb 16 06:59:50 CST 2017 FreeBSD 10.3-RELEASE-p16 The system is on a later version than the official release.
That seems to be in error, since this is the official release, correct? I got it directly from the public-facing web site.
Also, while downloading the new image I noticed an inaccuracy on the pfSense web site's download page (https://pfsense.org/download/) It incorrectly states:
"The embedded version is meant to be written to disc before use and is specifically tailored for use with any hardware using flash memory (mostly Compact Flash) rather than a hard drive. Flash memory can only handle a limited number of writes, so the embedded version runs read only from flash, with read/write file systems as RAM disks."
This is out of date and misleading. I was going to submit a bug report, but could not find a category for the web site, so the bug tracking system may not be the right place to report that, but I'm not sue who I should report it to. Hopefully someone reading this will know.
At least I'm up and running for now.
Unfortunately it looks like I need to start looking for a new firewall solution though, given the end-of-life thing. Bummer. :-( Thanks for the heads up on that though! Do we know when the actual target date is for ALIX support to end?
And I guess to answer my own question that I started this thread with: No! It is definitely not safe to upgrade to 2.3.3 on ALIX! (Well, not the automated, GUI way at least.) :-)
-
And I guess to answer my own question that I started this thread with: No! It is definitely not safe to upgrade to 2.3.3 on ALIX! (Well, not the automated, GUI way at least.)
I will (politely) disagree. The upgrade procedure is safe, it has not left me with a [broken|unbootable|bricked] system.
The issues are:
- It is slow (depends on the CF card)
- If it gets part way through downloading the upgrade files and something goes wrong with the download then the process is too "stupid" and starts all over again duplicating the slice… - annoying
- If it gets some CF card write error - but at least it gives up and does not switch to the half-written slice.
On the end of i386 support - yes, I also get annoyed at the waste in the electronics/computing industry. In small home/offices that are nowhere near getting 100Mbps internet the Alix runs fine. e.g. I have a 10Mbps connection and never notice any problem. It is a shame to chuck out the Alix board. But actually getting rid of crud CF cards is a good thing (but not strictly related to i386!)
-
I will (politely) disagree. The upgrade procedure is safe, it has not left me with a [broken|unbootable|bricked] system.
You're absolutely right, and I actually don't disagree with you: "safe" was a bad word choice on my part. (I did mention it was late… I think about 4:00 am here in Seattle by the time I finally finished up and posted that last night.) :-(
The upgrade certainly did not leave me with an unbootable or unusable system at any point. However, the gui-based upgrade does consistently fail and is not usable to upgrade, that was what I was (poorly) trying to express. So it is "safe", but it also just doesn't work, and the workaround I used certainly wasn't safe (but that's why we make backups.)
Although in the end I got it done, the upgrade (including my posts here on the forum and the upgrade itself) cost me many hours of time over the past few days. I wasn't really tracking it, but I'd guesstimate I've lost the equivalent of a full 8-hour workday to it. For an embedded system that is meant to be user-friendly with a web based administrative GUI, that's kind of lame in this day and age. If the GUI is going to provide a "an update is available, click here to install it" option, it should work, not lead the user down a rabbit hole that requires many hours of time, and fairly advance sysadmin abilities to resolve. (I'm perfectly comfortable with the process, but the majority of people I've provided IT support to over the years you would never want going anywhere near something like dd.) :-)
So, it's just frustrating. As I said in the beginning, I really do appreciate all the effort that has gone into pfSense, I don't mean any disrespect to the development team. It's still frustrating having to deal with all this for what should be a routine update though (and from my personal experience, every recent pfSense update has been very painful, some even more so than this one.)
- It is slow (depends on the CF card)
- If it gets some CF card write error - but at least it gives up and does not switch to the half-written slice.
You and your CF hating! (kidding.) :-) For what it's worth, I ordered yet another 4GB CF card last night. It should be here in a few days. I mainly bought it so I could try out alternatives to pfSense on my spare ALIX, but if I can find the time I'm also going to put 2.3.2 on it and try the 2.3.2 -> 2.3.3 update again, to see if it makes any difference. I'll report back if I do.
On the end of i386 support - yes, I also get annoyed at the waste in the electronics/computing industry. In small home/offices that are nowhere near getting 100Mbps internet the Alix runs fine. e.g. I have a 10Mbps connection and never notice any problem. It is a shame to chuck out the Alix board.
Exactly. I've run this on as fast as a 50Mbps Internet uplink. It's also my VPN termination when accessing my home network from outside, my DHCP server and local DNS resolver, my NRPE intermediary for monitoring my home network resources from my external monitoring server, etc. Excellent performance on all counts, in fact it's barely even trying. There's no need to replace it. In fact the only place that it seems to fall down is applying updates.
But actually getting rid of crud CF cards is a good thing (but not strictly related to i386!)
Oh, there you go again! Hehehe. :-P
I've noticed the ALIX boards do have solder points for an ATA header adjacent to the CF. Perhaps it would be possible to put something else in them? A hard drive seems like overkill though, actually feels like a step backwards for an embedded system.
-
For what it's worth, I ordered yet another 4GB CF card last night. It should be here in a few days. I mainly bought it so I could try out alternatives to pfSense on my spare ALIX, but if I can find the time I'm also going to put 2.3.2 on it and try the 2.3.2 -> 2.3.3 update again, to see if it makes any difference. I'll report back if I do.
Well, the new CF arrived and I finally had a chance to test it out today. I used dd to write a bit-for-bit copy of my previous 2.3.2 install onto it, so everything was exactly as before, and then I tried using the update button in the GUI to go from 2.3.2 -> 2.3.3 again. It failed again, in exactly the same way and place as the previous attempt failed.
This is a brand new Kingston card:
https://www.amazon.com/Kingston-CompactFlash-Memory-CF-4GB/dp/B000T9251O
The "old" card was a relatively new (~ 1 year) SanDisk card:
https://www.amazon.com/SanDisk-Compact-Frustration-Free-Packaging-SDCFHS-004G-AFFP/dp/B00GHBBJUG/
So I'm more convinced than ever that this has noting to do with "bad CF cards". I also find it hard to believe that a bad or slow CF card would produce an error like that (complaining that a SQL database was malformed.) It seems like a legitimate pfSense bug to me.
Given the end-of-life status of pfSense on i386 (not to mention, frankly, my frustration with all the problems I've had with pfSense on this platform and the general attitude I've experienced that the pfSense devs don't feel that any of it is on them), I'm not going to bother filing a bug report, I'm just going to move on. I wanted to post my results here though to let everyone know what my test results were.
Thanks to everyone that responded and helped out with this, it's sincerely appreciated!
-
I've noticed the ALIX boards do have solder points for an ATA header adjacent to the CF. Perhaps it would be possible to put something else in them? A hard drive seems like overkill though, actually feels like a step backwards for an embedded system.
I did that in the past and installed a HDD on the pin header.
That works but an ALIX is still a 32bit system and support for 32bit (as well as NanoBSD) is vanishing, though the team announced to give those versions at least security updates for some time if need be.
Haveing said that, running 2.3.3_p1 (on an ALIX) is perfectly fine until there's a future security problem not getting fixed anymore.
I successfully updated some older embedded systems to 2.3.3, but I was lazy and just wrote the image to a spare CF card and uploaded the config backup. Works just fine.
The CF Card is mounted read/write now all the time with the heavy writing being on ram-disk for /var and /tmp. That should sort out some of the hiccups you experience. -
I used dd to write a bit-for-bit copy of my previous 2.3.2 install onto it
If some package database was already corrupted on the first CF card, then it is going to be corrupted on the copy.
I would backup the config from the old card, put a fresh install on the new card, restore the config and proceed from there.the pfSense devs don't feel that any of it is on them
If this [refers to|includes me] then be aware that I am here in a totally voluntary capacity. I have received zero financial reward for anything I have done. I try to give the best information and diagnosis that I can. If that is not good enough, then honestly, tough luck - it is definitely not "on me". I am feeling tired of people taking free help on forums and then throwing it back.
-
I have an ALIX running 2.3.3-p1 and it still chugs along. It's not perfect, but it runs. The hardware was great for its time, but it's past EOL. If it still operates, keep using it.
We intend to support 2.3.x for a year or so after 2.4 releases. You're welcome to keep running 2.3.x as long as you want on there.
NanoBSD upgrades have always worked fine in our testing, and from our customers that have reported issues, most of them were due to faulty CF and the others were connectivity related or in very rare cases, a configuration issue preventing DNS from working during the upgrade.
Even new cards can be faulty, it's the luck of the draw there. We can't fix or code around broken disks. Sandisk is usually reliable and fast, at least the 200X ones I typically use. You have to pay a little more for them but they are worth it: SanDisk SDCFH-004G. But even the best CF is still CF.