Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PfSense Crashed

    Scheduled Pinned Locked Moved 2.1 Snapshot Feedback and Problems - RETIRED
    59 Posts 24 Posters 21.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      limecat
      last edited by

      In my case, attempting to upgrade thru the GUI triggered reboots 90% of the time.  This should fix the problem if anyone has run into it (run from SSH / console)

      For i386:

      8
      fetch http://snapshots.pfsense.org/FreeBSD_RELENG_8_3/i386/pfSense_HEAD/updates/pfSense-Full-Update-2.1-BETA1-i386-20130423-1530.tgz
      exit
      13
      2
      /root/pfSense-Full-Update-2.1-BETA1-i386-20130423-1530.tgz
      y
      
      

      For amd64:

      8
      fetch http://snapshots.pfsense.org/FreeBSD_RELENG_8_3/amd64/pfSense_HEAD/updates/pfSense-Full-Update-2.1-BETA1-amd64-20130423-0841.tgz
      exit
      13
      2
      /root/pfSense-Full-Update-2.1-BETA1-amd64-20130423-0841.tgz
      y
      
      

      This has been tested on a remote system over SSH, and works fine.

      I should add that if you are on the affected version from thursday, you need to downgrade ASAP; on one of my (virtual) systems the problem progressed until the vm no longer booted up.  It seems to get worse, probably due to repeated dirty unmounts of the filesystem.

      1 Reply Last reply Reply Quote 0
      • rcfaR
        rcfa
        last edited by

        @limecat

        I can confirm the "getting worse" part, and that the GUI access makes it even less stable.
        However there seem to be other things, maybe VPN related, that made my system not stable enough to be recoverable from the CLI, because by the time it was up and I did an slogin, it was about to crash.

        So the only way I could do this was to disconnect all network cables (physically), so there were no network events (packets, pings, VPN down, etc.) and then do a restore of a full backup from the physical console, which was a bit tricky due to the kind of hardware I use. (No keyboard and video port on the outside of the case).

        So your suggestion may not work for everyone.

        1 Reply Last reply Reply Quote 0
        • L
          limecat
          last edited by

          If you are in that state where it wont boot, this should work.  I have used it to remotely restore 3 boxes so far, and it seems to work well.

          Get an ISO of a "good" version (2.0.3, 2.1 as of april 23).  Boot up to it, and select "recovery".  Pick your drive, and continue.
          You will need to re-assign your interfaces to their adapters, dont worry about getting all of them correct as we will restore the config.
          Once you are at the standard "menu", run the following:

          8
          cp /tmp/hdrescue/cf/conf/config.xml /cf/conf/config.xml
          cp /tmp/hdrescue/cf/conf/config.xml /conf/config.xml
          rm /tmp/config.cache
          exit
          

          Your config should now be loaded.  Manually assign the proper IPs to your interfaces, and you should have proper web-gui access again.  Log in, and make a backup of your config.

          Continue with the installation, which should preserve your now in-memory configuration.

          I HIGHLY recommend that you A) confirm that the downloaded configuration is correct and that B) cat /cf/conf/config.xml shows your configuration.  Make SURE you have backups before proceeding with the install, which will involve unmounting and wiping your existing partition.

          1 Reply Last reply Reply Quote 0
          • M
            monkfish
            last edited by

            @markuhde - i know you didnt. was somebody else mangling my earlier comment about pfsense rocks. which it does!

            Moving swiftly on - all my 32-bit builds are happy running Thu Apr 25 09:08:19 EDT 2013, which I think was the last snapshot prior to the broken one. All good this end!

            1 Reply Last reply Reply Quote 0
            • L
              lucky
              last edited by

              Looking at today's build log, seems like the error from the weekend is gone (I think it was a sig 15 during a make world). A build seems to be running now.

              1 Reply Last reply Reply Quote 0
              • J
                jits
                last edited by

                Ahh..whew!! That's a good crash!

                At least it wasn't someone on the network trying to play master hacker.

                1 Reply Last reply Reply Quote 0
                • M
                  markuhde
                  last edited by

                  Anyone dare load today's build yet?

                  1 Reply Last reply Reply Quote 0
                  • M
                    mdima
                    last edited by

                    @markuhde:

                    Anyone dare load today's build yet?

                    Why, the 24th April build is running soooooo well… :D

                    1 Reply Last reply Reply Quote 0
                    • E
                      eri--
                      last edited by

                      The changes that caused the crashes have been reverted.
                      So the new snapshots are safe.

                      I would have been interested on your backtraces shown in the info when you rebooted pfSense, if you can get those.

                      1 Reply Last reply Reply Quote 0
                      • M
                        mdima
                        last edited by

                        I confirm… after a remote update the box is still there...

                        Thank Ermal to you and all the staff!

                        Michele

                        1 Reply Last reply Reply Quote 0
                        • rcfaR
                          rcfa
                          last edited by

                          @ermal:

                          The changes that caused the crashes have been reverted.
                          So the new snapshots are safe.

                          I would have been interested on your backtraces shown in the info when you rebooted pfSense, if you can get those.

                          So was it the build process crashing that caused the issue, or a code change?
                          If code change, what was it? Just curious as to what has such strange effects.

                          Oh, yes, and also, so far no issues with the new build…

                          1 Reply Last reply Reply Quote 0
                          • M
                            markuhde
                            last edited by

                            Thank you everybody! Updating now (no one's on the campground today so it's perfect for this).

                            1 Reply Last reply Reply Quote 0
                            • J
                              jits
                              last edited by

                              Hi.

                              How do you do a backtrace?

                              I'm doubtful if I can do it now. I re-installed from scratch and my latest backup were from December 2012. Serves me right!

                              In any case, on that particular system running an intel atom 1.3Ghz processor with Realtek daughterboard NIC card, all is well when rebooted with no network cables connected. The second I connect the LAN cable, there is a huge amount of data that starts scrolling up the screen, then it stops for half-a-sec, and reboots. Will do this repeatedly.

                              When cable is already connected, the screen halts at either NTP Configured or WAN Configured, then you hear the sound it makes when there is supposedly a successful reboot or restart.

                              1 Reply Last reply Reply Quote 0
                              • L
                                lucky
                                last edited by

                                Just tried today's build - 2.1-BETA1 (amd64) built on Mon Apr 29 09:14:28 EDT 2013.

                                So far, so good.

                                1 Reply Last reply Reply Quote 0
                                • F
                                  freebee
                                  last edited by

                                  sbflush_internal cc 0 || mb 0xc2651c00 || mbcnt 4352

                                  1 Reply Last reply Reply Quote 0
                                  • C
                                    Clear-Pixel
                                    last edited by

                                    @lucky:

                                    Just tried today's build - 2.1-BETA1 (amd64) built on Mon Apr 29 09:14:28 EDT 2013.

                                    So far, so good.

                                    Not showing a Apr 29 09:14:28 build  ???
                                    http://snapshots.pfsense.org/FreeBSD_RELENG_8_3/i386/pfSense_HEAD/livecd_installer/?C=M;O=D

                                    It's only showing  ???
                                    pfSense-LiveCD-2.1-BETA1-i386-20130429-0914.iso.gz 29-Apr-2013 09:43 85M

                                    Was planning on doing a fresh install but I'm a bit Scared.  :-\

                                    Maybe tomorrow do a fresh install off the latest snap shot ???

                                    HP EliteBook 2530p Laptop - Core2 Duo SL9600 @ 2.13Ghz - 4 GB Ram -128GB SSD
                                    Atheros Mini PCI-E as Access Point (AR5BXB63H/AR5007EG/AR2425)
                                    Single Ethernet Port - VLAN
                                    Cisco SG300 10-port Gigabit Managed Switch
                                    Cisco DPC3008 Cable Modem  30/4 Mbps
                                    Pfsense 2.1-RELEASE (amd64)
                                    –------------------------------------------------------------
                                    Total Network Power Consumption - 29 Watts

                                    1 Reply Last reply Reply Quote 0
                                    • L
                                      lucky
                                      last edited by

                                      @Clear-Pixel:

                                      It's only showing  ???
                                      pfSense-LiveCD-2.1-BETA1-i386-20130429-0914.iso.gz 29-Apr-2013 09:43 85M

                                      Was planning on doing a fresh install but I'm a bit Scared.  :-\

                                      Seems like you looked at the i386 build vs amd64, which is what I'm running. Not sure how i386 is working out.

                                      1 Reply Last reply Reply Quote 0
                                      • C
                                        Clear-Pixel
                                        last edited by

                                        Oops my bad…

                                        HP EliteBook 2530p Laptop - Core2 Duo SL9600 @ 2.13Ghz - 4 GB Ram -128GB SSD
                                        Atheros Mini PCI-E as Access Point (AR5BXB63H/AR5007EG/AR2425)
                                        Single Ethernet Port - VLAN
                                        Cisco SG300 10-port Gigabit Managed Switch
                                        Cisco DPC3008 Cable Modem  30/4 Mbps
                                        Pfsense 2.1-RELEASE (amd64)
                                        –------------------------------------------------------------
                                        Total Network Power Consumption - 29 Watts

                                        1 Reply Last reply Reply Quote 0
                                        • ?
                                          Guest
                                          last edited by

                                          @rcfa:

                                          If code change, what was it? Just curious as to what has such strange effects.

                                          My bet would be on this one:
                                          https://github.com/pfsense/pfsense-tools/commit/3fe8a9cd9297d64ab243fe3f38cea8b4ef147899

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            cmb
                                            last edited by

                                            @alexh:

                                            My bet would be on this one:
                                            https://github.com/pfsense/pfsense-tools/commit/3fe8a9cd9297d64ab243fe3f38cea8b4ef147899

                                            That's it, disabling those kernel patches gets us back to where we were last week before the bad snapshot.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.