Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense Plus generating corrupted backups

    General pfSense Questions
    6
    14
    1.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • hayescompatibleH
      hayescompatible
      last edited by

      I've had some issues with my SG-5100 recently which have forced me to reinstall pfSense a couple of times in the past week. I've discovered that since reinstalling pfSense from a recent backup that subsequent backups being generated now include duplicate <sshdata> sections in the XML backup file which pfSense doesn't like.

      When I import this backup file into pfSense to restore the configuration, the webConfigurator immediately crashes with a PHP error message related to the duplicate <sshdata> block (it's an XML error) and it appears that my only recourse is to reinstall pfSense. I can't even reset to factory defaults via console. Rebooting doesn't fix the problem either as it appears pfSense has already saved the problematic config.

      (I did not take a screenshot of the error message, I was in too much of a rush to get everything going again and did not think of documenting this problem. However, I am sure that this can be reproduced with a problematic backup file.)

      The host keys contained within the <sshdata> section are all different.

      I'm running pfSense Plus 22.01-RELEASE.

      Any ideas why pfSense is generating corrupted backups now? Do I need to reset the host SSH keys somewhere?

      S 1 Reply Last reply Reply Quote 0
      • S
        SteveITS Rebel Alliance @hayescompatible
        last edited by

        @hayescompatible I just checked a recent backup from a 3100 and it has only one <sshdata> section.

        But, see https://forum.netgate.com/topic/171966/not-posible-to-restore-backups which links to https://redmine.pfsense.org/issues/13132

        That shows "Plus Target Version: 22.05" but I don't see it in the release notes for 22.05.

        If it uploads the config as is (??) you should be able to edit it:
        https://docs.netgate.com/pfsense/en/latest/config/xml-configuration-file.html#edit-in-place

        Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
        When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
        Upvote 👍 helpful posts!

        hayescompatibleH 1 Reply Last reply Reply Quote 0
        • hayescompatibleH
          hayescompatible @SteveITS
          last edited by

          @steveits thanks for pointing out those links, especially being able to edit the config in place. What I did instead was to edit the config file before I uploaded it to a fresh pfSense install, in order to make sure there were no duplicate <sshdata> sections.

          I have a bunch of config file backups going back probably a year, I could check those to see when the duplicate <sshdata> sections started appearing, but I know I've successfully restored from backups before so it must be a recent thing.

          In any event, hopefully this fix was rolled into 22.05 as the bug suggests.

          S 1 Reply Last reply Reply Quote 0
          • S
            SteveITS Rebel Alliance @hayescompatible
            last edited by

            @hayescompatible Editing before uploading would make more sense, if one knows to do that. Since mine doesn't have the duplicate section, and yours and the other poster's does, that implies something is causing it. If you can pin it to something, that might be useful for others, and/or make a Redmine entry. It might be useful to know if it's in the "live" config file that way? Just thinking about how to pursue it...

            Pre-2.7.2/23.09: Only install packages for your version, or risk breaking it. Select your branch in System/Update/Update Settings.
            When upgrading, allow 10-15 minutes to restart, or more depending on packages and device speed.
            Upvote 👍 helpful posts!

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by

              That bug was fixed quite some time ago, but if your config already had it in there it may not have been cleaned up properly at the time.

              After import that section should have been removed, but it wasn't always removed.

              IIRC there was an additional fix that went into 22.05 but I can't remember for certain without digging through redmine.

              On 22.05 I did quite extensive testing of restoring SSH keys various ways and it's all solid now as far as I've been able to tell. When reinstalling using the memstick/iso the config recovery also now recovers SSH keys, which is also fun.

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              hayescompatibleH 1 Reply Last reply Reply Quote 0
              • hayescompatibleH
                hayescompatible @jimp
                last edited by

                @jimp I just generated a backup of my config with 22.05 and there are still duplicate <sshdata> sections in it. However, if I understand correctly, 22.05 and above will now know to use only one section and then subsequent backups will only contain the one section?

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, that's not what I would expect. If you try to restore it again I'd expect that to fail in the same way.

                  1 Reply Last reply Reply Quote 0
                  • jimpJ
                    jimp Rebel Alliance Developer Netgate
                    last edited by

                    Looking at https://redmine.pfsense.org/issues/13132 it should be cleaning that up as a part of the restore process on 22.05.

                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      I can't replicate that here in a backup.

                      Do you know the exact steps you took to reach that?

                      hayescompatibleH 1 Reply Last reply Reply Quote 0
                      • hayescompatibleH
                        hayescompatible @stephenw10
                        last edited by

                        @stephenw10 said in pfSense Plus generating corrupted backups:

                        I can't replicate that here in a backup.

                        Do you know the exact steps you took to reach that?

                        Checking my old backups archive, the duplicate <sshdata> sections began appearing after I reinstalled pfSense 22.01 to a new disk and restored an XML config file to it. Every backup taken since then (including with today's 22.05 release) is generating duplicate <sshdata> sections.

                        1 Reply Last reply Reply Quote 0
                        • jimpJ
                          jimp Rebel Alliance Developer Netgate
                          last edited by

                          The extra section was made then, but if you restore it to 22.05 it should work properly -- it will remove the duplicate, restore the keys, then remove the sshdata section entirely.

                          It's fixed when restoring, not when generating.

                          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                          Need help fast? Netgate Global Support!

                          Do not Chat/PM for help!

                          P hayescompatibleH 2 Replies Last reply Reply Quote 0
                          • P
                            p1erre @jimp
                            last edited by

                            @jimp this is what i got on the last 22.05 RC

                            [25-Jun-2022 11:51:00 Europe/Zurich] PHP Fatal error:  Uncaught Exception: XML error: SSHDATA at line 15302 cannot occur more than once
 in /etc/inc/xmlparse.inc:89
Stack trace:
#0 [internal function]: startElement(Resource id #27, 'SSHDATA', Array)
#1 /etc/inc/xmlparse.inc(188): xml_parse(Resource id #27, 'aCB023tWNBoI3S4...', true)
#2 /etc/inc/xmlparse.inc(149): parse_xml_config_raw('/conf/config.xm...', Array, 'false')
#3 /etc/inc/config.lib.inc(134): parse_xml_config('/conf/config.xm...', Array)
#4 /etc/inc/config.gui.inc(56): parse_config()
#5 /etc/inc/auth.inc(33): require_once('/etc/inc/config...')
#6 /etc/inc/ipsec.inc(26): require_once('/etc/inc/auth.i...')
#7 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#8 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#9 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#10 /etc/inc/config.inc(51): require_once('/etc/inc/notice...')
#11 /etc/inc/openvpn.inc(32): require_once('/etc/inc/config...')
#12 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#13 /usr/local/sbin/ in /etc/inc/xmlparse.inc on line 89
[25-Jun-2022 11:51:00 Europe/Zurich] PHP Warning:  fopen(): Filename cannot be empty in /etc/inc/notices.inc on line 101

                            This broke the whole system.

                            1 Reply Last reply Reply Quote 0
                            • hayescompatibleH
                              hayescompatible @jimp
                              last edited by

                              @jimp said in pfSense Plus generating corrupted backups:

                              The extra section was made then, but if you restore it to 22.05 it should work properly -- it will remove the duplicate, restore the keys, then remove the sshdata section entirely.

                              It's fixed when restoring, not when generating.

                              I can confirm 22.05 is still broken in this regard.

                              I created a backup of my running system in 22.05; the backup contained two <sshdata> sections.

                              I reinstalled 22.01 (latest ISO I have) then updated to 22.05 when prompted.

                              After rebooting, I restored the backed-up config file which completely broke pfSense and required another reinstall… which, coupled with the issue I reported here, was NOT fun.

                              At the console:

                              Fatal error: Uncaught Exception: XML error: SSHDATA at line 7349 cannot occur more than once
 in /etc/inc/xmlparse.inc:89
Stack trace:
#0 [internal function]: startElement(Resource id #26, 'SSHDATA', Array)
#1 /etc/inc/xmlparse.inc(188): xml_parse(Resource id #26, 'aEK1LX9+3feLBOO...', false)
#2 /etc/inc/xmlparse.inc(149): parse_xml_config_raw('/conf/config.xm...', Array, 'false')
#3 /etc/inc/config.lib.inc(134): parse_xml_config('/conf/config.xm...', Array)
#4 /etc/inc/config.gui.inc(56): parse_config()
#5 /etc/inc/auth.inc(33): require_once('/etc/inc/config...')
#6 /etc/inc/openvpn.inc(35): require_once('/etc/inc/auth.i...')
#7 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#8 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#9 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#10 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#11 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#12 /etc/inc/config.inc(51): require_once('/etc/inc/notice...')
#13 /etc/rc.banner(2 in /etc/inc/xmlparse.inc on line 89
PHP ERROR: Type: 1, File: /etc/inc/xmlparse.inc, Line: 89, Message: Uncaught Exception: XML error: SSHDATA at line 7349 cannot occur more than once
 in /etc/inc/xmlparse.inc:89
Stack trace:
#0 [internal function]: startElement(Resource id #26, 'SSHDATA', Array)
#1 /etc/inc/xmlparse.inc(188): xml_parse(Resource id #26, 'aEK1LX9+3feLBOO...', false)
#2 /etc/inc/xmlparse.inc(149): parse_xml_config_raw('/conf/config.xm...', Array, 'false')
#3 /etc/inc/config.lib.inc(134): parse_xml_config('/conf/config.xm...', Array)
#4 /etc/inc/config.gui.inc(56): parse_config()
#5 /etc/inc/auth.inc(33): require_once('/etc/inc/config...')
#6 /etc/inc/openvpn.inc(35): require_once('/etc/inc/auth.i...')
#7 /etc/inc/filter.inc(30): require_once('/etc/inc/openvp...')
#8 /etc/inc/ipsec.inc(25): require_once('/etc/inc/filter...')
#9 /etc/inc/gwlb.inc(27): require_once('/etc/inc/ipsec....')
#10 /etc/inc/functions.inc(35): require_once('/etc/inc/gwlb.i...')
#11 /etc/inc/notices.inc(26): require_once('/etc/inc/functi...')
#12 /etc/inc/config.inc(51): require_once('/etc/inc/notice...')
#13 /etc/rc.banner(2

                              Same error message when loading any page in the webConfigurator.

                              R 1 Reply Last reply Reply Quote 1
                              • R
                                rcoleman-netgate Netgate @hayescompatible
                                last edited by

                                @hayescompatible said in pfSense Plus generating corrupted backups:

                                ror: SSHDATA at line 7349 cannot occur more than once
                                in /etc/inc/xmlparse.inc:89

                                There's a known bug in the SSHDATA issue (https://redmine.pfsense.org/issues/13132) Check the redmine for details

                                Ryan
                                Repeat, after me: MESH IS THE DEVIL! MESH IS THE DEVIL!
                                Requesting firmware for your Netgate device? https://go.netgate.com
                                Switching: Mikrotik, Netgear, Extreme
                                Wireless: Aruba, Ubiquiti

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post