• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Method to save configuration after fatal trap / panic

Scheduled Pinned Locked Moved Official Netgate® Hardware
backuperrormellanoxrecoverysg-5100
13 Posts 3 Posters 1.9k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • H
    HydeTech
    last edited by HydeTech Nov 8, 2020, 9:12 AM Nov 8, 2020, 8:52 AM

    Our SG-5100 seems to have crashed. In another stroke of bad luck, I can't find a backup. Below, is how far it boots before stopping. Is there ANY possible way to retrieve the settings so I am not starting 100% from scratch? Netgate tech support is telling me I need to re-image. I'm dreading the many hours it's going to take to get this Firewall back to where I had it if I do that without being able to extract the settings.

    Alternate question: Any way around having to re-image based on where it's halting?

    I am able to interrupt the load of pfSense and get to the loader prompt. From this prompt, I can see the file system. Is there any documented method of saving off pertinent settings from the pfSense loader prompt or from the kernel?

    If I allow it to attempt a full load, it gets to this point and halts:

    mlx5en: Mellanox Ethernet driver 3.5.2 (September 2019)
    Fatal trap 1: privileged instruction fault while in kernel mode
    cpuid = 0; apic id = 04
    instruction pointer = 0x20:0xffffffff830c0a54
    stack pointer = 0x28:0xffffffff830c0a48
    frame pointer = 0x28:0xffffffff830c0a40
    code segment = base 0x0, limit 0xfffff, type 0x1b
    = DPL 0, pres 1, long 1, def32 0, gran 1
    processor eflags = interrupt enabled, resume, IOPL = 0
    current process = 0 (swapper)
    trap number = 1
    panic: privileged instruction fault
    cpuid = 0
    KDB: enter: panic
    [ thread pid 0 tid 100000 ]
    Stopped at kdb_enter+0x3b: movq $0,kdb_why

    1 Reply Last reply Reply Quote 0
    • H
      HydeTech
      last edited by Nov 8, 2020, 9:26 AM

      Is this what I am looking for? If so, how do I get them off the netgate and stored in a safe place?

      /cf/conf/backup/config-*.xml

      Are each of these a full backup of a prior state or do I need all of them?

      If I'm not on the right track, please let me know.

      H 1 Reply Last reply Nov 8, 2020, 9:29 AM Reply Quote 0
      • H
        heper @HydeTech
        last edited by Nov 8, 2020, 9:29 AM

        @HydeTech

        Each is a full state... You only need 1 of them.

        Is there a usb port available on the sg device?

        H H 2 Replies Last reply Nov 8, 2020, 9:32 AM Reply Quote 1
        • H
          HydeTech @heper
          last edited by HydeTech Nov 8, 2020, 9:41 AM Nov 8, 2020, 9:32 AM

          @heper Yes. There are two USB ports available. I'm just a complete amateur at using the console on this device. Would you mind pointing me in the right direction on how to mount a USB stick AND how to copy the files over to it? I know that sounds like Amateur Hour 101, but I'm fumbling around here. What format do I need the USB stick to be in for the Netgate to recognize it? (I'm a bit shocked that official support channels told me I had to re-image and that I was out of luck on my backups. Glad I didn't believe them and started digging). Thanks for your help.

          Alternatively, can I just restore from the loader prompt or kernel somehow?

          1 Reply Last reply Reply Quote 0
          • H
            heper
            last edited by heper Nov 8, 2020, 9:55 AM Nov 8, 2020, 9:49 AM

            i don't own an sg5100... so this might not be accurate

            1. format a usb drive with fat32 on a pc/laptop
            2. insert usb drive in sg5100
            3. on command line type
            [2.4.5-RELEASE][root@pfSense.lan]/root: dmesg
            

            at the end of the screen-output you should see something like this:

            da1 at umass-sim0 bus 0 scbus4 target 0 lun 0
            da1: <USB_STICK_BRAND 1.04> Fixed Direct Access SCSI-4 device
            da1: Serial Number WD-WXE508CAN263
            da1: 40.000MB/s transfers
            da1: 152627MB (312581808 512 byte sectors: 255H 63S/T 19457C)
            da1: quirks=0x2<NO_6_BYTE>
            

            this can be da0,da1,da2,da3,....

            when you found your device node above & verified is is indeed your usb drive, then you can proceed to mounting it:

            [2.4.5-RELEASE][root@pfSense.lan]/root: mount_msdosfs /dev/da1s1 /mnt/
            

            after mounting is succesful, you can copy de config

            [2.4.5-RELEASE][root@pfSense.lan]/root: cp /cf/conf/backup/config-kdjkldjfkl.xml /mnt/
            

            after that you need to unmount the usb drive.

            [2.4.5-RELEASE][root@pfSense.lan]/root: umount /mnt
            

            after above is complete, it is safe to remove the usb-drive & store the config somewhere safe on an other pc

            H 1 Reply Last reply Nov 8, 2020, 10:00 AM Reply Quote 0
            • H
              heper @heper
              last edited by heper Nov 8, 2020, 9:54 AM Nov 8, 2020, 9:53 AM

              @heper said in Method to save configuration after fatal trap / panic:

              @HydeTech
              (I'm a bit shocked that official support channels told me I had to re-image and that I was out of luck on my backups. Glad I didn't believe them and started digging).

              at the start you claimed that the device didn't boot. now it appears you've managed to get to the console.
              I'm sure the netgate staff would have assisted you in recovering the config if you told them you managed to get through the boot into a working consoleshell

              1 Reply Last reply Reply Quote 0
              • H
                HydeTech @heper
                last edited by Nov 8, 2020, 10:00 AM

                @heper said in Method to save configuration after fatal trap / panic:

                i don't own an sg5100... so this might not be accurate

                1. format a usb drive with fat32 on a pc/laptop
                2. insert usb drive in sg5100
                3. on command line type
                [2.4.5-RELEASE][root@pfSense.lan]/root: dmesg
                

                at the end of the screen-output you should see something like this:

                da1 at umass-sim0 bus 0 scbus4 target 0 lun 0
                da1: <USB_STICK_BRAND 1.04> Fixed Direct Access SCSI-4 device
                da1: Serial Number WD-WXE508CAN263
                da1: 40.000MB/s transfers
                da1: 152627MB (312581808 512 byte sectors: 255H 63S/T 19457C)
                da1: quirks=0x2<NO_6_BYTE>
                

                this can be da0,da1,da2,da3,....

                when you found your device node above & verified is is indeed your usb drive, then you can proceed to mounting it:

                [2.4.5-RELEASE][root@pfSense.lan]/root: mount_msdosfs /dev/da1s1 /mnt/
                

                after mounting is succesful, you can copy de config

                [2.4.5-RELEASE][root@pfSense.lan]/root: cp /cf/conf/backup/config-kdjkldjfkl.xml /mnt/
                

                after that you need to unmount the usb drive.

                [2.4.5-RELEASE][root@pfSense.lan]/root: umount /mnt
                

                after above is complete, it is safe to remove the usb-drive & store the config somewhere safe on an other pc

                How did you get here in the first place? I seem to be in a completely different system. Some kind of pfsense loader command prompt with very very few commands compared to a linux shell (which is what it appears you are in).

                Note that I have no IP based connectivity. I'm using the console USB connection on a COM port.

                1 Reply Last reply Reply Quote 0
                • H
                  heper
                  last edited by Nov 8, 2020, 10:02 AM

                  i can't know what you are seeing on your screen ....

                  H 1 Reply Last reply Nov 8, 2020, 10:18 AM Reply Quote 0
                  • H
                    HydeTech @heper
                    last edited by HydeTech Nov 8, 2020, 10:20 AM Nov 8, 2020, 10:18 AM

                    @heper said in Method to save configuration after fatal trap / panic:

                    i can't know what you are seeing on your screen ....

                    Sorry. I appreciate your help. Here's what I am seeing. This is a "pfSense loader prompt" that does not appear to be a linux shell. I can't seem to figure out how to get to a linux shell. It seems to be going right into booting pfSense and then I'm stuck there.

                    I wonder if I could create a linux boot USB stick and then mount the internal drive after booting with the USB based Linux.
                    Seems like I should be able to get to the Linux kernel that's alraedy on the Netgate, but I haven't figured out how so far.

                    Here's where I am.

                    loader.JPG

                    loader2.JPG

                    1 Reply Last reply Reply Quote 0
                    • H
                      HydeTech
                      last edited by Nov 8, 2020, 10:37 AM

                      @heper Unless you have an easier suggestion, I think I'm going to try creating a Ubuntu boot USB stick and try to retrieve the backup file(s) that way. I do need some sleep and so I'm signing off for now. If you have any additional thoughts I would definitely be appreciative. I will not be replying again for at least a few hours. I've had it for the night and we do have a backup firewall of sorts in place to keep us hobbling along until I get this resolved (an old Unifi security gateway). Thanks again.

                      1 Reply Last reply Reply Quote 0
                      • H
                        heper
                        last edited by Nov 8, 2020, 11:17 AM

                        @HydeTech said in Method to save configuration after fatal trap / panic:

                        you have any additional thoughts I would definitely be appreciative. I will not be replying again for at least a few hours. I've had it for the night and we do have a backup firewall of sorts in place to keep us hobbling along until I get this resolved (an old Unifi security

                        thats the bootloader ...i don't think it is possible to recover a config from within the bootloader.

                        you tried booting into single user mode right ?

                        it might be possible to boot an ubuntu usb drive, but i doubt it'll be straightforward or easy to do

                        H 1 Reply Last reply Nov 8, 2020, 7:14 PM Reply Quote 0
                        • H
                          HydeTech @heper
                          last edited by Nov 8, 2020, 7:14 PM

                          @heper said in Method to save configuration after fatal trap / panic:

                          @HydeTech said in Method to save configuration after fatal trap / panic:

                          you have any additional thoughts I would definitely be appreciative. I will not be replying again for at least a few hours. I've had it for the night and we do have a backup firewall of sorts in place to keep us hobbling along until I get this resolved (an old Unifi security

                          thats the bootloader ...i don't think it is possible to recover a config from within the bootloader.

                          you tried booting into single user mode right ?

                          it might be possible to boot an ubuntu usb drive, but i doubt it'll be straightforward or easy to do

                          I did try booting single user mode. It fails too. Wish me luck on what I'm about to start trying, which is the Ubuntu approach. I'll post my results, and if I'm successful, I'll post the steps I took so if someone else finds themselves in this predicament, maybe it will help them at some point in the future.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by Nov 9, 2020, 6:47 PM

                            You can recover the config from the pfSense install image as long as the partition is not completely destroyed, it gives you option before you install:
                            https://docs.netgate.com/pfsense/en/latest/backup/restore-during-install.html#recover-config-xml-from-existing-installation

                            Worst case you can cat the recovered config to the console from there and copy it out into a file.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            1 out of 13
                            • First post
                              1/13
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                              This community forum collects and processes your personal information.
                              consent.not_received