Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Method to save configuration after fatal trap / panic

    Official Netgate® Hardware
    backup error mellanox recovery sg-5100
    3
    13
    417
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      HydeTech last edited by HydeTech

      Our SG-5100 seems to have crashed. In another stroke of bad luck, I can't find a backup. Below, is how far it boots before stopping. Is there ANY possible way to retrieve the settings so I am not starting 100% from scratch? Netgate tech support is telling me I need to re-image. I'm dreading the many hours it's going to take to get this Firewall back to where I had it if I do that without being able to extract the settings.

      Alternate question: Any way around having to re-image based on where it's halting?

      I am able to interrupt the load of pfSense and get to the loader prompt. From this prompt, I can see the file system. Is there any documented method of saving off pertinent settings from the pfSense loader prompt or from the kernel?

      If I allow it to attempt a full load, it gets to this point and halts:

      mlx5en: Mellanox Ethernet driver 3.5.2 (September 2019)
      Fatal trap 1: privileged instruction fault while in kernel mode
      cpuid = 0; apic id = 04
      instruction pointer = 0x20:0xffffffff830c0a54
      stack pointer = 0x28:0xffffffff830c0a48
      frame pointer = 0x28:0xffffffff830c0a40
      code segment = base 0x0, limit 0xfffff, type 0x1b
      = DPL 0, pres 1, long 1, def32 0, gran 1
      processor eflags = interrupt enabled, resume, IOPL = 0
      current process = 0 (swapper)
      trap number = 1
      panic: privileged instruction fault
      cpuid = 0
      KDB: enter: panic
      [ thread pid 0 tid 100000 ]
      Stopped at kdb_enter+0x3b: movq $0,kdb_why

      1 Reply Last reply Reply Quote 0
      • H
        HydeTech last edited by

        Is this what I am looking for? If so, how do I get them off the netgate and stored in a safe place?

        /cf/conf/backup/config-*.xml

        Are each of these a full backup of a prior state or do I need all of them?

        If I'm not on the right track, please let me know.

        H 1 Reply Last reply Reply Quote 0
        • H
          heper @HydeTech last edited by

          @HydeTech

          Each is a full state... You only need 1 of them.

          Is there a usb port available on the sg device?

          H H 2 Replies Last reply Reply Quote 1
          • H
            HydeTech @heper last edited by HydeTech

            @heper Yes. There are two USB ports available. I'm just a complete amateur at using the console on this device. Would you mind pointing me in the right direction on how to mount a USB stick AND how to copy the files over to it? I know that sounds like Amateur Hour 101, but I'm fumbling around here. What format do I need the USB stick to be in for the Netgate to recognize it? (I'm a bit shocked that official support channels told me I had to re-image and that I was out of luck on my backups. Glad I didn't believe them and started digging). Thanks for your help.

            Alternatively, can I just restore from the loader prompt or kernel somehow?

            1 Reply Last reply Reply Quote 0
            • H
              heper last edited by heper

              i don't own an sg5100... so this might not be accurate

              1. format a usb drive with fat32 on a pc/laptop
              2. insert usb drive in sg5100
              3. on command line type
              [2.4.5-RELEASE][root@pfSense.lan]/root: dmesg
              

              at the end of the screen-output you should see something like this:

              da1 at umass-sim0 bus 0 scbus4 target 0 lun 0
              da1: <USB_STICK_BRAND 1.04> Fixed Direct Access SCSI-4 device
              da1: Serial Number WD-WXE508CAN263
              da1: 40.000MB/s transfers
              da1: 152627MB (312581808 512 byte sectors: 255H 63S/T 19457C)
              da1: quirks=0x2<NO_6_BYTE>
              

              this can be da0,da1,da2,da3,....

              when you found your device node above & verified is is indeed your usb drive, then you can proceed to mounting it:

              [2.4.5-RELEASE][root@pfSense.lan]/root: mount_msdosfs /dev/da1s1 /mnt/
              

              after mounting is succesful, you can copy de config

              [2.4.5-RELEASE][root@pfSense.lan]/root: cp /cf/conf/backup/config-kdjkldjfkl.xml /mnt/
              

              after that you need to unmount the usb drive.

              [2.4.5-RELEASE][root@pfSense.lan]/root: umount /mnt
              

              after above is complete, it is safe to remove the usb-drive & store the config somewhere safe on an other pc

              H 1 Reply Last reply Reply Quote 0
              • H
                heper @heper last edited by heper

                @heper said in Method to save configuration after fatal trap / panic:

                @HydeTech
                (I'm a bit shocked that official support channels told me I had to re-image and that I was out of luck on my backups. Glad I didn't believe them and started digging).

                at the start you claimed that the device didn't boot. now it appears you've managed to get to the console.
                I'm sure the netgate staff would have assisted you in recovering the config if you told them you managed to get through the boot into a working consoleshell

                1 Reply Last reply Reply Quote 0
                • H
                  HydeTech @heper last edited by

                  @heper said in Method to save configuration after fatal trap / panic:

                  i don't own an sg5100... so this might not be accurate

                  1. format a usb drive with fat32 on a pc/laptop
                  2. insert usb drive in sg5100
                  3. on command line type
                  [2.4.5-RELEASE][root@pfSense.lan]/root: dmesg
                  

                  at the end of the screen-output you should see something like this:

                  da1 at umass-sim0 bus 0 scbus4 target 0 lun 0
                  da1: <USB_STICK_BRAND 1.04> Fixed Direct Access SCSI-4 device
                  da1: Serial Number WD-WXE508CAN263
                  da1: 40.000MB/s transfers
                  da1: 152627MB (312581808 512 byte sectors: 255H 63S/T 19457C)
                  da1: quirks=0x2<NO_6_BYTE>
                  

                  this can be da0,da1,da2,da3,....

                  when you found your device node above & verified is is indeed your usb drive, then you can proceed to mounting it:

                  [2.4.5-RELEASE][root@pfSense.lan]/root: mount_msdosfs /dev/da1s1 /mnt/
                  

                  after mounting is succesful, you can copy de config

                  [2.4.5-RELEASE][root@pfSense.lan]/root: cp /cf/conf/backup/config-kdjkldjfkl.xml /mnt/
                  

                  after that you need to unmount the usb drive.

                  [2.4.5-RELEASE][root@pfSense.lan]/root: umount /mnt
                  

                  after above is complete, it is safe to remove the usb-drive & store the config somewhere safe on an other pc

                  How did you get here in the first place? I seem to be in a completely different system. Some kind of pfsense loader command prompt with very very few commands compared to a linux shell (which is what it appears you are in).

                  Note that I have no IP based connectivity. I'm using the console USB connection on a COM port.

                  1 Reply Last reply Reply Quote 0
                  • H
                    heper last edited by

                    i can't know what you are seeing on your screen ....

                    H 1 Reply Last reply Reply Quote 0
                    • H
                      HydeTech @heper last edited by HydeTech

                      @heper said in Method to save configuration after fatal trap / panic:

                      i can't know what you are seeing on your screen ....

                      Sorry. I appreciate your help. Here's what I am seeing. This is a "pfSense loader prompt" that does not appear to be a linux shell. I can't seem to figure out how to get to a linux shell. It seems to be going right into booting pfSense and then I'm stuck there.

                      I wonder if I could create a linux boot USB stick and then mount the internal drive after booting with the USB based Linux.
                      Seems like I should be able to get to the Linux kernel that's alraedy on the Netgate, but I haven't figured out how so far.

                      Here's where I am.

                      loader.JPG

                      loader2.JPG

                      1 Reply Last reply Reply Quote 0
                      • H
                        HydeTech last edited by

                        @heper Unless you have an easier suggestion, I think I'm going to try creating a Ubuntu boot USB stick and try to retrieve the backup file(s) that way. I do need some sleep and so I'm signing off for now. If you have any additional thoughts I would definitely be appreciative. I will not be replying again for at least a few hours. I've had it for the night and we do have a backup firewall of sorts in place to keep us hobbling along until I get this resolved (an old Unifi security gateway). Thanks again.

                        1 Reply Last reply Reply Quote 0
                        • H
                          heper last edited by

                          @HydeTech said in Method to save configuration after fatal trap / panic:

                          you have any additional thoughts I would definitely be appreciative. I will not be replying again for at least a few hours. I've had it for the night and we do have a backup firewall of sorts in place to keep us hobbling along until I get this resolved (an old Unifi security

                          thats the bootloader ...i don't think it is possible to recover a config from within the bootloader.

                          you tried booting into single user mode right ?

                          it might be possible to boot an ubuntu usb drive, but i doubt it'll be straightforward or easy to do

                          H 1 Reply Last reply Reply Quote 0
                          • H
                            HydeTech @heper last edited by

                            @heper said in Method to save configuration after fatal trap / panic:

                            @HydeTech said in Method to save configuration after fatal trap / panic:

                            you have any additional thoughts I would definitely be appreciative. I will not be replying again for at least a few hours. I've had it for the night and we do have a backup firewall of sorts in place to keep us hobbling along until I get this resolved (an old Unifi security

                            thats the bootloader ...i don't think it is possible to recover a config from within the bootloader.

                            you tried booting into single user mode right ?

                            it might be possible to boot an ubuntu usb drive, but i doubt it'll be straightforward or easy to do

                            I did try booting single user mode. It fails too. Wish me luck on what I'm about to start trying, which is the Ubuntu approach. I'll post my results, and if I'm successful, I'll post the steps I took so if someone else finds themselves in this predicament, maybe it will help them at some point in the future.

                            1 Reply Last reply Reply Quote 0
                            • stephenw10
                              stephenw10 Netgate Administrator last edited by

                              You can recover the config from the pfSense install image as long as the partition is not completely destroyed, it gives you option before you install:
                              https://docs.netgate.com/pfsense/en/latest/backup/restore-during-install.html#recover-config-xml-from-existing-installation

                              Worst case you can cat the recovered config to the console from there and copy it out into a file.

                              Steve

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post