Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    SG-3100 Died With Bad eMMC

    Scheduled Pinned Locked Moved Official Netgate® Hardware
    21 Posts 4 Posters 646 Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S Offline
      stephenw10 Netgate Administrator
      last edited by

      Yup it will boot from an SSD just fine. It's m.2 SATA though not PCIe (NVMe).

      1 Reply Last reply Reply Quote 0
      • J Offline
        jsylvia007
        last edited by

        OK... So...

        The recovery works fine, I think:

        Type the name of the destination device (mmcsd0,ada0) or press the Enter key to install on mmcsd0: ada0
        
        Selected M.2 device: ada0
        
        Are you sure you want to continue ? (y/N) Y
        
        Erasing the M.2 contents...
        Writing the firmware to M.2...
        (this may take a few minutes to complete)
        0+32768 records in
        512+0 records out
        2147483648 bytes transferred in 30.763006 secs (69807341 bytes/sec)
        Fixing M.2 labels...
        Fixing u-boot environment...
        dd: /dev/flash/spi0: No such file or directory
        Invalid input file: /root/env
        stat: No such file or directory
        Invalid input file: /root/newenv
        stat: No such file or directory
        
        Done!
        
        The system will halt now, please power off and remove the firmware
        recovery storage device.
        
        
        
        
        Shutdown NOW!
        shutdown: [pid 1926]
        Aug 13 19:41:29 Stopping cron.
        Waiting for PIDS: 1860.
        Stopping devd.
        Waiting for PIDS: 1611.
        Writing entropy file: .
        Writing early boot entropy file: .
        Terminated
        .
        Aug 13 19:41:30 3100-recovery syslogd: exiting on signal 15
        Waiting (max 60 seconds) for system process `vnlru' to stop... done
        
        Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 done
        All buffers synced.
        Uptime: 1m14s
        
        The operating system has halted.
        Power cycle or reset to reboot.
        

        Then I issued the command to change the boot from mmc to m2... That worked and it looks like it now tries to boot, BUT, all I'm left with is a flashing blue light for the green circle in the front, and the console says:

        Reset SCSI
        AHCI init for unit0
        Target spinup took 0 ms.
        SATA link 1 timeout.
        
        Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found
        scanning bus for devices...
          Device 0: (0:0) Vendor: ATA Prod.: ORICO Rev: X071
                    Type: Hard Disk
                    Capacity: 244198.3 MB = 238.4 GB (500118192 x 512)
        Found 1 device(s).
        ** Unrecognized filesystem type **
        ## Starting application at 0x00200000 ...
        

        And it just hangs there... Someone with some better knowledge help me out maybe? I dunno what I'm supposed to be seeing, but perhaps there's an issue. The m.2 SATA is brand new and was apparently recoginzed, etc.

        F 1 Reply Last reply Reply Quote 0
        • stephenw10S Offline
          stephenw10 Netgate Administrator
          last edited by stephenw10

          Was that drive new? It's possible it had some older filesystem on it that needs to be completely removed.

          You should see something like:

          Board configuration detected:
          Net:   
          |  port  | Interface | PHY address  |
          |--------|-----------|--------------|
          | egiga0 |   RGMII   |     0x00     |
          | egiga1 |   RGMII   |     0x01     |
          | egiga2 |   SGMII   |   In-Band    |
          egiga0 [PRIME], egiga1, egiga2
          Hit any key to stop autoboot:  0 
          
          Reset SCSI
          AHCI init for unit0
          Target spinup took 0 ms.
          SATA link 1 timeout.
          
          Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found
          scanning bus for devices...
            Device 0: (0:0) Vendor: ATA Prod.: NT-32 Rev: 1.09
                      Type: Hard Disk
                      Capacity: 30533.8 MB = 29.8 GB (62533296 x 512)
          Found 1 device(s).
          reading ubldr.bin
          228648 bytes read in 11 ms (19.8 MiB/s)
          ## Starting application at 0x00200000 ...
          Consoles: U-Boot console  
          Compatible U-Boot API signature found @0x7f4dc280
          
          FreeBSD/armv6 U-Boot loader, Revision 1.2
          (Tue Jul 10 10:26:23 EDT 2018 root@buildbot3)
          
          DRAM: 2048MB
          Number of U-Boot devices: 4
          U-Boot env: loaderdev not set, will probe all devices.
          Found U-Boot device: disk
            Probing all disk devices...
            Checking unit=0 slice=<auto> partition=<auto>...disk0: read failed, error=1
          
            Checking unit=1 slice=<auto> partition=<auto>... good.
          Booting from disk1s2a:
          Loading /boot/defaults/loader.conf
          console comconsole is invalid!
          no valid consoles!
          Available consoles:
              uboot
          /boot/kernel/kernel text=0x1b4 text=0x84e20c text=0xa04440 data=0xac168 data=0x0+0x4c000 syms=[0x4+0xae330+0x4+0x11aaee]
          |
          Hit [Enter] to boot immediately, or any other key for command prompt.
          Booting [/boot/kernel/kernel]...               
          /boot/dtb/armada-385-80300-0148-G00-X100.dtb size=0x67f4
          Loaded DTB from file 'armada-385-80300-0148-G00-X100.dtb'.
          Kernel entry at 0x400200...
          Kernel args: (null)
          SOC: Marvell 88F6820, TClock 250MHz, Frequency 1600MHz
            Instruction cache prefetch enabled, data cache prefetch disabled
          ---<<BOOT>>---
          GDB: debug ports: uart
          
          J 1 Reply Last reply Reply Quote 0
          • F Offline
            FSC830 @jsylvia007
            last edited by

            @jsylvia007 Hm, its long time ago when I did that, and actually I cant remember if I changed any boot environment values. 😕
            When looking in manual I found "By default, the M.2 SATA drive will then be the first drive
            recognized by pfSense software" and as far as I remember that was all, re-installing the software at ada0...
            Unfortunately I do not have access from my current location to the laptop I did connect to the console and logged all output when doing this.
            Where did you find an advise to change boot environment?

            Regards

            J 1 Reply Last reply Reply Quote 0
            • stephenw10S Offline
              stephenw10 Netgate Administrator
              last edited by

              You do have to manually update the uboot env when swtching boot device. Some while back the recovery installer was able to set it automatically but no longer does.

              1 Reply Last reply Reply Quote 0
              • J Offline
                jsylvia007 @FSC830
                last edited by

                @FSC830 - Its in the instructions now to perform it.

                1 Reply Last reply Reply Quote 0
                • J Offline
                  jsylvia007 @stephenw10
                  last edited by

                  @stephenw10 - It was a brand-new-in-package drive. Any chance I can dd the drive from the installer or recovery somehow? I don't want to pull it all apart again.

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S Offline
                    stephenw10 Netgate Administrator
                    last edited by

                    For reference:

                    Marvell>> printenv bootcmd
                    bootcmd=run setLED; run m2boot;
                    Marvell>> printenv m2boot 
                    m2boot=scsi reset; fatload scsi 0:$m2dev 200000 ubldr.bin; go 200000;
                    

                    So it's trying to load the boot loader from a fat partition on the drive but instead finding an unknown filesystem there.

                    You can try:

                    Marvell>> scsi init
                    AHCI init for unit0
                    Target spinup took 0 ms.
                    SATA link 1 timeout.
                    
                    Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found
                    scanning bus for devices...
                      Device 0: (0:0) Vendor: ATA Prod.: NT-32 Rev: 1.09
                                Type: Hard Disk
                                Capacity: 30533.8 MB = 29.8 GB (62533296 x 512)
                    Found 1 device(s).
                    Marvell>> scsi part 0
                    
                    Partition Map for SCSI device 0  --   Partition Type: DOS
                    
                    Part	Start Sector	Num Sectors	UUID		Type
                      1	4158      	69993     	00000000-01	0c Boot
                      2	74151     	62459145  	00000000-02	a5
                    

                    Then dig deeper with:

                    Marvell>> fatls scsi 0:1  
                       870156   u-boot.bin 
                       949520   u-boot.mmc 
                       228648   ubldr.bin 
                    
                    3 file(s), 0 dir(s)
                    
                    J 1 Reply Last reply Reply Quote 0
                    • J Offline
                      jsylvia007 @stephenw10
                      last edited by

                      @stephenw10 -

                      So... that worked? I typed out those commands and it showed me exactly what was posted...

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S Offline
                        stephenw10 Netgate Administrator
                        last edited by

                        Huh so it has the fat partiton on the drive and it contains the file?

                        I would try manually running the command in m2boot then. See exactly where it fails.

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S Offline
                          stephenw10 Netgate Administrator
                          last edited by

                          Oh wait what do you have m2dev set to?

                          Marvell>> printenv m2dev
                          m2dev=1
                          

                          And which slot did you add the drive to?

                          Try setting it to 1 (or 0 if it;s already 1).

                          J 1 Reply Last reply Reply Quote 0
                          • J Offline
                            jsylvia007 @stephenw10
                            last edited by

                            @stephenw10 - WINNER WINNER CHICKEN DINNER!!!!!!!!!!!!

                            J 1 Reply Last reply Reply Quote 0
                            • J Offline
                              jsylvia007 @jsylvia007
                              last edited by

                              Wait... Spoke too soon... Setting it to 1 looks like it's trying to boot / read from the eMMC and it's REALLY unhappy about that... Any way to stop it from doing that?

                              J 1 Reply Last reply Reply Quote 0
                              • J Offline
                                jsylvia007 @jsylvia007
                                last edited by

                                It says "External Config Loader Started... mmcsd0" and then it goes off to never, never land, which is exactly what I would expect because the MMC is TOAST.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S Offline
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, that's painful. It should eventually get past that. Once it does I would try to clear the partitions from the eMMC. If it allows that....

                                  J 1 Reply Last reply Reply Quote 0
                                  • J Offline
                                    jsylvia007 @stephenw10
                                    last edited by

                                    @stephenw10 - It did... It failed into a single user mode, I was able to issue a DD to the eMMC and after a reboot, it no longer sees the MMC device. Note that the DD made it VERY angry also, but I let it run for a bit to make sure that it destroyed as much of the partition table at the beginning as possible. It appears to be happy now!

                                    I'm going to run another factory reset, etc., just to make sure it doesn't get wonky.

                                    Note that when it's in a state with the failed eMMC, the USB terminal is also pretty unstable.

                                    Of note for other folks who stumble across this... Believe it or not, the config file was still there and intact. Before I started all of this, I saw the config, was able to cat it out with the device in read-only, but couldn't get access to it. I couldn't remount it, and because it was read-only, I couldn't mount a USB device on top of it, but I was still able to get it.

                                    I created a tmpfs on top of /mnt (the exact command escapes me, but I think it was mount -t tmpfs tmpfs /mnt. I think I had to make it read/write, but I can't remember off the top of my head how I did that piece (it might have been mount -u /mnt). From there, I was able to mount a USB stick to /mnt, and then copy the file from /cf/config/config.xml to the USB stick.

                                    It took me a LONG time to figure out the tmpfs "hack", but it worked!

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S Offline
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      As a last resort you can usually just cat the config file and copy/paste it from the console directly.

                                      1 Reply Last reply Reply Quote 1
                                      • F Offline
                                        FSC830
                                        last edited by

                                        The uboot commands sound somehow familiar to me, but actually I cant remember if I needed to do so.
                                        I cant even remember which version of pfSense was the curretn at this time when adding the m.2 SATA, 21.x or 22.x? 😕

                                        Regards

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.