SG-3100 M.2 install

mcury

Hi, I just installed a M.2 device in my SG-3100.
Then, run a reinstall and selected ada0 instead of the eMMC drive.

The install was successfully but now I don't know if its booting from eMMC or from the M.2.

[23.01-RELEASE][root@pfsense.home.arpa]/root: df -h
Filesystem                      Size    Used   Avail Capacity  Mounted on
/dev/diskid/DISK-D4EC8498s2a    7.0G    1.5G    5.0G    23%    /
devfs                           1.0K    1.0K      0B   100%    /dev
/dev/diskid/DISK-D4EC8498s1      34M    2.0M     32M     6%    /boot/u-boot
tmpfs                           4.0M    112K    3.9M     3%    /var/run
devfs                           1.0K    1.0K      0B   100%    /var/dhcpd/dev

[23.01-RELEASE][root@pfsense.home.arpa]/root: gpart show
=>      63  15269825  diskid/DISK-D4EC8498  MBR  (7.3G)
        63      4095                        - free -  (2.0M)
      4158     69993                     1  fat32lba  [active]  (34M)
     74151  15195737                     2  freebsd  (7.2G)

=>       0  15195737  diskid/DISK-D4EC8498s2  BSD  (7.2G)
         0      3673                          - free -  (1.8M)
      3673  15192064                       1  freebsd-ufs  (7.2G)

=>       63  234441585  ada0  MBR  (112G)
         63       4095        - free -  (2.0M)
       4158      69993     1  fat32lba  [active]  (34M)
      74151    4120153     2  freebsd  (2.0G)
    4194304  230247344        - free -  (110G)

=>      0  4120153  ada0s2  BSD  (2.0G)
        0     3673          - free -  (1.8M)
     3673  4116480       1  freebsd-ufs  (2.0G)

=>       63  234441585  diskid/DISK-2M1029S569F9  MBR  (112G)
         63       4095                            - free -  (2.0M)
       4158      69993                         1  fat32lba  [active]  (34M)
      74151    4120153                         2  freebsd  (2.0G)
    4194304  230247344                            - free -  (110G)

=>      0  4120153  diskid/DISK-2M1029S569F9s2  BSD  (2.0G)
        0     3673                              - free -  (1.8M)
     3673  4116480                           1  freebsd-ufs  (2.0G)

=>       1  30031871  da0  MBR  (14G)
         1     98304    1  fat32  (48M)
     98305   4194320    2  freebsd  [active]  (2.0G)
   4292625  25739247       - free -  (12G)

=>      0  4194320  da0s2  BSD  (2.0G)
        0       16         - free -  (8.0K)
       16  4194304      1  freebsd-ufs  (2.0G)

Edit: Tried to boot to shell (boot -s), but couldn't wipe the eMMC

# dd if=/dev/zero of=/dev/mmcsd0 bs=1M status=progress
dd: /dev/mmcsd0: Operation not permitted

# gpart destroy -F mmcsd0
gpart: geom 'mmcsd0': Operation not permitted

Edit2: Wiped the eMMC, but still couldn't boot from ada0

root@:/ # dd if=/dev/zero of=/dev/mmcsd0 bs=1M status=progress
dd: /dev/mmcsd0: end of device454 MiB) transferred 442.005s, 18 MB/s

7457+0 records in
7456+0 records out
7818182656 bytes transferred in 442.118910 secs (17683439 bytes/sec)
root@:/ #

Stuck at boot

Board configuration detected:
Net:   
|  port  | Interface | PHY address  |
|--------|-----------|--------------|
| egiga0 |   RGMII   |     0x00     |
| egiga1 |   RGMII   |     0x01     |
| egiga2 |   SGMII   |   In-Band    |
egiga0 [PRIME], egiga1, egiga2
Hit any key to stop autoboot:  0 
** No partition table - mmc 0 **
## Starting application at 0x00200000 ...

FSC830

Its half a year ago when I switched to M.2 devices, but as far as I remember I need nothing to do then the reinstall.
I did not wipe the eMMC and I cant remember to modify any boot sequence.

Regards

Edit: Will look up in my logs later if I can get more information.

mcury

@fsc830 said in SG-3100 M.2 install:

Edit: Will look up in my logs later if I can get more information.

Thanks FSC830, any help would be much appreciated.

I did some tests here by making manual edits to /etc/fstab, and its working, however, eMMC is still being used during the boot process.

I know that because I wiped the entire eMMC and the system no longer boots
I suppose there is something different with the 23.01 firmware.

So now what I have is:

23.01 system installed in the eMMC, clean install.
Once it booted to the eMMC, edited /etc/fstab and pointed both entries to the M.2 disk which already has 23.01 installed.
Reboot and it works but not exactly what I want since I don't want to use the eMMC.
I would like it completely disabled.

stephenw10

At the Marvell > prompt (u-boot) enter:

setenv bootcmd 'run setLED; run m2boot;'
saveenv
reset

It should now boot from the m.2 ssd everytime

Steve

mcury

@stephenw10 said in SG-3100 M.2 install:

setenv bootcmd 'run setLED; run m2boot;'
saveenv
reset

Thanks stephenw10, I tried that but it didn't work.
According to the TAC, there is an env missing here in my system.
Unfortunately I can't test right now, but definitely I'll try later

I'll try:

At this moment, printenv shows m2dev=0, TAC suggested me to change it to 1 like this:

setenv m2dev=1
setenv bootcmd 'run setLED; run emmcboot;'
saveenv
reset

Then, once this env is applied, I'll try as you suggested:

setenv bootcmd 'run setLED; run m2boot;'
saveenv
reset

stephenw10

Hmm, well that's odd. Yes m2dev should be 1 by default on the 3100.

mcury

@stephenw10 said in SG-3100 M.2 install:

Hmm, well that's odd. Yes m2dev should be 1 by default on the 3100.

I did something horrible..
Believe me or not, I ran the command, env default -f -a

As I didn't know how to restore defaults, I created everything again, from scratch, all setenvs one by one, then confirmed if everything was correct using meld app.

Luck that during the TAC follow up, they asked me to send them the printenv, so I could just replicate that in my SG-3100 exactly like it was before.. Now its booting again :)

Edit1:
The good thing about all this is that I learned a lot.. Nothing like fire to make us learn something...

Edit2: Boot from M.2 with eMMC empty working.

Thanks every one for the help, and I have to say something.

The guys in the TAC are amazing, they helped me a lot and I only have TAC Lite..
I can say that I'm definitely a happy customer. Thanks Netgate!

eyes

What did you do to get it to boot from the M.2?

I have the same. Its stuck in the boot

Starting application at 0x00200000 ...

Can you post a copy of your printenv? plz

/simon

mcury

@eyes said in SG-3100 M.2 install:

Can you post a copy of your printenv? plz

I sold my SG-3100, bought a SG-4100..

You probably need to change the setting m2dev.

Check this: https://forum.netgate.com/topic/178623/sg-3100-upgraded-storage-to-m-2-but-disk-is-still-reading-emmc-storage

stephenw10

Yup almost certainly that issue. ^

There is a known bug in the 3100 installer that fails to update the uboot values. It requires you change them manually.

Steve

eyes

@stephenw10

The address its trying to boot from is that on the M.2 or Ram of the 3100?

I tried booting with m2dev=0 and another instance with m2dev=1

same result. stuck at the memory address
Starting application at 0x00200000 ...

I still have a working system installed on the emmc, is that an issue for the m.2 to work?

From the instructions I should use m2dev=0
since I use the long m.2 2280 scsi in the J10 slot.

https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/m-2-sata-installation.html

stephenw10

Usually you only need to change the bootcmd value from emmcboot to m2boot.

stephenw10

It may also be booting from the m.2 and then mounting root from eMMC confusing the issue.

mcury

@eyes said in SG-3100 M.2 install:

From the instructions I should use m2dev=0

To be able to boot from the M2 SSD, m2dev should be 1.
and setenv bootcmd 'run setLED; run m2boot;'

I always wipe the entire eMMC disk when using SSD to avoid the problem described below:

https://docs.netgate.com/pfsense/en/latest/troubleshooting/multiple-disks.html

Edit:

In addition to the described above, I usually set:

Create the file /boot/loader.conf.local and add to it:
hint.mmcsd.0.disabled="1"

eyes

@mcury

I might want to try to wipe the drive then

Before i do that. This is the M.2 I use.

Sata III is working with the device right?

https://www.proshop.dk/SSD/Intenso-SSD-M2-2280-SATA-III-High-Performance-240GB/2628894

stephenw10

If you were able to install to it as ada0 then it should be fine.

eyes

What is the command in U-boot to erase the eMMC? if i want to wipe it clean.

I cant do it in root, since its the active partition

mcury

I don't have that written down here, but as far as I remember, you will need:

1- Connect to the console
2- Boot the device in single user mode:

At the SG-3100 console press any key to interrupt the boot loader when you see:
Hit [Enter] to boot immediately, or any other key for command prompt.

At the prompt enter: boot -s

That will boot in single user mode to a question asking for a path to the shell, just press return to reach the # prompt.

3- Run the command to wipe the eMMC as described in the link posted above;
Wipe the first 1MB of the disk
dd if=/dev/zero of=/dev/mmcsd0 bs=1M count=1 status=progress
or:
Wipe the entire disk:
dd if=/dev/zero of=/dev/mmcsd0 bs=1M status=progress

Then, boot through USB drive and perform a normal install using ada0 drive.

@stephenw10 can you confirm if these steps are correct ? I really don't recall exactly what I did at that time..

stephenw10

You should be able to run run m2boot manually then choose single user mode to reach a prompt you can erase the eMMC from.

Or at the uboot prompt try: mmc erase 0 400000

jimp

I recently updated the 3100 M.2 install doc with better info on all this, so make sure you are following the new doc and not older instructions:

https://docs.netgate.com/pfsense/en/latest/solutions/sg-3100/m-2-sata-installation.html