Installing new M.2 in Netgate 2100 - How to change boot drive?

[stephenw10]

The uboot envs are set by the installer automatically you don't have to set anything there.

Follow the reinstall instructions:
https://docs.netgate.com/pfsense/en/latest/solutions/netgate-2100/reinstall-pfsense.html

Just choose ada0 (the m.2 drive) when asked, step 7 in the instructions.

Steve

[pazmanian]

@stephenw10

Thanks for that... unfortunately, I got an error just after the erase started:

Erasing the AHCI contents.
trim: open failed: /dev/ada: Operation not permitted
Erasing the eMMC contents.
Writing the firmware to AHCI.
(this may take a few minutes to complete) dd: /dev/ada0: Operation not permitted**

...Why would it erase my eMMC if it errored on the open command? I would recommend not erasing the eMMC if it runs into any errors prior to that point.

It ended up erasing my eMMC & left me with an initialized eMMC. I reinstalled back onto the eMMC, and then had to restore my router config.

Any ideas how to resolve the OPEN FAILED error?

Thank you!
Paul

[stephenw10]

Hmm, odd. Does it actually appear as ada0 in the boot log?

It looks fine in the output you posted above though.

[pazmanian]

@pazmanian said in Installing new M.2 in Netgate 2100 - How to change boot drive?:

trim: open failed: /dev/ada: Operation not permitted

I am wondering if the fact that the M.2 SSD was formatted already is the reason why the trim failed? I decided to erase the SSD using trim myself. From what I saw this operation is a quick & direct way to erase an SSD:

trim /dev/ada0
gpart create -s GPT ada0
gpart show ada0
Which gets me to here:
=> 40 500118112 ada0 GPT (238G)
40 500118112 - free - (238G)

I'd like to try the 'run usbrecovery' again, but I really don't want it to erase my eMMC again until I know that the software is successfully copied to the SSD.

Does anyone know if usbrecovery has any command line options to NOT erase the eMMC? Any other suggestions?

Thanks!
Paul

[stephenw10]

The eMMC is always erased by the installer because otherwise you end up with two ZFS pools with the same name or two similar named root partitions in UFS. That results in mounting root from the wrong device.

You can boot the installer without erasing it in uboot by running:
run usbboot
But I don't recommend it. The only time I use that is if I know the eMMC is already blank.

Steve

[pazmanian]

@stephenw10

Bummer. Would it be possible to configure to remove the eMMC from being mounted so this does not occur? I'm guessing not since you would be doing this if it were possible...

Thank you!

[stephenw10]

It gets erased twice normally. Once in uboot, which you can skip, and once in the installer. Technically you could interrupt the installer and manually write the image to m.2 then update the uboot envs.
But there's a good reason it erases the eMMC. You don't want to have an install on both drives as it almost invariably causes problems.

It's unclear why the installer won't write to the m.2 in your case but I'd guess it's because there's a remaining filesystem on it from when you initially copied the installer image to it.
If you can I would try zeroing out the m.2 drive and then reinstalling from USB.

Steve

[pazmanian]

@stephenw10
I tried this (run usbboot) & it seemed to work ok, but on reboot it errored. The bootcmd does not mention the M.2 SSD.

  bootcmd=run setLED; run mmcboot; run usbboot; run net;

Any ideas?

Thanks!
Paul

This is the console output from the runusbboot & the reboot after that:

---

Select the file system to install (UFS/ZFS) or press the Enter key to install with ZFS:

Selected AHCI device: ada0 - ZFS

In addition to the AHCI, the eMMC is also erased by this operation.

Are you sure you want to continue ? (y/N) y

Erasing the AHCI contents...
Erasing the eMMC contents...
Writing the firmware to AHCI...
(this may take a few minutes to complete)
0+81920 records in
1280+0 records out
5368709120 bytes transferred in 85.842498 secs (62541390 bytes/sec)

Done!

The system will halt now, please power off and remove the firmware
recovery storage device.

Shutdown NOW!
shutdown: [pid 1132]
Waiting for system shutdown.
*** FINAL System shutdown message from root@pfSense-aarch64-recovery ***

System going down IMMEDIATELY

                                                                Aug 15 00:07:45 pfSense-aarch64-recovery shutdown[1132]: halt by root:

Stopping cron.
Waiting for PIDS: 1053.
Stopping devd.
Waiting for PIDS: 820.
Writing entropy file: .
Writing early boot entropy file: .
^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[A^[[ATerminated

AFTER REBOOT-------------------

U-Boot 2018.03-devel-1.2.0ROGUE2-01.00.00.02+ (Feb 07 2020 - 14:33:22 -0500)

Model: Silicom Rogue-2
CPU 1200 [MHz]
L2 800 [MHz]
TClock 200 [MHz]
DDR 750 [MHz]
DRAM: 3.4 GiB
Comphy chip #0:
Comphy-0: SGMII1 3.125 Gbps
Comphy-1: PEX0 2.5 Gbps
Comphy-2: SATA0 6 Gbps
PCIE-0: Link down
MMC: sdhci@d8000: 0
Loading Environment from SPI Flash... mvebu_spi_ofdata_to_platdata: ofdata clock 200000000, frequency 20000000
SF: Detected w25q32bv with page size 256 Bytes, erase size 4 KiB, total 4 MiB
OK
Model: Silicom Rogue-2
Net: eth0: neta@30000 [PRIME], eth1: neta@40000
Hit any key to stop autoboot: 0
Setting bus to 1
** No partition table - mmc 0 **
resetting USB...
USB0: Register 2000104 NbrPorts 2
Starting the controller
USB XHCI 1.00
USB1: USB EHCI 1.00
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 1 for devices... EHCI timed out on TD - token=0xd8d80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
EHCI timed out on TD - token=0x1f8c80
Device NOT ready
Request Sense returned 00 00 00
2 USB Device(s) found
scanning usb for storage devices... 1 Storage Device(s) found
** Bad device size - usb 0 **
BOOTP broadcast 1
DHCP client bound to address 75.73.200.145 (91 ms)
*** Warning: no boot file name; using '4B49C891.img'
Using neta@30000 device
TFTP from server 0.0.0.0; our IP address is 75.73.200.145; sending through gateway 75.73.200.1
Filename '4B49C891.img'.
Load address: 0x1000000
Loading: *
TFTP error: 'No such file or directory' (2)
Not retrying...
*** ERROR: serverip' not set *** ERROR: serverip' not set
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
No FDT memory address configured. Please configure
the FDT address via "fdt addr <address>" command.
Aborting!
ERROR: invalid device tree
ERROR: failed to install device tree

[stephenw10]

Ah is that a much older 2100?
Set the bootcmd manually like so:

setenv bootcmd 'run setLED; run mmcboot; run scsiboot; run usbboot; run net;'
saveenv

It should try to boot from each device in order and the eMMC is current blank so it should boot from scsi (m.2).

Steve

[pazmanian]

@stephenw10

Hi, I reinstalled to the eMMC since I needed a working network at home. I'll try it again this evening, thank you for this information. I hope my 2100 isn't considered old, I just got it in Feb2023. :-)

Paul

[stephenw10]

Hmm, odd. I expect it to have the scsiboot in the list then. Adding it should allow it to boot as expected.

[pazmanian]

@stephenw10

Success!

For the record, I used the following to format the M.2 SSD/SCSCI:

trim /dev/ada0
gpart create -s GPT ada0
gpart show ada0

Per your guidance I modified the bootcmd env var in Marvell (adding "run scsiboot"):

 setenv bootcmd 'run setLED; run mmcboot; run scsiboot; run usbboot; run net;' 

I then ran "run usbboot" to kick off my inserted usb thumb drive to install the firmware onto the M.2.

I left my usb drive in after reboot & it picked up the config.xml & correctly reconfigured my firewall, per the documentation.

Thanks,
Paul