Migrating Netgate 1100 from eMMC to USB flash storage to keep it FIT

[jared.silva]

I have an almost three year old Netgate 1100 with mostly normal use. I came across the PSA on reddit referring to https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-lifetime.html#checking-disk-health-lifetime I checked the eMMC and Type A is 0x07 (70%), Type B is 0x0b (110%), and Pre EOL is 0x01 (Normal). I ordered a Samsung FIT 128GB for $15 and went through the migration process. Note, the Samsung FIT does not support Diagnostics / S.M.A.R.T. Status / Information & Tests. Migration from the eMMC to USB flash storage was easy, and I spent about 2 hours doing research and 1 hour doing the actual migration. You should be comfortable using the console prior to doing the migration. The specific steps I followed are below:

Make a backup of your configuration. See https://docs.netgate.com/pfsense/en/latest/backup/configuration.html

Go through the reinstall process. See https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html and related links. Familiarize yourself with them prior to doing the migration. Specific notes related to the reinstall are below.

During Boot the Installation Media, only insert the USB memstick with the installation media, using the USB 2 port. After issuing the command "run usbboot", insert the target drive using the USB 3 port.

The documentation about https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html#determine-target-drive is not accurate, as you can select additional disk(s) for software installation:

┌────────────────────────────┤ Disk Selection ├─────────────────────────────┐
│                                                                           │
│ Select the disk(s) for software installation.                             │
│                                                                           │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │[ ] mmcsd0 7.3G <MMCHC DG4008 0.1 SN 6E124003 MFG 01/2018 by 69 0x0000>│ │
│ │[X] da0    120G <Samsung Flash Drive FIT>                              │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
├───────────────────────────────────────────────────────────────────────────┤
│                           [  OK  ]     [Cancel]                           │
└───────────────────────────────────────────────────────────────────────────┘

If there is a delay after 130/177 Extracting ca_root_nss-3.104 is done, be patient, it will get there.

After the installation finishes, reboot and then halt the boot. At the Marvell prompt, configure to boot from USB and then from eMMC. I got the commands from https://forum.netgate.com/post/1168353 or https://forum.netgate.com/topic/187951/sg-1100-update-from-23-09-1-to-24-03-keeps-failing/63

Marvell>> setenv bootcmd 'run usbboot; run emmcboot;'
Marvell>> saveenv
Saving Environment to SPI Flash... SF: Detected mx25u3235f with page size 256 Bytes, erase size 64 KiB, total 4 MiB
Erasing SPI flash...Writing to SPI flash...done
OK
Marvell>> run usbboot

When you log in to the web gui you will be prompted with the Copyright banner. You will also need to reinstall mmc-utils using the command line, as it will return command not found. You can confirm in the Disks area on the main page that the USB storage is being used and not the eMMC. Ignore the Setup Wizard, go to Diagnostics > Backup & Restore, and change the default password. Then go to Diagnostics > Backup & Restore again, and under Restore Backup, Browse..., and Restore Configuration. One more reboot and you will be up and running.

For other discussion on eMMC, see the following threads:
https://forum.netgate.com/topic/170128/emmc-write-endurance
https://forum.netgate.com/topic/195990/another-netgate-with-storage-failure-6-in-total-so-far

And additional documentation that may be useful:
https://www.netgate.com/supported-pfsense-plus-packages
https://docs.netgate.com/pfsense/en/latest/troubleshooting/disk-writes.html
https://docs.netgate.com/pfsense/en/latest/config/advanced-misc.html#ram-disk-settings

[stephenw10]

@jared-silva said in Migrating Netgate 1100 from eMMC to USB flash storage to keep it FIT:

The documentation about https://docs.netgate.com/pfsense/en/latest/solutions/sg-1100/reinstall-pfsense.html#determine-target-drive is not accurate, as you can select additional disk(s) for software installation

Yup this is only now possible using the Net Installer.

I would recommend using UFS and RAM disks when running from USB as it often has worse write life and ware leveling than eMMC. Though since it's replacable...

[andrew_cb]

@jared-silva I am glad the PSA helped you identify that the storage on your 1100 was at imminent failure status and you were able to avoid it.

Thank you for the instructions for how to migrate to USB flash storage - they should be helpful to others in a similar situation.

[Cabledude]

@jared-silva : I second what @andrew_cb says. Thank you for the writeup, so other folks can benefit from it.
I had already migrated to USB attached SSD (Samsung 120GB evo) and it’s working well.

@stephenw10 : thanks for pointing out the concerns re writes on SSD. I am in fact using RAM disks since I became aware of the eMMC issues. Having a 120GB SSD drive should make for ample drive life, would you agree?

[stephenw10]

It's much less of an issue on a USB attached SSD. The available write cycles and wear-leveling are usually far far better on an SSD compared to a USB flash drive.

So, yes, if you're running a 120GB SSD with RAM disks enabled you shouldn't have any concerns there.

[jared.silva]

I am totally comfortable spending a few bucks on a disposable USB flash drive to delay my going down the RAM disk and other related rabbit holes. This keeps it at the lower end of the prioritized To Do list.

[Cabledude]

@jared-silva I totally understand that, though configuring RAM disk settings is hardly going down the rabbit hole. It’s a one time setting and you can pretty much forget about it once it’s done. I can share my settings if you like.

[stephenw10]

@Cabledude said in Migrating Netgate 1100 from eMMC to USB flash storage to keep it FIT:

It’s a one time setting and you can pretty much forget about it once it’s done.

For a lot of users that's true. But for a large number of other users enabling RAM disks means compromising in other areas like what packages you have running or careful tuning of those packages to avoid exhausting the RAM disks. On the 1100 that's especially true because the available RAM to create disks from is relatively limited.

I you're not running packages then, yes, just enable RAM disks.

[keyser]

@stephenw10 Hi Stephen.

I have just tried to attempt to salvage a 1100 unit that has a very well worn - but not dead yet eMMC.

Im attaching a USB to SATA bridge with a 240Gb Sata Disk, but it does not become available as a install target in the installer.
The device is recognized and reported as “Da0” upon connect:
🔒 Log in to view

Any ideas? I’m thinking perhaps I’m not succeding in attaching it eraly enough before the installer takes hardware inventory and stores its list of possible targets? Or could it be which state (MBR/GPT and partitionlayout) the SSD is in?

[stephenw10]

Hmm, it shouldn't matter how it's formatted. You are using the Net Installer? It only gives you the option of the mmcsd0 device?

[keyser]

@stephenw10 Yes, its the netinstaller from the shop (just downloaded), and yes - it only shows the eMMC as an install target.

Edit: I will do a few more tests with connecting the USB SSD before the “Run usbboot” command, as well as connecting it later on after running the command.

[stephenw10]

Yup, also try some other USB drives to test. That particular setup might take too long to be recognised for some reason.

[jared.silva]

@keyser the net installer will try to boot USB3 over USB2, so I would try putting installation media into USB3 and your USB to SATA bridge into USB2, just for troubleshooting purposes to eliminate any timing issues.

[Cabledude]

@jared-silva said in Migrating Netgate 1100 from eMMC to USB flash storage to keep it FIT:

@keyser the net installer will try to boot USB3 over USB2, so I would try putting installation media into USB3 and your USB to SATA bridge into USB2, just for troubleshooting purposes to eliminate any timing issues.

Yes now that you mention it, I remember doing it this way too.