Another Netgate with storage failure, 6 in total so far
-
@andrew_cb thread went silent………
-
Well it is the weekend. Even I try to clock out occasionally.
-
The problem with eMMC would hardly ever have occurred if the eMMC size had been chosen appropriately. After all, TBW is proportional to this parameter. Essentially, in this sense, the difference from SSDs is minimal, except for the type of connection and the fact that SSDs can be replaced without hot air soldering skills . What seems most odd to me is the choice of such small eMMC sizes, especially considering that eMMC prices in those years didn’t significantly increase the final cost of the product.
-
I was directed to this post from 2021 about pfSense with ZFS writing 14-20GB per day.
This got me curious, so I checked our fleet and found that the range is 3-28GB per day, with an average of 18GB. That's about 2-3 DWPD.
This works out to an average of 6.5TB per year, and 19.7TB after 3 years.
Assuming each cell in the eMMC storage can be written 3000 times, 8GB can handle 24TB of writes before the storage is worn out.
This is the best-case scenario. Given the background filesystem behavior plus internal drive processing, writing 20TB of data of a 24TB maximum seems "good".
A 32GB eMMC could handle 96TB TBW, which would be 52GB per day maximum, and probably about 43GB (80%) per day real-world.Comparatively, a 250GB Samsung 860 EVO has a TBW value of 150TB, which works out to 82GB per day for 5years. Halving that to approximate a 120GB drive gives a TBW of 75TB and 41 GB per day for 5 years. This fits with the typical 0.3 DWPD rating of most SSDs.
Clearly, a larger storage drive, whether eMMC or SSD, greatly increases the lifespan of the storage.
If eMMC itself is not the problem, then the small 8GB appears to be a significant factor. Increasing the onboard eMMC storage to 16 or 32GB would greatly increase its lifespan and significantly reduce the number of premature device failures. -
Good morning @jwt . Just checking to see if Netgate is still looking into this and if you have any further comments to add.
-
My 2 1/2-year-old 6100 is showing over 100% usage, so I've ordered an NVMe to install. I did not have any logging-intensive applications running, so it's frustrating. The cheaper mini PCs I bought for other family members' houses are just chugging along with their SSDs, more memory, and faster CPUs.
Is there a guide on installing an NVMe and reinstalling pfSense on that drive out there somewhere? I think my drive arrives on Sunday, so that will be one of my weekend tasks.
-
@dstaylor There isn't a guide for the 6100. Try to find some forum threads on that since I seem to recall discussions. It's possible it's not an NVMe drive, too, I don't know, but some other models use SATA. Just in case you want to check before opening it.
-
@SteveITS I did see mention of NVMe for the 6100, and even a couple of pictures others had posted showing the drive installed. I just have not found any guide on doing an install and making sure it selects the new drive.
Guess I'll have my VM of 2.7.2 standing by in case the whole thing dies.
-
It is NVMe in the 4100/6100/8200. It's not hard to fit if you have any experience assembling PCs.
-
@dstaylor I feel your pain. The storage on your 6100 should not wear out so quickly!
Netgate staff say that it is not possible/supported/recommended for an end-user to install an SSD in the 4100 and 6100, so there is no official documentation.
"It's a time of purchase upgrade option."
Why does Netgate misleadingly advertise the slots when they have no use and the user is not supposed to touch them? Well, that is a real puzzler!
So if your Netgate device is 31 to 365 days old, you are SOL. Fortunately, this limitation does not apply to you since your device is out of warranty.
The actual SSD installation process is easy:
- Remove the rubber feet and then remove all 8 torx screws (2 front, 2 back, 4 under the feet).
- Remove the plastic filler panel that was held in by the screws.
- Gently separate the top and bottom half of your 6100.
- Install the M.2 NVMe drive into the slot.
- Carefully put the top and bottom halves together - pay attention to the LED lights and the 3 plastic "shrouds" that bend around the circuit board.
- Reinstall the screws and attach the feet.
The instructions from the 4200 are pretty similar and cover the software re-installation process.
This video of opening a 6100 should be helpful.
-
@jimp @stephenw10 @kphillips @marcosm @cmcdonald Would any of you care to comment on this thread? With 4.3k views here and over 60k views on Reddit, I hope that @jwt's comments do not represent Netgate's official and only response to the issues that have been raised.
-
@andrew_cb Some good points have been raised along with actionable suggestions to mitigate the issue. Thanks for the constructive feedback - the issue has our attention.
