/var is low on disk space
-
@JonH depends heavily on logging and packages, see
https://www.netgate.com/supported-pfsense-plus-packages
Edit: yeah ntopng, there you go, didn’t read down that far yet :)Also:
https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/m-2-sata-installation.html -
@SteveITS said in /var is low on disk space:
Perhaps the self-cleaning of old BEs should happen sooner
Indeed. I have been quite negligent in reviewing various changes and feature additions.
I did see that BEs existed but never gave it much thought in the past. Now I also see that my negligence has poured over to SDD lifetime.I'm in my mid-70's now and keeping up with tech has become quite difficult. And with the amount of security patches for various devices coming fast and furious of late I now see that it will be difficult if not impossible to keep a system running for my own expected lifetime without needing to upgrade the hardware (I mostly use an Apple Intel machine which is going to be unsupported in the very near future).
-
@JonH said in /var is low on disk space:
So 0x0b is "The disk has used 100%-110% of its estimated life time" and somewhat concerning.
and 0x01 is "The disk has consumed less than 80% of its reserved blocks" which is somewhat less concerning.Get an m.2 SSD ASAP.
Any mSATA m.2 SSD will work. https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/m-2-sata-installation.html
Do not wait. if the eMMC on this goes read-only the only method you have (that I have determined so far) is to use a re-soldering station to heat the eMMC and remove it from the board. it's messy. I haven't gotten a second 5100 with a read-only eMMC yet to see if there's another option to block the BIOS from seeing the chip.
-
-
@JonH said in /var is low on disk space:
@rcoleman-netgate said in /var is low on disk space:
Get an m.2 SSD ASAP.I strongly agree with that advice. My SG-5100 exhibited this same behavior when I upgraded to 23.01 back in March. I installed a compatible SSD within a couple days but unfortunately it was too late. The eMMC had completely failed at that point and prevented the unit from booting. My beloved SG-5100 was a boat anchor after only 20 months of use!
And the sad thing is, it really wasn't used that hard. Just a simple home environment with 40 or so client devices. The only write-heavy package installed was pfBlockerng.
I would highly recommend that you get a backup plan together in case your unit suffers the same fate. I had to scramble and go to Best Buy to get a POS consumer-grade Asus router just to get back online. Needless to say, the whole ordeal was pretty annoying an stressful. I work from home and this happened in the middle of a busy week. The wife wasn't too happy either...
Best of luck.
-
@azdeltawye said in /var is low on disk space:
I strongly agree with that advice. My SG-5100 exhibited this same behavior when I upgraded to 23.01 back in March. I installed a compatible SSD within a couple days but unfortunately it was too late. The eMMC had completely failed at that point and prevented the unit from booting.
Unfortunately for @azdeltawye, but (and I think I speak for both of us here) we hope to be fortune for others, this very same 5100 is the one I mentioned above where I extracted the eMMC from the board and it immediately started perform normally.
It's not a process I recommend everyone to do -- but for those who have the skills it can be done.
This is the eMMC on the board (the bottom side)
The board removed using a re-soldering station (a programmable heat gun)
The empty space (partially cleaned up)
The bottom of the eMMC.
-
@azdeltawye said in /var is low on disk space:
My beloved SG-5100 was a boat anchor after only 20 months of use!
Indeed. I got 16 mo's out of my 1st 5100, this one is going on 27 mo.
Same as you, my biggest write-hvy is pfBlockerNG. I am also simple home w/4 active devices but up to 30 because of smart devices.I was thinking of getting a SSD just to have more room, I had no idea how fragile the SSD was until now. I knew it was limited but didn't think it was this limited. A few minutes ago I cut my pfBlocker updates from 4x/day to once.
Thanks
-
@rcoleman-netgate said in /var is low on disk space:
use a re-soldering station to heat the eMMC and remove it from the board.
Referring to the later post with the photos of the removed eMMC can you clarify for me:
If an mSATA is installed it will supersede the eMMC, essentially making it electronically 'disappear'?But if it goes south then the mSATA won't do any good?
That is what I understand from what you posted, correct?
And the 32 GB SATAIII M.2 2242 SSD would be a suitable replacement?
As an aside, my 1st 5100 is still laying around, but if I read correctly, adding a mSATA probably won't help because if the eMMC is the cause then it's too late. Correct?
Is this likely to be a similar situation with my old 2440 w/ 20GB drive? Or is that a totally different situation? I know this may be impossible to answer, just fishing on the off chance I can get a working emergency spare. I'm ok with soldering but I don't have a re-soldering station, just a solder sucker and copper braid. And noticeably shakier hands than in the 'good old days'.
Neither of those units will POST and the only signs of life is the power light.
-
@JonH said in /var is low on disk space:
my biggest write-hvy is pfBlockerNG
Note NtopNG is marked "Requires SSD/HDD" in the linked package list above...
@rcoleman-netgate "...do not boot nor provide an output via the serial cable." Would that be similar for the 4860 model we discussed elsewhere? IIRC that doesn't have eMMC or at least can't use the "write life" software. Plus I'd expect the backup router to have way less disk writes.
-
@SteveITS said in /var is low on disk space:
Note NtopNG is marked "Requires SSD/HDD" in the linked package list above...
When I purchased my first 5100 in Nov 2019, to replace my dead 2440 w/ 20gb drive, I asked about boosting the product above the 8gb because I did not feel it was enough. The sales person said 8 was enough and they didn't have any upgrades for 5100.
I don't know when the "marked "Requires SSD/HDD"" was posted but at that time I probably would not have seen it even if it was posted. I stopped using NtopNG about 6 mo's ago but would like to run it again.
That linked list also mentions DarkStat. I do run that.
Thanks for this info.
-
@rcoleman-netgate said in /var is low on disk space:
Unfortunately for @azdeltawye, but (and I think I speak for both of us here) we hope to be fortune for others, this very same 5100 is the one I mentioned above where I extracted the eMMC from the board and it immediately started perform normally.
I am glad that my old 5100 has a chance at a second life.
I'm happy to help the cause!! -
@JonH said in /var is low on disk space:
Referring to the later post with the photos of the removed eMMC can you clarify for me:
If an mSATA is installed it will supersede the eMMC, essentially making it electronically 'disappear'?No. The BIOS POST has a step we haven't been able to find a way around -- a read-only eMMC will cause the system to boot very slowly (15-30 minutes to POST) or not at all.
And the 32 GB SATAIII M.2 2242 SSD would be a suitable replacement?
Probably.
As an aside, my 1st 5100 is still laying around, but if I read correctly, adding a mSATA probably won't help because if the eMMC is the cause then it's too late. Correct?
If, yes. If this is the case, and you are in the United States, and you are interested in having it used for a testing/trial process LMK and I can open DMs for you.
Is this likely to be a similar situation with my old 2440 w/ 20GB drive? Or is that a totally different situation? I know this may be impossible to answer, just fishing on the off chance I can get a working emergency spare. I'm ok with soldering but I don't have a re-soldering station, just a solder sucker and copper braid. And noticeably shakier hands than in the 'good old days'.
The RED LED of death? That's a CPU issue on the C2000 systems. The 5100 is not a C2000.
-
@SteveITS said in /var is low on disk space:
@rcoleman-netgate "...do not boot nor provide an output via the serial cable." Would that be similar for the 4860 model we discussed elsewhere? IIRC that doesn't have eMMC or at least can't use the "write life" software. Plus I'd expect the backup router to have way less disk writes.
The red LED of death is a C2K CPU issue. In theory you could use the same technique to replace the CPU but ... I am not about to do that experiment.
-
@rcoleman-netgate Hmm, no LED for me, just fan noise. Just curious. Never mind. :)
-
@rcoleman-netgate said in /var is low on disk space:
In theory you could use the same technique to replace the CPU but ... I am not about to do that experiment.
Hahaha. After replacing my wife's iPad battery I swore I'd never dive into these things again. And I didn't have to use my soldering iron for that one.
thanks
-
@SteveITS Then I don't know. I haven't had access to any of the C2K devices personally. You could give it a try if you have a reworking station.
-
@rcoleman-netgate said in /var is low on disk space:
interested in having it used for a testing/trial process
Can you explain what you mean by this?
-
@JonH said in /var is low on disk space:
Can you explain what you mean by this?
@stephenw10 and I have a theory that we might be able to bypass the POST check for the dead eMMC without removing the chip from the system -- which requires a reworking station to perform.
Finding 'dead' 5100s has been difficult. I've only so far gotten my hands on one (@azdeltawye's former unit) and it now sits on my desk as a testing unit.
-
@rcoleman-netgate said in /var is low on disk space:
Finding 'dead' 5100s has been difficult. I've only so far gotten my hands on one
Ryan:
OK, this sounds interesting to me, better than the landfill.
For my working 5100 I just ordered "Transcend 32GB SATA III 6GB/S MTS400S 42 mm M.2 SSD 400S Solid State Drive TS32GMTS400S" off of Amazon cause Sales said they no longer have any parts for 5100. I hope this is the correct drive.
The Netgate docs page on '5100 sata install' mentions "8. Locate the Thermal Pads that came with the SG-5100."
After all these years I cannot find the box (believe it or not, I usually keep them) so I have no thermal pads. Can you advise if they are necessary and on what I can look for in order to purchase? I asked Sales and they couldn't help me with that.
Finally, after requesting an image, how can I get pfSense loaded on the sata? I misplaced my notes on image install. Would this be via Etcher? I did this once before when I changed fs to zfs but I don't recall exactly what I did nor whether I used Etcher (I must have cause I have the app installed on my iMac). Will the sata boot first once it's installed?
I'll get back to you in a week or so on the dead 5100.
-
@JonH said in /var is low on disk space:
after requesting an image, how can I get pfSense loaded on the sata?
They (quite probably Ryan) will send you instructions with the file but they are also here:
https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/reinstall-pfsense.htmlPer the other page https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/m-2-sata-installation.html "By default, the M.2 SATA drive will be the first drive recognized..."
@rcoleman-netgate said in /var is low on disk space:
You could give it a try if you have a reworking station
We don't. I just would have expected a BIOS screen level boot attempt so it caught my eye that the 5100 doesn't output anything if the storage dies. It's unlikely to be storage due to wear, since it was our backup router so probably has 1% of the usage of the main. We have replacements arriving today anyway.