Help Needed: Upgrade to 2.4.3: /var/run filesystem full
-
Hi there, upgraded from 2.3.5 to 2.4.3 and am getting:
da0: Attempt to query device size failed: NOT READY, Medium not present da0: quirks=0x2 <no_6_byte>pid 119 (php-cgi), uid 0 inumber 7 on /var/run: filesystem full Failed to write core file for process php-cgi (error 28)</no_6_byte>
The system locks up serial and I'm stuck.
Running on a pcengines apu with a 24gb msata.
Not quite sure how to troubleshoot outside of doing a mass reinstall. If that's necessary, I'll go for it but hoping someone can help me with a different approach. Also, backups were not taken pre-upgrade. If re-install is necessary how might I go about pulling configs?
Thanks.
-
See what's taking up space on /var/run. It's probably something left over that shouldn't be there, nothing very large should.
Try:
du -ah /var/run
Steve
-
Thanks! How do I get into a shell if I get a locked up on boot?
-
Ah, so it did upgrade but only then showed that error…
It depends how far though the process you are. If you see the boot menu you can try booting to single user mode.
If not you might reach a shell from the error by escaping the script with ctl-c. No promises there though.
Steve
-
Indeed. I tried booting into single user with```
boot -s -
Ah, found https://doc.pfsense.org/index.php/Can_I_restore_a_backup_without_the_WebGUI which answers my second question.
-
Yes the recover config script in 2.4 is much improved. A full /var/run would not stop the config being read.
The only worry is that the filesystem may not actually be full, a bad drive can also sometimes present like that.
Steve
-
I'm having a ton of trouble mounting the disk. I'm starting to think it's hosed but I can still "ls" pfsense's boot (though I can't drop into a shell) any thoughts on how I might be able to get it to mount?
-
all good. via: http://ghantoos.org/2009/04/04/mounting-ufs-in-readwrite-under-linux/
-
So you did get the disc mounted. Since I have the same problem on one of 4 FWs I would bee interested to hear if you found out what took the space away or if it was another problem?
-
Yes, I'd be interested to see what rogue file filling /run.
Steve
-
As for my nightshift. I have talked with the support. There is no good way finding what happened at update. The suggestion was always have a serial console log running when you do the upgrade. Then you have a chance on catchen what was wrong.
In my situation there where only haproxy and securitata installed.
I ended up grabbing the last konfig (since I but carp in mainanance) from the old installation while booting from a usb stick with 2.4.3 on it.
I then did a reinstall and restored the config. Everything is now working.