PfSense 2.0.1 locking up (hard)
-
I have a SM-based pfsense with a D510 and a pair of Intel (em) on-board NICs. MBUF usage grows continually until exhausted, followed by a full lockup. The "kern.ipc.nmbclusters="131072"" setting puts your MBUF limit much higher than default and will postpone or prevent lockup, depending on how much you need. I think you can always set that number higher if you have the RAM to back it.
Thanks for the note… it's good to hear from people with similar hardware.
I have the same MB at home (not for this install though, I got the D525 for the client) -- and THEN bought the D510 since I had parts for it laying around and don't need the mem speed for it.
Your proposition gives me the willies. What? Am I supposed to schedule a reboot every weekend? Turn on the BMC watchdog? (need to test that at home first) Yikes.
I do have 4GB in the unit -- but I am running pfSense-2.0.1-embedded_AM64 off an SSD... so RAM isn't exactly wild-free-and-open. But, I'll bump up the bufs later.
right not seems to have a fairly static 5574/25600 and the client is getting their full 50Mb/s BW to the internet. So overall, provided the system stays up, we're pretty happy.
As always, the open source community does some excellent stuff.
stay tuned,
-Ben
-
Theoretically the MBUF usage should level off at some point. Mine are at 34652/131072 and growing with 21 days of uptime. This is approximately the same uptime I had 21 days ago when one of the NICs stopped passing traffic, so I have yet to see my MBUF usage level off on this system. Mind you I have hundreds of users, so I can only guess that that is a factor, because my internet connection is less than 40 Mbps.
-
I should add that increasing your max MBUFS is not going to put much of a dent in 4G of RAM. My system has 4G and no swap and is currently reporting 10% memory usage. My current states are reporting at 11795/389000 but I've seen them as high as 34000. I think memory is allocated on your max rather than current, so you can see that increasing your nmbclusters isn't likely to be a problem on your system.
-
Theoretically the MBUF usage should level off at some point. Mine are at 34652/131072 and growing with 21 days of uptime. This is approximately the same uptime I had 21 days ago when one of the NICs stopped passing traffic, so I have yet to see my MBUF usage level off on this system. Mind you I have hundreds of users, so I can only guess that that is a factor, because my internet connection is less than 40 Mbps.
Hmmm…
This user has 50Mb/s and about 3,000 PC's (Mac/Windows Mix - but mostly probably 90% Mac).
I just need to watch it. So far, the CPU has been pretty sleepy according to RRD. Peaks of 25% with most daily usage under 20%.
(I'm assuming that's system total consider this is a dual-core w/HyperThreading mb)Very nice.
The state table I've seen peak around 20,000 of 390,000. So.Also nice.
Thanks for the data. I'll keep sharing as I watch this systems first day in "production" mode. (scaaaarrryyy... hahahah)
-Ben
-
I should add that increasing your max MBUFS is not going to put much of a dent in 4G of RAM. My system has 4G and no swap and is currently reporting 10% memory usage. My current states are reporting at 11795/389000 but I've seen them as high as 34000. I think memory is allocated on your max rather than current, so you can see that increasing your nmbclusters isn't likely to be a problem on your system.
Agreed. I didn't think adding more MBUFs would be a big impact either. (thanks for the note though)
Cheers,
-ben
-
BTW,
Does MBUF's recent over idle time or do they allocate and then (ideally) stick at some average level?
(i.e. what kind of behavior should I see if everything were running correctly?)
-Ben
-
I've never seen them decrease except with a reboot. I think best case is that they level off and stop increasing at some point.
-
K.
Good to know - I'll be watching.
Thanks,
-ben
-
I have a SM-based pfsense with a D510 and a pair of Intel (em) on-board NICs. MBUF usage grows continually until exhausted, followed by a full lockup. The "kern.ipc.nmbclusters="131072"" setting puts your MBUF limit much higher than default and will postpone or prevent lockup, depending on how much you need. I think you can always set that number higher if you have the RAM to back it.
Forgive my possibly noobish question, but how do I set this setting?
Do I just add it on the: "System -> Advanced -> System Tunables" page?
Does this setting require a reboot of the system?
If no, does changing the setting drop any current states?
Much obliged,
Matt -
I have a SM-based pfsense with a D510 and a pair of Intel (em) on-board NICs. MBUF usage grows continually until exhausted, followed by a full lockup. The "kern.ipc.nmbclusters="131072"" setting puts your MBUF limit much higher than default and will postpone or prevent lockup, depending on how much you need. I think you can always set that number higher if you have the RAM to back it.
Forgive my possibly noobish question, but how do I set this setting?
Do I just add it on the: "System -> Advanced -> System Tunables" page?
You have to look at the link above. There's a file in /boot you need to create and put those parms in.
Does this setting require a reboot of the system?
If no, does changing the setting drop any current states?Yes. it needs a reboot. (so that should answer the 2nd part of that question)
It worked for me with no issues.. and I'm seeing a level off at about 7200 (7174) of 131072 after a day over heavy operation.
Shout if you need more,
-Ben
-
Some settings can be set from the system tunables page, however this one cannot. You must add it to /boot/loader.conf.local (create the file if it does not exist) and then reboot for it to take effect. Do not add it to /boot/loader.conf, as this will be overwritten on the next system upgrade.
-
Hi!
I have similar problem.
About month ago, when I updated to 2.0.1. version I have got situations, when pfsense freeezees.
Pfsense is installed on old HP E800 series server with PIII 1ghz CPU, 512Mb RAM and 17Gb SCSI hdd.
Firstly I though, that it is from bad RAM or HDD, but after changing them, everything continues.
I setted up another hardware, simple pc with 1,7Ghz Celeron Intel CPU, 768Mb RAM and 40Gb hdd.
Two days everything was ok, but then in one day it stops 3 TIMES!!
Then I thought, that it has got something common with ACPI, even I have got UPS.
Turned off ACPI support in pfsense.
Few days everything was OK after turning of ACPI in pfsense, but this night again it stops.
And I am a little bit desperate.
If hardware change doesn't help, it could be MBUFs?
Right now it is = 0.
In pfsense first page it shows 518/24896 MBUFs usage.Any Ideas?