pfsense unresponsive (resolved)
-
Huh... As in my tests pfBNG never ran more than 2-3min it is strange. The cron was run at 4:16 and SWAP got killed at 4:27. That would indicate 11min... Do the other logs show anything in that timeframe besides pfBNG? Otherwise pfB should have some logs about its cron/update, perhaps that will shed some light into it.
-
In my experience, the update process has always taken circa 15 minutes to run. The system continues to be responsive and the only issue is DNS not resolving for approximately a minute or so at the very end whilst Unbound is reloaded.
The process today seems to have failed at the “validating databases” portion.
I have copied into the attached file (its quite lengthy) the pfblockerNG log for today and at the very foot (under ##########) the previous days log from this point onward so one can see what should have subsequently happened.
It strikes me that something within this process (a combination of circumstances is causing an issue, but not consistently) and was why I was looking to see if it was possible to activate some form of enhanced logging within pfblockerNG.
-
You are pushing the limit with 1.2M entries for DNSBL on a 8GB system. Remove 100000 to 200000 entries to get more free memory.
You could use BBC_DGA_High/381953 instead of BBC_DGA/875641.
-
Ron, thanks for your observation and remedial suggestion.
I have been running this setup for some time (>12 months) and it is only recently that I have encountered issues with pfsense running out of memory.
The one thing I think slightly odd, is that if the number of entries for DNSBL was excessive I would expect it to consistently hang, whereas its seems to be random, which makes me wonder (as previously stated) if there another issue at play (accepting I am probably stressing the system with the load).
I have coincidentally within the last month or so added an additional spamhaus drop list feed to the pfblockerNG configuration, so as an initial step I have removed this. I think it will be a question of wait and monitor.
If the problem returns I shall revisit my lists as per your recommendation.
-
@farrina said in pfsense unresponsive:
I have been running this setup for some time (>12 months) and it is only recently that I have encountered issues with pfsense running out of memory.
Well those lists grow constantly, so sooner or later they can hit a point where your hardware reaches it's limits.
The one thing I think slightly odd, is that if the number of entries for DNSBL was excessive I would expect it to consistently hang, whereas its seems to be random, which makes me wonder (as previously stated) if there another issue at play (accepting I am probably stressing the system with the load).
There are other processes on pfSense that will use increased memory from time to time, so it's always going to be a bit random. If one process is close to the border of the memory limits and nothing else is happening it's going to work, if another process currently needs more memory at that moment something will fail.
So either reduce the load on the hardware or upgrade to hardware that can take the load.
-
Thanks for taking the trouble to respond to my query with your thoughts.
I shall report back in due course with any developments,
-
Well problem has reoccurred, symptoms as previously described.
I have disabled the BBC_DGA feed which has reduced the number of entries on the DNSBL list from circa 1.2 million to 464K so this should (in theory) free up significant memory during pblockerNG updates each morning.
I'm struggling to locate your recommended feed (BBC_DGA_High) do you have a link by any chance ?
Cheers
-
@farrina https://osint.bambenekconsulting.com/feeds/
https://osint.bambenekconsulting.com/feeds/dga-feed-high.gz -
Thanks Ron
-
Just a quick follow up to my post to advise that my "problem" has not reoccurred since taking RonpfS advice to reduce the number of entries in my pfblockerNG block files.
Once again my thanks to all who took the trouble to respond.
Cheers
