Sustained Unbound write I/O

[keyser]

@keyser I'm sorry that I'm ressurecting my old thread on this topic again, but I just installed 22.01 (ZFS reinstall) on my 6100 and that in turn updated pfBlockerNG to the latest version 3.1.0_1 version.

I'm sorry to report that has brought back the Unbound disk write issue (with the same config).
My box went from doing about 130KB/s writes to about 550KB/s now. About 30% of that comes curtesy of the ZFS filesystem, but still it's at least a tripeling of Unbound diskwrites...

I'm beginning to wonder if Netgate considered the wear ZFS would cause on the small eMMC. appliances. If the write endurance rating is "industry standard" on the built-in eMMc, this level of write IO will kill my box within 2 years (And that's only because it's a 16Gb SG-6100).
The same config in my SG-2100 will kill the eMMC within a year.

I think we have a huge problem here....

[slu]

@keyser
did you find any solution?

Same here, disable all I can find, unbound still writing.

[keyser]

@slu No, not specifically. It is related to doing python mode integration to unbound as you discovered.
In my case, I eventually removed pfBlockerNG without saving the config.
After reinstalling the latest 3.2.0.x and configuring it from scratch, the deadly writing levels never reappeared. So I sort of concluded it might have been crud from previous installs/upgrades that caused pfBlocker to behave like that.

[slu]

@keyser said in Sustained Unbound write I/O:

After reinstalling the latest 3.2.0.x and configuring it from scratch, the deadly writing levels never reappeared.

With python unbound mode or without the deadly writing never reappeared?

[keyser]

@slu said in Sustained Unbound write I/O:

With python unbound mode or without the deadly writing never reappeared?

The issue did not reappear with Python mode enabled - which is what you want as Python mode has some nice advantages.

[slu]

@keyser said in Sustained Unbound write I/O:

The issue did not reappear with Python mode enabled [...]

That's very interesting, no idea who to find out what trigger this issue.

[keyser]

@slu Not that I could find… I sort of concluded that somehow the python script had ended up doing all of its temporary data handling on actual disk rather than in a memory based datastructure. I have nothing to base that conclusion on as I painstaikingly tried to determine what file(s) all the writing was done into. But there was never any files that changed or grew during this - neither did pfSense Swap… so….

[slu]

@keyser
not easy to debug.

@NOCling you use python mode as well, right?
Do you see this as well?

[NOCling]

Yes, i use it.
But i run a RAM Disk to prevent Unbound killing my SSD.
I use a UPS and NUT on the pfsense, so no unexpected Power loss and the RAM Disk is safely written to the SSD.

[slu]

@NOCling said in Sustained Unbound write I/O:

Yes, i use it.
But i run a RAM Disk to prevent Unbound killing my SSD.

ah ok, I can't use RAM Disk in my setup.
So you know about the Unbound write problem?

[w0w]

@NOCling
Сould you please suggest the correct configuration for a RAM disk? Every time I've tried to set it up, it ended with messages about running out of space or other issues. I have 16GB of memory; is that generally enough?

[NOCling]

I use this settings on my 6100, but it depends on the list if its ok or not:

[Bob.Dig]

@NOCling I lose my pfBlocker aliases if I try the RAM-disk. Any hints?

[Bob.Dig]

@NOCling Do I have to wait til Friday to get an answer?

[mcury]

@Bob-Dig said in Sustained Unbound write I/O:

@NOCling I lose my pfBlocker aliases if I try the RAM-disk. Any hints?

force update in pfblockerNG and you will get your aliases back.

[Bob.Dig]

@mcury Yes, but after the next reboot is was gone again. So maybe I have to tick a box to save something? But saving will not be good for the health...
I do reboot daily. "Reboot does good" they say around my place.

[mcury]

@Bob-Dig said in Sustained Unbound write I/O:

@mcury Yes but after the next reboot is was gone. So maybe I have to tick a box to save something? But saving will not be good for the health...
I do reboot daily. "Reboot does good" they say around my place.

you can install cron and shellcmd packages.
check cron and get the command used to update pfblockerNG, them add it to shellcmd package to run after boot.

[Bob.Dig]

@mcury said in Sustained Unbound write I/O:

check cron and get the command used to update pfblockerNG, them add it to shellcmd package to run after boot.

Thanks. Have you edited the command? It doesn't seem to work for me. No files are shown when looking in logs in pfB. Only running "reload" manually will bring them back.

[mcury]

@Bob-Dig said in Sustained Unbound write I/O:

@mcury said in Sustained Unbound write I/O:

check cron and get the command used to update pfblockerNG, them add it to shellcmd package to run after boot.

Thanks. Have you edited the command? It doesn't seem to work for me. No files are shown when looking in logs in pfB. Only running "reload" manually will bring them back.

this one works for me, but I'm not using DNSBL, so I don't know the DNSBL entry, but should be something similar.

/usr/local/bin/php /usr/local/www/pfblockerng/pfblockerng.php cron

[Bob.Dig]

@mcury Looking in the firewall rules, the aliases do work. But I can't see any files in the log-section of pfB. After I run reload manually, I see those files... Don't know what to think about it. Shellcmd doesn't seem to make any difference about that.