pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!

HuskerDu

Thanks BBcan177 for your great work,

For info, get yellow icon for DNSBL widget due to following error.

2020-11-27 21:33:25,736|ERROR| [pfBlockerNG]: Failed get_q_name_qinfo: in method '_get_qname', argument 1 of type 'struct query_info *'

BBcan177

@HuskerDu
Do you have any of these options enabled in the DNS Resolver?

DHCP Registration:

Static DHCP
OpenVPN Clients

If so, disable those options, clear the py_error log and Force Reload - DNSBL and see if the error returns.

HuskerDu

@BBcan177
The options were removed after I've read the doc, but first launch after the install must have triggered the error.

Thanks, clearing the log file solved the yellow icon "issue".

Regards,

A Former User

@BBcan177

with v3.0.0 load on my Firewall is significant reduced when running large Set of DNSBL Lists (~3 Million Domains).

I don't see the unbound-control process spinning up every couple of seconds anymore.

Well Done

fmroeira86

@BBcan177

Thank you for your effort!

Everytime I do a RELOAD the skew on the backup CARP gets its value set to 0.

This causes the backup machine holding as CARP master when the Master machine comes online again.

I'm running 3.0.0_2

BBcan177

@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

Thank you for your effort!
Everytime I do a RELOAD the skew on the backup CARP gets its value set to 0.
This causes the backup machine holding as CARP master when the Master machine comes online again.
I'm running 3.0.0_2

Did you update the package on all Nodes?
Did you set the SKEW on all nodes correctly?

The package should not be syncing the SKEW to the other nodes.

See the relevant code here:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9349-L9352

provels

@bbcan177
I got an alert today that an update from 3.0.0_2 => 3.0.0_3 was available. I had already updated to 3.0.0_3.
But my package manager now shows 3.0.0_2 as current and wants me to revert.
Any problem here? Thanks for reading.

BBcan177

@provels
No Idea. That is outside of the pkg as its part of the pfSense package manager.

asmodeus

@bbcan177 After upgrading to 3.0.0_3 whitelisting domains from the list of blocked domains (e.g. /pfblockerng/pfblockerng_alerts.php?filterdnsbl=DNSBL_Malicious2) no longer worked - getting 504 Gateway Time-out errors after selecting the domain to whitelist and providing the options (wildcard or not etc.)

The entire UI is unresponsive after that and I can get it back up again via ssh with /etc/rc.php-fpm_restart. I have rolled back to 3.0.0_2 but I still have this issue.

Any ideas on what the issue might be?

provels

@bbcan177
Must have been a personal problem. Rebooted the FW, GUI still offered upgrade to _2 when _3 was already installed. Did pkg upgrade from shell, offered _2 to _3. Went ahead and installed, seems fine now. Go figure.

BBcan177

@asmodeus said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

@bbcan177 After upgrading to 3.0.0_3 whitelisting domains from the list of blocked domains (e.g. /pfblockerng/pfblockerng_alerts.php?filterdnsbl=DNSBL_Malicious2) no longer worked - getting 504 Gateway Time-out errors after selecting the domain to whitelist and providing the options (wildcard or not etc.)
The entire UI is unresponsive after that and I can get it back up again via ssh with /etc/rc.php-fpm_restart. I have rolled back to 3.0.0_2 but I still have this issue.
Any ideas on what the issue might be?

There are two different things here.

Whitelisting
Filtering the Alerts tab.

The URL that you posted is for Filtering events, not whitelisting? Write out the steps you took so I can try to reproduce.
It could just be a browser timeout issue if you are trying to view a lot of events? In Alert Settings, what did you set the DNSBL Events number to?

fmroeira86

@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

Thank you for your effort!
Everytime I do a RELOAD the skew on the backup CARP gets its value set to 0.
This causes the backup machine holding as CARP master when the Master machine comes online again.
I'm running 3.0.0_2

Did you update the package on all Nodes?
Did you set the SKEW on all nodes correctly?

The package should not be syncing the SKEW to the other nodes.

See the relevant code here:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9349-L9352

Yes.

I'm running the latest version and My main node ovewrites the skew to the secondary one.

BBcan177

@fmroeira86

Ok I see the issue.

As reference:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9350-L9351

Can you edit the file: /usr/local/pkg/pfblockerng/pfblockerng.inc, Lines 9350 and 9351

From:

// Remove CARP Advskew setting, if found
if (isset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew'])) {
    unset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew']);
}

To:

// Remove CARP Advskew setting, if found
if (isset($xml['pfblockerngdnsblsettings' ]['config'][0]['pfb_dnsvip_skew'])) {
   unset($xml['pfblockerngdnsblsettings']['config'][0]['pfb_dnsvip_skew']);
}

I missed adding ['config'][0] to each of those two lines... Sorry about that...

asmodeus

@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

The URL that you posted is for Filtering events, not whitelisting? Write out the steps you took so I can try to reproduce.

Sure:

Coming from the pfBlockerNG dashboard on my Status/Dashboard page, I use the alerts link to review blocked domains in a list. The link points to /pfblockerng/pfblockerng_alerts.php?filterdnsbl=DNSBL_Malicious2
Using this list (which takes a while to load) I want to whitelist a domain, so I click on the 'plus' sign to start the process.
I specify wildcard no, add description no.
Page loads for a while and ultimately I am getting the nginx timeout. (Even when this worked it takes 110 seconds or more to complete)
The page I attempted to whitelist is NOT whitelisted.
The entire pfsense UI is unresponsive at this point and I resolve by restarting php-fpm via ssh

The Log Settings are set to default (20000 lines), and the Alert Settings are set to default:1000 Stat Table entries and 100 Filter Limit. Are these the Alert Settings you were asking about?

BBcan177

@asmodeus said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

and the Alert Settings are set to default:1000 Stat Table entries and 100 Filter Limit. Are these the Alert Settings you were asking about?

From the Reports/Alerts tab, click the "+" on the right side of the page to open the "Alert Settings", the default for "Filter Limit Entries" is 100.

If you set it too high, you need to wait for the whole page to finish loading before trying to Whitelist.

There is also a new option in the Reports tab to "Pause" the auto page refresh. Its at the left side of each table heading.

asmodeus

@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

From the Reports/Alerts tab, click the "+" on the right side of the page to open the "Alert Settings", the default for "Filter Limit Entries" is 100.

Got it, that's what Filter Limit Entres is set to for me - 100. Is there a chance to speed up loading of this page before/after whitelisting?

fmroeira86

@bbcan177 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

@fmroeira86

Ok I see the issue.

As reference:
https://github.com/pfsense/FreeBSD-ports/blob/devel/net/pfSense-pkg-pfBlockerNG-devel/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L9350-L9351

Can you edit the file: /usr/local/pkg/pfblockerng/pfblockerng.inc, Lines 9350 and 9351

From:
// Remove CARP Advskew setting, if found
if (isset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew'])) {
    unset($xml['pfblockerngdnsblsettings']['pfb_dnsvip_skew']);
}
To:
// Remove CARP Advskew setting, if found
if (isset($xml['pfblockerngdnsblsettings' ]['config'][0]['pfb_dnsvip_skew'])) {
   unset($xml['pfblockerngdnsblsettings']['config'][0]['pfb_dnsvip_skew']);
}
I missed adding ['config'][0] to each of those two lines... Sorry about that...

Thank you.

Is this fix going to be released in the next version?

BBcan177

@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

Thank you.
Is this fix going to be released in the next version?

Once you confirm that it works :)

fmroeira86

@fmroeira86 said in pfBlockerNG-devel v3.0.0 - No longer bound by Unbound!:

if (isset($xml['pfblockerngdnsblsettings' ]['config'][0]['pfb_dnsvip_skew'])) {
unset($xml['pfblockerngdnsblsettings']['config'][0]['pfb_dnsvip_skew']);

Made the changes you suggested but it is not working.
Skew still gets to 0 on the secondary machine.

Bob.Dig

This post is deleted!