pfBlockerNG-devel v3.1.0_7 / v3.1.0_14

fireodo

@bbcan177

Hi,

after the update (3.1.0_6 -> 3.1.0_7) two problems arise:

When in a DNSBL group I like to save a local feed I get this error:

[PFB_FILTER - 2] Invalid URL (not allowed2) [ /root/database/youtubeads.txt ] [ 12/9/22 08:56:50 ]

Running Cron Update I get this error on this list:

[ EasyList ] Downloading update .. 200 OK
[PFB_FILTER - 17] Failed or invalid Mime Type: [text/x-asm|0]

Edit: A few lists get marked as empty even the source feed contains lots of IPs.

Thanks for looking over it,
fireodo

Gertjan

@fireodo said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

When in a DNSBL group i like to save a local feed I get this error:

You've created a local file, like /root/test.txt with some domain names :

-rw-r--r--  1 root  wheel  127 Dec  9 10:11 /root/test.tx

Contains :

test-domaine.fr
ykpbgktjmubxdig.cn
ykpgnuokpcemmhl.com
ykpredvcmvlcaql.com
ykpunetdy.cf
ykqjiqolrykupmn.com
ykrpwrhoonediyv.com

Added it to the DNSBL Groups like :

and

This gets loaded for me :

[ TEST_local_file_test ]	 Downloading update [ 12/9/22 10:18:41 ] ..
  ----------------------------------------------------------------------
  Orig.    Unique     # Dups     # White    # TOP1M    Final                
  ----------------------------------------------------------------------
  7        7          6          0          0          1                    
  ----------------------------------------------------------------------

Note : only one, the first domain name "test-domaine.fr" , get retained, as the other 6 are a copy of another DNSBL feed, just to create a x lines 'home made DNSBL local file'

and blocked (first domain in the list ) :

Here a trick : go here : /var/db/pfblockerng/dnsblorig and delete your-file.orig - and reload pfblockerng

fireodo

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

Added it to the DNSBL Groups like :

I even can save the group:

The URL is: /root/database/ioam.txt

I think that is a validation issue.

Gertjan

@fireodo

Language issue ?

The text in the red block means : you can NOT save that info.
Btw : I'm dutch, s I can read (decode) German.
It says : line 15 in your file has an issue, as it can not get resolved.

So, please, don't let us wait : what is this line 15 ?
An "umlaut" or whatever 'illegal' german character in a host name ?
File enconding issue like UTF8 or 16 ?

jdeloach

@bbcan177
I updated pfBlockerNG_devel v3.1.0_6 to v3.1.0_7 with no issues. Everything appears to be working okay.

Thanks for all the hard work you put into maintaining/upgrading this Great package.

fireodo

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

So, please, don't let us wait : what is this line 15 ?

its a local file:
/root/database/ioam.txt

fireodo

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

Language issue

No, in 3.1.0_6 there was no issue!
(I do not use anywhere on pfsense special characters/Umlaut)

Gertjan

@fireodo said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

its a local file:
/root/database/ioam.txt

Yeah, I know.
It's a file.
A files has lines.
Line 15 of that file has an issue.

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

So, please, don't let us wait : what is this line 15 ?

edit :
I'm using

for several weeks now.
pfSense 22.05.

fireodo

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

Line 15 of that file has an issue.

Not the line 15 in the file (btw. its a domain name) - the error refers to the feed position in the group!

Line 15 is the position in the group of the feed that points to the local file! (Changing the feed position in the group results in change of the line number in the error)

Gertjan

@fireodo

Ah, ok.

Then what is different ?
Can you create a file called /root/test.xt - as I showed above ?
Can you place the content, as I showed above (7 lines or so) ?
Add the file as a "DNSBL Groups" entry" as I've showed above ?

We both use the same pfBlocker-ng version.

fireodo

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

Then what is different ?

I guess the way the script validates the feed url.

Can you create a file called /root/test.xt - as I showed above ?

I did.

Can you place the content, as I showed above (7 lines or so) ?

Yes

Add the file as a "DNSBL Groups" entry" as I've showed above ?

Yes - same error (the content of the file doesnt matter at this point)

We both use the same pfBlocker-ng version.

You under 22.05 and me under 2.6.0 ...

Gertjan

Remember :

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

Here a trick : go here : /var/db/pfblockerng/dnsblorig and delete your-file.orig - and reload pfblockerng

Because (I think) : pfblockerng doesn't reload the file, whatever you change in the settings.
It will reload a file or URL after one hours, two hours or whatever you've set.
To really for it : delete the 'orig' files or even all of them : they will get reloaded. As this worked for me.

fireodo

@gertjan said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

As this worked for me.

Because (i guess) your v3.1.0_7 on 22.05 is more like v3.1.0_6 was on 2.6.0 ...

BBcan177

@fireodo said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

[PFB_FILTER - 2] Invalid URL (not allowed2) [ /root/database/youtubeads.txt ] [ 12/9/22 08:56:50 ]

You need to move these files to the "/usr/local/www" or "/var/db/pfblockerng/" folder, if you intend to keep hosting these files on pfSense. Click on the Blue infoblock icon for the Source Definitions for more details.

Running Cron Update I get this error on this list:

[ EasyList ] Downloading update .. 200 OK
[PFB_FILTER - 17] Failed or invalid Mime Type: [text/x-asm|0]

The new code now checks the file magic to determine to the file mime-type.

There is an issue in decoding the magic for some Easylist feeds. I suggest you use the Easylist Feeds that are in the Feeds tab as I have allowed some exceptions until the magic database is updated.

I worked with the Dev to address this issue here:
https://github.com/file/file/commit/31ee773f9cb78ff584872456ea1f139081a01761

Just need for that to be updated into FreeBSD and then into pfSense.

fireodo

@bbcan177 said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

@fireodo said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

[PFB_FILTER - 2] Invalid URL (not allowed2) [ /root/database/youtubeads.txt ] [ 12/9/22 08:56:50 ]

You need to move these files to the "/usr/local/www" or "/var/db/pfblockerng/" folder, if you intend to keep hosting these files on pfSense. Click on the Blue infoblock icon for the Source Definitions for more details.

Ah - i thought that it was just a example in the infoblock - I moved now the files in /var/db/pfblockerng and could save the settings - and works.

Running Cron Update I get this error on this list:

[ EasyList ] Downloading update .. 200 OK
[PFB_FILTER - 17] Failed or invalid Mime Type: [text/x-asm|0]

The new code now checks the file magic to determine to the file mime-type.

There is an issue in decoding the magic for some Easylist feeds. I suggest you use the Easylist Feeds that are in the Feeds tab as I have allowed some exceptions until the magic database is updated.

OK understood - I'll do so!

I worked with the Dev to address this issue here:
https://github.com/file/file/commit/31ee773f9cb78ff584872456ea1f139081a01761

Just need for that to be updated into FreeBSD and then into pfSense.

Thanks a lot and regards,
fireodo

provels

It seems I've had v3.1.0_7 on 22.05 for some time. Not sure how that happened. I probably got a txt saying there was an update available a while back and installed it from the shell. Is this a re-release that needs to be reinstalled? No biggie, everything works, though I did need to start unbound after.

emikaadeo

@provels said in pfBlockerNG-devel v3.1.0_7 / v3.1.0_14:

It seems I've had v3.1.0_7 on 22.05 for some time.

Same here but the packages are definitely not the same.
Comparing file /usr/local/pkg/pfblockerng/pfb_dnsbl.doh.conf on my pfSense+ 22.05 and the one from github commit are different.

EDIT: I think we should wait https://www.reddit.com/r/pfBlockerNG/comments/zg9ipo/comment/izjaun5

BBcan177

@emikaadeo
What hardware?
amd64, arm64, or armv7

emikaadeo

@bbcan177
amd64

SteveITS

@provels 3.1.0_7 came out in late October or thereabouts, to fix the CPU usage/logging bug.
https://redmine.pfsense.org/issues/13154#note-17