pfBlockerNG-devel v3.1.0_19/10

Draco

@bbcan177 said in pfBlockerNG-devel v3.1.0_19/10:

Add "application/json" to list of allowed file download mime-types

I had hoped this might let pfBlocker directly download a JSON list like the one found at Microsoft Azure IPs. This is a file I manually download and then use pfSense's GUI CMD interface to upload for pfBlocker (I set the format to AUTO). Ran this on 3.1.0_11 just now.

It didn't work. So what JSON-related things were enabled with this change?

Thanks!

nimrod

It happened again after after update.

This is the content of py_error.log

2023-01-24 16:36:57,206|ERROR| [pfBlockerNG]: Failed to load python module 'maxminddb': No module named 'maxminddb'
2023-01-24 16:36:57,206|ERROR| [pfBlockerNG]: Failed to load python module 'sqlite3': No module named '_sqlite3'

Despite these errors, everything is working fine.

cmcdonald

@nimrod

What is the output of:

pkg info py* unbound

renegade

@cmcdonald
Same problem on my side.

[22.05-RELEASE][admin@firewall.home]/root: pkg info py* unbound
pkg: No match.

cmcdonald

@renegade

Sorry, try this:

pkg info "py*" unbound

nimrod

@cmcdonald said in pfBlockerNG-devel v3.1.0_19/10:

@renegade

Sorry, try this:

pkg info "py*" unbound

Here it is:

[2.6.0-RELEASE][admin@pfSense.home.arpa]/root: pkg info "py*" unbound
py38-ply-3.11
py38-setuptools-57.0.0
py39-maxminddb-2.0.3
py39-setuptools-57.0.0
py39-sqlite3-3.9.9_7
python38-3.8.12_1
python39-3.9.9
unbound-1.13.2

cmcdonald

@nimrod Thanks. I see the problem. Testing a fix. Standby

cmcdonald

@nimrod can you also share pkg info unbound ?

nimrod

@cmcdonald said in pfBlockerNG-devel v3.1.0_19/10:

@nimrod can you also share pkg info unbound ?

Of course. Here it is:

[2.6.0-RELEASE][admin@pfSense.home.arpa]/root: pkg info unbound
unbound-1.13.2
Name           : unbound
Version        : 1.13.2
Installed on   : Mon Jan 31 21:24:27 2022 CET
Origin         : dns/unbound
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Categories     : dns
Licenses       : BSD3CLAUSE
Maintainer     : jaap@NLnetLabs.nl
WWW            : https://www.nlnetlabs.nl/projects/unbound
Comment        : Validating, recursive, and caching DNS resolver
Options        :
	DEP-RSA1024    : off
	DNSCRYPT       : off
	DNSTAP         : off
	DOCS           : off
	DOH            : on
	ECDSA          : on
	EVAPI          : off
	FILTER_AAAA    : off
	GOST           : on
	HIREDIS        : off
	LIBEVENT       : on
	MUNIN_PLUGIN   : off
	PYTHON         : on
	SUBNET         : off
	TFOCL          : off
	TFOSE          : off
	THREADS        : on
Shared Libs required:
	libexpat.so.1
	libnghttp2.so.14
	libpython3.8.so.1.0
	libevent-2.1.so.7
Shared Libs provided:
	libunbound.so.8
Annotations    :
	FreeBSD_version: 1203500
	build_timestamp: 2022-01-12T15:27:10+0000
	built_by       : poudriere-git-3.3.99.20211130
	cpe            : cpe:2.3:a:nlnetlabs:unbound:1.13.2:::::freebsd12:x64
	port_checkout_unclean: no
	port_git_hash  : 8df9544dcbab
	ports_top_checkout_unclean: yes
	ports_top_git_hash: 7046b65c0d41
	repo_type      : binary
	repository     : pfSense
Flat size      : 7.99MiB
Description    :
Unbound is designed as a set of modular components, so that also
DNSSEC (secure DNS) validation and stub-resolvers (that do not run as
a server, but are linked into an application) are easily possible.

Goals:
    * A validating recursive DNS resolver.
    * Code diversity in the DNS resolver monoculture.
    * Drop-in replacement for BIND apart from config.
    * DNSSEC support.
    * Fully RFC compliant.
    * High performance, even with validation enabled.
    * Used as: stub resolver, full caching name server, resolver library.
    * Elegant design of validator, resolver, cache modules.
          o provide the ability to pick and choose modules.
    * Robust.
    * In C, open source: The BSD license.
    * Smallest as possible component that does the job.
    * Stub-zones can be configured (local data or AS112 zones).

Non-goals:
    * An authoritative name server.
    * Too many Features.

WWW: https://www.nlnetlabs.nl/projects/unbound

cmcdonald

@nimrod Can you now try reinstalling pfBlockerNG-devel on 22.05/2.6, and repeat the above command pkg info "py*" unbound

nimrod

@cmcdonald said in pfBlockerNG-devel v3.1.0_19/10:

@nimrod Can you now try reinstalling pfBlockerNG-devel on 22.05/2.6, and repeat the above command pkg info "py*" unbound

I reinstalled it and here is the output:

[2.6.0-RELEASE][admin@pfSense.home.arpa]/root: pkg info "py*" unbound
py38-maxminddb-2.0.3
py38-ply-3.11
py38-setuptools-57.0.0
py38-sqlite3-3.8.12_7
py39-maxminddb-2.0.3
py39-setuptools-57.0.0
py39-sqlite3-3.9.9_7
python38-3.8.12_1
python39-3.9.9
unbound-1.13.2

cmcdonald

@nimrod That should be correct now. Clear the unbound errors and try again.

nimrod

@cmcdonald said in pfBlockerNG-devel v3.1.0_19/10:

@nimrod That should be correct now. Clear the unbound errors and try again.

Yup. That fixed it. Thank you sir.

BBcan177

@draco said in pfBlockerNG-devel v3.1.0_19/10:

I had hoped this might let pfBlocker directly download a JSON list like the one found at Microsoft Azure IPs. This is a file I manually download and then use pfSense's GUI CMD interface to upload for pfBlocker (I set the format to AUTO). Ran this on 3.1.0_11 just now.

The Link you posted is the HTML page. You need to use the direct link:

https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public_20230123.json

Keep in mind that this will parse all IPs in the json file. You could also create a new shell script to parse this JSON and get more refinement on which IPs to pull ( "Advanced Tunables - Post-Script Script" feature.)

yorke

@bbcan177

I figure out why i was getting those errors some package/feature on pfsense needed to be update (ie unbound and about 4 others ) once I ran the update and reboot and reinstall
PfblockerNG work, no more errors.
Thanks BBcan177

bigjohns97

@cmcdonald I am seeing the same error about missing python modules on 23.01 RC, was this fixed on that version as well?

nimrod

@bigjohns97 said in pfBlockerNG-devel v3.1.0_19/10:

@cmcdonald I am seeing the same error about missing python modules on 23.01 RC, was this fixed on that version as well?

Yes.

Draco

@bbcan177 said in pfBlockerNG-devel v3.1.0_19/10:

he Link you posted is the HTML page. You need to use the direct link:
https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public_20230123.json

Fair enough -- this means I will need to manually update the link each time, but better than copying the file from my computer up to pfSense each time, thanks!

I might have to write a screen-scraper to pull the latest URL off the download page...

bigjohns97

@nimrod Can you confirm what add-on's I should see because they differ than what is posted above.

pkg info "py*" unbound

py311-maxminddb-2.2.0_2
py311-setuptools-63.1.0
py311-sqlite3-3.11.1_8
py39-libzfs-1.1.2022081600
py39-setuptools-63.1.0
py39-yaml-5.4.1
python311-3.11.1_1
python39-3.9.15
unbound-1.17.0

nimrod

@bigjohns97

Is this before or after pfblocker reinstall ?