pfBlockerNG-devel v3.1.0_9 / v3.1.0_15

planedrop

@thexman I'll give this a shot and see how it goes, thank you!

JeGr

after upgrading to _9 installations that utilized the IOC lists from ThreatFox / abuse.ch
(https://threatfox.abuse.ch/export/)
won't download the list anymore. Be it the JSON file from the last 48 hours or the full data dump (zipped with "zip" not gzip), it always ends in a MIME Type Error:

[ Abuse_ThreatFox_v4 ]		 Downloading update .. 200 OK.
 PFB_FILTER - 18 | pfb_download Failed or invalid Mime Type Compressed: [application/x-decompression-error-gzip-Unknown-compression-format|0]

or

[ Abuse_ThreatFox_48h_v4 ]	 Downloading update .. 200 OK
[PFB_FILTER - 17] Failed or invalid Mime Type: [application/json|0]

 [ pfB_PRI1_v4 - Abuse_ThreatFox_48h_v4 ] Download FAIL
  DNSBL, Firewall, and IDS (Legacy mode only) are not blocking download.

That only happened recently after upgrading, before it was running fine with _6 or _7 I believe the systems were on. The old list from before the update was/is still being used so it worked before.

List URLs are working via shell/curl or in browser so no problem on that front. It's only when trying to download it with pfB that those Mime Type errors pop up.

Edit: Edit: Curl in shell sees normal content types: content-type: application/json or content-type: application/zip so no clue where that failed or invalid types come from.

Cheers
\jens

fireodo

@jegr

Hi and a good New Year.

Read this: Mime Types

I guess it adress your problem.

Cheers,
fireodo

BBcan177

@jegr
The latest code now validates the contents of all Compressed files before extraction to ensure that the file-mime type is allowed.

There is a currently an incompatibility with ZIP files and the 'file' mime-type magic database that validates the Compressed file type contents as "application/x-decompression-error-gzip-Unknown-compression-format".

I tried to see if the file maintainer could add functionality to fix this, but unfortunately I had no luck with that. So for now, I have no way to validate the ZIP file contents before extraction, so in the next version it will first Extract ZIP compressed files, and then perform the file-mime type validation on the extracted file. I will continue to see if I can find a way to validate before extraction.

The second part is that the next version will add "application/json" as a valid mime-type.

Thanks for the report!

Draco

@bbcan177 said in pfBlockerNG-devel v3.1.0_9 / v3.1.0_15:

The second part is that the next version will add "application/json" as a valid mime-type.

Does that mean a URL like MSFT Azure IP Blocks will be a valid download source? At present, I have to download this on my PC then upload it to /var/db/pfblockerng/deny on my pfSense box.

Note the MSFT Azure IP Blocks link sends you to a page where the file download starts; that page contains a link to the most current file (at present, https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public_20230102.json).

[edit: added more info on links]

sensei-two

Hi
I was reading about the DoH/DoT/DoQ Blocking feature in DNSBL SafeSearch of my
pfBlockerNG-devel v3.1.0_9.
As far as I have understood it, it blocks clients on my LAN to use DoH/DoT, so
I was wondering if this feature can also affect DoT queries from Unbound itself since I enabled DoT in its setting.

Thanks

xpxp2002

This post is deleted!

xpxp2002

@sensei-two I believe it does. I use unbound to funnel all public DNS queries to Cloudflare over DoT. But I took a different approach to blocking all other DoT/DoH.

I have pfB-NG creating a deny alias using these blocklists:
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall_ipv4
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall_ipv6

Then I created rules on the LAN-side interfaces that destination blocks the aliases that these blocklists create.

Unbound should be using your WAN interface to reach your DoT provider and it won't have an ingress interface (i.e. there's no LAN-to-WAN or WAN-to-LAN flow). The only way I think you could control the firewall's own egress-to-WAN would be using a floating rule.

sensei-two

@xpxp2002

I also use Unbound to funnel DNS queries to Cloudflare over DoT,
so, in your opinion, if I now enable the DoH/DoT/DoQ Blocking feature in pfng something might be wrong with my "legitimate" DNS queries over DoT, right?

I am interested in your approach to block DoT from LAN's clients, but I new to pfblockerNG, and I didn't use pfSense for a long time either, so bear with me, please.

Could you tell me how to set the deny alias using the blocklists above, please?
Should I delete my floating rules if I create the new rules on the LAN-side?
My floating rules:

Thanks

xpxp2002

@sensei-two It's been a while since I looked at the DoT/DoH blocking in pfB-NG, but I thought it creates floating rules, which would impact the unbound service's egress out of the WAN interface (as well as any other interface) unless you configure the floating rule to exempt the WAN address. The problem I see with that is that you can't generally edit auto-generated rules because they are overwritten after an update cron job.

Maybe that has since been fixed in pfB-NG? I don't know for sure. I can only speak to my solution, which I have tested and know works in my environment for the DoT and DoH providers in those blocklists.

I have pfBlocker-NG creating these aliases. These can probably be Alias Native instead of Alias Deny. My understanding of de-dup and reputation should be irrelevant in my config. IPv4 details shown here. I have an identical IPv6 one, as well.

Then assign these aliases to block rules near the top of your LAN side, before anything that might allow them.

sensei-two

@xpxp2002

Ok, I think I have understood how to set them, and
I'm going to give them a go.
This way you can do without the DoH/DoT/DoQ Blocking feature in DNSBL SafeSearch, can't you?
Out of curiosity. Did you also try the DoH/DoT/DoQ Blocking feature? If so, did you stumble upon some issues because of it?
Thanks

xpxp2002

@sensei-two I honestly don't remember. I probably did. I have a couple subnets where I do allow less restricted outbound access, so it's possible I did this just to have more control.

Looking at the feeds, I also threw these into the custom IP lists (IPv4 and IPv6, respectively) at the bottom since they weren't covered by the feeds themselves.
1.0.0.2/31
1.1.1.2/31
2606:4700:4700::1002
2606:4700:4700::1003
2606:4700:4700::1112
2606:4700:4700::1113

I noticed you mentioned DoQ, as well. For completeness, here's how I'm blocking everything DNS-related on my guest subnet except for my unbound resolver.

sensei-two

@xpxp2002

Very helpful. Thank you very much indeed

sensei-two

This post is deleted!

sensei-two

This post is deleted!

sensei-two

@xpxp2002

I came up with these firewall rules:

I had already set a NAT rule for dns redirection which makes its work:

In order to test the DoH/Dot block rules I enabled DoT in the Firefox browser of one on my clients, but it seems that it doesn't work.
The client gets access to webpage nonetheless.
What did I get wrong? Thanks

UPDATE!

I also see this log:

maybe the browser is being redirect to port 53 if DoH queries can't be resolved because of the new pf_rules I added.
Any thought?

JeGr

@sensei-two said in pfBlockerNG-devel v3.1.0_9 / v3.1.0_15:

maybe the browser is being redirect to port 53 if DoH queries can't be resolved because of the new pf_rules I added.
Any thought?

If you are talking about Browsers using DoH - there normally should be fallback to a system's DNS setting if no DoH can be established for exactly that case. AFAIR Firefox etc. have a setting to disallow that to "avoid" being redirected to a "bad/controlled DNS" but in a home/company setting that's exactly what you want because otherwise many internal domains/services are unavailable (as they are often non-public DNS entries). But that discussion has become way out of scope of that post here, where the topic is about the new version and problems/bugs that may happen with it. Please put posts about usage/config problems of a feature in a separate post (as that feature wasn't introduced in version 3.1.0_9 / _15) for better visibility :)

Cheers
\jens

sensei-two

@jegr
I might be wrong, but it is exactly what seems to have happened with Firefox too: it switched to my system DNS.
Anyway, as you say, I'll post about the issue elsewhere, to dive deeper into it possibly.
Thanks

smolka_J

I am copying this over another user noticed in 3.1.0_1 that I am seeing still currently present in 3.1.0_9. I have IPv4 whitelists that carried over from pfSense 21.05 my box shipped with, used to be able to edit and add IPs when I needed. Now I find that unless I am whitelisting IPs via the alerts tab itself, if I try to edit any of these custom lists manually I am facing this same "Warning: When using an Action setting of 'Permit Inbound or Permit Both', you must configure at least one of 'Advanced Inbound Custom Port/Destination' settings." as well as a "Improper Permit rules on the WAN can catastrophically impact the security of your network!" error detected. This is also verified when trying to create a new rule from scratch, even with attempting to set IPv4 main screen interface rules config for inbound rules to LAN instead of WAN as it was and should be:

Tueurdragon wrote on https://forum.netgate.com/topic/176439/error-on-permit-inbound-rule-ipv4-part :

Error on Permit Inbound rule IPv4 part
pfBlockerNG 1 2 192
TTueurdragon Dec 13, 2022, 11:19 AM
Error on Permit Inbound rule IPv4 part

Hi there,

I have a problem creating a Permit Inbound rule in the PFBlockerNG-devel module IPv4 part.

Indeed, I want to create a Whitelist before all the GEOIP blocking rules.

SETTING part
Here are the things that I provide:
Action: Permit Inbound
Update Frequency: never
Weekly: Monday
Auto-Sort Header field: Enable auto-sort
Enable logging: Enabled
States Removal: Enabled

Part Advanced Inbound Firewall Rules Settings
Custom DST Port: checkbox check , in the input field I enter an alias
Custom Destination: checkbox check, in the input field I enter an alias
Custom Protocol: TCP
Custom Gateway: I choose my gatewaygroup because I have several WANs in failover

Unfortunately I always get this error

The following input errors were detected:

Warning: When using an Action setting of 'Permit Inbound or Permit Both', you must configure at least one of 'Advanced Inbound Custom Port/Destination' settings.
===> WARNING <===
Improper Permit rules on the WAN can catastrophically impact the security of your network!
And the Custom DST Port and Custom Destination input fields are cleared.

Can you help me?
Thanks in advance,

0
TTueurdragon 28 days ago
Small clarification,

I have another PFSENSE firewall with the PFBlocker-NG module in version 3.1.0_1 and it works but obviously it no longer works on the PFBlocker-NG version 3.1.0_7

SteveITS

@smoke_a_j said in pfBlockerNG-devel v3.1.0_9 / v3.1.0_15:

When using an Action setting of 'Permit Inbound or Permit Both', you must configure at least one of 'Advanced Inbound Custom Port/Destination' settings

There was a different thread about this a few weeks ago. If you allow all inbound traffic to your WAN IP on all ports then the IPs specified can access pfSense via SSH, HTTPS (web GUI), DNS, etc. Meaning, attackers can continually guess passwords until they get in.

The warning is so you don't allow that.

If you really wanted to open your pfSense to the Internet you could create these as Alias Native which only creates the alias, and then create your own rules on the WAN interface using that alias.