pfBlockerNG v3.2.0_10 unable to download .txt blocklists

[fluvannait]

Prior to updating pfSense+ to 24.03 and pfBlockerNG to v3.2.0_10 I was able to use block lists from Blocklist Project on GitHub. Since the update, none of their lists work with pfBlockerNG because the URLs are links to .txt files like this (https://blocklistproject.github.io/Lists/alt-version/tiktok-nl.txt).
When forcing an update this is the error:

[ BL_TikTok ] Downloading update [ 05/3/24 10:33:53 ] .. 200 OK
[PFB_FILTER - 17] Failed or invalid Mime Type: [text/x-file|0]

[ DNSBL_BL_TikTok - BL_TikTok ] Download FAIL [ 05/3/24 10:35:03 ]
DNSBL, Firewall, and IDS (Legacy mode only) are not blocking download.

If I paste the link in a browser the contents display in the browser and doesn't try to download the tiktok-nl.txt file. I prefer the Blocklist Project lists because of their TikTok and Gambling blocklists. I have a mandate to block TikTok on all of my network, but the URLs of all of the blocklists I've found end in .txt and they all fail with the same error as above. Does anyone have any insight into this issue? In order to block TikTok I've had to copy the contents of the TikTok list and past it into the DNSBL Custom_List.
I forgot to mention that this problem exists across the the seven psSense+ appliances that I manage. Six of which are Netgate devices (1100, 2100, 3100, 4100, 6100 and two 7100's) and one Protecli Vault.

[fluvannait]

I also forgot to mention that this behavior only became apparent when I replaced an existing Netgate 1100 with a Netgate 2100 at one of my sites when the update to 24.03 didn't go well on the 1100 and I had to install pfBlockerNG from scratch. The downloads failed the first time I forced the update. On all of my other appliances, if they already had the blocklists in question they don't throw the error, but they don't ever update the lists. Any forced update (or cron job) they just show that the list(s) exists, but they never actually update. If I delete one of the blocklists, update and add the same blocklist back, it fails with the same error.

[BBcan177]

@fluvannait on each download, the downloaded file mime type is validated to ensure that it's an allowed file type.

It uses the file command and a magic database to determine the file mime type. However, I have seen it report an incorrect file mime type (ie: for easylists).

For now, you can manually edit /usr/local/pkg/pfblockerng/pfblockerng.inc and add that "text/x-file" mime type around line #259.

Reference:
https://github.com/BBcan177/FreeBSD-ports/blob/4de1e3c4e66d4e0bb510403cf06d99023478e087/net/pfSense-pkg-pfBlockerNG/files/usr/local/pkg/pfblockerng/pfblockerng.inc#L259

Will see about a longer term solution.

[michmoor]

@BBcan177 Does the edit of the .inc survive a reboot of the firewall or re-install of the package?
Is it saved as part of the .XML in any way in case one needs to restore a firewall?

[BBcan177]

@michmoor it will be replaced if you reinstall the package or perform a pfSense Upgrade.

[fluvannait]

@BBcan177 Thanks! That did the trick. I appreciate the quick response.