@Gertjan Thanks.

Maybe one core isn't that bad. We'll see.

Looking good so far.

Is there a way to keep the list contents and disable future updates in pfBlockerNG? I selected Never for Update Frequency but I'm still getting download errors. Under DNSBL Source Definitions there is an option to place lists on HOLD, would this allow them to continue working without updating any more?

I don't see any harm in keeping the list active at least until another similar replacement can be found.

@Artes Yup just checked that out. Thanks again.

@T-Monster This is really weird. When I started to go through the setup again with the use of a website describing the process, I saw that what I was seeing for my pfBlockerNG was different from what was on the website. I went back to the package manager, searched on pfBlockerNG and found a much newer version, which I installed. Everything seems fine now. I have no idea where the older version of pfBlockerNG came from.

@BBcan177 Well that is definitely an answer! I had no idea that MaxMind thought it was also in Brazil. IDK if this is right or wrong or if I should even be blocking so much in my firewall as there are datacenters all over the world (these are outbound rules) but I dont use Bing and that IP seems to be a Microsoft "bingbot" according to Google. If the next release shows this better in the logs then I am happy. Thanks so much!

Output from those commands on my box.

mmdblookup -f /usr/local/share/GeoIP/GeoLite2-Country.mmdb -i 191.232.139.2 country iso_code "IE" <utf8_string> grep "191.128.0.0" /usr/local/share/GeoIP/* /usr/local/share/GeoIP/GeoLite2-Country-Blocks-IPv4.csv:191.128.0.0/12,3469034,3469034,,0,0 grep "3469034" /usr/local/share/GeoIP/GeoLite2-Country-Locations-en.csv 3469034,en,SA,"South America",BR,Brazil,0 mmdblookup -f /usr/local/share/GeoIP/GeoLite2-Country.mmdb -i 191.232.139.2 { "continent": { "code": "EU" <utf8_string> "geoname_id": 6255148 <uint32> "names": { "de": "Europa" <utf8_string> "en": "Europe" <utf8_string> "es": "Europa" <utf8_string> "fr": "Europe" <utf8_string> "ja": "ヨーロッパ" <utf8_string> "pt-BR": "Europa" <utf8_string> "ru": "Европа" <utf8_string> "zh-CN": "欧洲" <utf8_string> } } "country": { "geoname_id": 2963597 <uint32> "is_in_european_union": true <boolean> "iso_code": "IE" <utf8_string> "names": { "de": "Irland" <utf8_string> "en": "Ireland" <utf8_string> "es": "Irlanda" <utf8_string> "fr": "Irlande" <utf8_string> "ja": "アイルランド" <utf8_string> "pt-BR": "Irlanda" <utf8_string> "ru": "Ирландия" <utf8_string> "zh-CN": "爱尔兰" <utf8_string> } } "registered_country": { "geoname_id": 3469034 <uint32> "iso_code": "BR" <utf8_string> "names": { "de": "Brasilien" <utf8_string> "en": "Brazil" <utf8_string> "es": "Brasil" <utf8_string> "fr": "Brésil" <utf8_string> "ja": "ブラジル連邦共和国" <utf8_string> "pt-BR": "Brasil" <utf8_string> "ru": "Бразилия" <utf8_string> "zh-CN": "巴西" <utf8_string> } } }

I started noticing very weird blocked functions in my home network. My music server would not resolve http(s) get requests for streaming music, though I could on other devices. I could not resolve a domain name by typing the base URL in the address bar (like typing purple.com and get the page to load), but I could find it in a search bar and navigate to them and perform nslookups on them. I updated the main pfSense OS, but could not update any packages. Other weirdness that was not easy to categorize.

I tried to reboot the pfSense via the web and command menu - neither worked. I forced a hardware reboot and noticed many repeated errors on the console and system logs which read:
pfr_update_stats: assertion failed

Here is my solution, which is working for now:

In the Firewall / pfBlockerNG / General page, tick the box for "Suppression - This will prevent Selected IPs from being blocked. Only for IPv4 lists (/32 and /24)." and click the Save button for this page. In the Firewall / pfBlockerNG / General page, untick: Enable pfBlockerNG and Keep settings to disable them both. then click the save button. In the Firewall / pfBlockerNG / General page, tick only the Enable pfBlockerNG to enable and leave the Keep settings unticked/disabled. In the Firewall / pfBlockerNG / Update page, select the Select 'Force' option Update and click Run. Copy the output from this into a text file.

Use the text file to separate the results to find four types of results:

No Domains Found

Terminated - Easylists can not be used

Anything which is not working, such as a 404 page not found or other error

Anything working can be ignored

Open EVERY DNS Group Name on the Firewall / pfBlockerNG / DNSBL Feeds page. Search for any of the feeds that are NOT working at all and paste the URL into a browser bar. If they do not resolve, delete them - don't forget to click the save button at the bottom. If they do resolve, see the next step

Open EVERY DNS Group Name on the Firewall / pfBlockerNG / DNSBL Feeds page. Search for any of the feeds that are listed as No Domains Found, or that did resolve to a list in a previous step, and paste the URL into a browser bar. If the list is just a bunch of IP addresses, then you have them on the wrong part of your firewall! To fix this:

Copy the URLs of any of the lists which were IP-based out of the DNSBL page and into a text file as a placeholder.

Move over to the Firewall / pfBlockerNG / IPv4 page and start a new Alias Name (or edit one you may already have there). Add each one of the URLs from your text file, giving it a unique header name (the last field) and make sure to set it to Auto & ON. .

Once all added, I set the List Action and update schedule to my preference and saved the page

For anything which results in 'Terminated - Easylists can not be used' I do not yet have a solution.

In the Firewall / pfBlockerNG / General page, untick: Enable pfBlockerNG and Keep settings to disable them both. then click the save button. In the Firewall / pfBlockerNG / General page, tick only the Enable pfBlockerNG to enable and leave the Keep settings unticked/disabled. In the Firewall / pfBlockerNG / Update page, select the Select 'Force' option Update and click Run. Repeat the process of reviewing the results to remove broken lists and move IP-based lists to the right IPv4 list page.

------ONCE satisfied with the results:

In the Firewall / pfBlockerNG / General page, untick: Enable pfBlockerNG and Keep settings to disable them both. then click the save button. In the Firewall / pfBlockerNG / General page, tick BOTH the Enable pfBlockerNG and Keep settings to enable them both. In the Firewall / pfBlockerNG / Update page, select the Select 'Force' option Update and click Run.

Sounds like this issue: #10414

Workaround: #901871 but maybe the firewall table is to small to use pfBlocker after the change...

@Gertjan Yeah, you got it all wrong, probably because of my English-writing-skills. 😉

@Co6aka I'm in the same boat as you. Upgraded from 2.4.4 to 2.4.5. Wound up with "Cannot allocate memory" errors & only the firewall could access the internet. Uninstalling/reinstalling pfBlocker_NG gets the LAN back online (I know it isn't a pfBlocker issue).

My table entries were at 20 million before upgrading - because I have a lot of lists and some of them are massive (each list does have a purpose). I think I worked up to 60M entries before setting this aside for the night.

I haven't tried breaking apart my lists into smaller aliases. After reading the relevant posts here and on Reddit, it doesn't seem likely to help. It'd still be the same number of IPs that need allocation.

(wild guess coming) Unless the issue is that the structures holding my massive aliases are buckling under the load. But, heck. I don't know.

I'm going to sleep on it. Maybe tomorrow I'll puzzle out where I should be looking for clues. Otherwise, I'll have to check into rolling back - wait for bigger brains to set our world right (yet) again.

Edit: box has 4GB RAM

Q: How do I calculate Firewall Maximum Table Entries (assume 100MB in aliastables dir)

Edit.2: I haven't been able to find a fix. Going to roll back.
and
I'm fairly impressed w/ the difficulty of locating a download link for
pfSense-CE-2.4.4-RELEASE-p3-amd64.iso.gz
Not giving up!

Edit3: Found a copy of 2.4.4 on Linuxtracker.org (not affiliated).
Installed a fresh copy. Restored from the backup I made using 2.4.5 (because, you know) and that worked just fine. Everything came right up; no issues at all.

I'm all good again. I'm also scared of upgrading any of my boxes to 2.4.5 but what can you do.
I still appreciate all the work that goes into this.

@RonpfS Found it. Thank you. Still haven't found the specific alert I'm looking for, but I at least know where to dig. Note that I haven't been looking all this time. I got side tracked doing something else. Thanks again for your help.

@revengineer
The is a log snippet above that to show the processing of that feed and the restart of Unbound. Take a look at those two sections of the pfblockerng.log.

FWIW: did a fresh install. Still core faults if I reset logs.

@BBcan177 Excellent! Working as expected now. Maybe someday there can be a button or comment explaining how to re-download from MaxMind because I didnt even know the command did that when I was looking at it. Thanks! Keep up the good work!

@Core7
I don't think bridging will work well with the pkg. I also have no other first hand experience doing that sorry.

No its for all versions

@GregBinSD said in Shallalist and UT1 lists not working on 2.4.5-RELEASE/pfBlockerNG-devel 2.2.5_29:

Can you tell me how long that might be?

The pfSense devs need to review and approve. Hopefully next week.

@Gertjan @t41k2m3

Thank you for the details. I’ll make the jump to the -devel package first then.

Are there any specific posts/blogs you would recommend to get up to speed on any critical changes or potential gotchas that might extend my maintenance window?

My router is usually hovering around 3% CPU and 19% memory utilization with pfblocker, squid, squidguard, snort, and a few other pkgs running. these stats are with no inbound OpenVPN client tunnels active or outbound IPsec VPN to my Oracle Cloud IaaS tenancy up. Still, plenty of resource capacity.