pfBlockerNG-devel pfsense 23.05.1
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
is this supposed to work on a Netgate 3100?
Yes, it does.
Look :
You saw the Count : mine is 196,981 ? That means that that list has "196,981" entries, or host names or 196981 lines.
Let me check :
The file, as pfBlocker downloaded it : https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (download it yourself, and count the lines, and get the size - for me its about 6,6 Mbytes).pfBlocker transforms this file for its internal use : it's here : /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt - now it's 11,4 Mbytes.
The thing is : The GUI, and pfBlocker, uses 'PHP' to handle the file creation, handling, parsing.
The error :
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 122616096 bytes)
Tells us that PHP (while executing 'something' in /usr/local/pkg/pfblockerng/pfblockerng.inc) tried to allocate (reserve) 134217728 (bytes) while PHP can only offer maximum 122616096 (bytes).
That was a no-go.Fast solution : don't use very big feeds or lists.
=> How do you know what list/feed is to big ?Example :
You see the URL ?
Open it in a browser, save the file and check ^^Next solution : To use big things, you need big equipment. Now you know why people use a Xenon Processor for their firewall, and install 16 Gbytes of ram, and fast, multiple SSD's.
And test before you decide https://www.netgate.com/appliances - go to the right and don't stop before the end ;)
Even the biggest Netgate appliance can have PHP out of memory errors, even with huge amounts of RAM, because the PHP process allocates just a part of the available RAM, not all the RAM.
There will always be an upper ceiling.Btw : A Netgate 3100, isn't that a device with an ARM processor ?
Do get a Intel Iron next time, even if you managed to download all these (many ?) feeds/lists, you'll be needing the horse power to process all these lists.
One of the tasks is : put all the files together (yes !) and remove entries that are listed more then one.
Remember : check the Firewall > pfBlockerNG > Update page, and do a manual force all update.
You want this process to go as fast as possibleFinal solution : if pfBlockerng was written in native language, like 'C', then it would be a binary executable. Only then you will able to handle Gigabytes files or more.
That is : if your download link, Internet connection, can support these type of files, as you have to download them also ... -
@Gertjan thank you for sharing your experience, are you using pfBlockerNG or pfBlockerNG-devel?
Actually I've setup an external solution that worked at first try:
https://forum.netgate.com/topic/182369/pi-hole/4I've removed all packages, will try again the whole config.
Thinking about resource usage it could be a good solution to keep filter host outside the SG3100 :
-
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Only then you will able to handle Gigabytes files or more.
is your 3100 a base 8GB or the upgraded 32GB?
Would be also interested in that actual version of pfBlockerNG you are running .. I see you mention 23.05.1 and pfBlockerNG-devel but not noticing you mention the release of that?
On a 2100 - no problem
not sure what being an ARM processor has to do with it...
The files size is fine.
-rw-r--r-- 1 root wheel 12613611 Aug 23 00:30 StevenBlack_ADs.txtI have one other small list so the overall count is a little higher: but still
all of var db is only 16M
and there is no memory crush
What mode are you running in "ubound" or "ubound python" mode ?
- maybe provide us page view of your DNSBL settings.
What other packages might you be running?
Do you have the current patch package installed?
-
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
is your 3100 a base 8GB or the upgraded 32GB?
base version
Now I'll try again from zero:
- System > Packages, Available Package > System_Patches 2.2.5.
- System > Packages, Available Package > pfBlockerNG-devel
- System > Patches: no entryes for pfblocker
- Firewall > pfBlockerNG Setup
UPDATE PROCESS START [ v3.2.0_6 ] [ 08/23/23 15:18:42 ] ===[ DNSBL Process ]================================================ ===[ DNSBL Virtual IP and/or Ports are not defined. Exiting ]====== Clearing all DNSBL Feeds Unbound stopped in 1 sec. Starting Unbound Resolver... completed [ 08/23/23 15:18:44 ] DNSBL update [ 0 | PASSED ]... completed ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] Downloading update .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 115 115 115 [ Pass ] ----------------------------------------------------------------- [ Abuse_SSLBL_v4 ] Downloading update [ 08/23/23 15:18:45 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 31 31 31 [ Pass ] ----------------------------------------------------------------- [ CINS_army_v4 ] Downloading update .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 15000 14999 14999 [ Pass ] ----------------------------------------------------------------- [ ET_Block_v4 ] Downloading update [ 08/23/23 15:18:47 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 1564 1453 1453 [ Pass ] ----------------------------------------------------------------- [ ET_Comp_v4 ] Downloading update [ 08/23/23 15:18:49 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 521 497 497 [ Pass ] ----------------------------------------------------------------- [ ISC_Block_v4 ] Downloading update [ 08/23/23 15:18:50 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 20 5 5 [ Pass ] ----------------------------------------------------------------- [ Spamhaus_Drop_v4 ] Downloading update [ 08/23/23 15:18:52 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 989 0 0 [ Pass ] ----------------------------------------------------------------- [ Spamhaus_eDrop_v4 ] Downloading update [ 08/23/23 15:18:53 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 348 336 336 [ Pass ] ----------------------------------------------------------------- [ Talos_BL_v4 ] Downloading update [ 08/23/23 15:18:54 ] .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 3609 3586 3586 [ Pass ] ----------------------------------------------------------------- Unable to apply rules. Outbound interface option not configured. ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_PRI1_v4 1 table created.21023 addresses added. ** Starting firewall filter daemon ** ===[ FINAL Processing ]===================================== [ Original IP count ] [ 22195 ] [ Final IP Count ] [ 21022 ] ===[ Deny List IP Counts ]=========================== 21023 total 14999 /var/db/pfblockerng/deny/CINS_army_v4.txt 3586 /var/db/pfblockerng/deny/Talos_BL_v4.txt 1453 /var/db/pfblockerng/deny/ET_Block_v4.txt 497 /var/db/pfblockerng/deny/ET_Comp_v4.txt 336 /var/db/pfblockerng/deny/Spamhaus_eDrop_v4.txt 115 /var/db/pfblockerng/deny/Abuse_Feodo_C2_v4.txt 31 /var/db/pfblockerng/deny/Abuse_SSLBL_v4.txt 5 /var/db/pfblockerng/deny/ISC_Block_v4.txt 1 /var/db/pfblockerng/deny/Spamhaus_Drop_v4.txt ====================[ Empty Lists w/127.1.7.7 ]================== Spamhaus_Drop_v4.txt ====================[ IPv4/6 Last Updated List Summary ]============== Aug 22 01:24 Spamhaus_Drop_v4 Aug 22 06:30 ET_Block_v4 Aug 22 22:22 ET_Comp_v4 Aug 23 12:38 Spamhaus_eDrop_v4 Aug 23 14:15 ISC_Block_v4 Aug 23 14:18 CINS_army_v4 Aug 23 15:10 Abuse_Feodo_C2_v4 Aug 23 15:10 Talos_BL_v4 Aug 23 15:15 Abuse_SSLBL_v4 =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 21023 /var/db/aliastables/pfB_PRI1_v4.txt pfSense Table Stats ------------------- table-entries hard limit 9000000 Table Usage Count 22236 UPDATE PROCESS ENDED [ 08/23/23 15:19:03 ]
-
Restarted widzard and changed ports, found this old post and updated config:
UPDATE PROCESS START [ v3.2.0_6 ] [ 08/23/23 15:57:52 ] ===[ DNSBL Process ]================================================ Saving new DNSBL web server configuration to port [ 8082 and 8442 ] Stopping Unbound Resolver. Unbound stopped in 2 sec. Starting Unbound Resolver... completed [ 08/23/23 15:57:56 ] DNSBL is disabled ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] exists. [ Abuse_SSLBL_v4 ] exists. [ CINS_army_v4 ] exists. [ ET_Block_v4 ] exists. [ ET_Comp_v4 ] exists. [ ISC_Block_v4 ] exists. [ Spamhaus_Drop_v4 ] exists. [ Spamhaus_eDrop_v4 ] exists. [ Talos_BL_v4 ] exists. ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload No Changes to Aliases, Skipping pfctl Update ===[ Kill States ]================================================== No matching states found ====================================================================== UPDATE PROCESS ENDED [ 08/23/23 15:57:59 ]
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
DNSBL Virtual IP and/or Ports are not defined. Exiting
above from your post.
then from the follow up post..
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
DNSBL is disabled
likely won't do anything if it is disabled.
try it enabled ..
then let's confirm if the error returns (I'd leave the Resolver Live Sync unchecked for now) (I don't have that option because I run python mode)The issue is likely configuration related,
I don't have a 3100 so hard for me do exact configuration, but you might find this helpful.
the link is referencing an 1100 which is smaller still (in terms of Ram)
https://forum.netgate.com/topic/179185/php-memory-allocation-error-in-pfblockerng-dnsbl
Clearly the first screen capture suggest the list is at least loading on the 1100 and showing a count of 195,149 (likely correct based on the date)
at the same time my 2100 has not been changed and is working. In this regard.
// Set memory limit to 512M on amd64. if ($ARCH == "amd64") { ini_set("memory_limit", "512M"); } else { ini_set("memory_limit", "128M"); } -
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
likely won't do anything if it is disabled.
thanks for point it out it was already enabled,then I've disable, now it's enabled again:
iginal Master Final ------------------------------ 31 24 24 [ Pass ] ----------------------------------------------------------------- [ CINS_army_v4 ] exists. [ 08/23/23 16:00:24 ] [ ET_Block_v4 ] exists. [ ET_Comp_v4 ] exists. [ ISC_Block_v4 ] Downloading update .. 200 OK. completed .. ------------------------------ Original Master Final ------------------------------ 20 5 5 [ Pass ] ----------------------------------------------------------------- [ Spamhaus_Drop_v4 ] exists. [ 08/23/23 16:00:26 ] [ Spamhaus_eDrop_v4 ] exists. [ Talos_BL_v4 ] exists. ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_PRI1_v4 8 addresses deleted. ===[ Kill States ]================================================== No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 22199 ] [ Final IP Count ] [ 21014 ] ===[ Deny List IP Counts ]=========================== 21015 total 14999 /var/db/pfblockerng/deny/CINS_army_v4.txt 3586 /var/db/pfblockerng/deny/Talos_BL_v4.txt 1449 /var/db/pfblockerng/deny/ET_Block_v4.txt 497 /var/db/pfblockerng/deny/ET_Comp_v4.txt 336 /var/db/pfblockerng/deny/Spamhaus_eDrop_v4.txt 118 /var/db/pfblockerng/deny/Abuse_Feodo_C2_v4.txt 24 /var/db/pfblockerng/deny/Abuse_SSLBL_v4.txt 5 /var/db/pfblockerng/deny/ISC_Block_v4.txt 1 /var/db/pfblockerng/deny/Spamhaus_Drop_v4.txt ====================[ Empty Lists w/127.1.7.7 ]================== Spamhaus_Drop_v4.txt ====================[ IPv4/6 Last Updated List Summary ]============== Aug 22 01:24 Spamhaus_Drop_v4 Aug 22 06:30 ET_Block_v4 Aug 22 22:22 ET_Comp_v4 Aug 23 12:38 Spamhaus_eDrop_v4 Aug 23 14:18 CINS_army_v4 Aug 23 15:10 Talos_BL_v4 Aug 23 15:15 ISC_Block_v4 Aug 23 15:55 Abuse_SSLBL_v4 Aug 23 16:00 Abuse_Feodo_C2_v4 ====================[ DNSBL Last Updated List Summary ]============== Aug 23 15:39 StevenBlack_ADs Aug 23 15:50 UT1_malware Aug 23 15:50 UT1_phishing Aug 23 15:50 UT1_publicite Aug 23 15:50 UT1_reaffected Aug 23 15:50 UT1_tricheur =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 21015 /var/db/aliastables/pfB_PRI1_v4.txt pfSense Table Stats ------------------- table-entries hard limit 9000000 Table Usage Count 22228 UPDATE PROCESS ENDED [ 08/23/23 16:00:32 ] [ Force Reload Task - All ] UPDATE PROCESS START [ v3.2.0_6 ] [ 08/23/23 16:15:48 ] ===[ DNSBL Process ]================================================ ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] exists. [ 08/23/23 16:15:50 ] [ Abuse_SSLBL_v4 ] exists. [ CINS_army_v4 ] exists. [ ET_Block_v4 ] exists. [ ET_Comp_v4 ] exists. [ ISC_Block_v4 ] exists. [ Spamhaus_Drop_v4 ] exists. [ Spamhaus_eDrop_v4 ] exists. [ Talos_BL_v4 ] exists. ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload No Changes to Aliases, Skipping pfctl Update ===[ Kill States ]================================================== No matching states found ====================================================================== UPDATE PROCESS ENDED [ 08/23/23 16:15:52 ] [ Force Reload Task - All ] UPDATE PROCESS START [ v3.2.0_6 ] [ 08/23/23 16:16:05 ] ===[ DNSBL Process ]================================================ ===[ GeoIP Process ]============================================ ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] Reload [ 08/23/23 16:16:08 ] . completed .. ------------------------------ Original Master Final ------------------------------ 119 119 119 [ Pass ] ----------------------------------------------------------------- [ Abuse_SSLBL_v4 ] Reload . completed .. ------------------------------ Original Master Final ------------------------------ 31 31 31 [ Pass ] ----------------------------------------------------------------- [ CINS_army_v4 ] Reload . completed .. ------------------------------ Original Master Final ------------------------------ 15000 14999 14999 [ Pass ] ----------------------------------------------------------------- [ ET_Block_v4 ] Reload [ 08/23/23 16:16:10 ] . completed .. ------------------------------ Original Master Final ------------------------------ 1564 1449 1449 [ Pass ] ----------------------------------------------------------------- [ ET_Comp_v4 ] Reload [ 08/23/23 16:16:12 ] . completed .. ------------------------------ Original Master Final ------------------------------ 521 497 497 [ Pass ] ----------------------------------------------------------------- [ ISC_Block_v4 ] Reload [ 08/23/23 16:16:13 ] . completed .. ------------------------------ Original Master Final ------------------------------ 20 5 5 [ Pass ] ----------------------------------------------------------------- [ Spamhaus_Drop_v4 ] Reload . completed .. ------------------------------ Original Master Final ------------------------------ 989 0 0 [ Pass ] ----------------------------------------------------------------- [ Spamhaus_eDrop_v4 ] Reload [ 08/23/23 16:16:14 ] . completed .. ------------------------------ Original Master Final ------------------------------ 348 336 336 [ Pass ] ----------------------------------------------------------------- [ Talos_BL_v4 ] Reload [ 08/23/23 16:16:15 ] . completed .. ------------------------------ Original Master Final ------------------------------ 3609 3586 3586 [ Pass ] ----------------------------------------------------------------- ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_PRI1_v4 8 addresses added. ===[ Kill States ]================================================== No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 22199 ] [ Final IP Count ] [ 21022 ] ===[ Deny List IP Counts ]=========================== ====================[ IPv4/6 Last Updated List Summary ]============== Aug 22 01:24 Spamhaus_Drop_v4 Aug 22 06:30 ET_Block_v4 Aug 22 22:22 ET_Comp_v4 Aug 23 12:38 Spamhaus_eDrop_v4 Aug 23 14:18 CINS_army_v4 Aug 23 15:10 Talos_BL_v4 Aug 23 15:15 ISC_Block_v4 Aug 23 15:55 Abuse_SSLBL_v4 Aug 23 16:00 Abuse_Feodo_C2_v4 ====================[ DNSBL Last Updated List Summary ]============== Aug 23 15:39 StevenBlack_ADs Aug 23 15:50 UT1_malware Aug 23 15:50 UT1_phishing Aug 23 15:50 UT1_publicite Aug 23 15:50 UT1_reaffected Aug 23 15:50 UT1_tricheur =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 21023 /var/db/aliastables/pfB_PRI1_v4.txt pfSense Table Stats ------------------- table-entries hard limit 9000000 Table Usage Count 22236 UPDATE PROCESS ENDED [ 08/23/23 16:51:43 ]
-
@Summer
So the list is loading per the dashboard --
but I don't see that you mention of the error ?
is it gone?confirm the files
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
are you using pfBlockerNG or pfBlockerNG-devel?
Although both packages are the same now.
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
ARM processor has to do with it...
Pure processor power.
ARM is great, of course, as at the end of the year the electricity bill will not be the same as a overpowered "Big Iron Intel" (Xenon) processor.@jrey said in pfBlockerNG-devel pfsense 23.05.1:
and there is no memory crush
That's the system memry.
PHP, as a process, uses only a small part of that. It's this small 'internal' PHP memory pool to overflows.@jrey said in pfBlockerNG-devel pfsense 23.05.1:
What other packages might you be running?
These :
acme backup Cron Filer - ipsec-profile-wizard - Notes - openvpn-client-* - Shellcmd - Systempaches
do nothing, and take no run time memory at all.
Ok, acme - a big shell (not PHP) , starts once a day to check if the certs have to be renewed. That's peanuts.
For me, pfBlockerng uses the most memory.@jrey said in pfBlockerNG-devel pfsense 23.05.1:
Do you have the current patch package installed?
And activated all the patches.
Not because I can, because I know why the patch is there - what it does.
( that is, that's what I hope ^^ )@jrey said in pfBlockerNG-devel pfsense 23.05.1:
On a 2100 - no problem
Hummm. That's interesting.
edit :
I saw your pfBlockerng manual update logs.
It took a whopping 35 minutes (start 16:16 - end 16:51)
That would be a big no-way for me.
Without entering into details : your system is not able to do what you want it to do.
35 minutes to download some list and sort them .... that's way to much.
Keep in mind : most lists don't have every xx minutes an update, some are not even updated every week. pfBlockerng is smart enough not to download again the same identical list.
Still : even as one list changes, they have to added together, and sorted out again.
That's why I have pfBlocker set to : sync ones a day, and not more.
Your are using the "Python mode", right ? -
@Gertjan
are the questions in the edit directed at me or @Summer ?so my comment "On a 2100 - no problem"
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Hummm. That's interesting.
how so?
works just fine.
yes the load time for Summer seems a bit long, but per the dashboard image provided it does appear to have the files (after the latest pass), (waiting on the list requested) that will tell us -- more perhaps
at the same time in ubound mode the service might then not be starting or is "confused" about the status, which could be the result of a port conflict with some other service running. (this issue has been commented on several times in the forum)
but I'm curious
perviously above in the thread you said
Now in the "edit" of the previous post you state
"Without entering into details : your system is not able to do what you want it to do."my 2100 has no issues with this or the other lists I use.
from the comments and screen captures that have been provided @Summer is using ubound -
I, on the other hand, am using python mode.
so the configs will be a "little" differentbut certainly it works - just fine - no issues here
-
Thanks you all for support,
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
but I don't see that you mention of the error ?
is it gone?Yes don't know why but it seems gone :)
How can I check for port conflict, I've tried to show listeing active ports but netstat output is confusing.
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
How can I check for port conflict, I've tried to show listeing active ports but netstat output is confusing.
sockstat
Try sockstat -4 for IPv4 onlySSH (or console) access of course, as that's far superior to 'fake' looking GUI console :
[23.05.1-RELEASE][root@pfSense.bhf.net]/root: sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS root sshd 23511 4 tcp4 192.168.1.1:22 192.168.1.6:49225 root php-fpm 42388 4 udp4 *:* *:* root php-fpm 52780 4 udp4 *:* *:* root php 87192 4 udp4 *:* *:* root php 86955 4 udp4 *:* *:* root lighttpd_p 86620 4 tcp4 10.10.10.1:443 *:* root lighttpd_p 86620 7 tcp4 10.10.10.1:80 *:* root php_pfb 86025 4 udp4 *:* *:* root upsd 77399 4 tcp4 192.168.1.1:3493 *:* root upsd 77399 6 tcp4 127.0.0.1:3493 *:* root upsd 77399 10 tcp4 192.168.1.1:3493 192.168.1.33:36240 root upsd 77399 14 tcp4 192.168.1.1:3493 192.168.1.6:49903 root radiusd 73058 10 tcp4 192.168.1.1:17074 192.168.1.33:3307 root radiusd 73058 22 udp4 127.0.0.1:18128 *:* root radiusd 73058 23 udp4 127.0.0.1:18127 *:* root radiusd 73058 24 udp4 *:1812 *:* root radiusd 73058 25 udp4 *:1816 *:* root radiusd 73058 26 udp4 *:1813 *:* avahi avahi-daem 70566 13 udp4 *:5353 *:* avahi avahi-daem 70566 15 udp4 *:24862 *:* root php-fpm 1331 4 udp4 *:* *:* root openvpn 27220 7 udp4 192.168.10.4:1194 *:* root php-fpm 12968 4 udp4 *:* *:* unbound unbound 60201 5 udp4 *:53 *:* unbound unbound 60201 6 tcp4 *:53 *:* unbound unbound 60201 9 tcp4 127.0.0.1:953 *:* root perl 14972 6 tcp4 *:4949 *:* root syslogd 70815 7 udp4 192.168.1.1:514 *:* root nginx 98000 5 tcp4 *:8003 *:* root nginx 97589 5 tcp4 *:8003 *:* root nginx 97406 5 tcp4 *:8003 *:* root nginx 97177 5 tcp4 *:8003 *:* root nginx 97024 5 tcp4 *:8003 *:* root nginx 96834 5 tcp4 *:8003 *:* root nginx 96618 5 tcp4 *:8003 *:* root nginx 96156 5 tcp4 *:8002 *:* root nginx 95852 5 tcp4 *:8002 *:* root nginx 95691 5 tcp4 *:8002 *:* root nginx 95501 5 tcp4 *:8002 *:* root nginx 95415 5 tcp4 *:8002 *:* root nginx 95151 5 tcp4 *:8002 *:* root nginx 95056 5 tcp4 *:8002 *:* dhcpd dhcpd 25040 20 udp4 *:52880 *:* dhcpd dhcpd 24812 12 udp4 *:67 *:* root nginx 14625 5 tcp4 *:443 *:* root nginx 14625 7 tcp4 *:80 *:* root nginx 14483 5 tcp4 *:443 *:* root nginx 14483 7 tcp4 *:80 *:* root nginx 14212 5 tcp4 *:443 *:* root nginx 14212 7 tcp4 *:80 *:* root ntpd 13581 21 udp4 *:123 *:* root ntpd 13581 23 udp4 192.168.1.1:123 *:* root ntpd 13581 27 udp4 192.168.2.1:123 *:* root ntpd 13581 29 udp4 192.168.100.1:123 *:* root ntpd 13581 31 udp4 192.168.10.4:123 *:* root ntpd 13581 35 udp4 127.0.0.1:123 *:* root ntpd 13581 36 udp4 10.10.10.1:123 *:* root ntpd 13581 39 udp4 192.168.3.1:123 *:* root sshd 64413 4 tcp4 *:22 *:* root php-fpm 400 4 udp4 *:* *:* ? ? ? ? tcp4 192.168.1.1:52798 192.168.1.33:3307 ? ? ? ? tcp4 192.168.1.1:53 192.168.1.26:55519 ? ? ? ? tcp4 192.168.1.1:38324 192.168.1.33:3307 ? ? ? ? tcp4 192.168.2.1:8002 192.168.2.224:49534The common mortals, aka : we all, normally don't have to bother with "ports used by which process".
Why do you think you have a port issue ? -
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Why do you think you have a port issue ?
I've read that dns forwarder should be disabled to make pfblocker work, I've tried + reload + update it but still same.
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS dhcpd dhcpd 73222 18 udp4 *:67 *:* nobody dnsmasq 21576 5 tcp4 192.168.24.1:53 *:* nobody dnsmasq 21576 7 tcp4 192.168.21.1:53 *:* root php_pfb 43824 7 udp4 *:* *:* unbound unbound 86021 8 udp4 *:53 *:* unbound unbound 86021 11 tcp4 *:53 *:* unbound unbound 86021 12 tcp4 127.0.0.1:953 *:* root syslogd 53247 10 udp4 *:514 *:* root php-fpm 81735 5 udp4 *:* *:* root nginx 38803 7 tcp4 *:459 *:* root openvpn 67560 8 udp4 *:20004 *:* root sshd 59577 5 tcp4 *:88 *:* root openvpn 55471 8 udp4 *:20003 *:* root php-fpm 738 5 udp4 *:* *:* root php-fpm 400 5 udp4 *:* *:* root php-fpm 399 5 udp4 *:* *:* root php-fpm 398 5 udp4 *:* *:* -
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
dns forwarder should be disabled
The dns forwarder or dnsmasq is still present in pfSense for historical reasons.
Long time ago, pfSense used dnsmasq for its DNS needs, and isn't needed anymore.
Earth isn't flat, isn't the center of the universe, and even the sun isn't :we (should) know better these days.
pfSense uses the resolver as Internet (and DNS) was meant to be used like that from day 1.Still, dnsmasq is present, and can be used. Just shut down the resolver (unbound).
pfBlocker needs the resolver, not dnsmasq.
pfSense, when installed, isn't DNS forwarding, and doesn't have the forwarder activated (dnsmasq).
Btw : both the forwarder dnsmasq and the resolver, unbound, can be used together.
Just keep in mind that these are server process, so they like to 'bind' to the same port, the famous '53' - UDP and (!) TCP.
So, example, select for your LAN unbound and dsnmasq OPT.
Both use the outbound WAN connection , that won't be an issue.edit :
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
nobody dnsmasq 21576 5 tcp4 192.168.24.1:53 :
nobody dnsmasq 21576 7 tcp4 192.168.21.1:53 :
root php_pfb 43824 7 udp4 : :
unbound unbound 86021 8 udp4 *:53 :
unbound unbound 86021 11 tcp4 *:53 :yeah, that a complete fail.
dnsmasq binds to interface "192.168.24.1", port 53 - only TCP ? (??? - better check your sources, only TCP is .... strange - are the 192.168.24.1 network clients aware of this severe limitation ?)and unbound (tries to) bind to 'all available interfaces', hence the "*.53", this time UDP and TCP, which is ok.
That will fail to bind to 192.168.24.1 .... and unboud should have told you that - that's why I always ask : where are the logs ?(which actually means : did you look at them ?)I understand now.
The admin created a major issue.
Happens.
Why do you (think you) need dnsmasq ? -
G Gertjan referenced this topic on
-
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Still, dnsmasq is present, and can be used. Just shut down the resolver (unbound).
pfBlocker needs the resolver, not dnsmasq.
Now DNS resolver (unbound) isn't running.
Shell Output - sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS dhcpd dhcpd 12567 17 udp4 *:67 *:* nobody dnsmasq 8322 8 udp4 LANADDRESS:53 *:* nobody dnsmasq 8322 9 tcp4 LANADDRESS:53 *:* root php_pfb 71633 7 udp4 *:* *:* root php-fpm 71033 5 udp4 *:* *:* root php-fpm 66669 5 udp4 *:* *:* root syslogd 53247 10 udp4 *:514 *:*Main problem is admin it's me ;-), to clear this mess: I should to provide local pfsense LAN address as DNS server to clients so DNS Forwarder need to be up.
Now Firewall > pfBlockerNG > Update > Reload All
UPDATE PROCESS START [ v3.2.0_6 ] [ 08/24/23 11:53:02 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL SafeSearch... enabled Loading DNSBL Whitelist... completed Loading TOP1M Whitelist... completed Blacklist database(s) ... exists. [ UT1_malware ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 43291 43291 0 0 0 43291 ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 3686 3686 2 13 0 3671 ---------------------------------------------------------------------- [ UT1_reaffected ] Reload [ 08/24/23 11:53:13 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 7 7 0 0 0 7 ---------------------------------------------------------------------- [ UT1_tricheur ] Reload [ 08/24/23 11:53:14 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 35 35 0 0 0 35 ---------------------------------------------------------------------- [ StevenBlack_ADs ] Reload [ 08/24/23 11:53:16 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 211630 211630 4484 119 0 207027 ---------------------------------------------------------------------- Saving DNSBL statistics... completed [ 08/24/23 11:53:53 ] ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 08/24/23 11:53:55 ] Stopping Unbound Resolver Unbound stopped in 1 sec. Starting Unbound Resolver... completed [ 08/24/23 11:53:58 ] DNSBL update [ 254079 | PASSED ]... completed [ 08/24/23 11:54:15 ] ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ MaxMind Database downloading and processing ( approx 4MB ) ... Please wait ... Download Process Starting [ 08/24/23 11:54:16 ] /usr/local/share/GeoIP/GeoLite2-Country.tar.gz 200 OK Download Process Ended [ 08/24/23 11:54:17 ] ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] Reload [ 08/24/23 11:54:17 ] . completed .. ------------------------------ Original Master Final ------------------------------ 109 109 109 [ Pass ] ----------------------------------------------------------------- ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_PRI1_v4 no changes. ===[ Kill States ]================================================== No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 22367 ] [ Final IP Count ] [ 21211 ] ===[ Deny List IP Counts ]=========================== 21212 total 15000 /var/db/pfblockerng/deny/CINS_army_v4.txt 3774 /var/db/pfblockerng/deny/Talos_BL_v4.txt 1462 /var/db/pfblockerng/deny/ET_Block_v4.txt 496 /var/db/pfblockerng/deny/ET_Comp_v4.txt 338 /var/db/pfblockerng/deny/Spamhaus_eDrop_v4.txt 109 /var/db/pfblockerng/deny/Abuse_Feodo_C2_v4.txt 29 /var/db/pfblockerng/deny/Abuse_SSLBL_v4.txt 3 /var/db/pfblockerng/deny/ISC_Block_v4.txt 1 /var/db/pfblockerng/deny/Spamhaus_Drop_v4.txt ====================[ Empty Lists w/127.1.7.7 ]================== Spamhaus_Drop_v4.txt ===[ DNSBL Domain/IP Counts ] =================================== 254079 total 207027 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt 43291 /var/db/pfblockerng/dnsbl/UT1_malware.txt 3671 /var/db/pfblockerng/dnsbl/UT1_publicite.txt 48 /var/db/pfblockerng/dnsbl/UT1_phishing.txt 35 /var/db/pfblockerng/dnsbl/UT1_tricheur.txt 7 /var/db/pfblockerng/dnsbl/UT1_reaffected.txt ====================[ IPv4/6 Last Updated List Summary ]============== Aug 22 01:24 Spamhaus_Drop_v4 Aug 23 06:30 ET_Block_v4 ====================[ DNSBL Last Updated List Summary ]============== Aug 23 15:39 StevenBlack_ADs =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 21212 /var/db/aliastables/pfB_PRI1_v4.txt pfSense Table Stats ------------------- table-entries hard limit 9000000 Table Usage Count 22425 UPDATE PROCESS ENDED [ 08/24/23 11:54:33 ]Now sockstats show:
Shell Output - sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS root php_pfb 68323 7 udp4 *:* *:* unbound unbound 26623 8 udp4 *:53 *:* unbound unbound 26623 11 tcp4 *:53 *:* unbound unbound 26623 12 tcp4 127.0.0.1:953 *:* dhcpd dhcpd 12567 17 udp4 *:67 *:* nobody dnsmasq 8322 8 udp4 LANADDRESS:53 *:* nobody dnsmasq 8322 9 tcp4 LANADDRESS:53 *:*
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
Now DNS resolver (unbound) isn't running.
That's sad - your are writing this in the a "pfBlockerng" forum, and that one (need' unbound, not the ancient forwarder.
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
unbound unbound 26623 8 udp4 *:53 :
unbound unbound 26623 11 tcp4 *:53 :
unbound unbound 26623 12 tcp4 127.0.0.1:953 :You said
Now DNS resolver (unbound) isn't running.
and then you show it is running.
-
That line says me that pfBLockerng says to you :
"Listen, this is not a question, more a condition" :
This is now explained :
DNSBL can't work if unbound isn't available.
I know, it is running as you've shown above. I suspect its a zombie process.
Kill it on the command line.Then : shut down dnsmasq - the forwarder.
Activate unbound with default settings.
This will bring you close to "No more pfBlocker issues". -
@Gertjan I got it!
Please look at basic concepts
- DNS Resolver MUST be enabled
- pfBlockerNG
To Utilize, Unbound DNS Resolver must be enabled.Thanks to all of you! Wish you a nice day
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
Yes don't know why but it seems gone :)
good.
so we know the files are present and have been processed. (we can talk about the time stamps (how far apart they are) later)
since you are looking at ports etc. I'll let that play out. Doesn't appear you have any package installed that will create a conflict with the ports you set and referenced above from the log "Saving new DNSBL web server configuration to port [ 8082 and 8442 ]" (there was a documented case where port 8442 was used by another listener and thus "confusing" DNSBL causing it not to start) that doesn't appear to be your situation.
From the screen capture, it looks like you have gone ahead and disabled the DNS Resolver and did that after posting your sockstat showing it running, so my take right now is that DNS Resolver is currently off ?
I'll ask then, what is your expectation with regards DNS?
or
what provides DNS to your LAN clients?
what provides DNS to your Netgate?