pfBlockerNG-devel pfsense 23.05.1
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
are you using pfBlockerNG or pfBlockerNG-devel?
Although both packages are the same now.
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
ARM processor has to do with it...
Pure processor power.
ARM is great, of course, as at the end of the year the electricity bill will not be the same as a overpowered "Big Iron Intel" (Xenon) processor.@jrey said in pfBlockerNG-devel pfsense 23.05.1:
and there is no memory crush
That's the system memry.
PHP, as a process, uses only a small part of that. It's this small 'internal' PHP memory pool to overflows.@jrey said in pfBlockerNG-devel pfsense 23.05.1:
What other packages might you be running?
These :
acme backup Cron Filer - ipsec-profile-wizard - Notes - openvpn-client-* - Shellcmd - Systempaches
do nothing, and take no run time memory at all.
Ok, acme - a big shell (not PHP) , starts once a day to check if the certs have to be renewed. That's peanuts.
For me, pfBlockerng uses the most memory.@jrey said in pfBlockerNG-devel pfsense 23.05.1:
Do you have the current patch package installed?
And activated all the patches.
Not because I can, because I know why the patch is there - what it does.
( that is, that's what I hope ^^ )@jrey said in pfBlockerNG-devel pfsense 23.05.1:
On a 2100 - no problem
Hummm. That's interesting.
edit :
I saw your pfBlockerng manual update logs.
It took a whopping 35 minutes (start 16:16 - end 16:51)
That would be a big no-way for me.
Without entering into details : your system is not able to do what you want it to do.
35 minutes to download some list and sort them .... that's way to much.
Keep in mind : most lists don't have every xx minutes an update, some are not even updated every week. pfBlockerng is smart enough not to download again the same identical list.
Still : even as one list changes, they have to added together, and sorted out again.
That's why I have pfBlocker set to : sync ones a day, and not more.
Your are using the "Python mode", right ? -
@Gertjan
are the questions in the edit directed at me or @Summer ?so my comment "On a 2100 - no problem"
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Hummm. That's interesting.
how so?
works just fine.
yes the load time for Summer seems a bit long, but per the dashboard image provided it does appear to have the files (after the latest pass), (waiting on the list requested) that will tell us -- more perhaps
at the same time in ubound mode the service might then not be starting or is "confused" about the status, which could be the result of a port conflict with some other service running. (this issue has been commented on several times in the forum)
but I'm curious
perviously above in the thread you said
Now in the "edit" of the previous post you state
"Without entering into details : your system is not able to do what you want it to do."my 2100 has no issues with this or the other lists I use.
from the comments and screen captures that have been provided @Summer is using ubound -
I, on the other hand, am using python mode.
so the configs will be a "little" differentbut certainly it works - just fine - no issues here
-
Thanks you all for support,
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
but I don't see that you mention of the error ?
is it gone?Yes don't know why but it seems gone :)
How can I check for port conflict, I've tried to show listeing active ports but netstat output is confusing.
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
How can I check for port conflict, I've tried to show listeing active ports but netstat output is confusing.
sockstat
Try sockstat -4 for IPv4 onlySSH (or console) access of course, as that's far superior to 'fake' looking GUI console :
[23.05.1-RELEASE][root@pfSense.bhf.net]/root: sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS root sshd 23511 4 tcp4 192.168.1.1:22 192.168.1.6:49225 root php-fpm 42388 4 udp4 *:* *:* root php-fpm 52780 4 udp4 *:* *:* root php 87192 4 udp4 *:* *:* root php 86955 4 udp4 *:* *:* root lighttpd_p 86620 4 tcp4 10.10.10.1:443 *:* root lighttpd_p 86620 7 tcp4 10.10.10.1:80 *:* root php_pfb 86025 4 udp4 *:* *:* root upsd 77399 4 tcp4 192.168.1.1:3493 *:* root upsd 77399 6 tcp4 127.0.0.1:3493 *:* root upsd 77399 10 tcp4 192.168.1.1:3493 192.168.1.33:36240 root upsd 77399 14 tcp4 192.168.1.1:3493 192.168.1.6:49903 root radiusd 73058 10 tcp4 192.168.1.1:17074 192.168.1.33:3307 root radiusd 73058 22 udp4 127.0.0.1:18128 *:* root radiusd 73058 23 udp4 127.0.0.1:18127 *:* root radiusd 73058 24 udp4 *:1812 *:* root radiusd 73058 25 udp4 *:1816 *:* root radiusd 73058 26 udp4 *:1813 *:* avahi avahi-daem 70566 13 udp4 *:5353 *:* avahi avahi-daem 70566 15 udp4 *:24862 *:* root php-fpm 1331 4 udp4 *:* *:* root openvpn 27220 7 udp4 192.168.10.4:1194 *:* root php-fpm 12968 4 udp4 *:* *:* unbound unbound 60201 5 udp4 *:53 *:* unbound unbound 60201 6 tcp4 *:53 *:* unbound unbound 60201 9 tcp4 127.0.0.1:953 *:* root perl 14972 6 tcp4 *:4949 *:* root syslogd 70815 7 udp4 192.168.1.1:514 *:* root nginx 98000 5 tcp4 *:8003 *:* root nginx 97589 5 tcp4 *:8003 *:* root nginx 97406 5 tcp4 *:8003 *:* root nginx 97177 5 tcp4 *:8003 *:* root nginx 97024 5 tcp4 *:8003 *:* root nginx 96834 5 tcp4 *:8003 *:* root nginx 96618 5 tcp4 *:8003 *:* root nginx 96156 5 tcp4 *:8002 *:* root nginx 95852 5 tcp4 *:8002 *:* root nginx 95691 5 tcp4 *:8002 *:* root nginx 95501 5 tcp4 *:8002 *:* root nginx 95415 5 tcp4 *:8002 *:* root nginx 95151 5 tcp4 *:8002 *:* root nginx 95056 5 tcp4 *:8002 *:* dhcpd dhcpd 25040 20 udp4 *:52880 *:* dhcpd dhcpd 24812 12 udp4 *:67 *:* root nginx 14625 5 tcp4 *:443 *:* root nginx 14625 7 tcp4 *:80 *:* root nginx 14483 5 tcp4 *:443 *:* root nginx 14483 7 tcp4 *:80 *:* root nginx 14212 5 tcp4 *:443 *:* root nginx 14212 7 tcp4 *:80 *:* root ntpd 13581 21 udp4 *:123 *:* root ntpd 13581 23 udp4 192.168.1.1:123 *:* root ntpd 13581 27 udp4 192.168.2.1:123 *:* root ntpd 13581 29 udp4 192.168.100.1:123 *:* root ntpd 13581 31 udp4 192.168.10.4:123 *:* root ntpd 13581 35 udp4 127.0.0.1:123 *:* root ntpd 13581 36 udp4 10.10.10.1:123 *:* root ntpd 13581 39 udp4 192.168.3.1:123 *:* root sshd 64413 4 tcp4 *:22 *:* root php-fpm 400 4 udp4 *:* *:* ? ? ? ? tcp4 192.168.1.1:52798 192.168.1.33:3307 ? ? ? ? tcp4 192.168.1.1:53 192.168.1.26:55519 ? ? ? ? tcp4 192.168.1.1:38324 192.168.1.33:3307 ? ? ? ? tcp4 192.168.2.1:8002 192.168.2.224:49534The common mortals, aka : we all, normally don't have to bother with "ports used by which process".
Why do you think you have a port issue ? -
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Why do you think you have a port issue ?
I've read that dns forwarder should be disabled to make pfblocker work, I've tried + reload + update it but still same.
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS dhcpd dhcpd 73222 18 udp4 *:67 *:* nobody dnsmasq 21576 5 tcp4 192.168.24.1:53 *:* nobody dnsmasq 21576 7 tcp4 192.168.21.1:53 *:* root php_pfb 43824 7 udp4 *:* *:* unbound unbound 86021 8 udp4 *:53 *:* unbound unbound 86021 11 tcp4 *:53 *:* unbound unbound 86021 12 tcp4 127.0.0.1:953 *:* root syslogd 53247 10 udp4 *:514 *:* root php-fpm 81735 5 udp4 *:* *:* root nginx 38803 7 tcp4 *:459 *:* root openvpn 67560 8 udp4 *:20004 *:* root sshd 59577 5 tcp4 *:88 *:* root openvpn 55471 8 udp4 *:20003 *:* root php-fpm 738 5 udp4 *:* *:* root php-fpm 400 5 udp4 *:* *:* root php-fpm 399 5 udp4 *:* *:* root php-fpm 398 5 udp4 *:* *:* -
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
dns forwarder should be disabled
The dns forwarder or dnsmasq is still present in pfSense for historical reasons.
Long time ago, pfSense used dnsmasq for its DNS needs, and isn't needed anymore.
Earth isn't flat, isn't the center of the universe, and even the sun isn't :we (should) know better these days.
pfSense uses the resolver as Internet (and DNS) was meant to be used like that from day 1.Still, dnsmasq is present, and can be used. Just shut down the resolver (unbound).
pfBlocker needs the resolver, not dnsmasq.
pfSense, when installed, isn't DNS forwarding, and doesn't have the forwarder activated (dnsmasq).
Btw : both the forwarder dnsmasq and the resolver, unbound, can be used together.
Just keep in mind that these are server process, so they like to 'bind' to the same port, the famous '53' - UDP and (!) TCP.
So, example, select for your LAN unbound and dsnmasq OPT.
Both use the outbound WAN connection , that won't be an issue.edit :
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
nobody dnsmasq 21576 5 tcp4 192.168.24.1:53 :
nobody dnsmasq 21576 7 tcp4 192.168.21.1:53 :
root php_pfb 43824 7 udp4 : :
unbound unbound 86021 8 udp4 *:53 :
unbound unbound 86021 11 tcp4 *:53 :yeah, that a complete fail.
dnsmasq binds to interface "192.168.24.1", port 53 - only TCP ? (??? - better check your sources, only TCP is .... strange - are the 192.168.24.1 network clients aware of this severe limitation ?)and unbound (tries to) bind to 'all available interfaces', hence the "*.53", this time UDP and TCP, which is ok.
That will fail to bind to 192.168.24.1 .... and unboud should have told you that - that's why I always ask : where are the logs ?(which actually means : did you look at them ?)I understand now.
The admin created a major issue.
Happens.
Why do you (think you) need dnsmasq ? -
G Gertjan referenced this topic on
-
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Still, dnsmasq is present, and can be used. Just shut down the resolver (unbound).
pfBlocker needs the resolver, not dnsmasq.
Now DNS resolver (unbound) isn't running.
Shell Output - sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS dhcpd dhcpd 12567 17 udp4 *:67 *:* nobody dnsmasq 8322 8 udp4 LANADDRESS:53 *:* nobody dnsmasq 8322 9 tcp4 LANADDRESS:53 *:* root php_pfb 71633 7 udp4 *:* *:* root php-fpm 71033 5 udp4 *:* *:* root php-fpm 66669 5 udp4 *:* *:* root syslogd 53247 10 udp4 *:514 *:*Main problem is admin it's me ;-), to clear this mess: I should to provide local pfsense LAN address as DNS server to clients so DNS Forwarder need to be up.
Now Firewall > pfBlockerNG > Update > Reload All
UPDATE PROCESS START [ v3.2.0_6 ] [ 08/24/23 11:53:02 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL SafeSearch... enabled Loading DNSBL Whitelist... completed Loading TOP1M Whitelist... completed Blacklist database(s) ... exists. [ UT1_malware ] Reload . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 43291 43291 0 0 0 43291 ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 3686 3686 2 13 0 3671 ---------------------------------------------------------------------- [ UT1_reaffected ] Reload [ 08/24/23 11:53:13 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 7 7 0 0 0 7 ---------------------------------------------------------------------- [ UT1_tricheur ] Reload [ 08/24/23 11:53:14 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 35 35 0 0 0 35 ---------------------------------------------------------------------- [ StevenBlack_ADs ] Reload [ 08/24/23 11:53:16 ] . completed .. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 211630 211630 4484 119 0 207027 ---------------------------------------------------------------------- Saving DNSBL statistics... completed [ 08/24/23 11:53:53 ] ------------------------------------------------------------------------ Assembling DNSBL database...... completed [ 08/24/23 11:53:55 ] Stopping Unbound Resolver Unbound stopped in 1 sec. Starting Unbound Resolver... completed [ 08/24/23 11:53:58 ] DNSBL update [ 254079 | PASSED ]... completed [ 08/24/23 11:54:15 ] ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ MaxMind Database downloading and processing ( approx 4MB ) ... Please wait ... Download Process Starting [ 08/24/23 11:54:16 ] /usr/local/share/GeoIP/GeoLite2-Country.tar.gz 200 OK Download Process Ended [ 08/24/23 11:54:17 ] ===[ IPv4 Process ]================================================= [ Abuse_Feodo_C2_v4 ] Reload [ 08/24/23 11:54:17 ] . completed .. ------------------------------ Original Master Final ------------------------------ 109 109 109 [ Pass ] ----------------------------------------------------------------- ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload Updating: pfB_PRI1_v4 no changes. ===[ Kill States ]================================================== No matching states found ====================================================================== ===[ FINAL Processing ]===================================== [ Original IP count ] [ 22367 ] [ Final IP Count ] [ 21211 ] ===[ Deny List IP Counts ]=========================== 21212 total 15000 /var/db/pfblockerng/deny/CINS_army_v4.txt 3774 /var/db/pfblockerng/deny/Talos_BL_v4.txt 1462 /var/db/pfblockerng/deny/ET_Block_v4.txt 496 /var/db/pfblockerng/deny/ET_Comp_v4.txt 338 /var/db/pfblockerng/deny/Spamhaus_eDrop_v4.txt 109 /var/db/pfblockerng/deny/Abuse_Feodo_C2_v4.txt 29 /var/db/pfblockerng/deny/Abuse_SSLBL_v4.txt 3 /var/db/pfblockerng/deny/ISC_Block_v4.txt 1 /var/db/pfblockerng/deny/Spamhaus_Drop_v4.txt ====================[ Empty Lists w/127.1.7.7 ]================== Spamhaus_Drop_v4.txt ===[ DNSBL Domain/IP Counts ] =================================== 254079 total 207027 /var/db/pfblockerng/dnsbl/StevenBlack_ADs.txt 43291 /var/db/pfblockerng/dnsbl/UT1_malware.txt 3671 /var/db/pfblockerng/dnsbl/UT1_publicite.txt 48 /var/db/pfblockerng/dnsbl/UT1_phishing.txt 35 /var/db/pfblockerng/dnsbl/UT1_tricheur.txt 7 /var/db/pfblockerng/dnsbl/UT1_reaffected.txt ====================[ IPv4/6 Last Updated List Summary ]============== Aug 22 01:24 Spamhaus_Drop_v4 Aug 23 06:30 ET_Block_v4 ====================[ DNSBL Last Updated List Summary ]============== Aug 23 15:39 StevenBlack_ADs =============================================================== Database Sanity check [ PASSED ] ------------------------ Masterfile/Deny folder uniq check Deny folder/Masterfile uniq check Sync check (Pass=No IPs reported) ---------- Alias table IP Counts ----------------------------- 21212 /var/db/aliastables/pfB_PRI1_v4.txt pfSense Table Stats ------------------- table-entries hard limit 9000000 Table Usage Count 22425 UPDATE PROCESS ENDED [ 08/24/23 11:54:33 ]Now sockstats show:
Shell Output - sockstat -4 USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS root php_pfb 68323 7 udp4 *:* *:* unbound unbound 26623 8 udp4 *:53 *:* unbound unbound 26623 11 tcp4 *:53 *:* unbound unbound 26623 12 tcp4 127.0.0.1:953 *:* dhcpd dhcpd 12567 17 udp4 *:67 *:* nobody dnsmasq 8322 8 udp4 LANADDRESS:53 *:* nobody dnsmasq 8322 9 tcp4 LANADDRESS:53 *:*
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
Now DNS resolver (unbound) isn't running.
That's sad - your are writing this in the a "pfBlockerng" forum, and that one (need' unbound, not the ancient forwarder.
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
unbound unbound 26623 8 udp4 *:53 :
unbound unbound 26623 11 tcp4 *:53 :
unbound unbound 26623 12 tcp4 127.0.0.1:953 :You said
Now DNS resolver (unbound) isn't running.
and then you show it is running.
-
That line says me that pfBLockerng says to you :
"Listen, this is not a question, more a condition" :
This is now explained :
DNSBL can't work if unbound isn't available.
I know, it is running as you've shown above. I suspect its a zombie process.
Kill it on the command line.Then : shut down dnsmasq - the forwarder.
Activate unbound with default settings.
This will bring you close to "No more pfBlocker issues". -
@Gertjan I got it!
Please look at basic concepts
- DNS Resolver MUST be enabled
- pfBlockerNG
To Utilize, Unbound DNS Resolver must be enabled.Thanks to all of you! Wish you a nice day
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
Yes don't know why but it seems gone :)
good.
so we know the files are present and have been processed. (we can talk about the time stamps (how far apart they are) later)
since you are looking at ports etc. I'll let that play out. Doesn't appear you have any package installed that will create a conflict with the ports you set and referenced above from the log "Saving new DNSBL web server configuration to port [ 8082 and 8442 ]" (there was a documented case where port 8442 was used by another listener and thus "confusing" DNSBL causing it not to start) that doesn't appear to be your situation.
From the screen capture, it looks like you have gone ahead and disabled the DNS Resolver and did that after posting your sockstat showing it running, so my take right now is that DNS Resolver is currently off ?
I'll ask then, what is your expectation with regards DNS?
or
what provides DNS to your LAN clients?
what provides DNS to your Netgate? -
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
From the screen capture, it looks like you have gone ahead and disabled the DNS Resolver and did that after posting your sockstat showing it running, so my take right now is that DNS Resolver is currently off ?
I don't understand how the "background" magic work but the unbound were started by the pfblocker and it was missing other unbound process from DNS Resolver.
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
so my take right now is that DNS Resolver is currently off ?
No DNS Resolver is UP and Running.
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
I'll ask then, what is your expectation with regards DNS?
Just looking for a simple Ad filter
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
what provides DNS to your LAN clients?
Now it is DNS Resolver on pfsense LAN that act both a resolver and forwarder
@jrey said in pfBlockerNG-devel pfsense 23.05.1:
what provides DNS to your Netgate?
Just local ISP
-
@Summer said in pfBlockerNG-devel pfsense 23.05.1:
Just local ISP
As an answer to
what provides DNS to your Netgate?
Most ISP, using a DHCP WAN, PPPOE or whatever, have one or several DNS available for their clients.
That's part the "get our connection, get our box, and we do the rest" deal. Just perfect for grandma and the other, vast majority, of their clients.
In the past, there were good reasons why they had to do this.
A huge minority of the ISP users use their own router firewall supper gadget (we call it pfSense), which gives you a big pallet with tools so you can do some fine control on your Internet traffic.
One of the first advantages is : now you can get the DNS as it was always intended be the used : you can resolve. Don't underestimate this 'resolve' word here. Understanding it means that you know what that "DNS" thing is all about.Out of the box, when you install and start pfSense the first time, and your ISP used IPv4 and DHCP, you must do this to make everything work :
Nothing.
( I agree, it might be a wise thing that you change the GUI password - and that's it - and I'm serious )
pfSEnse is like any other router out their : plug and play.
Your connection works, and not known host names are lookup up using the original, official way : resolving.
With "original, official" I mean : example : you want to know the phone number of Paul.
Some say : let's ask Jack - as he had Paul on the phone recently.
I prefer this way : "let's ask Paul". Because I guess Paul is probably the best source for this info.
Match this example to pfSense using DNS resolving.
Some : What's the addresses of microsoft.com ? => Let's ask Google.
Others : What's the addresses of microsoft.com ? => Let's ask micosoft.comNow .... you agree ? No needs to finish "Havard" to get this answer here.
It is, of course, not wrong to use the DNS of your ISP. Or to use 8.8.8.8. The bottom line is : it's a choice.
-
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Out of the box, when you install and start pfSense the first time, and your ISP used IPv4 and DHCP, you must do this to make everything work :
Nothing.Glad to know everything can run out of the box.
My out of the box start version was 2.3, since that updates and updates and I don't really know what each change has done.
@Gertjan said in pfBlockerNG-devel pfsense 23.05.1:
Now .... you agree ? No needs to finish "Havard" to get this answer here.
That's fine, but sometimes a reading of glossary on netgate docs could save a day ;-)
-
@Summer quoted in pfBlockerNG-devel pfsense 23.05.1:
when you install and start pfSense the first time, and your ISP used IPv4 and DHCP
Not true in "every" case. your connection could have been a static IP and DHCP would not be in play in that case. (don't really recall you saying ISP or connection type until the question was asked)
The line of questioning was more to lead on the path of thinking about how things work in your specific case. (Learning)
Without getting into the details of how or why, my DNS for example, is all local (internal, behind the wall), fully isolated from the internet. Blazing fast DNS response times. I currently have 40-50 devices behind the 2100 - and it doesn't even have to work hard. It's all about how you approach things, with a specific goal in mind. Plan it out.
I'd would like, if you don't mind, to cycle back on what was perceived to be a long download time for you and the file. (because yes that ~30min time for you ahowed does seem excessive)
what kind of speed is your WAN?
I, for example, typically download this file in 1-2 seconds max - even on a "congested" day it might take 4 seconds (yes, the file only downloads when needed, but still )
(start) Thu 24 Aug 2023 11:26:38 EDT % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6671k 100 6671k 0 0 17.0M 0 --:--:-- --:--:-- --:--:-- 17.3M (end) Thu 24 Aug 2023 11:26:38 EDT