Microsoft Updates Not Progressing
-
Mmm, well I'd still remove those and retest. The fact that it showed any entries makes it my number 1 suspect!
-
@stephenw10 I just went to and they are now gone, I guess they purge on their own after a period of time I take it.
-
Yes, they do. You can set that timeout though and it can be disabled entirely.
https://docs.netgate.com/pfsense/en/latest/packages/snort/setup.html#managing-blocked-hosts
-
@stephenw10 Onething I have noticed with this 6100 is it's dead slow when reloading pfblockerNG lists. I made some changes and reloaded "All" and so far 38 minutes to run that CRON job. I expected a bit better performance overall.
-
Hmm, yeah, something not right there. That should take a few minutes at most. Something is probably blocking access to the lists and has to timeout. I would expect to see a bunch of errors in the update log.
-
Just don't.
PHP is parsing that file. PHP isn't the tool you want to parse such a file.I've activated that DNSBL_UT1 on my 4100-MAX : even the restart of unbound took 30 seconds to start.
When you see things like this in the pfblokcerng 'update' log, you know you should back of.
TLD analysis........xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx completed [ 09/12/22 17:13:06 ]
** TLD Domain count exceeded. [ 800000 ] All subsequent Domains listed as-is **
Anyway .... a list with 4+ million DNSBL ...... and only 4 hits ?
[ Force Reload Task - All ] UPDATE PROCESS START [ v3.1.0_4 ] [ 09/12/22 17:07:16 ] ..... UPDATE PROCESS ENDED [ 09/12/22 17:16:27 ]Yep, 10 minutes, as it needed 6 minutes to download this U1 file.
And now I have to deal with :
so using this U1 feed is a no-go for 4 Gbytes.
-
@gertjan I'll go have a look thank you.
-
@gertjan Looks like the "Adult" category is the large database.
-
-
-
@gertjan That helped. it now took 2 min 49 seconds to "Reload All" which took almost 40minutes with the adult list added. To help with adult site, I simply changed my cloudflare setting to 1.1.1.3 and 1.0.0.3. Thanks
-
That with python mode enabled?
-
@stephenw10 Unbound Mode only
-
Ah yes, you probably need python mode for a list that large. That's not something I've tried (yet) I only use a few small lists for add-blocking.
-
@stephenw10 Ahh I see, well being my first setup and installation I think I have more than enough probably, maybe too much, so the device definitely feels way more responsive, and when it finally arrives at its destination when I am finished I can monitor in their real environment and tighten or loosen up the network. Just trying to make sure the devices are getting their correct IP addressing and they can see only what they need to. I will check the Microsoft updates when I get home and see if these changes have affected anything. But we have addressed issues I hadn't even looked into yet so very good. Appreciate all the help so far.
-
@stormgate said in Microsoft Updates Not Progressing:
Unbound Mode only
Using this mode, after sorting, removing the doubles, all DNSBL info is placed in one file ( see it here - it(s the 'biggest' file ) and unbound is informed to read it during startup.
Unbound isn't fast at reading these files, so on every restart, it will take time, timing during your network has no DNS.The "python mode" is using a python script ( as the developers of unbound chose python as a script/API access for unbound ) : unbound loads the small python script, and can continue.
This python scripts handles the big DNSBL file access, and also adds some new features, like regex filtering, AAAA filtering, and gives you a way to exclude some LAN devices from being DNSBL filtered.
Python mode is well tested by now. -
Hmm, so your test was using python mode and it still caused huge delays with that massive list @Gertjan?
-
@gertjan Thank you, I will give it a try, as the issue with microsoft updates is far from fixed. One PC has been running since 5PM yesterday to get its updates and its at 20% on only one of them. So simply changing the mode is all I need to do to implement python?
-
Yup. See: https://forum.netgate.com/post/947354
-
@stephenw10 said in Microsoft Updates Not Progressing:
Hmm, so your test was using python mode and it still caused huge delays with that massive list @Gertjan?
Yep, I'm using python mode.
Most of the 'huge file' treatment doesn't change, as pfblockerng still has to download de DNSBL file.
Sort it.
Adding it to the 'main' DNSBL file, while taking care of 'doubles'.
This is done using PHP, and has nothing to with the 'Python mode" which is actually just a set of function used in the resolve process within unbound.PHP isn't famous for it text file parse power ;)
