Speed up Updates and Reloads

Stewart

When I reload or update pfBlocker it takes a REALLY long time. I just built a new box and it took 7:30 for a Reload All. How can I shorten that? Much of the time, ~4 minutes, is TLD Analysis and Finalizing. I am using the IP section for IPv4 PRI1, and GEOIP for Top Spammers and North America. We are also using the standard DNSBL lists and a few categories such as Hacking, Spyware, Warez, etc. I do have TLD on. I understand that I'm putting a lot into this little box but what is the limiting factor? It doesn't appear to be CPU as it sits mostly idle during the process. RAM doesn't seem heavily impacted. I would think drive speed but the numbers seem good:

/root: diskinfo -tv /dev/ada0
/dev/ada0
        512             # sectorsize
        120034123776    # mediasize in bytes (112G)
        234441648       # mediasize in sectors
        4096            # stripesize
        0               # stripeoffset
        232581          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.
        KINGSTON SUV500MS120G   # Disk descr.
        50026B7782C95AB3        # Disk ident.
        Yes             # TRIM/UNMAP support
        0               # Rotation rate in RPM
        Not_Zoned       # Zone Mode

Seek times:
        Full stroke:      250 iter in   0.029203 sec =    0.117 msec
        Half stroke:      250 iter in   0.016129 sec =    0.065 msec
        Quarter stroke:   500 iter in   0.034352 sec =    0.069 msec
        Short forward:    400 iter in   0.024513 sec =    0.061 msec
        Short backward:   400 iter in   0.022884 sec =    0.057 msec
        Seq outer:       2048 iter in   0.120875 sec =    0.059 msec
        Seq inner:       2048 iter in   0.120608 sec =    0.059 msec

Transfer rates:
        outside:       102400 kbytes in   0.285311 sec =   358907 kbytes/sec
        middle:        102400 kbytes in   0.260321 sec =   393361 kbytes/sec
        inside:        102400 kbytes in   0.280713 sec =   364785 kbytes/sec

Here are the system specs:

System 	PC Engines APU2
Version 	2.4.5-RELEASE-p1 (amd64)
CPU Type 	AMD GX-412TC SOC - 4 CPUs: 1 package(s) x 4 core(s)
SSD:  Kingston SUV500MS/120G
pfBlockerNG-devel     3.0.0_8

Thanks for the help!

BBcan177

@stewart

TLD is a complex process of analyzing all the domains and deciding if those domains should be wildcard blocked. That is a long time to process in your box, so seems to be way under powered?
You can add more TLD Blacklist TLDs, so adding say "cn", "ru", that will block all sub-domains of those TLDs, and save some processing time.

https://www.spamhaus.org/statistics/tlds/
https://trends.netcraft.com/cybercrime/tlds

Stewart

@bbcan177 I get that it's a complicated process, but what is being taxed? Is it all loaded in RAM and it's the RAM speed limiting it? I have the same problem with Quickbooks in every install we've done. It's runs slow but nothing we see is being overly taxed. If I were to build a new system and upgrade from the APU2 units, what would need to be sped up to alleviate the issue?