4100/6100 Base or Max

[mcury]

@dennypage said in 4100/6100 Base or Max:

I don't myself see a delay associated with pfBlockerNG. I do see a delay associated with restoring /var (from the backup created at shutdown) of maybe 20 seconds, but overall this is a minor thing.

hm.. with the SG-3100, DNSBL and pfblockerNG enabled, could take up to two minutes, just with the wizard stuff..

@dennypage said in 4100/6100 Base or Max:

The other issues listed are associated with configuring the ram disks incorrectly, rather than recurring issues associated with each reboot.

Indeed.. I used RAM disk for a long time, and I can say that I was able to sell my SG-3100 with the eMMC working because of it.. 6 years using that device.
It is a life saver, but I prefer to use a SSD for that and keep my eMMC intact when the time to replace/sell happens.

[Cabledude]

@Cabledude said in 4100/6100 Base or Max:

This is UFS, pfBlocker DNSBL AdsBasic, IP PRI1, no GeoIP, avahi, 8 vlans, night time so the rest of the family is asleep. I will re-enable geoIP top spammers all countries for a couple of hours and then disable pfblocker and see what that does to the graph

Here we go. Around 23:45 (11:45PM) I added the GeoIP "Top Spammers" list, and played with settings for 1.5 hours, then went to bed. It did raise the CPU usage a little but after 2 AM it was largely the same as before.

Then at 6AM pfBlocker OFF. Note that CPU activity is almost zero now. Here is a one hour view:

And here is the RAM usage:

So after pfBlocker OFF there is a huge RAM release.

[JonathanLee]

@Cabledude said in 4100/6100 Base or Max:

I can recommend it only if you use the cache and want the proxy to cache running.

Could you please do some noob translate and elaborate? I don't understand any of this. But I don't use proxies.

The proxy when using SSL intercept will cache (keep on the SSD) the what passes through the firewall and eventually you will have a nice percentage of traffic that the cache will serve when requests are sent to the firewall. When the item requested is already on the firewall, the firewall will send the information without redownloading it from the Internet. Again this requires certificates installed to work. It accelerates traffic. In the past this was known as content acceleration. Again it now has extra security tools to protect from viruses and invasive containers. The MAX has a big SSD to use with the cache, the bigger the cache the more you can save. Most Internet traffic is the same logos, scripts, photos over and over on website and some changes each day on the website. The cache saves the stuff that is repeated over and over wasting energy, and network bandwidth. It holds it locally.

Dynamic Example: Windows updates Squid uses what's called "dynamic caching" this is for when a huge update is needed for multiple devices. Leading to the network doesn't need to download that same update five or six times once for every laptop on your network, it's the same update. So, the firewalls proxy just caches that and saves it, so it just send it over and over when requested. This works, I have tested it. It even works in reverse for server traffic being requested.

This works great, I use it all the time, plus its energy friendly.

It does require some extra config.

Ref:
https://wiki.squid-cache.org/SquidFaq/SquidLogs

[dennypage]

@Cabledude said in 4100/6100 Base or Max:

So after pfBlocker OFF there is a huge RAM release.

I don’t really see much of a drop off in you graphs. Wire and Active are the most important measures, and there isn’t that much change there. Inactive (allocated but not being used) drops a bit, but that isn’t nearly as important.

Are you using DNSBL? If so, are you using Python mode?

[Cabledude]

Thank you @dennypage for having a look!

@dennypage said in 4100/6100 Base or Max:

I don’t really see much of a drop off in you graphs. Wire and Active are the most important measures, and there isn’t that much change there. Inactive (allocated but not being used) drops a bit, but that isn’t nearly as important.

Free goes up from 13 to 36 after stopping pfBlocker. I figured that is a lot, but maybe I need to learn more about memory management.

Are you using DNSBL? If so, are you using Python mode?

Unbound mode

Thanks,

[Gertjan]

@Cabledude said in 4100/6100 Base or Max:

Unbound mode

Why ? can you motivate this choice ?

If it is performance, memory utilization, etc you're after :

NLLab, the authors of Unbound, said themselves : "use Python mode, that's why we've included it".

[Cabledude]

@Gertjan said in 4100/6100 Base or Max:

@Cabledude said in 4100/6100 Base or Max:
Unbound mode

Why ? can you motivate this choice ?

Ummmm…well, how shall I put this. Okay I’ll be honest and say I just left it at its default setting and didn’t notice that the python option existed. Thank you for pointing this out. I will certainly try the python setting now.

Maybe I’ve been too far conditioned to assuming autodetect mechanisms, i.e. to expect the software to select the right version depending on the environment.
However pfSense isn’t quite the “automatic” firewall, it expects us to do research and manually configure the settings most appropriate for our situation or use case.

[dennypage]

@Cabledude said in 4100/6100 Base or Max:

Maybe I’ve been too far conditioned to assuming autodetect mechanisms, i.e. to expect the software to select the right version depending on the environment.
However pfSense isn’t quite the “automatic” firewall, it expects us to do research and manually configure the settings most appropriate for our situation or use case.

For the most case, the defaults are good and do not require changing. FWIW, pfBlockerNG defaults to Unbound Mode because prior versions of Unbound could crash based upon pfBlockerNG's configuration. This has since been fixed.

[stephenw10]

If you enable pfBlocker with a large number of lists then you will see an increase in CPU usage for a given volume of traffic. pf has to filter against all the list of IPs for pfBlocker auto-rules and that can be non-trivial.

[Cabledude]

@stephenw10 Hi Steve yes I understand that, however this setup is as basic as can be: AdsBasic DNSBL and PRI1 IP lists. The CPU usage is considerable and the CPU graph goes completely flat after shutting down pfblocker. For this reason I am hesitant to get the 2100 as the CPU is the same as the 1100 CPU.

I will try to switch to python mode and see if that makes a substantial difference.

[dennypage]

@Cabledude said in 4100/6100 Base or Max:

this setup is as basic as can be: AdsBasic DNSBL and PRI1 IP lists

Just for reference, the PRI1 IP list is currently 93,947 entries, and the AdsBasic list is a whopping 202,613 entries.

FYI, All the Steven Black lists are pretty large... You might want to consider switching from AdsBasic to Ads. The Ads list is only 9,621 entries, but it hits all the high points.

Edit: EasyList Privacy is another good list at 50,231 entries.

[stephenw10]

Obviously it will only be additional load when traffic is passing and opening new states. Or when it has to reload the ruleset or update the lists.

If it's just idling without any traffic that shouldn't show any significant extra load.

[Gertjan]

@Cabledude

To remove the 'pfBlockerng' load :

( A DNSBL example - I don't use any IP list)

because : most, if not all, of the work of pfBlockerng is : collecting / scanning the log files, and making nice charts, graphs etc.
But, after some testing and observing, and you're sure it works, why not silence all this stuff ?

I'm using a 4100 MAX - it's doing 'close to nothing' when I look at the CPU stats :

or these stats. Check also the 'DNS stats'.

[Cabledude]

@Gertjan
Have you ever measured your 4100’s power draw from the wall socket?

[Cabledude]

@Gertjan said in 4100/6100 Base or Max:

NLLab, the authors of Unbound, said themselves : "use Python mode, that's why we've included it".

So I switched to Python mode.

I started with the smaller Ads list but I added the IP PRI1 list and the worst of the GeoIP lists just to see what that would do to CPU:

This is the graph now:

Which is to say there is less CPU activity than when using unbound mode.

Just to add I started with the EasyList but that feed stubbornly won’t load.

[Edit: my bad. EasyList running fine now]

[dennypage]

@Cabledude You can see the CPU much better if you turn off "processes" (click on the green circle).

[Cabledude]

@dennypage that is pretty neat, never knew about that, thanks!

[Cabledude]

@Gertjan
So here is my CPU graph after switching to python mode, ditching AdsBasic and tweaking the feeds:

Here are my current feeds:

So my feeds list now consists of the entire PRI1 list, a custom ingress list for my home NAS email server (not in use atm), about 6 GeoIP countries and the EasyList.

At this point the CPU load is below 5% average which seems very doable, so I'm swinging back to the 2100 which appears to be quite adequate for this load.

[RobbieTT]

When you buy a 'base' rather than a Max it is a trivial task to add a suitable SSD. It would take a number of lifetimes to exceed the physical write life of my NVMe.

You know, I don't think I ever set Python mode... I will have to check and adjust if required.

️

[Cabledude]

@JonathanLee and to all in this topic:

I decided to give the SG-2100 a chance, mostly based on @JonathanLee 's recommendation and personal experience.
I went with the SSD model.
When I ordered Netgate just released the new SG-2100 with 128GB SSD and my unit in fact came with the larger drive.

No issues so far and idling away at < 3% user util. RAM used 14% of 3388GB.

My ISP upped the cable internet to 400 down / 50 up (from 200/40). Speed tests don't make the SG-2100 sweat.

Power draw 5W steady, peaking to 6W incidentally, on a cheap smart plug in Home Assistant.

I have two weeks to evaluate and if I should prefer the 4100 I can return the 2100 under €25 restocking fee. So far no reason to.

Thanks,