6100 SLOW in comparison to Protectli FW6E

bmeeks

@manilx:
Your problem has a simple fix. Using more threads and a high core-count CPU requires more TCP stream-memcap memory be allocated. Go to the FLOW/STREAM tab and increase the amount of Stream Memory Cap under Stream Engine Settings. Try 256 MB and continue increasing in 128 MB or 256 MB chunks until Suricata starts succesfully.

This line in the log is the problem:

21/2/2023 -- 20:14:32 - <Error> -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - failed to setup/expand stream session pool. Expand stream.memcap?

Likely this is triggered by a change in the 6.0.10 binary upstream. Previously Suricata on pfSense was running the 6.0.8 binary. The default is 128 MB, and that may not be enough with some hardware setups.

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@manilx Also changing back to autofp (when suricata doesn't start any longer) and trying to start on the interface results in it NOT starting.
Will revert to working snapshot! Not further testing for the time being.

I have posted a Release Notes thread for the new Suricata package here: https://forum.netgate.com/topic/178138/suricata-6-0-10_1-update-for-pfsense-plus-23-01-release-notes. You can check it out for details on what has changed.

Your startup error is solely due to not having enough TCP stream memory. Raising the Stream Memcap value will fix you up. You were likely already sitting on the edge. This is a not too uncommon occurrence when users have high core-count CPUs.

manilx

@bmeeks OK. Doubled it. Suricata starts. Did speedtest. Can't find any difference from default setting actually (with this unit).

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@bmeeks OK. Doubled it. Suricata starts. Did speedtest. Can't find any difference from default setting actually (with this unit).

Glad it started up. I was pretty confident the Stream Memory increase would fix it.

As for speed, if I recall you were already pretty much at link saturation, so not much room for improvement. But now, it should not experience any random stalls. The other Suricata developer and I are pretty confident we found and fixed that problem. It was the same problem that plagued OPNsense as well when using Suricata.

manilx

@bmeeks Awesome!
I will report here if I find any further issues.

manilx

@manilx I got this error during the night. Never got this before:

pfsense: pfSense.home.lan
PHP ERROR: Type: 1, File: /usr/local/pkg/suricata/suricata_check_for_rule_updates.php, Line: 820, Message: Uncaught ValueError: gettext(): Argument #1 ($message) is too long in /usr/local/pkg/suricata/suricata_check_for_rule_updates.php:820
Stack trace:
#0 /usr/local/pkg/suricata/suricata_check_for_rule_updates.php(820): gettext('- Snort rules: ...')
#1 {main}
thrown

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@manilx I got this error during the night. Never got this before:

pfsense: pfSense.home.lan
PHP ERROR: Type: 1, File: /usr/local/pkg/suricata/suricata_check_for_rule_updates.php, Line: 820, Message: Uncaught ValueError: gettext(): Argument #1 ($message) is too long in /usr/local/pkg/suricata/suricata_check_for_rule_updates.php:820
Stack trace:
#0 /usr/local/pkg/suricata/suricata_check_for_rule_updates.php(820): gettext('- Snort rules: ...')
#1 {main}
thrown

Hmm... never seen that one pop up before. I will look into it. First guess is something changed in the behavior of the third-party gettext() library used for language translation.

bmeeks

@manilx:
One more question, please. Where are you located, and is English perhaps not the primary language there?

Trying to figure out why this has not cropped up in neither my testing nor from the testing done by others in the snapshots development cycle.

manilx

@bmeeks I'm located in Portugal. But my pfsense and all config is in english

manilx

@manilx P.S. And this is new to the latest suricata.

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@bmeeks I'm located in Portugal. But my pfsense and all config is in english

So all the displayed text on the firewall is in English as well? What rules packages do you have enabled on the GLOBAL SETTINGS tab? Do you have an Extra Rules downloads added? I will need to reproduce your rules download setup to see if I can duplicate the error.

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@manilx P.S. And this is new to the latest suricata.

Yes, but the underlying pfSense GUI subsystem has undergone a huge update from PHP 7.4 to PHP 8.1. That is where all the GUI issues with packages and even pfSense itself are coming from. Thousands of lines of PHP code had to be changed in order to be compatible with PHP 8.1. Some got missed. A lot of stuff showed up and got fixed during the DEVEL snapshot testing, but not always everything. Different users can have unique configurations that trigger a PHP issue with the change to 8.1.

manilx

@bmeeks Yes! All in english.
ScreenShot 2023-02-22 at 13.58.11.png

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@bmeeks Yes! All in english.

Okay... that helps. So if you enable the Snort Subscriber Rules you get the error. I will try and reproduce in my test virtual machine.

manilx

@bmeeks Yes. Just enable and run Update(Force)

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@bmeeks Yes. Just enable and run Update(Force)

Well, it worked fine for me just now in a newly updated 2.7 snapshot virtual machine. That machine should be pretty much the same as a current 23.01 machine.

Here is the output from the rules update run (I am in the US Eastern Time Zone):

Starting rules update...  Time: 2023-02-22 09:09:34
	Downloading Emerging Threats Open rules md5 file...
	Checking Emerging Threats Open rules md5 file...
	There is a new set of Emerging Threats Open rules posted.
	Downloading file 'emerging.rules.tar.gz'...
	Done downloading rules file.
	Downloading Snort VRT rules md5 file...
	Checking Snort VRT rules md5 file...
	There is a new set of Snort rules posted.
	Downloading file 'snortrules-snapshot-29200.tar.gz'...
	Done downloading rules file.
	Downloading Feodo Tracker Botnet C2 IP rules file...
	Done downloading rules file.
	Extracting and installing Feodo Tracker Botnet C2 IP rules...
	Feodo Tracker Botnet C2 IP rules were updated.
	Downloading ABUSE.ch SSL Blacklist rules file...
	Done downloading rules file.
	Extracting and installing ABUSE.ch SSL Blacklist rules...
	ABUSE.ch SSL Blacklist rules were updated.
	Downloading Extra Malsilo_Rules rules md5 file...
	Checking Extra Malsilo_Rules rules md5 file...
	Downloading Extra Malsilo_Rules rules file...
	Done downloading rules file.
	Extracting and installing Malsilo_Rules IP rules...
	Extra Malsilo_Rules rules were updated.
	Extracting and installing Emerging Threats Open rules...
	Installation of Emerging Threats Open rules completed.
	Extracting and installing Snort rules...
	Installation of Snort rules completed.
	Copying new config and map files...
	Updating rules configuration for: WAN ...
	Live-Reload of updated rules is enabled...
	Live-Reload of updated rules requested for WAN.
	Updating rules configuration for: LAN ...
	Live-Reload of updated rules is enabled...
	Live-Reload of updated rules requested for LAN.
	Updating rules configuration for: IOT_NET ...
	Live-Reload of updated rules is enabled...
	Live-Reload of updated rules requested for IOT_NET.
The Rules update has finished.  Time: 2023-02-22 09:09:47

Here is the Snort Rules section of the GLOBAL SETTINGS tab:

Do you have the same Snort Rules file configured? Are you using snortrules-snapshot-29200.tar.gz?

manilx

@bmeeks Using snortrules-snapshot-31470.tar.gz

manilx

@manilx ScreenShot 2023-02-22 at 14.16.42.png

bmeeks

@manilx said in 6100 SLOW in comparison to Protectli FW6E:

@bmeeks Using snortrules-snapshot-31470.tar.gz

Very bad!!! Never use Snort 3 rules with Suricata. And never Snort 3 stuff on pfSense.

Change that to snortrules-snapshot-29200.tar.gz, and your problem will go away.

pfSense is not configured for Snort 3 stuff at all.. Snort on pfSense remains on the 2.9.x branch.

Look at this Help Text printed immediately under the line where you specify a Snort Rules file for download:

Enter the rules tarball filename (filename only, do not include the URL.)
Example: snortrules-snapshot-29151.tar.gz
DO NOT specify a Snort3 rules file! Snort3 rules are incompatible with Suricata and will break your installation!

manilx

@bmeeks OK! Always used those...

Changed and update run fine. THX!