@bmeeks I has been quite some time since the other VLAN's were setup and certainly at least a major version or two ago.

Interesting point regarding the VLAN's on igb3 being seen via promiscuous mode. Perhaps I should drop the VLAN's on igb3 off the snort interface list altogether.

It's not clear what happened to cause the problem, but I was able to "fix" the problem, by adding yet another VLAN (99), associating that with igb3, and low and behold the pve VLAN (111) was available to add within the snort gui - but the newly added VLAN 99 interface is not showing up! Probably something was corrupted over time. I will see how this works, and perhaps look at removing the VLAN's on igb3 within snort to streamline the configuration.

Thanks for the reply and suggestions.

OpenAppID works by examining the SNI in the packet header. Here is a quick explanation of SNI (server name identification) from Cloudfare: https://www.cloudflare.com/learning/ssl/what-is-sni/.

Currently SNI is usually not encrypted, thus it can be seen and interpreted by IDS/IPS tools such as Snort and Suricata. There is a push to move to encrypted SNI. Here is a Cloudfare article describing that process: https://www.cloudflare.com/learning/ssl/what-is-encrypted-sni/. Should ESNI take hold and be widely adopted, Layer 7 IDS/IPS tools could suffer a fatal blow unless MITM (man-in-the-middle) breaking of encryption is utilized.

As I mentioned previously, you will need to reduce the number of rotated blocks.log files you keep on hand. The code reads all of those files from disk into RAM, then correlates them with the current alerts.log file. The purpose is to show how many times a given IP has been blocked. This was done because in the past users wanted to have more "history" available for a blocked IP.

I recommend to users that they enable the option on the GLOBAL SETTINGS tab to automatically remove blocked IPs after an interval, and my suggested interval is 1 hour (or 3 hours max). There is no point in keeping an IP in the block table forever in my view. If Suricata blocked it once, it will block it the next time it attempts to connect. Let that table automatically clear itself out every hour or every 3 hours and you won't have the out-of-memory issue.

The auto-clear routine will not remove IP addresses that are seeing active traffic. It will only remove an IP that has been in the table for the interval chosen AND that IP has not seen any traffic during that interval. For example, if you select 1-hour as the automatic clear interval, then it will only remove IPs from the block table that have not been the source of any traffic for at least the past hour. If an IP is continuing connection attempts, then it will not be removed by the auto-clear routine.

@hsid

Can you share your setup for the Modem - Pfense - Switch - Clients.

In my setup, i have the pfsense currently just for testing pfblockerng and Suricata.

Mikrotik has the DNS and DHCP, this can stay has is. Without double nat.

Switch has the vlans setup has a router on a stick to the Mikrotik.

@SteveITS said in Suricata interfaces on HA setup need to be identical:

Perhaps I am misunderstanding but I don't see any paths in config.xml?

The paths are hard-coded into the template files (and in a few cases the PHP source files themselves). They are not recorded in the config.xml.

The package source code files for Suricata are here: https://github.com/pfsense/FreeBSD-ports/tree/devel/security/pfSense-pkg-suricata/files/usr/local/pkg/suricata

and here: https://github.com/pfsense/FreeBSD-ports/tree/devel/security/pfSense-pkg-suricata/files/usr/local/www/suricata

Feel free to modify them and submit a pull request to add the feature if you would like. Just be sure to fully test the new package with several types of configurations to be sure the migration does not break someone's existing install.

@bmeeks I created a new thread: https://forum.netgate.com/topic/185153/suricata-interfaces-on-ha-setup-need-to-be-identical

Thanks. I was able to reduce the logging to focus on what I was looking for and they are much less noisy than default and working for what we need.

@farazb59 The “stream” events ruleset seems to generate a lot of false positives. Consider just turning it off, which is what we do.

Curious how any traffic goes through the secondary, if it hasn’t become master?

@bitslammer said in pfSense Suricata Crashes on Malformed Block List Entry:

@bmeeks I'm guessing the rules that I enabled have grown over time so I'll try to trim them. Oddly this doesn't happen every time which you'd kind of expect. It's a Netgate 3100 so it looks like I need to look at some other options since this isn't upgradeable. Thanks for the quick reply adn happy holidays.

Several memory parameters have new increased minimums in Suricata 7.x. You are probably seeing the impact of those on the SG-3100. Same issue exists for SG-1100 users, too. 4GB is the new minimum, and even that might get cramped with lots of rules (more than 15,000).

If I were spec'ing a box today for someone who wanted to run IDS/IPS (and most folks want to run pfBlockerNG with DNSBL, too), then I would set 8 GB as a new minimum RAM requirement. The new default ZFS install will also chew up much more RAM than the old UFS setup.

Edit: also looking once again at your log snippet post, I see it seemed to be updating the rules as I see a "rule reload" message. RAM usage will increase during rule swaps, especially if "live rule swap" is enabled.

@gwaitsi I don’t think we ever tried it. Our LAGG was to abstract the interfaces to facilitate HA replacement but that’s no longer a thing…HA can have different hardware now. So no more LAGG.

When we tried inline many years ago we found it broke Remote Desktop over time, no idea why. At the time one NIC was Realtek which isn’t ideal. But, haven’t been in a situation to experiment much again.

@SteveITS

re: scanning on both interfaces...

well, i suppose if i can limit it then the firewall on the windows box that it's port-forwarding to would take care of it.. but in a perfect world i would prefer to have suricata scanning the one port on my wan that i have open.

i've thought about putting that box on it's own vlan, i believe i can do that. i'm just not advanced/savvy enough to where i can whittle it all down to what's needed and what isn't

@feins said in Snort intermittent Crash and snort Deamon stopped.:

@bmeeks

I never create any rules myself all the rules are from the Snort Lan Categories.
The only thing i did is to disable the rules from alerts that cause my application been block only.

Here the rules from the syntax 12558.

alert udp $HOME_NET any -> any 53 (msg:"ET INFO DYNAMIC_DNS Query to a *.is-a-teacher .com Domain"; content:"|01|"; offset:2; depth:1; content:"|00 01 00 00 00 00 00|"; distance:1; within:7; content:"|0c|is-a-teacher|03|com|00|"; nocase; distance:0; fast_pattern; reference:url,help.dyn.com/list-of-dyn-dns-pro-remote-access-domain-names/; classtype:bad-unknown; sid:2042426; rev:2; metadata:attack_target Client_and_Server, created_at 2022_12_08, deployment Perimeter, former_category INFO, performance_impact Low, signature_severity Informational, updated_at 2022_12_08, mitre_tactic_id TA0011, mitre_tactic_name Command_And_Control, mitre_technique_id T1568, mitre_technique_name Dynamic_Resolution;)

I'm working on a Suricata issue at the moment, so give me a little time to reconfigure my test VM for Snort and I will test this rule. It appears to be coming from the ET-INFO category. Looking over it, I don't see any problem with the syntax.

@bmeeks said in Starting suricata, failing netmap, on "oversized" HW (multiple processors/cores + RAM):

I suspect a reboot will be required

Yeah I thought so too, since it's still just in test mode, it's often restarted without consequence...

@bmeeks
Thanks! I had 100M /tmp tmpfs and run exactly into this issue.
After increasing /tmp to 400M it worked!

Here we go again, with three different IP's all showing up as Alerts in the Alerts tab (as they should)

0cfb3856-c273-4460-8cd9-e0aed9c006d8-image.png

But also showing up in the Blocked IPs tab (which they shouldn't):

e9a8579e-9524-445e-a39a-7f14a3e4ec80-image.png

After a Halt of the router for few minutes and a restart Snort interface en/disable seems to work again.
A reboot earlier did not help t o fix the problem.
A poweroff for a few minutes did.

@Maltz hmmm, I will try that and monitor.