@SteveITS said in Suricata process dying due to hyperscan problem:

https://docs.netgate.com/pfsense/en/latest/releases/2-7-1.html#troubleshooting

Thank you for the link. This resolved my problem.

Regards,
Tim

Drifter worries me this could make firewalls have issues if they are really inspecting stack after delivery

@RobertK-1 We've found the stream events cause a lot of issues so routinely disabled those. The disable.conf file as noted above, is permanent. Otherwise for your case the rule IDs may change and/or they add more.

@darkphox said in Sanity check on suppressing alerts:

I have a few specific rules that alert very frequently. I'd like them to continue functioning (drop connections) without spamming my alerts tab, if possible.

Hmm... no way to do that directly in the GUI on pfSense. You could perhaps custom modify those particular rules to try adding the noalert option to the actual rule text.

@darkphox said in Sanity check on suppressing alerts:

After about a year of functioning normally, it began blocking my internal device IP when a rule triggered (when using either the default or a custom passlist).

I've been trying to chase this down. Unfortunately I have never been able to replicate the problem in my test environment. Not being able to replicate the issue makes troubleshooting it and finding a solution very difficult. I'm relegated to bascially guessing for potential causes. So far my guesses have not found the true issue as random users experience random blocks of pass list hosts.

@jimp still happening intermittently;

only on the interface rules page only when changing rules to view from the dropdown gives a blank screen when it happens and the web developers screen returns nothing on the blank screen. has only started since I changed the interface from legacy to inline

not able to repeat consistently, so i have not been able to see the web developerer screen before/after it happens yet.

@MaxBishop said in Suricata - bans LAN device -new behavior on new pf install:

@bmeeks

OK, how do I get it.

Here is the link: https://drive.google.com/file/d/1L-rCf8rF-_C93TFISOx4iWPRgW95sFww/view?usp=sharing

This will pull from my Google Drive folder.

Here are the instructions for installing (and then later removing) the test binary.

To begin, download the suricata-7.0.2_7.pkg file from the link above and transfer it to your firewall placing it in the /root directory. IMPORTANT: make sure you transfer the file in binary (unaltered) form! So, if using WinSCP for the transfer from a Windows PC, choose "Binary" for the transfer type.

Stop all running Suricata instances by executing this command from a shell prompt on the firewall:

/usr/local/etc/rc.d/suricata.sh stop Install the updated version of the Suricata binary using the command below at a shell prompt on the firewall: pkg-static install -f /root/suricata-7.0.2_7.pkg

That command forcibly updates the binary portion of Suricata with a new package leaving the GUI portion unaltered.

Return to the pfSense GUI and restart Suricata on the interfaces using the icons on the INTERFACES tab.

Report back if there is any change in behavior. I sort of don't really expect a change, but maybe we get lucky. This has proven to be an extraordinarily difficult nut to crack in the past (evidenced by the fact I still have not found a true root cause and thus effective solution). Not being able to reproduce it on my end is what makes finding the bug so hard. I have consulted with the upstream Suricata developers, and they told me the Radix Tree code is thread-safe.

Be sure you leave the passlist-debugging: yes option set in suricata.yaml to give me the maximum level of debugging log messages to work with.

To revert, you will need to first remove the Suricata package, verify the updated binary was also removed, then install the package again from the pfSense menu under SYSTEM > PACKAGE MANAGER.

Remove the package using the SYSTEM > PACKAGE MANAGER menu option.

Next, run this command from a shell prompt:

pkg-static delete suricata-7.0.2_7

That insures the updated test binary is truly removed. If you receive a "not found" or "not installed" error, that simply means the updated binary was removed when the package was removed.

Return to the SYSTEM > PACKAGE MANAGER menu and install Suricata again from the official pfSense repo. This will pull down the current RELEASE package version.

@Ramosel try to add the false positive to the surpass list. It won’t block it anymore on Snort

@wolfsden3 have you inspected your surpass list and make sure you don’t have extra space or a mistake in that list. I had an issue with one doing this for AppID the surpass list had a half deleted rule i removed and it was causing issues for me until I deleted it and corrected the spaces. I kept having it block a app until I fixed it.

https://rules.emergingthreatspro.com/open-nogpl/snort-2.9.0/emerging.rules.tar.gz.md5

6b3a1466f57848cb5d0924c76bdc97ec

This is the correct hash

@eldog

FYI, I removed the CARP interfaces from my configuration and the problem went away, even on non-carp addresses. I have a separate installation of PfSense with almost identical hardware that does not use CARP and I have never had a problem with it. Seems to point the finger at CARP generically.

@bmeeks I has been quite some time since the other VLAN's were setup and certainly at least a major version or two ago.

Interesting point regarding the VLAN's on igb3 being seen via promiscuous mode. Perhaps I should drop the VLAN's on igb3 off the snort interface list altogether.

It's not clear what happened to cause the problem, but I was able to "fix" the problem, by adding yet another VLAN (99), associating that with igb3, and low and behold the pve VLAN (111) was available to add within the snort gui - but the newly added VLAN 99 interface is not showing up! Probably something was corrupted over time. I will see how this works, and perhaps look at removing the VLAN's on igb3 within snort to streamline the configuration.

Thanks for the reply and suggestions.

OpenAppID works by examining the SNI in the packet header. Here is a quick explanation of SNI (server name identification) from Cloudfare: https://www.cloudflare.com/learning/ssl/what-is-sni/.

Currently SNI is usually not encrypted, thus it can be seen and interpreted by IDS/IPS tools such as Snort and Suricata. There is a push to move to encrypted SNI. Here is a Cloudfare article describing that process: https://www.cloudflare.com/learning/ssl/what-is-encrypted-sni/. Should ESNI take hold and be widely adopted, Layer 7 IDS/IPS tools could suffer a fatal blow unless MITM (man-in-the-middle) breaking of encryption is utilized.

As I mentioned previously, you will need to reduce the number of rotated blocks.log files you keep on hand. The code reads all of those files from disk into RAM, then correlates them with the current alerts.log file. The purpose is to show how many times a given IP has been blocked. This was done because in the past users wanted to have more "history" available for a blocked IP.

I recommend to users that they enable the option on the GLOBAL SETTINGS tab to automatically remove blocked IPs after an interval, and my suggested interval is 1 hour (or 3 hours max). There is no point in keeping an IP in the block table forever in my view. If Suricata blocked it once, it will block it the next time it attempts to connect. Let that table automatically clear itself out every hour or every 3 hours and you won't have the out-of-memory issue.

The auto-clear routine will not remove IP addresses that are seeing active traffic. It will only remove an IP that has been in the table for the interval chosen AND that IP has not seen any traffic during that interval. For example, if you select 1-hour as the automatic clear interval, then it will only remove IPs from the block table that have not been the source of any traffic for at least the past hour. If an IP is continuing connection attempts, then it will not be removed by the auto-clear routine.

@hsid

Can you share your setup for the Modem - Pfense - Switch - Clients.

In my setup, i have the pfsense currently just for testing pfblockerng and Suricata.

Mikrotik has the DNS and DHCP, this can stay has is. Without double nat.

Switch has the vlans setup has a router on a stick to the Mikrotik.

@SteveITS said in Suricata interfaces on HA setup need to be identical:

Perhaps I am misunderstanding but I don't see any paths in config.xml?

The paths are hard-coded into the template files (and in a few cases the PHP source files themselves). They are not recorded in the config.xml.

The package source code files for Suricata are here: https://github.com/pfsense/FreeBSD-ports/tree/devel/security/pfSense-pkg-suricata/files/usr/local/pkg/suricata

and here: https://github.com/pfsense/FreeBSD-ports/tree/devel/security/pfSense-pkg-suricata/files/usr/local/www/suricata

Feel free to modify them and submit a pull request to add the feature if you would like. Just be sure to fully test the new package with several types of configurations to be sure the migration does not break someone's existing install.

@bmeeks I created a new thread: https://forum.netgate.com/topic/185153/suricata-interfaces-on-ha-setup-need-to-be-identical

Thanks. I was able to reduce the logging to focus on what I was looking for and they are much less noisy than default and working for what we need.

@farazb59 The “stream” events ruleset seems to generate a lot of false positives. Consider just turning it off, which is what we do.

Curious how any traffic goes through the secondary, if it hasn’t become master?