Great. Back up and running now. Thanks!

@harmisist said in Snort ignoring pass lists:

I fixed the pass list on both interfaces. I do have several incoming services, so I need both. When I set Snort up several months ago I followed this KB article which shows them adding the pass list to the external interface. It seems to be working now, thanks for the reply.

That should fix it for you, but DO NOT confuse the EXTERNAL_NET variable with the physical external interface (WAN, in your case). They are not the same thing at all. While it is true that most of the !HOME_NET addresses will come into your network via the WAN, that does not mean when you see EXTERNAL_NET to think that only applies to your physical WAN.

Go read some of those Google tutorials I mentioned in my first post and learn what those two variables really mean within Snort. I don't mean to sound rude or patronizing with this statement, but your first action that caused your initial issue, and then your second reply to my post about the solution, leads me to believe you do not understand how Snort should be configured yet. Reading some of those tutorials will help you grasp the key concepts.

@theowolf said in Snort enable_react problem:

okay thanks for checking!

Best,
Theo

And to elaborate a bit more ... DAQ's netmap mode in Snort today requires that you dedicate two physical NIC interfaces to the connection: one as input and the other as output. The netmap module in DAQ then bridges those two physical interfaces, but with Snort sitting between the two operating in Inline IPS mode. So Snort then can either pass on, or drop, packets destined for the other interface.

The main issue that makes this unattractive on a UTM-type firewall such as pfSense is the requirement of using two physical interfaces. So a typical minimal firewall would need four physical NIC interfaces: LAN, WAN and then another pair for the Snort-DAQ netmap bridge pair. That's a little wasteful of physical NICs in most situations.

What I am looking into is patching DAQ's netmap module so that it can use the special "host stack" connection provided by native netmap on FreeBSD and other operating systems. This requires changes to DAQ's netmap module source code. If I can get this to work, then Snort Inline IPS mode can be configured the same as Suricata is done today on pfSense. That mode creates a pipe between the physical NIC interface and the host network stack, so the IPS can exist on the same interface as the LAN or WAN.

See my reply to you similar question here. To repeat the answer from there, "no, there is really no need to use the OpenAppID rules in a home network".

As you surmised, those rules are primarily aimed at identifying various traffic types and are not designed to detect and stop malicious software. Mostly they are to help IT Security admins enforce corporate computing policies such as no or limited access to social media during work hours and other similar policies.

I've tried most of that thread, but no luck. Looks like CPU just can't keep up. Thanks for the suggestions!

@bmeeks

Thanks. I have an old customer connection that still uses PPPoE which will hopefully be gone soon. I disabled the rules, and it is no longer flooding my syslog server. Everything else seems to be working as it should.

Thanks for the info Bill. I appreciate your help and guidance.

@bmeeks said in Suricata fails to start after pfSense reboot:

@kiekar
Start with these two Sticky Posts here in the IDS/IPS sub-forum.

How Automatic SID Management and User Rule Overrides Work in Snort and Suricata
About the New Block-on-Drops-Only Option in Suricata

To get some examples of SID MGMT, go to the SID MGMT tab and click the "enable" checkbox. That will populate the page with content. Open any of the example files and you will find comments inside each that explain the options. The internal syntax is copied from PulledPork, so you can Google the PulledPork utility used with Snort to get some examples of what you can do with the enablesid.conf, disablesid.conf and dropsid.conf files.

Great, thanks so much. You've been a great help

@s0m3f00l said in Suricata Not logging signature matches | | Suricata 4.0.13_11 with pfsense 2.4.4-RELEASE-p1 (arm):

@bmeeks , It appears to have worked. After deleting the package from the GUI "/usr/local/etc/suricata" was left behind. I removed the directory and its contents and then everything reinstalled correctly without intervention. I will wait for the next update and test again.

@bmeeks, thanks for putting up with me sir. You are extraordinarily helpful and supremely knowledgeable about suricata and snort on PFSENSE, and the community would be lost without your input.

Thanks, again.

Glad you got it working. There was likely a file that had been modified and contained "bad content" being left there. When pkg removes software, it compares the md5 hash of the file being removed to the hash the file had when installed. If different, pkg assumes the user modified the file so it leaves it alone. So simply removing and re-installing the package was not removing that modified file. Manually removing the directory and file gets rid of the malformed file. My guess is that it was a different version of the references.config file that was being left behind. That file then would get used with the next installation, so the problem persisted for you.

The alert simply describes a condition where a potential buffer overflow was observed. This could be a false positive or it could represent an actual attempt at compromise. To figure out which it was, start by doing research on Google about the rule that fired. Use the GID and SID to narrow the search. The GID is 124 and the SID is 3. GID 124 is the SMTP preprocessor.

More than likely it is a false positive. If you get one or two now and then, I would chalk it up to false positives. If you get several in a row, or get them quite often, I might investigate further.

The permanent fix for this issue was merged in Suricata package version 4.1.2_2 which is now posted. This issue is resolved.

@wafflez19
Go to the FLOW/STREAM tab and start increasing the TCP Stream Flow Memcap setting. The default is 32 MB (if I recall correctly), but with high core-count processors the default value may need doubling or even quadrupling in order for Suricata to start. The default value works fine on dual and quad-core processors, but higher core counts need much more Stream Memory. In your case, witih 16 cores, I would start with 256 MB and go up from there until Suricata starts reliably.

Search this sub-forum for the same error (stream memcap) and you should find posts similar to yours with the solution. One of the posts in the past included the formula to use for calculating the amount of required memory based on your CPU core count.

Generic Protocol Command Decode : ET INFO Session Traversal Utilities for NAT (STUN Binding Response)

What time of day do you have configured for your automatic update? I've found that anything around midnight US Eastern time will frequently fail as that is apparently when the file is being updated on the Amazon Web Services site. No proof of this theory, just an idea ... 😕 .

The fact a manual update suceeds for you leads me to think you may have that midnight problem. Try moving the update to some other time. I use 0130 (1:30 AM US Eastern Time) and mine never fails. A long time ago, my midnight updates frequently failed.

Earlier this week, late at night while testing some Snort code changes, I was uninstalling and re-installing Snort on a virtual machine over and over. Things were going great until around midnight (about 15 minutes before and after, to be exact), then the rules download would fail with the 500 error for the MD5 file just like you are getting. I continued my coding and testing anyway since I didn't need the Snort Subscriber rules for testing, and after about 12:30 AM the downloads started working again.

@msf2000 said in Suricata frequent crashes:

What can I do to minimize the risk of these bus errors? Any memory settings? Rule sets to enable/disable?

No, nothing really unless you knew precisely which rule or rules were causing the problem. But to complicate things even more, it could be the data in a network packet that causes the issue. It will be pretty random.

I know it's not the answer you want, but the only way for now to really not have the problem is to run Suricata on hardware with an Intel-based CPU. That can be an Atom, i5, i7, etc.

@johnpoz said in DNS Servers being blocked:

@justme2 said in DNS Servers being blocked:

With the roots and TLD servers, they only see the "lower level(s)". A forwarder sees the entire FQDN requested - of every request.

RE: minimization, correct. Should have more clearly (or perhaps more generically correctly) stated that the roots and TLD infrastructure may see the full FQDN, but no other 3rd party would have full visibility of all the queries generated by an organization. Generally, roots/TLD infrastructure are uninterested parties due to mandatory involvement.

Pfsense unbound config is pretty good out of the box.. I change it to not listen on all, and only the interfaces I want and only query outbound on interface(s) I need. Also change from transparent mode to static type. I also turn off the automatic ACLs and do my own.

Yes, when it comes to recursion performance - unbound is particularly good. When tuned on a dedicated box for load - it can be phenomenal for recursion.

The issue I encounter seems related to others who have tried to run inline mode without netmap support; traffic passes for a brief period of time and then all traffic flow stops.

@boobletins Ideally I would like to solve both of the issues, but the WAN latency is a higher priority. I think they are interrelated, but I have no idea which is causing the other.

dmesg | grep interrupt

igb0: Using MSIX interrupts with 5 vectors igb1: Using MSIX interrupts with 5 vectors igb0: Using MSIX interrupts with 5 vectors igb1: Using MSIX interrupts with 5 vectors igb0: Using MSIX interrupts with 5 vectors igb1: Using MSIX interrupts with 5 vectors

vmstat -i

interrupt total rate irq1: atkbd0 6 0 irq9: acpi0 2 0 irq16: ehci0 3845256 1 irq23: ehci1 1831351 1 cpu0:timer 2673498411 1035 cpu1:timer 377015655 146 cpu3:timer 376025919 146 cpu2:timer 375885908 146 irq264: isci0 1 0 irq266: igb0:que 0 186663825 72 irq267: igb0:que 1 39564243 15 irq268: igb0:que 2 46355532 18 irq269: igb0:que 3 49144499 19 irq270: igb0:link 2 0 irq271: igb1:que 0 285667733 111 irq272: igb1:que 1 67109876 26 irq273: igb1:que 2 43993156 17 irq274: igb1:que 3 62085685 24 irq275: igb1:link 8 0 irq276: ahci0 9574415 4 Total 4598261483 1781