Thanks.

First, in order to detect the Eicar virus file you will need a Snort or Emerging Threats rule specifically designed for detecting the file, and you need that rule signature enabled. Secondly, and more importantly, if you are downloading the file from a web site and the web site is SSL encrypted (in other words, it is an HTTPS URL), then the virus payload will not be detected because it is encrypted with the end-to-end encryption of SSL between your browser and the web site.

Neither Snort nor Suricata will, out-of-the-box, detect that file. And most especially they can't detect it when it comes over an encrypted connection such as SSL.

If this is a surprise to you, then welcome to the wonderful world of encryption and VPNs since that technology has pretty much totally neutered deep packet inspection (DPI) technology. The only method to do DPI with encrypted traffic is to set up a MITM (man-in-the-middle) interception. And that comes with its own Pandora's Box of technological and ethical problems.

Hello OP, how did your filebeats setup go?

I have run into this great log apocalypse and am searching for a solution. Really wish filebeats had made it into a package by now but I guess not.

@bmeeks said in News from Snort:

@fireodo said in News from Snort:

@bmeeks said in News from Snort:

@fireodo said in News from Snort:
That's where I would look first, by running a full memory hardware diagnostic if you have one.

No issues (after running memtest86+) and all back to normal (I mean exit on signal 11 😀 )

Thanks anyway for your time and all the best,
fireodo

@Daniel1972 said in Strange behavior on "Update Your Rules Set" at Snort Service:

Thanks bmeeks.

Iam gonna upgrade and see what happends.

This is my production firewall, so it's a critical task, probably I'll take my time to do it.

Thanks again.

Keeping your firewall updated is very important to overall network security. Older software versions can have unpatched vulnerabilities.

I don't know what else is installed on your firewall, but the following things are common causes of rule download issues:

Using a RAM disk, especially for /tmp. Snort needs 256 MB of free space in /tmp to safely download, extract and install updated rules. Many users configure a RAM disk and wind up shooting themselves in the foot because the disk size is too small and they run out of space during the rules update. This does not show up on the Dashboard, though, because at the end of an update Snort will cleanup after itself. So any space that was in use is returned. You can check the pfSense system log to see if there are any disk-space related errors.

Running Squid or any of the Squid related packages. These frequently interfere with the Snort rules downloads.

There have been reports of some IP lists used in pfBlockerNG causing problems, particularly if the list contains any AWS IP space. The Snort team hosts their rules files on Amazon Web Services IP space.

@costanzo said in running Suricata/Snort on a SG-1100 not a good idea ?:

@informel I purchased a SG-1100 for my house and highly recommend it. I am running the following with no problem:

pfBlockerNG devel (This is awesome - no more ads!)
Snort (running only on the WAN)
Acme (For let's encrypt)
Avahi (so I can use AirPlay and AirPrint)
OpenVPN
7 VLANs
DNS Resolver (on by default)

My ISP is through Comcast and I have a 100/20 connection.

As far as performance, I have seen no loss of speed. For example, I run a xfinity speed test and I get the speed I am paying for. OpenVPN from my iPhone works with no disconnects. Sometimes I forget to turn it off only to find it still running.

I did run in to some issues with the Snort service stopping during updates, but after following suggestion from bmeeks to use the "connectivity" IPS profile I haven't had this issue since. I think Snort is a very resource needy service.

Again, for a small environment like a home, in my opinion, is an excellent choice. I work in IT and this is what I run at home.

Hopefully this isn't so old that you don't see this, but I'm wondering what you mean when you say you are running Snort on WAN only? I'm reading that the 1100 can be under-powered for IDS but as it's my house I don't think I need really crazy rules in place, I just want to know if/when something happens. i can only really afford the SG1100 right now and it would be great to hear your thoughts on this (and how it's going, a year later)

@itNGO said in How to skip/bypass fully ignore subnet.:

@bmeeks 👍 Thank you..... Perfect working now....

One other point I should have mentioned. PASS rules are one of the very first things looked at, so when traffic matches a PASS rule the Suricata inspection pipleline is very quickly exited. PASS rules are checked for matches before any other rule, so traffic matching a PASS rule quickly exits the inspection engine, and thus there is minimal performance impact. It's not zero impact, but it is minimal.

Yes, you have the current version of pfSense RELEASE. And FreeBSD-11.3 is not using the iflib subsystem.

I'm not sure what could up with your setup. Thus far I've gotten no other reports like yours. Unfortunately I do not have anything to test on that has that NIC in it.

@bmeeks

Reinstalled Snort 3.2.9.14.1 with no issues.

Thanks Bill for the quick fix to the package.

Some of the things the Smart TV and IoT vendors are doing today with networking is just plain weird! I wonder sometimes if their software developers really and truly understand networking ???

@kiokoman said in Suricata-5.0.3 Package Update -- Release Notes (pfSense-2.5 DEVEL):

it will also be added to the README of the plugin https://github.com/influxdata/telegraf/issues/7843

👍

@wolfsden3 said in Snort borked again! Barnyard2!:

@bmeeks said in Snort borked again! Barnyard2!:

mysql57-client-5.7.30_1

I mended it! LOL

pkg install -f mysql57-client-5.7.30_1 Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. Checking integrity... done (0 conflicting) The following 1 package(s) will be affected (of 0 checked): Installed packages to be REINSTALLED: mysql57-client-5.7.30_1 [pfSense] Number of packages to be reinstalled: 1 Proceed with this action? [y/N]: y [1/1] Reinstalling mysql57-client-5.7.30_1... [1/1] Extracting mysql57-client-5.7.30_1: 100%

I just reinstalled the client and got lucky. It fired right up after doing that.

Hopes this helps someone else too!

Glad that fixed it for you, but as you said -- "you got lucky". Your system is broken someplace or it would not have thrown that error. You may continue to have difficulties with packge updates in the future if your pkg database is somehow corrupt.

@jpgpi250 said in exclude (disable) ET DNS Query...:

@bmeeks Thank you for your time, effort and very extensive answer.
For some reason, step 2 (RULES tab) appears to be unnecessary, after I executed step one, I checked the RULES tab entries, they were already marked as 'user disabled'.
The script, that runs overnight, and caused the alerts, did no longer cause any alerts, so the method explained above, has been successfully implemented.

Thanks again.

I think you misunderstood my reply. I was showing you that there are three different ways to accomplish disabling that rule. Any single one of the three is all you need to do.

@pet1975 said in suricata fail to launch after update:

i just pm you regarding and email :-)

Imported your configuration into my test virtual machine and then installed the Suricata package. It failed to complete installation with the current package version posted in the pfSense packages repository (Suricata v5.0.2_3). It failed the same as it has with you previously. Nothing shows under the SERVICES menu for Suricata after the package installation.

However, installing the latest Suricata package version that I am currently testing was successful. That package version is 5.0.3, and it will be available soon for the pfSense-2.5 DEVEL branch and then a bit later for the pfSense-2.4.5 RELEASE branch.

Even better news is I found what the root problem is, and it is what I had suspected. If you have the option on the GLOBAL SETTINGS tab checked to enable download of the GeoLite2 database, but your MaxMind database license key is invalid, that download will fail. The current PHP script, when detecting that failure performs an exit() call instead of a return() call. Calling exit() in PHP terminates the currently running script. That is in turn prematurely terminating the Suricata package installation PHP script so that the remainder of the installation (putting the entry under the SERVICES menu) fails to complete. Here is the error from the system log -- (I changed the order to show the recent event first, so read the entries from the bottom up for the chronological sequence)

Jul 11 11:37:55 php 92290 [Suricata] ERROR: GeoLite2-Country IP database update check failed. The GeoIP database was not updated! Jul 11 11:37:55 php 92290 [Suricata] ALERT: The Account ID or License Key for MaxMind GeoLite2 is invalid. Jul 11 11:37:54 php 92290 [Suricata] Checking for updated MaxMind GeoLite2 IP database file...

So the root cause of your issue is that your MaxMind GeoLite2 database download license key is invalid. If you fix this, then your Suricata installation will complete even with the current version. To fix this, go to this URL: https://myfirewall_ip/suricata/suricata_global.php. Replace "myfirewall_ip" with the correct value. Once on the tab, scroll down to this area --

Suricata_GeoLite2_Settings.png

If you do not want to use GeoIP rules with Suricata, then uncheck the box for GeoLite2 DB Update. If you do want to use GeoIP rules, then you will need to enter a valid license key. The current key you will see listed is not valid. In the screenshot above, I deliberately obfuscated your key for privacy. Save the changes you make on this screen and then run the remove and reinstall Suricata sequence again. It should complete successfully and show up under the SERVICES menu.

In the upcoming 5.0.3 Suricata version I have changed the GeoLite2 database install code so that a failure to download the database does not terminate the rest of the installation script.

@Cool_Corona said in Bring pfsense/suricata to its knees and eventually die?? No reboot options/recovery available?:

@bmeeks How to make the system aware of loader.conf.local??

AFAIK its using loader.conf and there is no such file in the folder?? Do I need to create and point loader.conf.local somewhere?

I see @DaddyGo has already provided an answer, but I will add to it.

It is common practice on many Unix-type distros to use a *.local version of a configuration file. The OS will look for such a file, and if it sees it in the same place as the system version of that file (for example, in /boot/), then it will append the contents of the *.local file onto the content in the parent file (the one without ".local" on the end).

The purpose of *.local files is to allow user customizations to be added that survive operating system upgrades. During an upgrade, the regular loader.conf file will get overwritten by a new version. But a loader.conf.local file will not get overwritten. It is up to the user to create the *.local file when such a feature is needed.

Snort and Suricata both use the rules.emergingthreats.net and rules.emergingthreatspro.com URLs for downloading ET rules, so there should be no impact. They are not using any hard-coded IP addresses.

However, users running other packages with large IP blocklists (in particular pfBlockerNG or pfBlockerNG-devel) will need to scour the IP lists being used by that package to be sure the AWS infrastructure IP ranges that get assigned to Emerging Threats are not on a block list. Some of those lists can be overly broad at times and block legitimate traffic.

I was a little to hasty.

Errors coming again...

Despite changing the dev.netmap.buf_size="16384"

Output of sysctl -a | grep netmap still looks the same and no change to any values.

Its like its hardcoded and cant be changed??

device netmap
dev.netmap.ixl_rx_miss_bufs: 0
dev.netmap.ixl_rx_miss: 0
dev.netmap.iflib_rx_miss_bufs: 0
dev.netmap.iflib_rx_miss: 0
dev.netmap.iflib_crcstrip: 1
dev.netmap.bridge_batch: 1024
dev.netmap.default_pipes: 0
dev.netmap.priv_buf_num: 4098
dev.netmap.priv_buf_size: 2048
dev.netmap.buf_curr_num: 163840
dev.netmap.buf_num: 163840
dev.netmap.buf_curr_size: 2048
dev.netmap.buf_size: 2048
dev.netmap.priv_ring_num: 4
dev.netmap.priv_ring_size: 20480
dev.netmap.ring_curr_num: 200
dev.netmap.ring_num: 200
dev.netmap.ring_curr_size: 36864
dev.netmap.ring_size: 36864
dev.netmap.priv_if_num: 2
dev.netmap.priv_if_size: 1024
dev.netmap.if_curr_num: 100
dev.netmap.if_num: 100
dev.netmap.if_curr_size: 1024
dev.netmap.if_size: 1024
dev.netmap.ptnet_vnet_hdr: 1
dev.netmap.generic_rings: 1
dev.netmap.generic_ringsize: 1024
dev.netmap.generic_mit: 100000
dev.netmap.generic_hwcsum: 0
dev.netmap.admode: 0
dev.netmap.fwd: 0
dev.netmap.txsync_retry: 2
dev.netmap.mitigate: 1
dev.netmap.no_pendintr: 1
dev.netmap.no_timestamp: 0
dev.netmap.verbose: 0
dev.netmap.ix_rx_miss_bufs: 0
dev.netmap.ix_rx_miss: 0
dev.netmap.ix_crcstrip: 0

Thanks @bmeeks

Yes I am seeing many of those IKE alerts consistently, including from my son's iPhone :( !

Thank you for the explanation around CNAT, had no idea. None of the IPs in the snippet above contain my WAN IP.

Makes total sense re the noise caught when Suricata is applied to the WAN interface, clearly shows my ISP isn't doing a terribly clean job, but hey they're cheap so I can't complain :)