@michmoor Depends on the setup but often not.

Well, for anyone that might run across the same issue. I know my problem is a larger ruleset for the interfaces on Suricata, so when I try to rebuild multiple interfaces it pushed PHP over the arbitrary 512G memory limit. I found a solution from others with log sizes causing their problem and a solution listed above to increase memory.

I wasn't thinking, so their solution didn't work for me because I was changing the wrong .inc file. My error was for /usr/local/pkg/suricata/suricata.inc and not /etc/inc/config.inc, which is why it wasn't acting like it increased the memory limit. Once I changed the 512M in suricata.inc to 1024M, I was able to rebuild all of the interfaces at the same time without the error triggering.

This is probably more of a "hack" solution, but I don't want to modify my SID management files just to fit the memory when I have more than enough memory to spare and it isn't taking that much extra for Suricata to go from 512M to 1G. One thing to note for anyone trying this is to make sure to write a script or a cron job to sed the suricata.inc periodically in case there was a package update that overwrote the system. Here is my cron job:

sed -i'.inc' -e 's/ini_set("memory_limit", "512M");/ini_set("memory_limit", "1024M");/g' /usr/local/pkg/suricata/suricata.inc

This seems to have solved my issue, but I'll keep an eye out and modify this response in case it occurs again under the same circumstances.

Thank you both for taking the time to review the problem and for not making me feel stupid for asking my question. You both have made some good suggestions here. I'm going to take them all into consideration and figure out what my next steps. Thanks again!

@bmeeks I will delete and create a new entry there, thanks for pointing me out to the right direction.

@michmoor Snort has a checkbox "Enable Packet Captures" to automatically capture packets, that create alerts. I thought I would have a look at the DNS query packet (i.postimg.cc) and enabled the checkbox & restarted Snort for the interface. When entered the "bad" URL into browser, to my surprise PC's IP was not blocked.

I reverted the checkbox and restarted Snort and PC will still not get blocked. So seems like Snort behavior is back to normal for now. Strange.

@audax10 said in Suricata blocking google.com:

As far as I understand, Legacy Mode blocks the host, while Inline blocks (drop/reject) the triggered rules. This is a big difference. I noticed that Google Maps connects to the same host as "ET INFO Android Device Connectivity Check" rule. So when the host is blocked rather than the rule specification, we will probably block more than we need. Is it correct @bmeeks ?

Yes, Inline IPS Mode is much more selective as it drops individual packets instead of entirely blocking a host by its IP address. However, when using Inline IPS Mode, you must change the action of rules that you wish to block traffic from ALERT to DROP. Otherwise, they will just generate alerts and not drop or block the traffic. Rule actions are default set to ALERT. In Legacy Mode, any alert is interpreted as a block. But that is NOT the case when using Inline IPS Mode. In that mode rule actions are interpreted literally! So if the action is ALERT, that's all that happens. To block traffic in Inline IPS Mode, the rule action must be changed to DROP.

Changing to Inline IPS Mode without also changing rule actions via SID MGMT or some other means will result in nothing being blocked.

The easiest way to manage this is to use the SID MGMT tab features. You can also alter the action of individual rules on the RULES tab and on the ALERTS tab once Inline IPS Mode is enabled. If you are new to this, you should read through this long Sticky Post first: https://forum.netgate.com/topic/143812/snort-package-4-0-inline-ips-mode-introduction-and-configuration-instructions. While a little of the information in that thread is probably dated by now, the majority of it is still applicable.

@stormgate said in Snort and VoIP:

@steveits Thank you so if my Vlan assignments are ix1.50, ix1.30 etc... then my common interface would be assigned ix1 for the Snort interface.
snort.PNG

I will answer for Steve -- "yes, this will be sufficient". Just remember that the same rules are being applied for all of the VLANs on that physical interface. But usually that is fine.

The Snort package offers the OpenAppID feature which can be useful blocking social media sites. You will want to be sure your NIC hardware (the specific driver, actually) fully supports the FreeBSD netmap kernel device. Netmap is used for Inline IPS Operation in Snort, and if you use OpenAppID you definitely want the ability to selectively drop individual traffic flows rather than simply block an entire IP address with Legacy Mode Blocking.

The official Netgate Snort package documentation can be found here: https://docs.netgate.com/pfsense/en/latest/packages/snort/setup.html. Setting up OpenAppID is at the bottom of the linked page.

@fartypants said in Snort keep blocking IPs on suppress list!:

Just wanted to say thanks to BMeeks for suggestion re- run-away snorts above.

Been smacking my head against a similar-but-different problem for days, and that's what it was.

Whodathunkit?

Si.

Multiple, but duplicate, Snort (and Suricata) processes can happen from either of these things:

Something causes the WAN IP to change rapidly or the WAN interface cycles down and back up repeatedly. This causes a built in pfSense script to fire that restarts all packages. Because Snort and Suricata both can take a while to start, rapid back-to-back execution of "restart all packages" can result in multiple instances of Snort or Suricata running on the same interface.

Configuring the Service Watchdog package to monitor Snort or Suricata. Service Watchdog does not understand how the Snort and Suricata packages work, thus it cannot properly monitor them. It also does not understand that both packages will stop and restart themselves when doing rule updates. Service Watchdog simply sees the Snort or Suricata daemon stop, so it immediately restarts it. Service Watchdog does not know that Snort (or Suricata) are in the process of restarting themselves from a rules update, so when it issues its own "start" command you can wind up with two or more processes running on the same physical interface. Service Watchdog should never be configured to monitor either of the two IDS/IPS packages!

If I understand your post correctly, you have devices on your internal networks (LAN) that communicate with a database server located elsewhere on the Internet (accessible via your WAN).

If this true, then you need to simply add the IP address of the remote DB server to a Pass List by creating a list on the PASS LIST tab, accepting the default checked options, adding the IP address of the remote DB to the list using the controls at the bottom of the EDIT LIST screen, then save the new list. Now go to the INTERFACE SETTINGS tab in Snort for your WAN (since your are running Snort on that interface) and select the newly created Pass List in the drop-down selector there. Save that change and restart Snort on the interface.

You do NOT need to be changing the HOME_NET nor EXTERNAL_NET variable settings. Changing those is almost never required. And changing them from the defaults without a full understanding of what they are for and how they work will result in a setup that will NOT trigger rules properly. The fact you altered them in an attempt to solve the problem you describe indicates you may not understand what those parameters are actually for. They define the networks to be protected (HOME_NET) and the networks that are assumed hostile (EXTERNAL_NET). The default setup puts every address/network not defined in HOME_NET in EXTERNAL_NET. Literally, in the PHP code, $EXTERNAL_NET is defined as !$HOME_NET (the leading '!' character indicates a logical NOT operation).

@bmeeks Yah, was running version 6.0.4_1 but I'll dig some more into the NIC and iflib stuff and see if there is a known incompatibility or other issue that I might have missed. Just for posterity currently running the following firmware on the X710 NIC:
sysctl dev.ixl.1.fw_version
dev.ixl.1.fw_version: fw 8.1.63299 api 1.12 nvm 8.10 etid 800093ea oem 1.267.0

@steveits I love multi threading, I have been researching this with Java for some time now with the university, but it's not fully taught, so I get hints from the Professor and book recommendations, and a lot of trial and error. I have a couple binary search methods that split the lists between different threads and cores to help aid in searching. It's amazing to see it work. The concurrent threads does cause confusion, again Class CyclicBarrier helps with making the threads run exactly at the same time or use of volatile variables helps for multi objects that pass in variables in the method headers. Again that is Java not Python. Python has to have something also. Thanks for the reply.

As a side note, I just tried to add a dropped entry into the Suppress list and when clicking the + button, it's not adding the entry. It used to work but now it doesn't. Odd.
GID-SID.png

Unless you have open and forwarded ports on your firewall, it is going to drop all unsolicited inbound traffic anyway. So, really Suricata is not doing anything worthwhile running on the WAN (unless you do have open and forwarded inbound ports on the WAN).

One big issue when it runs on the WAN is that it sees outbound traffic after NAT has been applied, and it sees inbound traffic before NAT is unwound. So, you can never see what internal host (on the LAN, for example) is participating in the conversation. But when you run Suricata on the LAN, then it sees the real native host IP address and the alert logs will show both the local internal host's IP address as well as that of the external participant in the traffic. So, much easier to see what internal hosts may have issues when you run Suricata on the LAN.

This diagram illustrates what I mean. This is how traffic flows with Suricata running in Legacy Blocking Mode. Suricata gets a copy of every packet, examines that packet, and then determines if it should signal a block. But notice the firewall gets the original packet. So, if the firewall drops it anyway due to the default rule dropping unsolicited inbound traffic, there is no point in having Suricata issue a second block for the same traffic.

ids-ips-network-flow-legacy-mode.png

@mrpete said in Suricata Home/External Net - HOW do you make the External list???:

@bmeeks THANKS!

I do want it to be an exact negation.
HAH. THANK YOU.
Bad assumption on my part (I assumed "default" meant "default Home Net" in both places!)

Simple suggestion: modify the External Net dropdown... something like:
from: "default"
to: "default: negate selected Home Net"

Just to be clear, when you set it to "default" that does in fact result in EXTERNAL_NET being an exact one-for-one negation of each entry in HOME_NET. So if you customize HOME_NET to something else other than the default values, then EXTERNAL_NET will follow automatically so long as it remains set for "default". But I can do a better job of explaining that in the help note underneath the drop-down selector.

Pure speculation here, but there was a patch submitted to FreeBSD upstream that fixed an issue where the traffic counters would not register at all when a netmap device was in use. That caused the counters to always show zero traffic.

A fix for that was submitted to FreeBSD upstream by, I believe, the OPNSense team. That fix then made its way into pfSense with a recent base OS update. Maybe the fix has a side-effect for emulation mode operation ??

Might be something you want to report upstream in FreeBSD. But it may be specific to your particular setup with Proxmox. Maybe other Proxmox users with a pfSense VM can chime in here.

@bmeeks We do a lot of work for [unnamed] clients who highly value the security of their information. My consultants are very attuned to those requirements, so I have little concern about their device healthcare. We have very few other visitors, so I didn't pay much attention to who was on the wifi until I discovered that the janitorial service phones were connecting. Now we have a separate guest wifi, direct to Internet on a separate circuit.

pfsense seems much happier since I took Snort off the Wan and cutback severely on the rulesets being used on the LANs.

My wife works for one of those companies that provides the hardware and controls the software. They're still running Win7! They offered to put their access software on her phone, but she would have to sign a document that basically gave them ownership of the phone. Hah!

I, too, think the current Windows Defender does a pretty good job. We happen to have a long term subscription for Malwarebytes, so we stick with it, at least for now. The Linux boxes run ClamAV. None of those are exposed outside the firewall.

Thank you, again, and SteveITS. You saved me a lot of wasted time and a significant addition to my collection of gray hairs!

Rog

@bmeeks I think I found it! So I waited until it blocked it again. Then I searched the snort2c list and found the remote IP of the server. So I confirmed it was getting blocked. Then I searched the alerts, but I increased the list count to 5000 instead of 500 and turned off auto-refresh. Then I found this new alert:

#SURICATA HTTP URI terminated by non-compliant character
suppress gen_id 1, sig_id 2221029

So I suppressed that one, removed the IP in the snort2c table, and the camera came back up!

So now, I just have to see if it holds! I will set the firewall back up to allow that IP and see if it makes a difference since I don't think changing it to the alias with all their IPs made a difference.

Thanks for the patient help on this!

Here is the site with the camera:

Lovington Weather

hola, necesito saber porque al utilizar suricata en pfsense y configurar el modo ips y cargarle reglas en una interfaz al iniciar suricata en dicha interfaz me bloquea el acceso web y me sale ERROR 502 Bad Gategay