Suricata is blocking LAN and WAN IPs

Bob.Dig

@bmeeks For me, I don't run snort because it has no Block On DROP Only mode.

xm4rcell0x

@bob-dig me too.
It's such a big feature in suricata !
@bmeeks can it be implemented in snort package?

bmeeks

@xm4rcell0x said in Suricata is blocking LAN and WAN IPs:

@bob-dig me too.
It's such a big feature in suricata !
@bmeeks can it be implemented in snort package?

No, because at the API hook where the custom output plugin gets called by the parent binary, Snort does not make the rule action available to test. Plus, Snort only updates its internal "drop" variables when true inline IPS mode operation is enabled in the Snort DAQ. So the short answer is "no", Snort cannot implement a "block on DROP only" option when using Legacy Mode blocking. The Snort binary is just not plumbed up the same as Suricata.

xm4rcell0x

@bmeeks ok, and what do you think about the "checksum offload" ? Disabling it seems has resolved the OP's problem. @Bob-Dig do you have this option enabled?

Bob.Dig

@xm4rcell0x said in Suricata is blocking LAN and WAN IPs:

@Bob-Dig do you have this option enabled?

Yes,tried both, didn't changed anything.

xm4rcell0x

@bmeeks
I don't know why but I've tried to unlock my internal IP, but before that i have changed the Drop action from Drop to Alert.
Cleared the Block Table, restarted suricata on the LAN interface , but it continues to trigger the Drop action instead if the Alert one, so i have uninstalled Suricata (i've followed your instructions)

Now i would a fresh install but there are some files in

usr/local/etc/suricata

usr/local/etc/snort

/suricata-6.0.3_2.txz

/root/suricata-6.0.3_2.txz

(also see the email)
How can i remove these files and folders ? I'm not a linux guy but i know there is the "remove" command, but i don't want to mess anything, that's my production environment.

pfSense-local-lan-Services-Suricata-Alerts (1).png

As you can see, on the right corner there the Alert action selected, but on the left for some rules it is still on the Drop action (already tried to change from that corner).

bmeeks

@xm4rcell0x said in Suricata is blocking LAN and WAN IPs:

@bmeeks ok, and what do you think about the "checksum offload" ? Disabling it seems has resolved the OP's problem. @Bob-Dig do you have this option enabled?

All of the hardware options for checksum offloading, TCP Segmentation, and Large Receive should be disabled. Check the corresponding boxes under SYSTEM > ADVANCED > Networking in the pfSense menu. I belive a reboot is required for all of those changes to become effective.

bmeeks

@xm4rcell0x said in Suricata is blocking LAN and WAN IPs:

@bmeeks
I don't know why but I've tried to unlock my internal IP, but before that i have changed the Drop action from Drop to Alert.
Cleared the Block Table, restarted suricata on the LAN interface , but it continues to trigger the Drop action instead if the Alert one, so i have uninstalled Suricata (i've followed your instructions)

Now i would a fresh install but there are some files in
usr/local/etc/suricata

usr/local/etc/snort

/suricata-6.0.3_2.txz

/root/suricata-6.0.3_2.txz 
(also see the email)
How can i remove these files and folders ? I'm not a linux guy but i know there is the "remove" command, but i don't want to mess anything, that's my production environment.

As you can see, on the right corner there the Alert action selected, but on the left for some rules it is still on the Drop action (already tried to change from that corner).

I sent you a reply via email about removing the files. They are harmless, but removing them is easy. See the email I sent.

For your DROP versus ALERT problem, how did you initially change the rule or rules to DROP? Did you use SID MGMT, or did you click the icon on the ALERTS tab or on the RULES tab when listing the contents of that category? Whichever of those methods you used, simply reverse the process.

On the RULES tab, when listing the contents of a category, there is a reset-to-defaults button that will restore all the rules in the currently selected category to the rule vendor's defaults. That would mean all the rules in the category would be reset to ALERT. If you don't want to do that, go find the SID in question in the list and click the icon under the Action column. In the pop-up dialog, choose the option to restore the default action. Then click Save. Back on the RULES tab, click Apply to send the change to the running Suricata process.

xm4rcell0x

@bmeeks thank you for the email :)
I have changed from the Alert tab, i usually don't change the values from the Rules tab.
If I can help you in any way, let me know Bill!

bmeeks

@xm4rcell0x :

I'm sorry, but I just absolutely cannot duplicate this issue. I've cloned your firewall as closely as possible by including every single IPv4 subnet you have listed in your Pass List. I created 4 interfaces on a virtual machine. I assigned all of your IP subnets to either physical ports on the virtual machine or VLANs defined on a port.

Here is the Pass List I'm using on my test virtual machine. Notice that it explicitly lists the 10.10.20.0/24 subnet which you say is your LAN. I have that IP subnet configured on a firewall interface. The firewall is 10.10.20.1, and I have a Kali Linux virtual machine on the same virtual network at 10.10.20.198 duplicating the IP address of your blocked LAN host.

I have the ET Scan rules enabled with some of them set to DROP. Here are the alerts generated by an nmap scan of the firewall's LAN interface (the 10.10.20.1 IP address) from that Kali Linux machine at IP 10.10.20.198.

I received no block of the Kali Linux host at IP 10.10.20.198. Here is the BLOCKS tab after running the scan (and generating the alerts shown above).

As you see, the Kali Linux host is not being blocked. This is expected as its IP address is within the subnet 10.10.20.0/24 defined in the Pass List.

The direction of traffic will not matter, meaning it is not material whether the LAN host is the source of the traffic, or the destination of the traffic. The blocking plugin simply looks at the IP address in the packet. But just to be sure, I tested three times with the Which IP to Block parameter set to each possible value (SRC, DST, BOTH). They all worked properly. The LAN host was not blocked.

bmeeks

@xm4rcell0x:

Another test for you to try for me. What kind of machine is that 10.10.20.198 host? Is it just a workstation -- perhaps Windows?

If so, and you can access a COMMAND prompt on that PC, I want to run this test.

Go to the RULES tab for the LAN interface in the Suricata GUI on the firewall.
In the Category drop-down, choose "custom.rules".
In the blank textbox that opens, copy and paste in this simple rule:

drop icmp any any -> $HOME_NET any (msg:"Test drop of icmp-request"; flow:to_server; classtype:bad-unknown; sid:20010936; rev:1;)

Click Save when you are done.
Back on the INTERFACES tab in Suricata, restart the LAN instance (I am assuming that is the 10.10.20.0/24 interface).
Clear the BLOCKS tab in Suricata.
Now go to that 10.10.20.198 host, and in a COMMAND prompt window type:

ping 10.10.20.1

or whatever the firewall's LAN IP address is. The idea is to ping the firewall's LAN interface from a host on the LAN to trigger the custom DROP rule.

Check if the host is immediately blocked, or if the first ping succeeds. I would like to establish if it takes some time for the IP to get blocked when triggering DROP rules, or if it gets blocked immediately.

If you can't use the 10.10.20.198 host, then any other host on that network will work as well.

bmeeks

@xm4rcell0x

I did have one other idea pop into my head after my previous posts above. There is one more place in the custom blocking plugin code I just found where I think some random data could cause the errant blocks.

I am creating an additional "fix" I would like for you to test same as you did the first time. Check your email for another test binary package arriving soon. I will include the installation instructions again in the email.

Thanks

Bob.Dig

@bmeeks Interestingly for me the problem is not occurring if I run those rules on the WAN-Interface with Suricata. Also please note that in my case the "offender" is a local Windows, as those rules are privacy-focused rules and not against attacks from outside.
So maybe this is part of the problem in my case.

1:2025275 emerging-info.rules
drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET INFO Windows OS Submitting USB Metadata to Microsoft"

1:2027390 emerging-user_agents.rules
drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET USER_AGENTS Microsoft Device Metadata Retrieval Client User-Agent";

You can trigger those if you (re-)connect a bluetooth device to Windows.

xm4rcell0x

@bob-dig me too, on the WAN side no problem at all.
I'm testing the newer binary from @bmeeks and i have some news.
Suricata only blocks the parent interface's IPs, so in my case the subnet 10.10.20.0/24, but not the VLAN on that interface as you can see in the screenshot.
I have also tried to Drop a Rule where the Src is 192.168.1.10 (igb0 interface, that's my ISP modem) and the destination is 10.10.20.201 (my desktop), 192.168.1.10 wasn't blocked.
That's another rule, where the VLAN 69 (an IOT device, ip: 172.168.69.33) wasn't blocked by suricata.

pfSense-local-lan-Services-Suricata-Alerts (2).png

Bob.Dig

@xm4rcell0x Thanks. Kinda the same here, I only noticed it on parent interface or interface without vlans. But also could be a coincidence.

bmeeks

@bob-dig said in Suricata is blocking LAN and WAN IPs:

@bmeeks Interestingly for me the problem is not occurring if I run those rules on the WAN-Interface with Suricata. Also please note that in my case the "offender" is a local Windows, as those rules are privacy-focused rules and not against attacks from outside.
So maybe this is part of the problem in my case.
1:2025275 emerging-info.rules
drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET INFO Windows OS Submitting USB Metadata to Microsoft"
1:2027390 emerging-user_agents.rules
drop http $HOME_NET any -> $EXTERNAL_NET any (msg:"ET USER_AGENTS Microsoft Device Metadata Retrieval Client User-Agent";
You can trigger those if you (re-)connect a bluetooth device to Windows.

The type of rule and the direction of traffic flow have no significance. What is happening is that when the IP addresses from the rule are compared to the IPs and subnets stored in the Pass List Radix Tree, the comparison seems to fail for you. I don't know why, because it does not fail for me. When the comparison fails, the blocking code thinks the IP address is not in the Pass List and thus blocks it. That's the problem. At the point this decision is made in the code, it is not looking at anything other than the IP address. Nothing else.

bmeeks

So you are getting blocks of hosts residing on the parent interface (the LAN, for example), but not on VLANs defined on that parent? Is that correct?

That can be a hint. It seems I need to work on setting up an environment where I can use actual VLANs for more testing.

xm4rcell0x

@bmeeks yes, that's what I've seen for now.

bmeeks

@xm4rcell0x said in Suricata is blocking LAN and WAN IPs:

@bmeeks yes, that's what I've seen for now.

Can you please list for me (or else screenshot it from the firewall) the physical interfaces defined and then the VLANs assigned to each one? I need to know which of the subnets in your Pass List are VLANs and which are on parent interfaces.

Also provide a list of all the packages you have installed. You mentioned once about having DNSBL (from pfBlockerNG-devel) and HAProxy. Do you still have those installed?

This will help me duplicate your setup better.

xm4rcell0x

@bmeeks
I have these packages:
Acme, Avahi, bandwidth, ntopng, freeradius3, Cron, haproxy-devel, pfblockerng-devel , suricata, pimd, service watchdog . If you need to take a look for my packages configuration let me know.

These are the interfaces and the VLANs.

Screenshot_20211003_181548_com.android.chrome.jpg Screenshot_20211003_181609_com.android.chrome.jpg

bmeeks

@xm4rcell0x said in Suricata is blocking LAN and WAN IPs:

@bmeeks
I have these packages:
Acme, Avahi, bandwidth, ntopng, freeradius3, Cron, haproxy-devel, pfblockerng-devel , suricata, pimd, service watchdog . If you need to take a look for my packages configuration let me know.

These are the interfaces and the VLANs.

Thanks for all of the details. Let me work on creating an environment in my test suite that duplicates yours as best I can. I need to be able to reliably reproduce the problem you, @Bob-Dig and others are having with Pass List IPs getting blocked. Once I can reproduce the problem, then fixing it should be easy.

bmeeks

@xm4rcell0x:

One more request please -- will you screen-capture your Suricata interfaces for me? Just go to the INTERFACES tab in Suricata and post up a copy of that tab so I can see how many instances you are running. I want to duplicate everything as fully as possible ... . Feel free to obfuscate any real public IP addresses, but I don't believe anything private will show up there. It should just show the interfaces and their names and the current Suricata status.

xm4rcell0x

@bmeeks
Yeah no problem. I only have LAN and WAN, i wanted to add iot and guest, but I ran in this problem so I didn't.
Screenshot_20211003_193657_com.android.chrome.jpg
All default settings. I have the et-scan , p2p on the lan (only for testing purposes), on the wan i have p2p, ciarmy, compromised.

bmeeks

@xm4rcell0x:
On all of your Suricata interfaces except for the WAN (because it is PPPoE), you could be running Inline IPS Mode. That would do away with the need for a Pass List. Looking at your physical interfaces, all of them except the PPPoE virtual interface on the WAN are netmap-compatible.

I recommend you consider changing your LAN interface over to Inline IPS Mode. That will automatically over all of your defined VLANs as well. You already have selected rules configured for the DROP action, so you would not need to change anything in your rules configuration.

Simply stop Suricata on the LAN, go to the INTERFACE SETTINGS tab for the LAN and change the blocking mode from Legacy to Inline IPS. I also recommend scrolling down a little farther and changing the Suricata Runmode from autofp to workers. That improves performance a little bit when using the netmap kernel device. Save the changes, then go back and start Suricata.

xm4rcell0x

@bmeeks thanks Bill, I'll give it a try. I'm still scared from 2y ago when I disable my checksum offload on the ix0 and my pfSense crashed.
Later this month I'll surely give it a try, i have to put a newer CPU so that's not a problem if it will crash.

Bob.Dig

For me it even had happened in inline-Mode.

Running it on WAN in Legacy Mode for many days now, no problem at all (WAN IP never got blocked).
One Difference between WAN and LAN would be that there are absolutely no VLANs on WAN in my setup...

xm4rcell0x

I haven't tried it yet
At the end of the month I'll surely try to switch to inline.

bmeeks

@bob-dig said in Suricata is blocking LAN and WAN IPs:

For me it even had happened in inline-Mode.

Running it on WAN in Legacy Mode for many days now, no problem at all (WAN IP never got blocked).
One Difference between WAN and LAN would be that there are absolutely no VLANs on WAN in my setup...

There is not even a blocking module loaded when using Inline IPS Mode, so it would be impossible for Suricata to block an IP completely using Inline IPS Mode. Add to that the Pass List is not used at all in that mode (in fact, the option is hidden when Inline IPS Mode is selected for an interface).

So not sure what you are talking about here. The only thing I can imagine is you had DROP rules triggering on some traffic. And it is expected for them to block the traffic they trigger on. They do this by dropping individual packets from the triggering stream, not by putting the host's IP address in the snort2c blocking table. Inline IPS Mode does not block by IP address, it simply drops (or does not forward) select packets that match rules with the DROP action.

Bob.Dig

@bmeeks said in Suricata is blocking LAN and WAN IPs:

The only thing I can imagine is you had DROP rules triggering on some traffic.

Yup, that's it.

Problem for me was and is, because those "rules" I use gets triggerd also when Windows boots. And when my machine got blocked and I tried another machine, it also could get blocked (or dropped) which really gave me headache, because all my browser are only on Windows machines (real or VMs).

bmeeks

@bob-dig said in Suricata is blocking LAN and WAN IPs:

@bmeeks said in Suricata is blocking LAN and WAN IPs:

The only thing I can imagine is you had DROP rules triggering on some traffic.

Yup, that's it.

Problem for me was and is, because those "rules" I use gets triggerd also when Windows boots. And when my machine got blocked and I tried another machine, it also could get blocked (or dropped) which really gave me headache, because all my browser are only on Windows machines (real or VMs).

Yes, but that is not really the thrust of this thread. The problem other users in this thread were describing occurred with Legacy Mode where the Pass List and the snort2c IP blocking table are used. Neither of them have any relevance to Inline IPS Mode. So it's crossing up apples and oranges to use an old analogy.

Inline IPS Mode does not "block an IP". It selectively drops certain packets. That is an important distinction! Legacy Mode literally blocks an IP address entirely. No traffic of any kind can come and go from a blocked IP address in Legacy Mode. Contrast that to Inline IPS Mode where only specific rule-triggering packets are dropped. The host at that IP is still perfectly free and open to swap other kinds of data across the network, because its IP address is not blocked.

So for example, with Inline IPS Mode I might choose to block icmp-reply packets from IP address 1.2.3.4 with a DROP rule. So no icmp-reply packets from that host will ever get passed. But that host can still send TCP or UDP packets freely (because the IP is not blocked, only selected packets are dropped). It can even send icmp-request packets if I'm not blocking those, but it can't send icmp-reply packets due to the drop rule. But if I was using Legacy Mode Blocking and had the same rule in place, the very first icmp-reply packet that crossed the network from IP 1.2.3.4 would result in IP address 1.2.3.4 being added to the snort2c blocking table. Now ALL traffic from that host, including TCP and UDP traffic, will get blocked. That's because Legacy Mode blocks at the IP level, not at the packet level.

Bob.Dig

@bmeeks Now I got you, thanks for the detailed explanation. So it must have been different then I thought.