Suricata blocking IPs on passlist, legacy mode blocking both

btspce

@sgnoc The internal dns is also the primary dns for this pfsense install. And the WAN CARP VIP is also being blocked within seconds of suricata starting on wan which triggers a failover resulting in both firewalls becomes master and all traffic dies until both firewalls are rebooted. This is very severe if any random internal ip can be blocked at any moment as it could result in all sorts of issues including management lockout.

SteveITS

Are you two running Suricata on WAN? Have you tried moving it to LAN, where IIRC it’s not even going to see the WAN IP.

sgnoc

@SteveITS No issues running on WAN. I like running the bad reputation lists on my WAN. I'm having issues with internal interfaces blocking internal IPs within ranges or specifically listed on the default IP Pass List.

btspce

@SteveITS Yes WAN plus two other internal interfaces.
But due to suricata blocking the wan vip within seconds on start we have disabled suricata on wan.
But today one of our internal dns servers was blocked by suricata running since yesterday on an internal interface.
This is how we found out that the passlist was not working and sure enough the carp wan vip was blocked by suricata in the logs on wan.
This dns server is the primary pfsense dns server and is on the passlist.
So the conclusion is that there is still issues with the passlist not working randomly and there is a high risk that any random internal ip could be blocked at any moment with this version 7.0.2_3.

We are stopping suricata on all interfaces until this is resolved.

SteveITS

@sgnoc said in Suricata blocking IPs on passlist, legacy mode blocking both:

I like running the bad reputation lists on my WAN.

OK. We put those in a pfBlocker feed or alias and just create a rule.

I’ll watch on ours, AFAIK no internal IPs blocked yet. But thanks for the heads up. Threads like these recent Suricata threads are helpful to know what’s going on.

AlternateShadow

Suricata is running only on internal interfaces here and DST addresses on the pass list are getting blocked.

SteveITS

Is everyone using the Default pass list or a custom list? We're using a custom list with all the "Auto-Generated IP Addresses" boxes checked, and our "trusted" alias added.

btspce

@SteveITS Threads like these is why we waited with the suricata upgrade but unfortunately there were still bugs left. Still grateful for @bmeeks great work.
Keep an eye on your lan ip so it doesn't get randomly blocked (!) . IP's from both the default passlist and the IP Pass List can be blocked as it is now.

We are using a custom passlist with all the "Auto-Generated IP Addresses" boxes checked and trusted alias added aswell.

SteveITS

@btspce What rule is blocking your internal IPs? I'm wondering if it's not something we have enabled.

Our LAN IP has actually shown up but not been blocked... I suppressed a "SURICATA SSH invalid banner" alert yesterday from an internal network scanner/probe IP and it didn't block either.

I'd upgraded Suricata and set it back to Auto the day before.

btspce

@SteveITS Our WAN VIP and our DNS internal IP were both found in suricatas block list and was very much blocked until removed.
Suricata works very well in that regard :)

btspce

@SteveITS

WAN VIP
[Block Dst] [] [1:2402000:6860] ET DROP Dshield Block Listed Source group 1 [] [Classification: Misc Attack] [Priority: 2] {TCP}
[Block Dst] [] [1:2402000:6860] ET DROP Dshield Block Listed Source group 1 [] [Classification: Misc Attack] [Priority: 2] {TCP}

DNS
[Block Dst] [] [1:2035465:4] ET INFO Observed Discord Domain in DNS Lookup (discord .com) [] [Classification: Misc activity] [Priority: 3] {UDP}

SteveITS

@btspce FWIW we don't have either of those enabled...DShield is covered by the ET_Block feed in pfBlocker (so plain fw rule) and "info" is usually meant as informational/observation per Bill and we'd seen a lot of false positives so we don't have those enabled. So, small possibility it's rule related but I would think not.

"when I enable pass list debugging, everything starts working as normal"

Knowing absolutely nothing about the code, maybe thread/timing related?

btspce

@SteveITS Well @bmeeks already found and fixed two bugs related to the passlist randomly not working higher up in this thread which was included in the latest suricata version as I understands it so another one seems likely at this point. I'm waiting for Bill to chime in but it's weird you don't see any issues yet.

Anyway suricata should not be blocking whitelisted ip's.

sgnoc

@SteveITS I'm using the default pass list on all of my interfaces.

bmeeks

@btspce and @sgnoc:
I need some additional information from both of you to help narrow this down.

Post the full output of the suricata.log file for the impacted interface (or interfaces if several). You can easily view that file and copy its contents to the clipboard for pasting here on the forum under the LOGS VIEW tab in Suricata. To make reading the file easier, once you paste its contents into your post, highlight all the text you just pasted with your mouse and then click the "Code" icon at the top of the post submission dialog. That icon looks like this: </>.
Use the DIAGNOSTICS > EDIT FILE menu choice in pfSense and browse to the configuration directory for an impacted Suricata interface and paste the full contents of the pass_list file back here. You will find the file under /usr/local/etc/suricata/suricata_xxx_yyyyy on the firewall. Again, use the DIAGNOSTICS > EDIT FILE menu choice to browse to the file and open it. Paste the contents back here. To format the pasted text so it's easier to read, do the same thing as step #1 above: highlight all of the pasted in text and click the Code icon (</>) to format it.
Are you using VLANs on the impacted interfaces? If so, how many?

Turn on the pass list debugging option as described in this post of mine higher up in this thread: https://forum.netgate.com/topic/184858/suricata-blocking-ips-on-passlist-legacy-mode-blocking-both/8.

I examined the Pass List logic pretty much all day yesterday, but I am not finding anything obviously wrong. Whatever is happening is subtle because not all users are impacted.

btspce

@bmeeks Hello Is there any other way I can send these files to you so we don't have to show our internal/external ip adresses for the whole world ?

bmeeks

@btspce said in Suricata blocking IPs on passlist, legacy mode blocking both:

@bmeeks Hello Is there any other way I can send these files to you so we don't have to show our internal/external ip adresses for the whole world ?

Yes, you can send them to my Gmail account. Here is first part of the address. The second part is of course gmail.com.

billmeeks8

btspce

@bmeeks Email sent

bmeeks

@btspce said in Suricata blocking IPs on passlist, legacy mode blocking both:

@bmeeks Email sent

Confirmed receipt with a reply. Thank you for sending the data.

sgnoc

@bmeeks I'm trying to get this information for you. The trouble I seem to be having is it only happens when pass list debugging is off. When I turned on pass list debugging on the interface, the problem goes away, at least with one interface. I'm waiting to see if another interface with debugging on will alert, but it doesn't alert that often.

I'll continue to try and get you the above information as soon as possible.