Suricata blocking IPs on passlist, legacy mode blocking both

btspce · Dec 22, 2023, 4:39 PM

@SteveITS Our WAN VIP and our DNS internal IP were both found in suricatas block list and was very much blocked until removed.
Suricata works very well in that regard :)

btspce · Dec 22, 2023, 4:54 PM

@SteveITS

WAN VIP
[Block Dst] [] [1:2402000:6860] ET DROP Dshield Block Listed Source group 1 [] [Classification: Misc Attack] [Priority: 2] {TCP}
[Block Dst] [] [1:2402000:6860] ET DROP Dshield Block Listed Source group 1 [] [Classification: Misc Attack] [Priority: 2] {TCP}

DNS
[Block Dst] [] [1:2035465:4] ET INFO Observed Discord Domain in DNS Lookup (discord .com) [] [Classification: Misc activity] [Priority: 3] {UDP}

SteveITS · Dec 22, 2023, 5:04 PM

@btspce FWIW we don't have either of those enabled...DShield is covered by the ET_Block feed in pfBlocker (so plain fw rule) and "info" is usually meant as informational/observation per Bill and we'd seen a lot of false positives so we don't have those enabled. So, small possibility it's rule related but I would think not.

"when I enable pass list debugging, everything starts working as normal"

Knowing absolutely nothing about the code, maybe thread/timing related?

btspce · Dec 22, 2023, 5:27 PM

@SteveITS Well @bmeeks already found and fixed two bugs related to the passlist randomly not working higher up in this thread which was included in the latest suricata version as I understands it so another one seems likely at this point. I'm waiting for Bill to chime in but it's weird you don't see any issues yet.

Anyway suricata should not be blocking whitelisted ip's.

sgnoc · Dec 22, 2023, 9:42 PM

@SteveITS I'm using the default pass list on all of my interfaces.

bmeeks · Dec 23, 2023, 3:25 PM

@btspce and @sgnoc:
I need some additional information from both of you to help narrow this down.

Post the full output of the suricata.log file for the impacted interface (or interfaces if several). You can easily view that file and copy its contents to the clipboard for pasting here on the forum under the LOGS VIEW tab in Suricata. To make reading the file easier, once you paste its contents into your post, highlight all the text you just pasted with your mouse and then click the "Code" icon at the top of the post submission dialog. That icon looks like this: </>.
Use the DIAGNOSTICS > EDIT FILE menu choice in pfSense and browse to the configuration directory for an impacted Suricata interface and paste the full contents of the pass_list file back here. You will find the file under /usr/local/etc/suricata/suricata_xxx_yyyyy on the firewall. Again, use the DIAGNOSTICS > EDIT FILE menu choice to browse to the file and open it. Paste the contents back here. To format the pasted text so it's easier to read, do the same thing as step #1 above: highlight all of the pasted in text and click the Code icon (</>) to format it.
Are you using VLANs on the impacted interfaces? If so, how many?

Turn on the pass list debugging option as described in this post of mine higher up in this thread: https://forum.netgate.com/topic/184858/suricata-blocking-ips-on-passlist-legacy-mode-blocking-both/8.

I examined the Pass List logic pretty much all day yesterday, but I am not finding anything obviously wrong. Whatever is happening is subtle because not all users are impacted.

btspce · Dec 23, 2023, 3:35 PM

@bmeeks Hello Is there any other way I can send these files to you so we don't have to show our internal/external ip adresses for the whole world ?

bmeeks · Dec 23, 2023, 3:39 PM

@btspce said in Suricata blocking IPs on passlist, legacy mode blocking both:

@bmeeks Hello Is there any other way I can send these files to you so we don't have to show our internal/external ip adresses for the whole world ?

Yes, you can send them to my Gmail account. Here is first part of the address. The second part is of course gmail.com.

billmeeks8

btspce · Dec 23, 2023, 3:57 PM

@bmeeks Email sent

bmeeks · Dec 23, 2023, 4:17 PM

@btspce said in Suricata blocking IPs on passlist, legacy mode blocking both:

@bmeeks Email sent

Confirmed receipt with a reply. Thank you for sending the data.

sgnoc · Dec 23, 2023, 4:19 PM

@bmeeks I'm trying to get this information for you. The trouble I seem to be having is it only happens when pass list debugging is off. When I turned on pass list debugging on the interface, the problem goes away, at least with one interface. I'm waiting to see if another interface with debugging on will alert, but it doesn't alert that often.

I'll continue to try and get you the above information as soon as possible.

bmeeks · Dec 23, 2023, 4:22 PM

@sgnoc said in Suricata blocking IPs on passlist, legacy mode blocking both:

@bmeeks I'm trying to get this information for you. The trouble I seem to be having is it only happens when pass list debugging is off. When I turned on pass list debugging on the interface, the problem goes away, at least with one interface. I'm waiting to see if another interface with debugging on will alert, but it doesn't alert that often.

I'll continue to try and get you the above information as soon as possible.

I will take it either way (with and/or without the pass list debugging). I'm really struggling to understand what relationship the pass list debugging option has, though. I have gone through the code multiple times trying to see if anything different happens relative to blocking with that enabled versus disabled, and I am not finding it.

sgnoc · Dec 25, 2023, 12:02 AM

@bmeeks Well, I finally got back to my network. I attempted to start from a fresh suricata install and have had nothing but trouble since. I completely uninstalled Suricata, and then did a fresh install. Now I'm right back to my WAN interface blocking my WAN IP again, like it did in a previous post of mine on this topic.

I've tried uninstalling and reinstalling, restarting the Suricata service, and also restarting the pfSense router. Nothing so far has resolved it. I've had to disable blocking on the WAN interface so I can keep my network going. I have never had this or other interface internal IP blocking issues previous to this major version of Suricata, so I'm stumped. I've collected as much as possible from the logs, but without having the pass list debugging enabled. In this case it is easy, the WAN IP was not put in the default pass list. The interface was up and operational when suricata was installed, and the WAN Gateway is in the list, just not the WAN IP. I have tried disabled blocking, restarting the interface, then enabled and restarting again, but the default pass IP list is not updating with the WAN IP.

I am using VLANs on the internal interfaces, but not the WAN interface. On the internal switch of the XG-7100, I'm using 8 VLANs (not using default VLAN 1), ix0 is the WAN (no VLAN) and ix1 is going to the downstream switches using 6 VLANS (not using default VLAN 1).

As a note of what I saw in the log, it appears for whatever reason the WAN IP was added and removed from the IP Pass List multiple times, with an ending result of being deleted, causing the IP to be blocked on the next alert.

I still have blocking enabled on on other interfaces to do testing, but I have to keep my WAN up, so I don't want to do too much testing unless it is specific with it.

Here is what I've collected.

*** EDIT *** I have to put this log in its own post, because if individual post length limitations

Default Pass List IPs:

10.10.5.0/24
10.10.5.101/32
10.10.6.0/24
10.10.7.0/24
10.10.8.0/24
10.10.9.0/24
10.10.10.0/24
10.10.11.0/24
10.10.15.0/24
10.10.25.0/24
10.10.31.0/29
10.10.32.0/29
10.10.33.0/29
10.10.34.0/29
10.10.35.0/29
10.10.36.0/29
10.10.37.0/29
10.10.45.0/24
10.10.55.0/24
10.10.60.0/29
<WAN Gateway>/32
fe80:6::/64
fe80:7::/64
fe80:8::/64
fe80:9::/64
fe80:10::/64

sgnoc · Dec 25, 2023, 12:02 AM

@bmeeks

Suricata.log for WAN interface (replaced actual WAN IP and WAN Gateway with aliases):

[102572 - Suricata-Main] 2023-12-23 23:56:17 Notice: suricata: This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
[102572 - Suricata-Main] 2023-12-23 23:56:17 Info: cpu: CPUs/cores online: 4
[102572 - Suricata-Main] 2023-12-23 23:56:17 Info: suricata: Setting engine mode to IDS mode by default
[102572 - Suricata-Main] 2023-12-23 23:56:18 Info: app-layer-htp-mem: HTTP memcap: 67108864
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Creating automatic firewall interface IP address Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix0 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix0 IPv4 address <WAN IP> to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lo0 IPv6 address 0000:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lo0 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lo0 IPv4 address 127.0.0.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.5 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.5 IPv4 address 10.10.5.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.15 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.15 IPv4 address 10.10.15.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.25 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.25 IPv4 address 10.10.25.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.45 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.45 IPv4 address 10.10.45.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.31 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.31 IPv4 address 10.10.31.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.32 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.32 IPv4 address 10.10.32.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.33 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.33 IPv4 address 10.10.33.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.34 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.34 IPv4 address 10.10.34.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.35 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.35 IPv4 address 10.10.35.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.36 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.36 IPv4 address 10.10.36.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.37 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.37 IPv4 address 10.10.37.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface lagg0.38 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6da to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.55 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.55 IPv4 address 10.10.55.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.60 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d9 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ix1.60 IPv4 address 10.10.60.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns1 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns1 IPv4 address 10.10.6.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns1 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns2 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns2 IPv4 address 10.10.7.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns2 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns3 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns3 IPv4 address 10.10.8.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns3 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns4 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns4 IPv4 address 10.10.9.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns4 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns5 IPv6 address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns5 IPv4 address 10.10.10.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface ovpns5 IPv6 address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Adding firewall interface tun_wg0 IPv4 address 10.10.11.1 to automatic interface IP Pass List.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: logopenfile: alert-pf output device (regular) initialized: block.log
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Loading and parsing Pass List from: /usr/local/etc/suricata/suricata_57861_ix0/passlist.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Pass List /usr/local/etc/suricata/suricata_57861_ix0/passlist processed: Total entries parsed: 26, IP addresses/netblocks/aliases added to No Block list: 26, IP addresses/netblocks ignored because they were covered by existing entries: 0.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: pfSense Suricata Custom Blocking Module initialized: pf-table=snort2c  block-ip=both  kill-state=yes  block-drops-only=yes  passlist-debugging=no
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: alert-pf: Created Interface IP Address change monitoring thread for auto-whitelisting of firewall interface IP addresses.
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: logopenfile: fast output device (regular) initialized: alerts.log
[102298 - Suricata-Main] 2023-12-23 23:56:18 Info: logopenfile: http-log output device (regular) initialized: http.log
[120645 - Suricata-IM#01] 2023-12-23 23:56:18 Info: alert-pf: Firewall Interface IP Address Change Monitor Thread IM#01 has successfully started.
[102298 - Suricata-Main] 2023-12-23 23:56:26 Error: detect-tls-ja3-hash: ja3 support is not enabled
[102298 - Suricata-Main] 2023-12-23 23:56:26 Error: detect: error parsing signature "alert tls $HOME_NET any -> $EXTERNAL_NET any (msg:"ET JA3 Hash - Suspected Cobalt Strike Malleable C2 M1 (set)"; flow:established,to_server; ja3.hash; content:"eb88d0b3e1961a0562f006e5ce2a0b87"; ja3.string; content:"771,49192-49191-49172-49171"; flowbits:set,ET.cobaltstrike.ja3; flowbits:noalert; classtype:command-and-control; sid:2028831; rev:1; metadata:affected_product Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint, created_at 2019_10_15, deployment Perimeter, former_category JA3, malware_family Cobalt_Strike, confidence Low, signature_severity Major, updated_at 2019_10_15, mitre_tactic_id TA0011, mitre_tactic_name Command_And_Control, mitre_technique_id T1001, mitre_technique_name Data_Obfuscation;)" from file /usr/local/etc/suricata/suricata_57861_ix0/rules/suricata.rules at line 7933
[102298 - Suricata-Main] 2023-12-23 23:56:26 Error: detect-tls-ja3s-hash: ja3(s) support is not enabled
[102298 - Suricata-Main] 2023-12-23 23:56:26 Error: detect: error parsing signature "drop tls $EXTERNAL_NET any -> $HOME_NET any (msg:"ETPRO JA3 Hash - Suspected ASYNCRAT Server Cert (ja3s)"; flow:established,to_client; flowbits:isset,ETPRO.asyncrat.flowbit; ja3s.hash; content:"b74704234e6128f33bff9865696e31b3"; fast_pattern; reference:url,github.com/NYAN-x-CAT/AsyncRAT-C-Sharp; classtype:command-and-control; sid:2842478; rev:1; metadata:attack_target Client_Endpoint, created_at 2020_05_08, deployment Perimeter, former_category JA3, performance_impact Low, confidence Low, signature_severity Major, updated_at 2020_05_08;)" from file /usr/local/etc/suricata/suricata_57861_ix0/rules/suricata.rules at line 8028
[102298 - Suricata-Main] 2023-12-23 23:56:49 Error: detect-parse: no terminating ";" found
[102298 - Suricata-Main] 2023-12-23 23:56:49 Error: detect: error parsing signature "drop tcp-pkt $HOME_NET any -> $EXTERNAL_NET any (msg:"ET MALWARE Win32/DarkVision RAT CnC Checkin M2"; flow:established,to_server; dsize:4; content:"|7c 02 00 00|" from file /usr/local/etc/suricata/suricata_57861_ix0/rules/suricata.rules at line 32725
[102298 - Suricata-Main] 2023-12-23 23:56:49 Info: detect: 2 rule files processed. 32727 rules successfully loaded, 117 rules failed
[102298 - Suricata-Main] 2023-12-23 23:56:49 Info: threshold-config: Threshold config parsed: 0 rule(s) found
[102298 - Suricata-Main] 2023-12-23 23:56:49 Info: detect: 32727 signatures processed. 199 are IP-only rules, 7465 are inspecting packet payload, 24937 inspect application layer, 106 are decoder event only
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.http.rtf.download' is checked but not set. Checked in 2815709 and 10 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ETPRO.RTF' is checked but not set. Checked in 2020700 and 8 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.Multimedia.Download' is checked but not set. Checked in 2827897 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.MP4.Download' is checked but not set. Checked in 2827898 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'OLE.CompoundFile' is checked but not set. Checked in 2815527 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ETPRO.wget.UA' is checked but not set. Checked in 2820973 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.BonitaDefaultCreds' is checked but not set. Checked in 2036817 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.Keitaro1' is checked but not set. Checked in 2831446 and 2 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.PROPFIND' is checked but not set. Checked in 2049438 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.raiffeisenapk' is checked but not set. Checked in 2828074 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ETPRO.w32unknown' is checked but not set. Checked in 2816366 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.genericphish' is checked but not set. Checked in 2850094 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.kumquat' is checked but not set. Checked in 2044067 and 1 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'ET.gadu.loginsent' is checked but not set. Checked in 2008299 and 0 other sigs
[102298 - Suricata-Main] 2023-12-23 23:56:49 Warning: detect-flowbits: flowbit 'file.onenote' is checked but not set. Checked in 61666 and 1 other sigs
[102298 - Suricata-Main] 2023-12-23 23:58:18 Info: runmodes: Using 1 live device(s).
[120661 - RX#01-ix0] 2023-12-23 23:58:19 Info: pcap: ix0: running in 'auto' checksum mode. Detection of interface state will require 1000 packets
[120661 - RX#01-ix0] 2023-12-23 23:58:19 Info: pcap: ix0: snaplen set to 1518
[102298 - Suricata-Main] 2023-12-23 23:58:19 Notice: threads: Threads created -> RX: 1 W: 4 FM: 1 FR: 1   Engine started.
[120645 - Suricata-IM#01] 2023-12-23 23:58:19 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[120645 - Suricata-IM#01] 2023-12-23 23:58:19 Info: alert-pf: Deleted address <WAN IP> from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:19 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[120645 - Suricata-IM#01] 2023-12-23 23:58:19 Info: alert-pf: Added address <WAN IP> to automatic firewall interface IP Pass List.
[120661 - RX#01-ix0] 2023-12-23 23:58:20 Info: checksum: No packets with invalid checksum, assuming checksum offloading is NOT used
[120645 - Suricata-IM#01] 2023-12-23 23:58:21 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[120645 - Suricata-IM#01] 2023-12-23 23:58:21 Info: alert-pf: Deleted address <WAN IP> from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:22 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[120645 - Suricata-IM#01] 2023-12-23 23:58:22 Info: alert-pf: Added address <WAN IP> to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:24 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[120645 - Suricata-IM#01] 2023-12-23 23:58:24 Info: alert-pf: Deleted address <WAN IP> from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:24 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[120645 - Suricata-IM#01] 2023-12-23 23:58:24 Info: alert-pf: Added address <WAN IP> to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Deleted address 10.10.6.1 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Added address 10.10.6.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Deleted address 10.10.7.1 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:50 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Added address 10.10.7.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Added address 10.10.9.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:51 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:52 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:52 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:52 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:52 Info: alert-pf: Added address 10.10.10.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:52 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:52 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Added address 10.10.6.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns1.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Deleted address 10.10.7.1 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Added address 10.10.7.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns2.
[120645 - Suricata-IM#01] 2023-12-23 23:58:53 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Deleted address 10.10.9.1 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Added address 10.10.9.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns4.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Deleted address 10.10.10.1 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Deleted address fe80:0000:0000:0000:0000:0000:0000:0001 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:54 Info: alert-pf: Deleted address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 from automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:55 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:55 Info: alert-pf: Added address fe80:0000:0000:0000:0208:a2ff:fe0e:f6d8 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:55 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:55 Info: alert-pf: Added address 10.10.10.1 to automatic firewall interface IP Pass List.
[120645 - Suricata-IM#01] 2023-12-23 23:58:55 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ovpns5.
[120645 - Suricata-IM#01] 2023-12-23 23:58:55 Info: alert-pf: Added address fe80:0000:0000:0000:0000:0000:0000:0001 to automatic firewall interface IP Pass List.

sgnoc · Dec 25, 2023, 12:21 AM

New information. I just got a block on a lagg from the XG-7100 internal switch. It blocked an internal 10.10.33.2 IP from the lag0.33 subnet 10.10.33.0/29. I'm just getting debugging enabled and restarting interfaces, but this his before I was able.

Default Pass List IPs:

10.10.5.0/24
10.10.5.101/32
10.10.6.0/24
10.10.7.0/24
10.10.8.0/24
10.10.9.0/24
10.10.10.0/24
10.10.11.0/24
10.10.15.0/24
10.10.25.0/24
10.10.31.0/29
10.10.32.0/29
10.10.33.0/29
10.10.34.0/29
10.10.35.0/29
10.10.36.0/29
10.10.37.0/29
10.10.45.0/24
10.10.55.0/24
10.10.60.0/29
<WAN Gateway>/32
fe80:6::/64
fe80:7::/64
fe80:8::/64
fe80:9::/64
fe80:10::/64

Block Log:

12/24/2023-19:05:34.343962  [wDrop] [**] [1:2032981:2] ET SCAN Bing Webcrawler User-Agent (BingBot) [**] [Classification: Not Suspicious Traffic] [Priority: 3] {TCP} 172.70.175.165:55592 -> 10.10.33.2:80

Suricata.log (too large to post without uploading a file):
Suricata_lagg0.33.Block.txt

sgnoc · Dec 25, 2023, 2:04 AM

@bmeeks I did some additional testing and got a block logged with pass list debugging on. This was on the WAN interface. I haven't gotten one logged on an internal interface, but this shows where the WAN IP was blocked.

Default IP Pass List:

10.10.5.0/24
10.10.5.101/32
10.10.6.0/24
10.10.7.0/24
10.10.8.0/24
10.10.9.0/24
10.10.10.0/24
10.10.11.0/24
10.10.15.0/24
10.10.25.0/24
10.10.31.0/29
10.10.32.0/29
10.10.33.0/29
10.10.34.0/29
10.10.35.0/29
10.10.36.0/29
10.10.37.0/29
10.10.45.0/24
10.10.55.0/24
10.10.60.0/29
<WAN Gateway>/32
fe80:6::/64
fe80:7::/64
fe80:8::/64
fe80:9::/64
fe80:10::/64

passlist_debug.log:

12/24/2023-19:07:07.155889  Pass List debugging enabled. Processing file: /usr/local/etc/suricata/suricata_57861_ix0/passlist.
12/24/2023-19:07:07.156041  Added IPv4 netblock 10.10.5.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.5.0/24.
12/24/2023-19:07:07.156064  Added IPv4 address 10.10.5.101/32 from Pass List.
12/24/2023-19:07:07.156095  Added IPv4 netblock 10.10.6.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.6.0/24.
12/24/2023-19:07:07.156106  Added IPv4 netblock 10.10.7.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.7.0/24.
12/24/2023-19:07:07.156114  Added IPv4 netblock 10.10.8.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.8.0/24.
12/24/2023-19:07:07.156121  Added IPv4 netblock 10.10.9.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.9.0/24.
12/24/2023-19:07:07.156129  Added IPv4 netblock 10.10.10.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.10.0/24.
12/24/2023-19:07:07.156136  Added IPv4 netblock 10.10.11.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.11.0/24.
12/24/2023-19:07:07.156144  Added IPv4 netblock 10.10.15.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.15.0/24.
12/24/2023-19:07:07.156151  Added IPv4 netblock 10.10.25.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.25.0/24.
12/24/2023-19:07:07.156158  Added IPv4 netblock 10.10.31.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.31.0/29.
12/24/2023-19:07:07.156166  Added IPv4 netblock 10.10.32.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.32.0/29.
12/24/2023-19:07:07.156173  Added IPv4 netblock 10.10.33.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.33.0/29.
12/24/2023-19:07:07.156180  Added IPv4 netblock 10.10.34.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.34.0/29.
12/24/2023-19:07:07.156187  Added IPv4 netblock 10.10.35.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.35.0/29.
12/24/2023-19:07:07.156215  Added IPv4 netblock 10.10.36.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.36.0/29.
12/24/2023-19:07:07.156224  Added IPv4 netblock 10.10.37.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.37.0/29.
12/24/2023-19:07:07.156232  Added IPv4 netblock 10.10.45.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.45.0/24.
12/24/2023-19:07:07.156239  Added IPv4 netblock 10.10.55.0/24 to IPv4 Radix Tree created from Pass List entry 10.10.55.0/24.
12/24/2023-19:07:07.156246  Added IPv4 netblock 10.10.60.0/29 to IPv4 Radix Tree created from Pass List entry 10.10.60.0/29.
12/24/2023-19:07:07.156253  Added IPv4 address <WAN Gateway>/32 from Pass List.
12/24/2023-19:07:07.156277  Added IPv6 netblock fe80:0006:0000:0000:0000:0000:0000:0000/64 to IPv6 Radix Tree created from Pass List entry fe80:6::/64.
12/24/2023-19:07:07.156291  Added IPv6 netblock fe80:0007:0000:0000:0000:0000:0000:0000/64 to IPv6 Radix Tree created from Pass List entry fe80:7::/64.
12/24/2023-19:07:07.156300  Added IPv6 netblock fe80:0008:0000:0000:0000:0000:0000:0000/64 to IPv6 Radix Tree created from Pass List entry fe80:8::/64.
12/24/2023-19:07:07.156310  Added IPv6 netblock fe80:0009:0000:0000:0000:0000:0000:0000/64 to IPv6 Radix Tree created from Pass List entry fe80:9::/64.
12/24/2023-19:07:07.156322  Added IPv6 netblock fe80:0010:0000:0000:0000:0000:0000:0000/64 to IPv6 Radix Tree created from Pass List entry fe80:10::/64.
12/24/2023-19:07:07.156340  Completed processing Pass List /usr/local/etc/suricata/suricata_57861_ix0/passlist. Total entries parsed: 26, Unique IP addresses/netblocks/aliases added to Radix Trees: 26, IP addresses/netblocks ignored because they were covered by existing Radix Tree entries: 0.

12/24/2023-19:11:38.942701  Thread: W#01  SRC IP: 194.26.135.109 did not match any Pass List entry, so adding to block list.
12/24/2023-19:11:38.964387  Thread: W#01  Successfully added IP: 194.26.135.109 to pf table snort2c for blocking.
12/24/2023-19:11:39.052857  Thread: W#01  Successfully killed any open states for IP: 194.26.135.109, so any stateful traffic is blocked.
12/24/2023-19:11:38.942701  Thread: W#01  DST IP: <WAN IP> covered by Pass List entry <WAN IP>/32 - not blocking.
12/24/2023-19:11:53.817090  Thread: W#03  SRC IP: 77.90.185.73 did not match any Pass List entry, so adding to block list.
12/24/2023-19:11:53.877467  Thread: W#03  Successfully added IP: 77.90.185.73 to pf table snort2c for blocking.
12/24/2023-19:11:53.964781  Thread: W#03  Successfully killed any open states for IP: 77.90.185.73, so any stateful traffic is blocked.
12/24/2023-19:11:53.817090  Thread: W#03  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:11:53.964897  Thread: W#03  Successfully added IP: <WAN IP> to pf table snort2c for blocking.
12/24/2023-19:11:54.052646  Thread: W#03  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:12:13.266186  Thread: W#04  SRC IP: 35.203.211.174 did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:13.464580  Thread: W#04  Successfully added IP: 35.203.211.174 to pf table snort2c for blocking.
12/24/2023-19:12:13.551139  Thread: W#04  Successfully killed any open states for IP: 35.203.211.174, so any stateful traffic is blocked.
12/24/2023-19:12:13.266186  Thread: W#04  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:13.637498  Thread: W#04  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:12:18.425677  Thread: W#01  SRC IP: 35.203.211.7 did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:18.514789  Thread: W#01  Successfully added IP: 35.203.211.7 to pf table snort2c for blocking.
12/24/2023-19:12:18.602906  Thread: W#01  Successfully killed any open states for IP: 35.203.211.7, so any stateful traffic is blocked.
12/24/2023-19:12:18.425677  Thread: W#01  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:18.689395  Thread: W#01  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:12:35.402347  Thread: W#04  SRC IP: 65.49.20.108 did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:35.709159  Thread: W#04  Successfully added IP: 65.49.20.108 to pf table snort2c for blocking.
12/24/2023-19:12:35.794935  Thread: W#04  Successfully killed any open states for IP: 65.49.20.108, so any stateful traffic is blocked.
12/24/2023-19:12:35.402347  Thread: W#04  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:35.880853  Thread: W#04  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:12:40.452282  Thread: W#02  SRC IP: 167.94.145.80 did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:40.774064  Thread: W#02  Successfully added IP: 167.94.145.80 to pf table snort2c for blocking.
12/24/2023-19:12:40.860822  Thread: W#02  Successfully killed any open states for IP: 167.94.145.80, so any stateful traffic is blocked.
12/24/2023-19:12:40.452282  Thread: W#02  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:40.947516  Thread: W#02  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:12:40.918625  Thread: W#03  SRC IP: 62.204.41.63 did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:41.273533  Thread: W#03  Successfully added IP: 62.204.41.63 to pf table snort2c for blocking.
12/24/2023-19:12:41.360033  Thread: W#03  Successfully killed any open states for IP: 62.204.41.63, so any stateful traffic is blocked.
12/24/2023-19:12:40.918625  Thread: W#03  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:41.446572  Thread: W#03  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:12:51.957100  Thread: W#03  SRC IP: 77.90.185.166 did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:52.415512  Thread: W#03  Successfully added IP: 77.90.185.166 to pf table snort2c for blocking.
12/24/2023-19:12:52.502160  Thread: W#03  Successfully killed any open states for IP: 77.90.185.166, so any stateful traffic is blocked.
12/24/2023-19:12:51.957100  Thread: W#03  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:12:52.588762  Thread: W#03  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:13:17.273336  Thread: W#04  SRC IP: 80.66.83.171 did not match any Pass List entry, so adding to block list.
12/24/2023-19:13:17.672138  Thread: W#04  Successfully added IP: 80.66.83.171 to pf table snort2c for blocking.
12/24/2023-19:13:17.758300  Thread: W#04  Successfully killed any open states for IP: 80.66.83.171, so any stateful traffic is blocked.
12/24/2023-19:13:17.273336  Thread: W#04  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:13:17.844843  Thread: W#04  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:13:33.413564  Thread: W#02  SRC IP: 77.90.185.127 did not match any Pass List entry, so adding to block list.
12/24/2023-19:13:33.854843  Thread: W#02  Successfully added IP: 77.90.185.127 to pf table snort2c for blocking.
12/24/2023-19:13:33.940725  Thread: W#02  Successfully killed any open states for IP: 77.90.185.127, so any stateful traffic is blocked.
12/24/2023-19:13:33.413564  Thread: W#02  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:13:34.026931  Thread: W#02  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:13:46.250811  Thread: W#04  SRC IP: 198.235.24.144 did not match any Pass List entry, so adding to block list.
12/24/2023-19:13:46.513589  Thread: W#04  Successfully added IP: 198.235.24.144 to pf table snort2c for blocking.
12/24/2023-19:13:46.600316  Thread: W#04  Successfully killed any open states for IP: 198.235.24.144, so any stateful traffic is blocked.
12/24/2023-19:13:46.250811  Thread: W#04  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:13:46.687333  Thread: W#04  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:14:06.084828  Thread: W#03  SRC IP: 31.220.1.83 did not match any Pass List entry, so adding to block list.
12/24/2023-19:14:06.197129  Thread: W#03  Successfully added IP: 31.220.1.83 to pf table snort2c for blocking.
12/24/2023-19:14:06.283613  Thread: W#03  Successfully killed any open states for IP: 31.220.1.83, so any stateful traffic is blocked.
12/24/2023-19:14:06.084828  Thread: W#03  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:14:06.370284  Thread: W#03  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:14:38.394887  Thread: W#04  SRC IP: 77.90.185.92 did not match any Pass List entry, so adding to block list.
12/24/2023-19:14:38.518387  Thread: W#04  Successfully added IP: 77.90.185.92 to pf table snort2c for blocking.
12/24/2023-19:14:38.605196  Thread: W#04  Successfully killed any open states for IP: 77.90.185.92, so any stateful traffic is blocked.
12/24/2023-19:14:38.394887  Thread: W#04  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:14:38.692102  Thread: W#04  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:15:00.905606  Thread: W#04  SRC IP: 77.90.185.14 did not match any Pass List entry, so adding to block list.
12/24/2023-19:15:01.321169  Thread: W#04  Successfully added IP: 77.90.185.14 to pf table snort2c for blocking.
12/24/2023-19:15:01.407791  Thread: W#04  Successfully killed any open states for IP: 77.90.185.14, so any stateful traffic is blocked.
12/24/2023-19:15:00.905606  Thread: W#04  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:15:01.494809  Thread: W#04  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.
12/24/2023-19:15:37.899907  Thread: W#02  SRC IP: 95.214.55.244 did not match any Pass List entry, so adding to block list.
12/24/2023-19:15:38.293385  Thread: W#02  Successfully added IP: 95.214.55.244 to pf table snort2c for blocking.
12/24/2023-19:15:38.379657  Thread: W#02  Successfully killed any open states for IP: 95.214.55.244, so any stateful traffic is blocked.
12/24/2023-19:15:37.899907  Thread: W#02  DST IP: <WAN IP> did not match any Pass List entry, so adding to block list.
12/24/2023-19:15:38.465667  Thread: W#02  Successfully killed any open states for IP: <WAN IP>, so any stateful traffic is blocked.

Suricata.log:

Suricata_ix0_SuricataLog.txt

bmeeks · Dec 25, 2023, 2:17 AM

@sgnoc:
To make some sense of this you will need to correlate times for your WAN IP from two different logs.

First, find all the times in suricata.log where the WAN IP address is deleted by the interface monitoring thread. You should be able to find those quickly by searching for your WAN IP as a string in that log.

Second, find the times from the Pass List debugging log where the WAN IP was blocked.

Here is what I think is happening. Your WAN IP (and all of the firewall interface IP addresses) are not part of the default Pass List you see in the GUI. Instead, a separate monitoring thread within the custom blocking module runs in a continuous loop subscribed to the kernel routing messages. The kernel sends that thread a notice each time an IP address is removed from or added to a firewall interface. The thread in turn either adds that firewall interface IP to or removes it from the Radix Tree structure that stores the Pass List IP addresses and netblocks.

The blocking portion of the custom module receives alerting packets, pulls out the SRC and DST IP addresses, searches for those in the Radix Tree structure, and if not found will block the IP. If the IP is either found directly in the Radix Tree, or it is determined to be contained within a netblock defined in the Radix Tree, then it is not blocked. The Radix Tree is used for very fast lookups of IP addresses.

This is a multithreaded operation as Suricata has multiple threads reading packets and processing alerts. These multiple threads are in turn looking up IP addresses in the two Radix Trees continually (one Radix Tree for IPv4 addresses and another for IPv6 addresses). If the firewall interface monitoring thread that is looking at the kernel routing messages sees that your WAN IP has changed (or has been removed for some reason), then that thread will remove the WAN IP (the old one) from the Radix Tree. If later the WAN IP returns either with the same value or a new value, the Radix Tree is appropriately updated. But if, during the interval between the WAN IP being removed and then added back at some later point, some other thread processed an alert containing the WAN IP (that was just deleted), then the IP is going to get blocked because it's not currently in the Radix Tree. If you read the Pass List debugging log carefully, you can see which threads blocked the WAN IP and which did not. I'm thinking this is all related to the rapid number of changes going on with your interface IPs (and all those OpenVPN addresses, too). Your WAN IP is being periodically cycled for some reason, and that results in it being removed from the Radix Tree (same as Pass List) for a bit. Then later it gets added back, but that short time it's missing is long enough for one of the running Suricata packet processing threads to process an alert with the IP address in it and thus trigger a block.

So, the first order of business in your case is to figure out why your interface IP addresses are changing so often. As I mentioned seeing in the log snippet you sent me, there are dozens and dozens of lines of interface IP address changes happening. That is not normal, and will definitely confuse the Suricata custom blocking module threads. They expect only very infrequent interface IP changes such as might happen if a PPPoE WAN interface cycled or your ISP renewed your WAN DHCP address with a new and different one. The threading logic is not expecting nearly as many changes per minute as I am seeing in your logs. I don't know if this is an issue with your CARP configuration, something with all the OpenVPN instances I see, something with the WireGuard Tunnel, or something related to LAGG interfaces that has cropped up in the recent pfSense release. But all those interface IP address ups and downs are the cause of your Pass List issue I suspect.

You say this worked in the past, but did you check back then to see if you had nearly as many interface IP changes as you do now? They would always have been logged in the suricata.log as that logic has been around for quite a while.

sgnoc · Dec 25, 2023, 2:32 AM

@bmeeks So that was confusing me too, but to add even more confusion, my WAN IP very rarely changes. I keep the same WAN IP for months in most cases before it updates to a new IP that I keep for some more months at a time. On top of that, ALL of my internal interfaces never change IP addresses for the interfaces, ever. They have been the same for years. I've never had any issues with Suricata previously (since I started running on this platform in 2019), so I've never had a need to look at the suricata logs to see if there were these IPs being added/deleted so often.

I'm not sure why it is happening or how, but I can assure you the IP does not change at all on the internal interfaces, and only changes a few times a year on the WAN.

I don't see any correlation that I can recognize between the suricata logs and the added/deleted logs on the debug file.

Side note, Merry Christmas! I didn't figure I would get any response until after.

sgnoc · Dec 25, 2023, 2:47 AM

One thought just came to mind. I believe this specific alert/block happened while I was loading suricata on the other interfaces, which were not completely up yet.

It seems that each time I load/reload a single suricata interface, it causes all of the other suricata instances to reload some portions. Could that be causing Suricata to think the IPs are being updated when they aren't?

Each time I reload a single interface, I see these logs in the main system log, and there is a set of these for ALL interfaces by a single interface restart/start. I never understood why restarting one interface did this for all interfaces.

2023-12-24 19:31:04.068717-05:00 	php 	86831 	[Suricata] Building new sid-msg.map file for 00_WAN...
2023-12-24 19:31:03.828809-05:00 	php 	86831 	[Suricata] Enabling any flowbit-required rules for: 00_WAN...
2023-12-24 19:30:57.681301-05:00 	php 	86831 	[Suricata] Updating rules configuration for: 00_WAN ...

I just checked my other interface, the lag0.33. Where it was blocking the internal 10.10.33.2 address before I turned on the pass list debugging, now that pass list debugging is on, the 10.10.33.2 address is no longer being blocked when an alert hits. I've had two alerts hit since I restarted the interface with the pass list debug enabled and it only blocked the external address, not the internal address. So something different is happening for me with the pass list debugging that is not triggering the internal IP address to be blocked, where it does before the debug was enabled. I'm baffled.

bmeeks · Dec 26, 2023, 8:09 PM

@sgnoc:
When you make changes to interfaces inside pfSense, then pfSense itself will send all packages a "restart all packages" command. This is done in the event IP addresses or subnets change on configured interfaces and packages might need to know they are changing. Even when nothing "really" changes, the message is still sent. Suricata will see this command from pfSense and dutifully restart itself. It uses the shell script in /usr/local/etc/rc.d/suricata.sh to do this. That shell script restarts all configured Suricata interfaces.

Also, a few versions back Suricata upstream made some changes in how the PCAP function works. The changes were to address a bug that cropped up occasionally when the underlying interface PCAP was running on restarted. I believe a side-effecct of this bug fix is that now when Suricata starts PCAP on some interface types it will cause a brief "reset" of the interface. That might cause it to appear to have gone down and come righ back up.

When I said your WAN IP was changing, I don't literally mean a different one. I mean the kernel is cycling the interface for some reason and sending the interface monitoring thread in Suricata's custom blocking module a notification of removing and adding an IP address to an interface. The kernel sends two messages: one deleting the IP address from the interface; and then another adding it back. 9 times out of 10 it adds back the same IP. But the monitoring thread can't assume that. So, when it gets the "DELETE ADDRESS" message it removes the IP from the internal Radix Tree. Later, when it gets the "ADD ADDRESS" message, it will put it back in the tree. But in the interval between those two messages on a very busy network it's entirely possible for one of the other Suricata packet processing threads to have received a packet with the WAN IP and go ahead and block it since for a brief second it is removed from the Radix Tree and would not be found on the "pass list" when looked up.

These are the messages I'm talking about where the automatic Interface Monitoring Thread (thread ID IM#01) is removing your WAN IP from the internal pass list then later adding it back. It appears to do this multiple times. During the interval when the WAN IP has been automatically removed, then it most certainly can be blocked (and it is, according to the Pass List debugging log).

[117623 - Suricata-IM#01] 2023-12-24 19:11:23 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[117623 - Suricata-IM#01] 2023-12-24 19:11:23 Info: alert-pf: Deleted address <WAN IP> from automatic firewall interface IP Pass List.
[117623 - Suricata-IM#01] 2023-12-24 19:11:24 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[117623 - Suricata-IM#01] 2023-12-24 19:11:24 Info: alert-pf: Added address <WAN IP> to automatic firewall interface IP Pass List.
[117623 - Suricata-IM#01] 2023-12-24 19:11:25 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[117623 - Suricata-IM#01] 2023-12-24 19:11:25 Info: alert-pf: Deleted address <WAN IP> from automatic firewall interface IP Pass List.
[117623 - Suricata-IM#01] 2023-12-24 19:11:26 Info: alert-pf: Monitor Thread IM#01 received notification of IP address change on interface ix0.
[117623 - Suricata-IM#01] 2023-12-24 19:11:26 Info: alert-pf: Added address <WAN IP> to automatic firewall interface IP Pass List.

It's getting removed because the pfSense kernel is sending the monitoring thread a notification telling it that. The first line of log text is from the kernel routing message subscription. It says an IP change happened on the ix0 interface. As I said earlier, here "change" does not necessarily mean a new IP. It might simply mean the interface was flapping. Shortly thereafter the kernel sent another message about ix0 and resulted in putting the IP address back. At that point the IM#01 monitoring thread would have put the WAN IP back in the Radix Tree (same as Pass List). But then shortly thereafter, the process repeats and the address is removed again. You need to find out what's going on that is causing all the interface flapping. That's why things are getting blocked, I think.

Because Suricata is multithreaded, this can be a bit hard to visualize in your head. But consider that in your case Suricata has 4 threads processing packets, comparing them to the rule signatures, and generating alerts when something matches. Those threads are labeled in the Pass List Debug Log as W#01 through W#04. You can follow them in the Pass List Debug Log. A totally separate thread is running at the same time listening for Kernel Routing messages from pfSense. That thread is automatically adding or removing firewall interface IPs from the pass list depending on whether the kernel says "I'm deleting an IP" or "I'm adding an IP". So that thread (called #IM01, for "interface monitoring thread #01) is constantly deleting and then adding back firewall interface IP addresses in your case because something appears to be flapping (interfaces going up and down). So, now it's a random race with the four packet processing worker threads. Does one of them see a packet with your WAN IP in it during the brief interval where the interface monitoring thread has deleted the WAN IP from the internal pass list? If so, then the WAN IP gets blocked.

Looking in your Pass List Debug log I see instances where a worker thread found the WAN IP in the pass list and did NOT block it, and then later a different thread checked and the WAN IP was NOT in the pass list at that instant and thus got blocked.