@steveits Thank you so much for you advice!! I tracked the location of that IP and just emailed the company, explaining what's happening. The most intriguing part is that the IP is from the agency that registers and maintains all".br" websites here in Brazil, maybe they were victims of some kind of attack.

@dobby_ Yes, it appears, if they can get it out of user space...

Actually just registered to reply to this and say thank you.

While some people think its pointless to preserve the snort2c table, i find it highly useful.

@michmoor said in Suricata and having a Oinkmaster code:

@bmeeks so I’m trying to understand the logic as it’s shown In the gui. If I do not select no Snort rule categories ,does snort VRT rules run under the hood? It seems that way implicitly.

I don't fully understand your question here. If you do not choose any Snort category files under CATEGORIES, and you don't enable an IPS Policy, then no Snort VRT rules will be used at all (ignoring OpenAppID here as those rules are totally different). The rules may still be downloaded and updated if you have the download option enabled, but they will not be selected for use on any interface unless the appropriate category is selected on that interface. Note that SID MGMT, if used, can also automatically enable categories including Snort VRT rules.

But if you enable an IPS Policy, that will automatically choose a collection of Snort VRT rules and enable them. Remember I posted somewhere up above that the Snort VRT rules (and ONLY the Snort VRT rules) contain a special metadata tag that associates a rule with one or more pre-defined IPS policies. Also, for each assigned policy, the metadata contains a suggested action (ALERT or DROP) for each rule. So when you check the box in the GUI to enable an IPS Policy, the PHP code will automatically scan the Snort VRT rules and pull in all the rules tagged with the IPS policy metadata matching the policy you chose on the CATEGORIES tab. You can easily see these metadata tags by opening some of the Snort VRT rules and searching for the string "policy {policy_name}-ips", {policy_name} is replaced by "connectivity, balanced, etc.". When IPS Policy is enabled, manual selection of Snort VRT categories on the CATEGORIES tab is disabled. The logic there is the user is handing over the Snort VRT rule selections to the enabled IPS policy logic. However, if desired, you can still use the SID MGMT features to add additional Snort VRT rules.

@bmeeks ok so using two IDS in none blocking mode maybe work but not as an IPS. That makes sense. Was curious if snort just for OpenAppID use case could still be used but doesn’t seem like it. Ok fair enough

@michmoor said in appid metadata - unknown:

@bmeeks I think i found the source of an issue. It seems that if there is the first match on any appID that is where snort stops reviewing other appID rules. This is a bit of an issue.
For example, appID xbox_live and/or xbox_live_sites. The very first match for these apps seems to be "Microsoft" which is correct but not wholly accurate.

Its the same for appID disney_plus. Snort will match on appID "iTunes" for this.

I thought I was conflicting with SIDs but even picking an unusually large number such as 99991, doesn't matter as it never gets matched.

The reason Yelp or TikTok were successful entries is that there was no previous match for those in the rules I believe

Perhaps the original rules were not well optimized in the sense they were not granular enough. By that I mean identifying the app as "Microsoft" when something like "XBox" would be more granular.

@bmeeks I don't have a proxy set up yet because there's a lot about its configuration that i'm unsure about.

I run suricata in inline mode on the lan interface. I also run unbound with full DoT to cloudflare 1.1.1.2 for dns. i have nat rules and blocks set up so that, unless a client is running a vpn from there device, dns is handled by unbound.

i'm not accessing anything yet from the outside, so.. (still learning this stuff.. babysteps)
My goal is to be able to perform a full-on mitm setup, however i'm still confused with the whole cert thing, wpad, transparent proxy thing.. (the more i read up, the more questions i have).

i thought it was supposed to work as follows: there are basically 2 certs (for simplicity sake i'll refer to the certs as follows):

-A WAN-side cert, so pfsense can de-encrypt/re-encrypt traffic as it receives/sends client data with the outside world,

and

-A LAN-side cert so that it could de-encrypt/re-encrypt traffic as it receives/sends data with the clients on the lan-side for https traffic.

that way, the moment my data hits pfsense on the lan-side, it is decrypted.. pfsense in all it's glory can process all packets to its fullest capability, and then before sending the data out to the internet, re-encrypts it.. and reverse (obviously) when it receives data.

Normally, i'd leave it as is, however as i fill my head with all this 'stuff', i realize that these packages really aren't doing much any more because of the encryption. and since it's just me and my family on my home network, i'd like to be able to use these facets..

also, this stuff is kinda fun! frustrating at times.. but fun..

Also i realize that email is the means of most attack vectors, and it's all basically double-encrypted, such that even with a full-on mitm proxy, email and certain apps can't be de-encrypted, and im fine with that. however regular web traffic i'd like to have scanned and processed. yeah, it's a lot, and those who know would say it's complete overkill and not serving a purpose, however it's all learning to me.. experience that i can take with me where ever i go..

@gertjan It is. Topping out at around 1gbit/s. Pipe is 10gbit/s.

Without Suricata running the FW fares well and load sit below 25% on CPU and 4% RAM.

When SC is running then it dies instantly. Both legacy and inline mode.

@daddygo hi, sorry, I come back to this topic again. Actually, i am also using expressvpn too.

What i don't understand.

vpn is not a wan interface outsiders on the same vpn access point, can probe the pfsense no? especially as bogon/
private networks are not blocked.

don't we want to have visibility to attacks from th expressvpn network? Isn't that what Suricata helps out with?

@michmoor said in OpenAppID - what is the application?:

@bmeeks Thanks Bill. Now you are just showing off your search skills :)

Thanks for this. Im understanding the structure here of how the app writing takes place. Not to difficult to piece together.

You can only detect what you can see. I am not looking forward to TLS1.4 and more from a AppID perspective. ha ha.

Yeah, not too complicated once you dig into it a little. What I call the AppID stubs that download regularly from the Snort VRT contain the metadata and detection pieces for identifying specific apps. The AppID text rules then reference that metadata and application names in the AppID stubs to generate alerts when specific app traffic comes through. It takes both to make the whole. And the text rules are usually the responsibility of the firewall admin, but that starter list included in the Snort package helps you get a basic setup working out of the box. But as mentioned up above, that starter package is a bit dated now as the maintainer is no longer updating it.

@bmeeks and bookmarked

@bmeeks Just chiming in to note this has been implemented via feature request #10809, working just fine.

Received one alert via Telegram on Dec 11 last year, all good.

Suricata new rule categories are available: - Emerging Threats Open rules: threatview_CS_c2.rules

@bmeeks
This looks quite cool...
I'll try some things out tomorrow during office hours - so I can have a look on the results more or less in livetime.
Keep you informed

@bmeeks Thanks a lot for the detailed answer. I will go ahead with Suricata IPS in pfSense with Legacy-mode blocking on, then send those alerts to system log and then send it to the Security Onion over syslog.
Security Onion can parse pfSense logs out of the box and then have custom Kibana Dashboards.

I have not done so personally, but the general idea is you install a filebeat client on pfSense and configure that client to ingest the EVE JSON logs on the firewall. You will find them under /var/log/suricata/suricata_xxxx/ on the firewall where the suricata_xxxx part will be a unique subdirectory for each configured Suricata interface that includes the physical interface name combined with a random UUID number.

Here is a general overview link for filebeat: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-overview.html.

You have to tell the client what kind of logs to ingest and where to pull them from. Here is the documentation for that: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html. Unfortunately, many of these packages change their input formatting somewhat frequently, so you will likely have to dig around on Google to find the "current" config parameters needed to ingest Suricata 6.0.x EVE logs.

Here is the documentation for establishing an output destination for filebeat: https://www.elastic.co/guide/en/beats/filebeat/current/configuring-output.html.

The basic process is to install and configure the client on the pfSense firewall, and then configure the client to output (or send) ingested log data from the pfSense firewall over the network to your ELK stack running on the Ubuntu machine.

I added a note to my internal Suricata package issue tracking to investigate a primitive means to at least allow display of the captured payloads in the GUI if that option is enabled. There will be only limited usefulness there, though, because the vast majority of the time the payload is going to be encrypted.

I prefer not to futher contaminate this thread with this conversation because your problem has nothing to do with "commented out" rules. Yours is a completely different issue. Feel free to create a new post thread if you want to continue this.

@michmoor said in clarification needed on direction of rules and usefulness:

@bmeeks Understood. In my mind, because the firewall is stateful I was thinking that the rules although are evaluated from EXTERNAL to INTERNAL, there was some value of having the rule applied on the internal lan as the firewall would see the return traffic.

So are you saying it operates from a purely stateless way, rule evaluated from source to destination (external to internal) and that's it. There needs to be a rule matching from HOME to External?

Lastly, from your standpoint, PFblockerNG and some of the ET rule sets have IP blocking. I assume there is overlap. Does it matter if the suricata rules are enabled for those IPs as well? Assuming there is no resource constraint on the firewall.

I am not understanding why you bring in stateful connections and rules. It seems to me you are confusing the IDS/IPS with the firewall. They are not the same at all! In fact, most commercial enterprises run the IDS/IPS on a completely separate box and not on the firewall.

The IDS/IPS (Suricata in your case), is totally ignorant of firewall states and even firewall rules. It simply evaluates packets as they flow across the physical interface Suricata is monitoring. And it is only looking at the IP header info in the packet to determine how far to evaluate that packet against the Suricata rules (not the firewall rules, the firewall rules are totally irrelevant to Suricata).

Suricata, especially when running in IPS mode, does not interact with the firewall engine at all. Not one bit. Actually Suricata sits out in front of the firewall. Here are two diagrams that show Suricata lives "outside", on the NIC-side of the firewall and thus sees traffic before the firewall rules have been applied.

ids-ips-network-flow-ips-mode.png
ids-ips-network-flow-legacy-mode.png

In terms of your pfBlocker question, if you use only simple IP lists in Suricata, then there would be perhaps 100% overlap between what it does and what pfBlocker does. But most folks would not do that. If you use pfBlocker, then you might apply other rules in Suricata that are not simple IP lists. But as we discussed much earlier in this thread, encryption severely hinders the ability of Suricata or any IDS/IPS package to peer into the packet payload.

@bmeeks Thanks. That’s what I figured but wanted to be sure it wasn’t some newbie mistake.

May move to all VLANs and not run anything on the parent interface so I can separate things more at the cost of system resources. I’m sure the over spec 7100 can handle that for my home 🤪.