GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.

[JonathanLee]

Hello fellow Netgate community members,

I wanted to share something with you as you have all shared so much with me over the years.

I have found an amazing way to make AppID work for what you need quickly without all the issues.

I simply only use custom rules to block what I need.

For example I only want to detect and block ChatGPT or Itunes. This can be anything you want. As long as it is listed inside the appMapping.data file.

1. Enable AppID

(Only use OpenAppID not text rules)

2. Enable preprocessor under edit interface

(make sure both radio options are checked)

3. Wait for stats log to start populating. This log is what AppID has detected so far.

(IMAGE: LOGS)

There is no alert rules that is the major issue AppID will create stats logs all day with detections. However, if you want to make it block at a granular AppID level we must create some alert rules.
Next Example I want Itunes and ChatGPT

5. Go to the following path in edit. We are not really changing anything in this file we are just trying to find the actual text rule to cross reference with log.stats above.

Path:    /usr/local/etc/snort/appid/odp/appMapping.data

Now that you have referenced the text rule the lowercase side copy this for your custom rule.

6. Create a custom alert on rule.

(use the custom entry area mine already has some rules I have been testing)

Here is some that seem to work for my system. Notice Itunes it's sid starts at 1,000,000 this is where you want to start your custom rules so you do not interfere with Snorts rules already in place.

alert tcp any any -> any any (msg:"iTunes"; appid:itunes; sid:1000000; rev:1)
alert tcp any any -> any any (msg:"iCloud"; appid:icloud; sid:1000001; rev:1)
alert tcp any any -> any any (msg:"LinkedIn"; appid:linkedin; sid:1000002; rev:1)
alert tcp any any -> any any (msg:"ChatGPT"; appid:chatgpt; sid:1000003; rev:1)
alert tcp any any -> any any (msg:"Amazon Ad System"; appid:amazon_ad_sys; sid:1000004; rev:1)
alert tcp any any -> any any (msg:"DoubleClick"; appid:doubleclick; sid:1000005; rev:1)

7. Wait for the alerts to show up and set them to block if needed notice the sid again 1000000-5 are now being listed in my alerts tab.

Thus you can use Snort and block at an AppID granular detection level similar to a url blocker.

8. Now that your getting Snort to alarm change your custom rule from alert to drop.

drop tcp any any -> any any (msg:"DoubleClick"; appid:doubleclick; sid:1000005; rev:1)

doubleclick net is now gone forever on your network. Or if your a school and have an english class you can set it to block ChatGPT forever on specific subnets and force students to write their own papers.

I hope you enjoy my researching of AppID inside of Snort for use with Netgate's pfSense firewall software.

9. Thumbs up this post and have a good day.

[michmoor]

@JonathanLee Great job on documenting this and giving people another way of blocking applications on pfsense

[JonathanLee]

@michmoor I think inline and legacy also have differences when and if you can drop or reject items. However you can see alerts and that is a huge plus to know what is going on for items you don't want access like ChatGPT for example especially for businesses that have employees doing online training to make sure they are not using it during a paid training over employees that study take notes and do the right thing.

[bmeeks]

Be careful using OpenAppID with Legacy Mode Blocking. That blocking mode is an "all or nothing" proposition -- meaning each ALERT equals a block of that host (assuming the host is not covered by a Pass List entry). That can result in completely blocking a host that might have other traffic you want to pass or access. A Legacy Mode Block of a host is a total block of all traffic on all ports for that host. The mode blocks the IP address completely.

Inline IPS Mode is much better suited for OpenAppID because then you can tailor the rule actions to have granular security. You can have some rules drop (and thus block) traffic while other rules simply generate alerts in the log (but do not block the traffic). Inline IPS Mode can selectively drop particular packets while continuing to allow other packets from a host not matching an IPS rule to flow freely.

[JonathanLee]

@bmeeks I can not get inline to run on LAN for Netgate's official SG-2100 MAX and it needs to run inline for AppID to block.

The other mode only detects and alerts

Updated

Same Results will not work with lan

[bmeeks]

@JonathanLee said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:

Same Results will not work with lan

That's because the physical NIC in that box is not supported by the FreeBSD native netmap device. These are the NICs that are supported for native mode operation:

'cc', 'cxl', 'cxgbe', 'em', 'igb', 'em', 'lem', 'ix', 'ixgbe', 'ixl', 're', 'vtnet', 'ena', 'ice', 'igc', 'bnxt', and 'vmx'

Inline IPS Mode uses the netmap device in the kernel, and that device is not supported by all NIC drivers. The move to the iflib network driver wrapper library helped expand the supported devices, but still not every NIC type is supported as not all of them have been ported to iflib.

[Bob.Dig]

How hard would it be to create an alert for every AppID when they are already named in the log file anyway? Just curious.

[bmeeks]

@Bob-Dig said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:

How hard would it be to create an alert for every AppID when they are already named in the log file anyway? Just curious.

I don't understand exactly what you are suggesting. The code already generates an alert for every AppID when the associated text rule is triggered. AppID works just like all the other Snort rules. If you do not have a text rule for a particular app, then it won't trigger an alert.

I think folks still have a major misunderstanding about OpenAppID and how it works. It has an app detection engine written in Lua, but then uses associated text rules created by the user (or someone else) to actually inspect traffic against. In that regards it works exactly like any other rule. No rule, then no alert.

[JonathanLee]

@Bob-Dig

Here is, the fully converted appMapping.data to text rule file alert ... Use only what you need or if you have the memory use them all. I have a 2100-MAX so I do not have the memory for all of them.

The pfSense Snort AppID de-cipher sorcerer's code file: --> textrules.txt
The pfSense Snort AppID de-cipher sorcerer's code file with case sensitive messages: --> textrules2.txt
Sid range: 1000000 - 1003371

Total 3,371 AppID rules you can use with the custom option.

I converted it with a Java program I just made. The message is the same as the appid match it makes it easier.

Some of the ieee items are bigger but they seem to match.

(you can load the full set into custom however you will have to comb it yourself after as it will generate an app id alert for everything)

I would only use what you need or need blocked.

[michmoor]

@JonathanLee This is good at least to gain that extra visibility on what applications are seen so i think there is value in that.
The downside is that because these arent categorized in any way you dont have the easy button of saying 'block all social media'. Thats ultimately the way to do it. But this is for sure half the battle.
Interesting enough, Cisco OpenAppId has 5,995 so there are some missing from the list. There isnt a 1to1 relationship with whats available from Snort and whats available on Cisco's service.

[JonathanLee]

@michmoor I am very thankful they shared what they have with the open source community. For a small non enterprise network, or some individuals that are working from home, something like this really helps with cyber security. Thank you Cisco, Snort, and pfSense.

[michmoor]

@JonathanLee @bmeeks

Are the application signatures still getting updated?
appMapping.data hasn't been updated since Jun 9 2023

[JonathanLee]

@michmoor I don't know, real question is is how is ChatGPT and other new AI sites listed in the database that is downloaded...

[michmoor]

@JonathanLee where do you see these new apps ?

[JonathanLee]

@michmoor I pay for a snort subscription, it’s not free. I have a custom adapted list that uses app id based on the databases. But keep in mind my list is not a free list. We have the paid stuff. It’s great.

[michmoor]

@JonathanLee I have a paid Snort VRT list as well. Not sure how that affects open appid

[bmeeks]

Guys:

I've posted on this innumerable times ... --

OpenAppID consists of two distinct pieces of information. One is LUA scripts containing what are called rule stubs. These contain most of the logic for interpreting applications. These rule stubs are updated by the Snort VRT. Each time you download updates for your rules those stub rules will come down along with the other Snort rules.

But in order for AppID to actually trigger alerts, you must provide text-based detection rules. Originally the Snort team intended for users to create their own text-based rules (the second of the two required pieces for OpenAppID to work). But when I added OpenAppID to the pfSense Snort package, a professor at a University in Brazil volunteered to craft and maintain a package of text-based rules for pfSense users to grab. So, I configured the package to use those rules. Over the years, that University team has disbanded and those rules are no longer updated. They have not been updated in quite a long time. Quite a few years ago we moved the location of those rules to a Netgate server because the University in Brazil was using geoblocking and thus users in several countries were blocked from downloading the rules.

Again, proper operation of OpenAppID requires both the rule stubs LUA scripts AND text rules written to use the keywords defined in the LUA scripts.

[michmoor]

@bmeeks Thanks for having paitence with us

For the Lua scripts i assume you mean these fils located here

/usr/local/etc/snort/appid/odp/lua

Should we care about these LUA scripts or focus on the appMapping.data file which basedon my research is the main file used to figure out which apps are identified and how to notate them in the text rule.

Then there is the other part which is appMapping.data that is found here:/usr/local/etc/snort/appid/odp

The text rules I'm not concerned about.

I am trying to understand how often AppID is updated. When I list the directory, the lua files show 2023. ChatAI apps are nonexistent except for OpenAI, so the list doesn't seem maintained.

[bmeeks]

@michmoor said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:

@bmeeks Thanks for having paitence with us

For the Lua scripts i assume you mean these fils located here

/usr/local/etc/snort/appid/odp/lua

Should we care about these LUA scripts or focus on the appMapping.data file which basedon my research is the main file used to figure out which apps are identified and how to notate them in the text rule.

Then there is the other part which is appMapping.data that is found here:/usr/local/etc/snort/appid/odp

The text rules I'm not concerned about.

I am trying to understand how often AppID is updated. When I list the directory, the lua files show 2023. ChatAI apps are nonexistent except for OpenAI, so the list doesn't seem maintained.

Everything under /usr/local/etc/snort/appid/ is automatically updated by the normal rules update job. But, and this is key, unless your text rules properly reference the keywords and options made available by all the files in this subdirectory, then OpenAppID will not function or not function properly. That's why I keep making such a strong point about how OpenAppID is totally different from the regular Snort rules. The regular rules don't have a dependency. You craft the rule and it will work (assuming it is syntatically correct). But OpenAppID does not work the same. If you have the OpenAppID files from Snort but did not create any text rules to reference the OpenAppID files, then nothing works. On the other hand, if you write the text rules but don't download and install the OpenAppID stubs and supporting files, then nothing works.

I keep seeing posts and questions from users that indicate very few really understand how OpenAppID works and what parts and pieces it needs to function correctly.

And yes, because the text rules for OpenAppID on pfSense have not been updated in years, there are new applications that text rules do not exist for in the pfSense package but the new app is defined in the rules stubs downloaded from the VRT. But even though a new app may exist in the rules stubs, if there is no corresponding text rule then app detection for that app will not work. Conversely, over the years some app names have changed (or typos were fixed) within the OpenAppID rules stubs from upstream. But the text rules contributed by that University team in Brazil have not been updated, so you will likely see errors from those when starting up Snort.

Anyone serious about using OpenAppID in Snort on pfSense really should be creating their own text rules for app detection. The available set from years ago is likely not adequate. I know there are many errors in those rules in terms of app names, for example.

[JonathanLee]

@michmoor You have to add your own text files, I created a java program to do the for me based on the database that is downloaded. So that might be why mine has ChatGPT. Yes that list is updated but it does no good unless you generate the text files to match them. I just made a java program to parse over that database and generated a text file based on that. My text file is above but I have not updated that list in a while. I have to run the database in it again.