GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.
-
@Bob-Dig said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:
How hard would it be to create an alert for every AppID when they are already named in the log file anyway? Just curious.
I don't understand exactly what you are suggesting. The code already generates an alert for every AppID when the associated text rule is triggered. AppID works just like all the other Snort rules. If you do not have a text rule for a particular app, then it won't trigger an alert.
I think folks still have a major misunderstanding about OpenAppID and how it works. It has an app detection engine written in Lua, but then uses associated text rules created by the user (or someone else) to actually inspect traffic against. In that regards it works exactly like any other rule. No rule, then no alert.
-
Here is, the fully converted appMapping.data to text rule file alert ... Use only what you need or if you have the memory use them all. I have a 2100-MAX so I do not have the memory for all of them.
The pfSense Snort AppID de-cipher sorcerer's code file: --> textrules.txt
The pfSense Snort AppID de-cipher sorcerer's code file with case sensitive messages: --> textrules2.txt
Sid range: 1000000 - 1003371Total 3,371 AppID rules you can use with the custom option.
I converted it with a Java program I just made. The message is the same as the appid match it makes it easier.
Some of the ieee items are bigger but they seem to match.
(you can load the full set into custom however you will have to comb it yourself after as it will generate an app id alert for everything)I would only use what you need or need blocked.
-
@JonathanLee This is good at least to gain that extra visibility on what applications are seen so i think there is value in that.
The downside is that because these arent categorized in any way you dont have the easy button of saying 'block all social media'. Thats ultimately the way to do it. But this is for sure half the battle.
Interesting enough, Cisco OpenAppId has 5,995 so there are some missing from the list. There isnt a 1to1 relationship with whats available from Snort and whats available on Cisco's service. -
@michmoor I am very thankful they shared what they have with the open source community. For a small non enterprise network, or some individuals that are working from home, something like this really helps with cyber security. Thank you Cisco, Snort, and pfSense.
-
J JonathanLee referenced this topic on
-
J JonathanLee referenced this topic on
-
M mcury referenced this topic on
-
J JonathanLee referenced this topic on
-
J JonathanLee referenced this topic on
-
J JonathanLee referenced this topic on
-
Are the application signatures still getting updated?
appMapping.data hasn't been updated since Jun 9 2023 -
@michmoor I don't know, real question is is how is ChatGPT and other new AI sites listed in the database that is downloaded...
-
@JonathanLee where do you see these new apps ?
-
@michmoor I pay for a snort subscription, it’s not free. I have a custom adapted list that uses app id based on the databases. But keep in mind my list is not a free list. We have the paid stuff. It’s great.
-
@JonathanLee I have a paid Snort VRT list as well. Not sure how that affects open appid
-
Guys:
I've posted on this innumerable times ...
--OpenAppID consists of two distinct pieces of information. One is LUA scripts containing what are called rule stubs. These contain most of the logic for interpreting applications. These rule stubs are updated by the Snort VRT. Each time you download updates for your rules those stub rules will come down along with the other Snort rules.
But in order for AppID to actually trigger alerts, you must provide text-based detection rules. Originally the Snort team intended for users to create their own text-based rules (the second of the two required pieces for OpenAppID to work). But when I added OpenAppID to the pfSense Snort package, a professor at a University in Brazil volunteered to craft and maintain a package of text-based rules for pfSense users to grab. So, I configured the package to use those rules. Over the years, that University team has disbanded and those rules are no longer updated. They have not been updated in quite a long time. Quite a few years ago we moved the location of those rules to a Netgate server because the University in Brazil was using geoblocking and thus users in several countries were blocked from downloading the rules.
Again, proper operation of OpenAppID requires both the rule stubs LUA scripts AND text rules written to use the keywords defined in the LUA scripts.
-
@bmeeks Thanks for having paitence with us
For the Lua scripts i assume you mean these fils located here
/usr/local/etc/snort/appid/odp/luaShould we care about these LUA scripts or focus on the appMapping.data file which basedon my research is the main file used to figure out which apps are identified and how to notate them in the text rule.
Then there is the other part which is
appMapping.datathat is found here:/usr/local/etc/snort/appid/odpThe text rules I'm not concerned about.
I am trying to understand how often AppID is updated. When I list the directory, the lua files show 2023. ChatAI apps are nonexistent except for OpenAI, so the list doesn't seem maintained.
Firewall: NetGate,Palo Alto-VM,Juniper SRX
Routing: Juniper, Arista, Cisco
Switching: Juniper, Arista, Cisco
Wireless: Unifi, Aruba IAP
JNCIP,CCNP Enterprise -
@michmoor said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:
@bmeeks Thanks for having paitence with us
For the Lua scripts i assume you mean these fils located here
/usr/local/etc/snort/appid/odp/luaShould we care about these LUA scripts or focus on the appMapping.data file which basedon my research is the main file used to figure out which apps are identified and how to notate them in the text rule.
Then there is the other part which is
appMapping.datathat is found here:/usr/local/etc/snort/appid/odpThe text rules I'm not concerned about.
I am trying to understand how often AppID is updated. When I list the directory, the lua files show 2023. ChatAI apps are nonexistent except for OpenAI, so the list doesn't seem maintained.
Everything under
/usr/local/etc/snort/appid/is automatically updated by the normal rules update job. But, and this is key, unless your text rules properly reference the keywords and options made available by all the files in this subdirectory, then OpenAppID will not function or not function properly. That's why I keep making such a strong point about how OpenAppID is totally different from the regular Snort rules. The regular rules don't have a dependency. You craft the rule and it will work (assuming it is syntatically correct). But OpenAppID does not work the same. If you have the OpenAppID files from Snort but did not create any text rules to reference the OpenAppID files, then nothing works. On the other hand, if you write the text rules but don't download and install the OpenAppID stubs and supporting files, then nothing works.I keep seeing posts and questions from users that indicate very few really understand how OpenAppID works and what parts and pieces it needs to function correctly.
And yes, because the text rules for OpenAppID on pfSense have not been updated in years, there are new applications that text rules do not exist for in the pfSense package but the new app is defined in the rules stubs downloaded from the VRT. But even though a new app may exist in the rules stubs, if there is no corresponding text rule then app detection for that app will not work. Conversely, over the years some app names have changed (or typos were fixed) within the OpenAppID rules stubs from upstream. But the text rules contributed by that University team in Brazil have not been updated, so you will likely see errors from those when starting up Snort.
Anyone serious about using OpenAppID in Snort on pfSense really should be creating their own text rules for app detection. The available set from years ago is likely not adequate. I know there are many errors in those rules in terms of app names, for example.
-
@michmoor You have to add your own text files, I created a java program to do the for me based on the database that is downloaded. So that might be why mine has ChatGPT. Yes that list is updated but it does no good unless you generate the text files to match them. I just made a java program to parse over that database and generated a text file based on that. My text file is above but I have not updated that list in a while. I have to run the database in it again.
-
@JonathanLee going to DM you for details
-
@michmoor In Snort's OpenAppID context, "appMapping.data is a file that maps application names to their corresponding AppID identifiers, which are used for creating rules to identify and control application traffic."
-
@bmeeks said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:
appMapping.data
Do you know when this is updated again?
appMapping.data
I have the subscription for Snort but I have not seen this update in a while now.
-
@JonathanLee said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:
Do you know when this is updated again?
appMapping.data
I have the subscription for Snort but I have not seen this update in a while now.
No. That file comes down as part of the AppID stub rules update. It's up to the Snort VRT for when they update it on their end. I don't know as I have not looked into this for quite some time, but it could be that the Snort VRT is slowly deprecating updates for legacy Snort 2.9.x stuff in favor of Snort3. Snort3 and 2.9.x are not compatible and cannot share things like rules files.
At some point for sure upstream Snort will discontinue updates for the Snort 2.9.x code tree. This is why I've urged Snort users on pfSense to move to Suricata. Of course Suricata does not have AppID support, so Snort users would lose that feature after migration. On the other hand, Suricata has much more intensive logging. If you continue to use Snort 2.9.x on pfSense, then expect to be doing much more hand-holding of the package and your own software maintenance.
-
I have a question. Why can't we get Snort 3 on pfsense since version 2 is being depreciated? I know there's Suricata but like to have options.
-
@Darkk said in GUIDE: Snort's AppID custom rules Quick Guide to blocking. Example shows OpenAI ChatGPT or Itunes.:
I have a question. Why can't we get Snort 3 on pfsense since version 2 is being depreciated? I know there's Suricata but like to have options.
Snort and Suricata are both volunteer maintained packages. That means a volunteer contributes the programming effort required to create and maintain the package without compensation and without any involvement from the pfSense developer team other than that team manually merging code changes submitted by the maintainer into the official pfSense repository. For Snort, I assumed the maintainer role for that package many years ago when the original developer grew weary and moved on. I wanted certain features to be available in the package (flowbits primarily), and so I added the necessary code and submitted the Pull Request to GitHub. It was accepted, and so I offered a few more updates such that over the years I became the defacto maintainer for Snort. For Suricata, I created that package from scratch, submitted the Pull Request to the Netgate team, and have been maintaining it since as a volunteer contributer sharing my work with the pfSense community for free.
For both packages, I am getting ready to step aside. There is no compensation for volunteer maintainers, and since I retired from an IT role in a Fortune 500 US company several years ago, I am slowly disentangling myself from committments to more fully enjoy my retirement. That's one of the reasons you have seen fewer Snort and Suricata updates over the last year.
I tried on two separate occasions in the recent past to create a Snort3 package for pfSense. But I gave up in frustration both times. Partly because my heart was not totally in it for the reasons outlined above (my retirement), but also because it's very hard to do since Snort3 is a radical departure code-wise from Snort 2.9.x on the binary side. It will require rewriting things in C++ and adapting all the old Snort 2.9.x configuration parameters over to Lua scripting. The use of Lua will require substantial changes in the GUI package code.
The only way Snort3 will come to pfSense is if either some other new volunteer steps forward to create the required package, or if Netgate decides to take it over. Everything needed is available on the pfSense GitHub repo here -
Snort Binary Package Code: https://github.com/pfsense/FreeBSD-ports/tree/devel/security/snort
Snort GUI Package Code: https://github.com/pfsense/FreeBSD-ports/tree/devel/security/pfSense-pkg-snortIf Snort3 is important to you, perhaps you might consider stepping into the volunteer maintainer role like I did many years ago and then sharing your work with the pfSense community ...
. -
I didn't mean any disrespect. I and others do appreciate the work that was put into it. I totally understand your situation and respect that.
