suricata (o)DoH rules

SteveITS

@jpgpi250 Do they need to be Suricata? pfBlockerNG-devel has feeds for DoH for IPv4 and 6. (TheGreatWall_DoH_IP, https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall_ipv4)

Also isn't DoH usually port 443 not 53? Or DoT on 853.

michmoor

@steveits i agree. there is a bit of overlap here. Doesnt hurt having it in Suricata but then just use the lists thats used in that package to feed Suricata with custom SIDs

jpgpi250

@steveits

I already have suricata, using several other rule sets.
I already have pfblockerng, only use the IP part, because pfblockerng requires unbound to run, unbound is running on a separate machine, highly customized, using options pfsense doesn't support (using the webinterface). I asked this on this forum once, anything not entered using the web interface is unsupported, see here.
I maintain (and share) a much larger (o)DoH list on GitHub, daily updated. TheGreatWall is one of the many lists I use to compile my list. You can find it here, manual here, also refered to in this topic
The suricata rules can be generated easily from the data in the database (also on Github). I would provide it for those administrators who can't use the other available options (IP list, RPZ)

jpgpi250

@bmeeks

FYI

on the suricata forum, topic here, I recieved a reply pointing me to this GitHub repository, with a link to https://sidallocation.org/

From what I can find on the web, there is no regulatory organization to avoid conflicts, best effort ...

bmeeks

@jpgpi250 said in suricata (o)DoH rules:

@bmeeks

FYI

on the suricata forum, topic here, I recieved a reply pointing me to this GitHub repository, with a link to https://sidallocation.org/

From what I can find on the web, there is no regulatory organization to avoid conflicts, best effort ...

Thank you for the links. I was not aware of that effort on GitHub, but maybe it will help to provide some needed regulation/cooperation with SID allocations.

jpgpi250

@bmeeks
just to keep you informed...

got a mail from Emerging Threats Support (support AT emergingthreats.net

apparently, they are behind the GitHub repository
<quote>
In concert with a few other interested parties we created this: https://github.com/sidallocation/sidallocation.org...
</quote>

another link they included, which may (or may not) interest you:
<quote>
Do you have interest in including this list as part of our ET Open rules? (https://rules.emergingthreatspro.com/open/)
</quote>
Any idea if the ETOpen rules come with a delay, e.g. no PRO subscription?

Thanks again, for your time, effort and great work. Much appreciated.

jpgpi250

@bmeeks

Things are coming together, however still some questions (I want to get it all correct, before publishing something).

Are there any naming conventions for suricata rule files, now using 'DOH.suricata-rules'?
As you will understand, I can't build IP lists if a suricata ruleset rejects the DNS queries to get the addresses. I currently have. All my rules are applied to WAN only, but that obviously will not work. I was hoping to deploy the DOH ruleset on a specific (not WAN) interface, disabling all other rules. That seems to work, apart from the default suricata rules, they are always listed in the active rules list. Is it possible to disable the suricata rules for a specific interface (I tried, using sid management (1:2200000-1:2299999), but the entire network crashed, so I assume this isn't the proper way). Goal is to have only the DOH list active on that interface, easier for trouble shooting.
Would you submit a ruleset to the 'emerging threats pro organization' (see quote in the previous post), or keep this separate (GitHub)?

Thanks again for your time and effort.

bmeeks

@jpgpi250 said in suricata (o)DoH rules:

@bmeeks

Things are coming together, however still some questions (I want to get it all correct, before publishing something).

Are there any naming conventions for suricata rule files, now using 'DOH.suricata-rules'?

The file extension for rules files is usually .rules. To keep things separated in the Suricata package on pfSense the code prepends the rules file source to the name when unpacking and storing the archive on the box. So for example, the Emerging Threats Open rule category files will have emerging- prepended to the file name that exists in the gzip archive. The Snort Subscriber rules, if enabled, are stored with snort_ prepended to each category file name. And finally, for user-specific custom rules archives, extrarule- is prepended to the category file name. This is done in the event you may have duplicate category file names from two different rules vendors.

So for your new DoH category, maybe include DoH. For example, DoH.rules would work.

As you will understand, I can't build IP lists if a suricata ruleset rejects the DNS queries to get the addresses. I currently have. All my rules are applied to WAN only, but that obviously will not work. I was hoping to deploy the DOH ruleset on a specific (not WAN) interface, disabling all other rules. That seems to work, apart from the default suricata rules, they are always listed in the active rules list. Is it possible to disable the suricata rules for a specific interface (I tried, using sid management (1:2200000-1:2299999), but the entire network crashed, so I assume this isn't the proper way). Goal is to have only the DOH list active on that interface, easier for trouble shooting.

You can exclude by category file name as well as SID, so just use the category file name in SID MGMT. You can also selectively display and disable the built-in rules on the RULES tab if I recall correctly.

Would you submit a ruleset to the 'emerging threats pro organization' (see quote in the previous post), or keep this separate (GitHub)?

If the Emerging Threats team wants to include your ruleset, I would let them. Perhaps they will also assist with maintaining it. Distribution would certainly be more widespread. So if they included it in their ET-Open rules package, it would just appear as a new category on the CATEGORIES tab and you would no longer need your custom extra rules configuration on the GLOBAL SETTINGS tab.

jpgpi250

@bmeeks

this is what I came up with, feel free to comment.
updated daily, so this may become less effective over time.

edit
updated, due to a sid calculation error (script)
sample: DOHrules.zip
/edit

also submitted to the Emerging Threats team, but most of us are ETOpen users...

bmeeks

@jpgpi250 said in suricata (o)DoH rules:

@bmeeks

this is what I came up with, feel free to comment.
updated daily, so this may become less effective over time.

sample: DOHrules.zip

also submitted to the Emerging Threats team, but most of us are ETOpen users...

That's a great contribution to the community. Thank you!

Emerging Threats (now part of ProofPoint) publishes both the ET-Pro and ET-Open rules. I thought they told you it could be added to the ET-Open rule set. That one is free. If they only package it as part of their ET-Pro set, then most certainly you should be compensated by them for that as ET-Pro is a very expensive paid subscription.

jpgpi250

@bmeeks

I downloaded this pdf/ebook.

In section 6.2 (Testing performance and correctness of written rules), they mention a command:

suricata -S mynew.rules -l /tmp/analysis --engine-analysis

The result is a file /tmp/rules_analysis.txt, every rule has a comment like this:

== Sid: 27995001 ==
reject dns $HOME_NET any -> $EXTERNAL_NET 53 (msg:"(o)DoH Query for dns.google"; dns.query; content:"dns.google"; nocase; fast_pattern; classtype:bad-unknown; sid:27995001; rev:1; metadata:affected_product Any, attack_target Client_Endpoint, database_domainlist_id(s) 0, updated_at 2022_11_11;)
    App layer protocol is dns.
    Rule contains 1 content options, 0 http content options, 0 pcre options, and 0 pcre options with http modifiers.
    Fast Pattern "dns.google" on "dns request query (dns_query)" buffer.
    Warning: TCP rule without a flow or flags option.
             -Consider adding flow or flags to improve performance of this rule.

edit
the above command achieves the primary test, I want to excecute. Manually changed a sid (to verify duplicate sids in the file are detected), result:

12/11/2022 -- 12:55:52 - <Notice> - This is Suricata version 6.0.4 RELEASE running in USER mode
12/11/2022 -- 12:55:52 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "reject dns $HOME_NET any -> $EXTERNAL_NET 53 (msg:"(o)DoH Query for cloudflare-dns.com"; dns.query; content:"cloudflare-dns.com"; nocase; fast_pattern; classtype:bad-unknown; sid:27995001; flow:to_server; rev:1; metadata:affected_product Any, attack_target Client_Endpoint, database_domainlist_id(s) 1, updated_at 2022_11_12;)"
12/11/2022 -- 12:55:52 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "reject dns $HOME_NET any -> $EXTERNAL_NET 53 (msg:"(o)DoH Query for cloudflare-dns.com"; dns.query; content:"cloudflare-dns.com"; nocase; fast_pattern; classtype:bad-unknown; sid:27995001; flow:to_server; rev:1; metadata:affected_product Any, attack_target Client_Endpoint, database_domainlist_id(s) 1, updated_at 2022_11_12;)" from file /root/DOH.rules at line 14

/edit

the test also produces a file /tmp/rules.json, content of the first line:

{"raw":"reject dns $HOME_NET any -> $EXTERNAL_NET 53 (msg:\"(o)DoH Query for dns.google\"; dns.query; content:\"dns.google\"; nocase; fast_pattern; classtype:bad-unknown; sid:27995001; rev:1; metadata:affected_product Any, attack_target Client_Endpoint, database_domainlist_id(s) 0, updated_at 2022_11_11;)","id":27995001,"gid":1,"rev":1,"msg":"(o)DoH Query for dns.google","app_proto":"dns","requirements":["flow"],"flags":["sp_any","applayer","toserver","prefilter"],"pkt_engines":[],"engines":[{"name":"dns_query","direction":"toserver","is_mpm":true,"app_proto":"dns","progress":1,"matches":[{"name":"content","content":{"pattern":"dns.google","nocase":true,"negated":false,"starts_with":false,"ends_with":false,"is_mpm":true,"fast_pattern":true}}]}],"lists":{}}

any idea what I need to change /add to eliminate this warning, without having to add things I really don't need? Or do I need to add all the additional entries?

edit
added (test): sid:27995001; flow:to_server; rev:1;
tested, an alert is triggered (using dig from in house recursive DNS server).

the warning is still there, I found this, may explain the warning?
/edit

Thanks (again)

bmeeks

@jpgpi250, I would agree the bug report you linked posted by the lead upstream Suricata developer is the likely cause of the warning. I would disregard it for now.

jpgpi250

@bmeeks

haven't heard anything from Emerging Threats, so I put the files on Github, daily updated between 04 and 05 AM CET, new entries of course only when the source lists change.

rules (with md5): https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOH/DOH.rules

I've added a section suricata to the DOH documentation. Could you please read section 10 (DOH Suricata rules), It describes how to add the rules (pfsense), this to verify I haven't made any mistakes, or missed something (not an experienced user myself).

Thanks

bmeeks

@jpgpi250 said in suricata (o)DoH rules:

I've added a section suricata to the DOH documentation. Could you please read section 10 (DOH Suricata rules), It describes how to add the rules (pfsense), this to verify I haven't made any mistakes, or missed something (not an experienced user myself).

My only comment about the Suricata setup is it will work best when using Inline IPS Mode, so you might want to highlight that configuration. The reason is that only Inline IPS Mode can selectively drop packets. Legacy Mode will block ALL traffic to/from the target IP address, and that is likely not what you want. Inline IPS Mode can selectively drop only the specific packets matching the DNS DoH rules.

But not all NIC hardware currently supports netmap operation in FreeBSD. Most hardware does, but there are still some exceptions. The netmap device is used for Inline IPS Mode operation, so your NIC must support netmap in order to work with that mode.

jpgpi250

@bmeeks

@bmeeks said in suricata (o)DoH rules:

it will work best when using Inline IPS Mode, so you might want to highlight that configuration

done, added to the documentation (with credits in change log).

Last (hopefully) question. In my documentation, I explained the need to "press the "enable all" button, while looking at the rules (Interface Tab / “WAN Rules”, select “extrarule-DOH.rules”).

As new list entries and / or CNAME_info will probably be added in the future, new rules, with new sids will be added. Will these rules automatically become enabled, or is there something else I need to do to ensure all rules, including new, will always be enabled?

bmeeks

@jpgpi250 said in suricata (o)DoH rules:

Last (hopefully) question. In my documentation, I explained the need to "press the "enable all" button, while looking at the rules (Interface Tab / “WAN Rules”, select “extrarule-DOH.rules”).
As new list entries and / or CNAME_info will probably be added in the future, new rules, with new sids will be added. Will these rules automatically become enabled, or is there something else I need to do to ensure all rules, including new, will always be enabled?

Clicking the Enable All button is not required unless you have rules in the archive that are "default disabled". Let me explain that term once more. I have discussed what that means in some other past threads, but will repeat it again so readers won't have to go searching for the explanation.

Many rules in some of the categories from the various vendors are "default disabled" in the distributed gzip archives. The rules are disabled by placing a semicolon as the first character on the line, and therefore Snort or Suricata skips loading of that rule when parsing the category file. Rules can be shipped "default disabled" for a number of reasons. The two most common are: (1) the rule tends to false positive in many environments; and (2) the rule addresses a very old threat for which patches have been made available for quite some time. There are a few other possible reasons for a rule to ship commented out in a category. What the Enable All button does is go through and remove any leading semicolons on all the rules in a category file. This "enabling" all of them regardless of how they initially shipped in the gzip archive.

But Enable All can be wasteful of config.xml space as it will put each and every GID:SID pair for those "forced enabled" rules into the file. The following paragraph explains why you do not need to do this.

So back to your question. So long as you do not ship your rules with a leading semicolon at the start of a line (which would mark it as "default disabled"), then all the rules in a category file will be loaded and processed each time the category file is scanned. And when you update the file and add additional rules, they will automatically be parsed and loaded so long as they do not have a leading semicolon marking them as commented out (or default disabled). Simply enabling the category on the CATEGORIES tab, or referencing the category file name in a SID MGMT conf file, is sufficient to ensure all the rules in the file are always parsed and loaded (again assuming you, as the author, do not ship with some rules commented out).

jpgpi250

current status:

the sid range has been approved, see here.
PR to add ruleset to suricata-intel-index/index.yaml at master · OISF/suricata-intel-index · GitHub, no response yet...

Further attemp to block DoH, using suricata:

a discussion on the pi-hole forum, regarding SVCB queries, read here.

TLDR; My opinion, SVCB data can be used by “rogue” applications to bypass the system configured DNS server, therefore, I would like to add a suricata rule that blocks SVCB (type 64) DNS queries.

question on the suricata forum, here, no replies yet.
question (issue on the stamus ( authors of "The Security Analysts Guide to Suricata") forum, here, no replies yet.

What I've come up with myself (sid is test - custom):

alert dns any any -> $EXTERNAL_NET 53 (msg:"SVCB query (DoH)"; content:"|00 01 00 00 00 00 00|"; content:"|00 00 40 00 01|"; fast_pattern; distance:3; classtype:external-ip-check; sid:1000002; rev:1; metadata:affected_product Any, attack_target Client_Endpoint, updated_at 2022_12_28;)

more details on what this rule implies here.

any ideas, comments?

edit
command to trigger the rule:

dig @94.140.15.15 _dns.resolver.arpa svcb +short

wireshark capture in zip: svcb query.zip
/edit