@bmeeks Thanks for the input. I like having the additional rules, which don't seem to affect Suricata at all other than initially building the yaml files, but I know you are correct in finding a better way to optimize my rules. I'll have to go through and audit the ruleset vs what I have in the SID management. I basically have a SID drop file and SID disable, with default enable on the rulesets. Here are the SID drop/disable rules for WAN.
I have similar SID files for 5 other interfaces that I manage/monitor with Suricata separately. I'm sure if I go through and find what rules I need on WAN, then reduce the rules even more for what I have on the internal interfaces, it would be a lot better. I won't need some of the WAN rules on internal interfaces, so only would then need rules appropriate for local interface to local interface, since the WAN rule will get anything in/outbound, if I'm thinking about it correctly. So I'll focus on that first.
I'm not sure really where the memory usage is the highest when building the yaml files. Does the bulk of the memory usage come from loading the various rules, and then more memory is used when adding in the SID management files? If I know how the memory is consumed, I can try and optimize focusing on that, too.
Here is the WAN disable SID:
# WAN Disable Ruleset
# ET Pro Rules
etpro-chat,etpro-dns,etpro-games,etpro-icmp,etpro-icmp_info,etpro-inappropriate,etpro-info,etpro-p2p
etpro-policy,etpro-tor
# SNORT Rules
snort_app-detect,snort_chat,snort_content-replace,snort_dns,snort_icmp-info,snort_icmp,snort_info
snort_multimedia,snort_p2p,snort_policy,snort_x11
# Individual Disabled Ruleset
# FIOS Guide
1:2840787 # ETPRO HUNTING Request for config.json
# Suricata Stream
1:2210008 # SURICATA STREAM 3way handshake SYN resend different seq on SYN recv
1:2210016 # SURICATA STREAM CLOSEWAIT FIN out of window
1:2210029 # SURICATA STREAM ESTABLISHED invalid ack
1:2210038 # SURICATA STREAM FIN out of window
1:2210042 # SURICATA STREAM TIMEWAIT ACK with wrong seq
1:2210044 # SURICATA STREAM Packet with invalid timestamp
1:2210045 # SURICATA STREAM Packet with invalid ack
1:2210050 # SURICATA STREAM reassembly overlap with different data
1:2210054 # SURICATA STREAM excessive retransmissions
# Breaks webpage/NEST
1:2221010 # SURICATA HTTP unable to match response to request
# Breaks NEST
1:2018383 # ET EXPLOIT Possible OpenSSL HeartBleed Large HeartBeat Response from Common SSL Port (Outbound from Client)
# Generated from LTE_Extender
1:2221045 # SURICATA HTTP Unexpected Request body
# Weatherflow
1:2229001 # SURICATA MQTT PUBLISH not seen before PUBACK/PUBREL/PUBREC/PUBCOMP
# Noisy / Misc
1:2023883 # ET DNS Query to a *.top domain - Likely Hostile
1:2027390 # ET USER_AGENTS Microsoft Device Metadata Retrieval Client User-Agent
1:2027757 # ET DNS Query for .to TLD
1:2027758 # ET DNS Query for .cc TLD
1:2200036 # SURICATA TCP option invalid length
1:2200070 # SURICATA FRAG IPv4 Fragmentation overlap
1:2200073 # SURICATA IPv4 invalid checksum
1:2200075 # SURICATA UDPv4 invalid checksum
1:2200076 # SURICATA ICMPv4 invalid checksum
1:2210002 # SURICATA STREAM 3way handshake right seq wrong ack evasion
1:2210004 # SURICATA STREAM 3way handshake SYNACK resend with different ack
1:2210010 # SURICATA STREAM 3way handshake wrong seq wrong ack
1:2210015 # SURICATA STREAM CLOSEWAIT ACK out of window
1:2210020 # SURICATA STREAM ESTABLISHED packet out of window
1:2210023 # SURICATA STREAM ESTABLISHED SYNACK resend with different ACK
1:2210024 # SURICATA STREAM ESTABLISHED SYNACK resend with different seq
1:2210026 # SURICATA STREAM ESTABLISHED SYN resend
1:2210030 # SURICATA STREAM FIN invalid ack
1:2210035 # SURICATA STREAM FIN2 FIN with wrong seq
1:2210036 # SURICATA STREAM FIN2 invalid ack
1:2210046 # SURICATA STREAM SHUTDOWN RST invalid ack
1:2221014 # SURICATA HTTP missing Host header
1:2221017 # SURICATA HTTP invalid response field folding
1:2221021 # SURICATA HTTP response header invalid
1:2224003 # SURICATA IKEv2 weak cryptographic parameters (PRF)
1:2224004 # SURICATA IKEv2 weak cryptographic parameters (Auth)
1:2224005 # SURICATA IKEv2 weak cryptographic parameters (Diffie-Hellman)
1:2229002 # SURICATA MQTT SUBSCRIBE not seen before SUBACK
1:2229005 # SURICATA MQTT message seen before CONNECT/CONNACK completion
1:2230003 # SURICATA TLS invalid handshake message
1:2230010 # SURICATA TLS invalid record/traffic
1:2260000 # SURICATA Applayer Mismatch protocol both directions
1:2260002 # SURICATA Applayer Detect protocol only one direction
Here is the WAN Drop Sid:
# WAN Drop Ruleset
# Snort GPLv2 Community Rules Drop
GPLv2_community
# Feodo Tracker Botnet C2 Rules
feodotracker
# Abuse.ch SSL Blacklist Rules
sslblacklist_tls_cert
# ET Pro Rules
etpro-activex,etpro-adware_pup,etpro-attack_response,etpro-botcc,etpro-ciarmy,etpro-coinminer
etpro-compromised,etpro-current_events,etpro-dos,etpro-drop,etpro-dshield
etpro-exploit,etpro-exploit_kit,etpro-ftp,etpro-hunting,etpro-imap,etpro-ja3,etpro-malware,etpro-misc
etpro-mobile_malware,etpro-netbios,etpro-phishing,etpro-pop3,etpro-rpc
etpro-scan,etpro-shellcode,etpro-smtp,etpro-sql,etpro-telnet,etpro-tftp,etpro-threatview_CS_c2
etpro-trojan,etpro-user_agents,etpro-web,etpro-worm
# Snort Ruleset
snort_attack-response,snort_backdoor,snort_bad-traffic,snort_blacklist,snort_botnet-cnc,snort_browser
snort_ddos,snort_deleted,snort_dos,snort_experimental,snort_exploit-kit,snort_file,snort_finger
snort_ftp,snort_indicator,snort_local,snort_malware,snort_misc,snort_mysql,snort_netbios,snort_os
snort_other-ida,snort_phishing-spam,snort_pop,snort_protocol,snort_pua,snort_rpc,snort_rservices
snort_scan,snort_server,snort_shellcode,snort_smtp,snort_snmp,snort_specific-threats
snort_spyware-put,snort_sql,snort_telnet,snort_tftp,snort_virus,snort_voip,snort_web
# Individual Ruleset
1:2210008 # SURICATA STREAM 3way handshake SYNACK resend with different ack
1:2210026 # SURICATA STREAM ESTABLISHED SYN resend