• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Suricata with custom memory_limit crashing on upgrade

Scheduled Pinned Locked Moved IDS/IPS
4 Posts 2 Posters 487 Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S
    sgnoc
    last edited by Feb 22, 2023, 3:14 PM

    I attempted an upgrade from Suricata 6.0.8 to 6.0.10 today. I had to use a larger memory limit (1024M from 512M) in the /usr/local/pkg/suricata/suricata.inc as described in:

    https://forum.netgate.com/topic/176467/suricata-sid-managment-rebuild-causing-php-memory-error

    The problem is, when the new suricata package is installed, the new suricata.inc is going back to the default 512M, which isn't large enough to rebuild the yaml files. I've tried to run a sed as soon as the suricata.inc is written, but it isn't working.

    I keep getting this error:

    Cleaning up after rules extraction... done.
    The Rules update has finished.
    Generating suricata.yaml configuration file from saved settings.
    Generating YAML configuration file for 00_WAN... done.
    Generating YAML configuration file for 05_LAN...
    Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 51232048 bytes) in /usr/local/pkg/suricata/suricata.inc on line 1066
    PHP ERROR: Type: 1, File: /usr/local/pkg/suricata/suricata.inc, Line: 1066, Message: Allowed memory size of 536870912 bytes exhausted (tried to allocate 51232048 bytes)pkg-static: POST-INSTALL script failed
    >>> Cleaning up cache... done.
    

    Are there any other options for me to get Suricata running again? The system is crashing before getting to where I can get in to the Suricata GUI.

    I really wish the memory_limit was a tunable feature that wouldn't be overwritten.

    Thanks!

    S 1 Reply Last reply Feb 22, 2023, 4:03 PM Reply Quote 0
    • S
      sgnoc @sgnoc
      last edited by Feb 22, 2023, 4:03 PM

      Ok, well I think I found a solution. I was able to manually uninstall suricata, then remove but save the <suricata></suricata> XML from the config.xml file.

      When I reinstalled suricata with the default settings, I then changed the memory from 512 back to 1024, and replaced the generated <suricata></suricata> XML from the config file with the original settings. I wasn't able to start suricata entirely, but it let me start each interface one at a time.

      It looks like Suricata is back on track, so I'll monitor for anything out of the ordinary.

      1 Reply Last reply Reply Quote 0
      • B
        bmeeks
        last edited by bmeeks Feb 22, 2023, 7:25 PM Feb 22, 2023, 7:25 PM

        While altering the allocated PHP memory is a workaround, you should address the root cause of the problem instead.

        Sounds like you need to rethink what you are doing in your SID MGMT settings. Exhausting the PHP memory should not be happening there unless something really strange is going on with what you have configured there.

        How about posting your SID MGMT conf files and let me have a look?

        S 1 Reply Last reply Feb 23, 2023, 2:19 AM Reply Quote 0
        • S
          sgnoc @bmeeks
          last edited by Feb 23, 2023, 2:19 AM

          @bmeeks Thanks for the input. I like having the additional rules, which don't seem to affect Suricata at all other than initially building the yaml files, but I know you are correct in finding a better way to optimize my rules. I'll have to go through and audit the ruleset vs what I have in the SID management. I basically have a SID drop file and SID disable, with default enable on the rulesets. Here are the SID drop/disable rules for WAN.

          I have similar SID files for 5 other interfaces that I manage/monitor with Suricata separately. I'm sure if I go through and find what rules I need on WAN, then reduce the rules even more for what I have on the internal interfaces, it would be a lot better. I won't need some of the WAN rules on internal interfaces, so only would then need rules appropriate for local interface to local interface, since the WAN rule will get anything in/outbound, if I'm thinking about it correctly. So I'll focus on that first.

          I'm not sure really where the memory usage is the highest when building the yaml files. Does the bulk of the memory usage come from loading the various rules, and then more memory is used when adding in the SID management files? If I know how the memory is consumed, I can try and optimize focusing on that, too.

          Here is the WAN disable SID:

          # WAN Disable Ruleset
          
          # ET Pro Rules
          etpro-chat,etpro-dns,etpro-games,etpro-icmp,etpro-icmp_info,etpro-inappropriate,etpro-info,etpro-p2p
          etpro-policy,etpro-tor
          
          # SNORT Rules
          snort_app-detect,snort_chat,snort_content-replace,snort_dns,snort_icmp-info,snort_icmp,snort_info
          snort_multimedia,snort_p2p,snort_policy,snort_x11
          
          # Individual Disabled Ruleset
          
          # FIOS Guide
          1:2840787 # ETPRO HUNTING Request for config.json
          
          # Suricata Stream
          1:2210008 # SURICATA STREAM 3way handshake SYN resend different seq on SYN recv
          1:2210016 # SURICATA STREAM CLOSEWAIT FIN out of window
          1:2210029 # SURICATA STREAM ESTABLISHED invalid ack
          1:2210038 # SURICATA STREAM FIN out of window
          1:2210042 # SURICATA STREAM TIMEWAIT ACK with wrong seq
          1:2210044 # SURICATA STREAM Packet with invalid timestamp
          1:2210045 # SURICATA STREAM Packet with invalid ack
          1:2210050 # SURICATA STREAM reassembly overlap with different data
          1:2210054 # SURICATA STREAM excessive retransmissions
          
          # Breaks webpage/NEST
          1:2221010 # SURICATA HTTP unable to match response to request
          
          # Breaks NEST
          1:2018383 # ET EXPLOIT Possible OpenSSL HeartBleed Large HeartBeat Response from Common SSL Port (Outbound from Client)
          
          # Generated from LTE_Extender
          1:2221045 # SURICATA HTTP Unexpected Request body
          
          # Weatherflow
          1:2229001 # SURICATA MQTT PUBLISH not seen before PUBACK/PUBREL/PUBREC/PUBCOMP
          
          # Noisy / Misc
          1:2023883 # ET DNS Query to a *.top domain - Likely Hostile
          1:2027390 # ET USER_AGENTS Microsoft Device Metadata Retrieval Client User-Agent
          1:2027757 # ET DNS Query for .to TLD
          1:2027758 # ET DNS Query for .cc TLD
          1:2200036 # SURICATA TCP option invalid length
          1:2200070 # SURICATA FRAG IPv4 Fragmentation overlap
          1:2200073 # SURICATA IPv4 invalid checksum
          1:2200075 # SURICATA UDPv4 invalid checksum
          1:2200076 # SURICATA ICMPv4 invalid checksum
          1:2210002 # SURICATA STREAM 3way handshake right seq wrong ack evasion
          1:2210004 # SURICATA STREAM 3way handshake SYNACK resend with different ack
          1:2210010 # SURICATA STREAM 3way handshake wrong seq wrong ack
          1:2210015 # SURICATA STREAM CLOSEWAIT ACK out of window
          1:2210020 # SURICATA STREAM ESTABLISHED packet out of window
          1:2210023 # SURICATA STREAM ESTABLISHED SYNACK resend with different ACK
          1:2210024 # SURICATA STREAM ESTABLISHED SYNACK resend with different seq
          1:2210026 # SURICATA STREAM ESTABLISHED SYN resend
          1:2210030 # SURICATA STREAM FIN invalid ack
          1:2210035 # SURICATA STREAM FIN2 FIN with wrong seq
          1:2210036 # SURICATA STREAM FIN2 invalid ack
          1:2210046 # SURICATA STREAM SHUTDOWN RST invalid ack
          1:2221014 # SURICATA HTTP missing Host header
          1:2221017 # SURICATA HTTP invalid response field folding
          1:2221021 # SURICATA HTTP response header invalid
          1:2224003 # SURICATA IKEv2 weak cryptographic parameters (PRF)
          1:2224004 # SURICATA IKEv2 weak cryptographic parameters (Auth)
          1:2224005 # SURICATA IKEv2 weak cryptographic parameters (Diffie-Hellman)
          1:2229002 # SURICATA MQTT SUBSCRIBE not seen before SUBACK
          1:2229005 # SURICATA MQTT message seen before CONNECT/CONNACK completion
          1:2230003 # SURICATA TLS invalid handshake message
          1:2230010 # SURICATA TLS invalid record/traffic
          1:2260000 # SURICATA Applayer Mismatch protocol both directions
          1:2260002 # SURICATA Applayer Detect protocol only one direction
          

          Here is the WAN Drop Sid:

          # WAN Drop Ruleset
          
          # Snort GPLv2 Community Rules Drop
          GPLv2_community
          
          # Feodo Tracker Botnet C2 Rules
          feodotracker
          
          # Abuse.ch SSL Blacklist Rules
          sslblacklist_tls_cert
          
          # ET Pro Rules
          etpro-activex,etpro-adware_pup,etpro-attack_response,etpro-botcc,etpro-ciarmy,etpro-coinminer
          etpro-compromised,etpro-current_events,etpro-dos,etpro-drop,etpro-dshield
          etpro-exploit,etpro-exploit_kit,etpro-ftp,etpro-hunting,etpro-imap,etpro-ja3,etpro-malware,etpro-misc
          etpro-mobile_malware,etpro-netbios,etpro-phishing,etpro-pop3,etpro-rpc
          etpro-scan,etpro-shellcode,etpro-smtp,etpro-sql,etpro-telnet,etpro-tftp,etpro-threatview_CS_c2
          etpro-trojan,etpro-user_agents,etpro-web,etpro-worm
          
          # Snort Ruleset
          snort_attack-response,snort_backdoor,snort_bad-traffic,snort_blacklist,snort_botnet-cnc,snort_browser
          snort_ddos,snort_deleted,snort_dos,snort_experimental,snort_exploit-kit,snort_file,snort_finger
          snort_ftp,snort_indicator,snort_local,snort_malware,snort_misc,snort_mysql,snort_netbios,snort_os
          snort_other-ida,snort_phishing-spam,snort_pop,snort_protocol,snort_pua,snort_rpc,snort_rservices
          snort_scan,snort_server,snort_shellcode,snort_smtp,snort_snmp,snort_specific-threats
          snort_spyware-put,snort_sql,snort_telnet,snort_tftp,snort_virus,snort_voip,snort_web
          
          # Individual Ruleset
          1:2210008 # SURICATA STREAM 3way handshake SYNACK resend with different ack
          1:2210026 # SURICATA STREAM ESTABLISHED SYN resend
          
          1 Reply Last reply Reply Quote 0
          • 5 5p9 referenced this topic on Jul 29, 2024, 11:02 AM
          4 out of 4
          • First post
            4/4
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
            This community forum collects and processes your personal information.
            consent.not_received