Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Netgate hardware

    Scheduled Pinned Locked Moved
    Official Netgate® Hardware
    4
    15
    1.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bmeeksB
      bmeeks @chrysmon
      last edited by bmeeks

      @chrysmon said in Netgate hardware:

      I have a question about pfSense hardware (not necessary Netgate) and you are very suitable person to respond. How many CPU cores and what CPU speed are required for pfSense to do the following:

      inter VLAN routing on 2 SFP+ 10Gbit interfaces and other 4 1Gbit including WAN. There are 5 VLANs on each SFP+. There are firewall rules everywhere. No 10Gbit endpoints pass through pfSense. The network has about 500 MAC addresses.
      Internet connection 150-200 Mbps symmetric.
      OpenVPN (2-3 connections simultaneously all the day long, occasionally others). There are 5 OpenVPN servers.
      Packages: Suricata, pfBlockerNG, Wireguard, Zabbix and some others not relevant here.

      Sorry, but I have no idea how answer those questions. You might want to take this up with the Netgate sales team.

      C 1 Reply Last reply Reply Quote 0
      • C
        chrysmon @bmeeks
        last edited by

        @bmeeks Sorry for the Watchdog, I'm new in the forum.
        The question is a CPU suggestion for the configuration specified. Because you know each package is multi core or not, etc.

        bmeeksB 1 Reply Last reply Reply Quote 0
        • bmeeksB
          bmeeks @chrysmon
          last edited by bmeeks

          @chrysmon said in Netgate hardware:

          Because you know each package is multi core or not, etc.

          No, I only know about the two packages I manage. Suricata is multithreaded while Snort is not. Those are the only ones I know about as those are the only two I maintain.

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            I will add that we do test packages especially the popular packages like Suricata. But we cannot test every package in every configuration on all hardware types. Unfortunately. 😉

            bmeeksB 1 Reply Last reply Reply Quote 1
            • bmeeksB
              bmeeks @stephenw10
              last edited by

              @stephenw10 said in Netgate hardware:

              I will add that we do test packages especially the popular packages like Suricata. But we cannot test every package in every configuration on all hardware types. Unfortunately. 😉

              Thanks @stephenw10. I did not know that the Netgate team ran formal tests with Suricata.

              I just assumed the occasional tickets that appear on Redmine with Netgate folks as the author were the result of client troubleshooting efforts by the paid support Help Desk staff.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by stephenw10

                The testing we can do there is fairly limited. But, for example, if Suricata failed to install or failed to start with a basic config that should be caught by release testing.

                More informally a number of us run test configs for Snort and Suricata with more complex configs. But that combinations of config and architecture are large!

                C 1 Reply Last reply Reply Quote 0
                • C
                  chrysmon @stephenw10
                  last edited by

                  @stephenw10 Maybe for longer than a day I had no Suricata crashes. Today I updated the package and it crashed twice since then. Version 7.0.2_1. I restart manually after deleting the /var/run/suricata_whatever

                  bmeeksB 1 Reply Last reply Reply Quote 0
                  • bmeeksB
                    bmeeks @chrysmon
                    last edited by bmeeks

                    @chrysmon said in Netgate hardware:

                    @stephenw10 Maybe for longer than a day I had no Suricata crashes. Today I updated the package and it crashed twice since then. Version 7.0.2_1. I restart manually after deleting the /var/run/suricata_whatever

                    Are you seeing the same HyperScan error as the users are reporting in this thread: https://forum.netgate.com/topic/184101/suricata-process-dying-due-to-hyperscan-problem?

                    If so, please post your current configuration and any log messages from the pfSense system log (under STATUS > SYSTEM LOGS) and the suricata.log (under LOGS VIEW in the Suricata GUI) into a new message in that thread. I would like to consolidate all the Suricata "stops" into a common thread if possible as I think they are all related to something with the HyperScan library and Suricata's interaction with it.

                    C 1 Reply Last reply Reply Quote 0
                    • C
                      chrysmon @bmeeks
                      last edited by

                      @bmeeks Yes, The last log entry in the STATUS > SYSTEM LOGS is:

                      suricata 17648 [209498] <Error> -- Hyperscan returned fatal error -1.

                      This is from the suricata.log:

                      [107176 - Suricata-Main] 2023-11-25 12:48:26 Notice: suricata: This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
                      [107176 - Suricata-Main] 2023-11-25 12:48:26 Info: cpu: CPUs/cores online: 12
                      [107176 - Suricata-Main] 2023-11-25 12:48:26 Info: suricata: Setting engine mode to IDS mode by default
                      [107176 - Suricata-Main] 2023-11-25 12:48:26 Info: app-layer-htp-mem: HTTP memcap: 671088640
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Creating automatic firewall interface IP address Pass List.
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: alert-pf output device (regular) initialized: block.log
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Loading and parsing Pass List from: /usr/local/etc/suricata/suricata_5401_igb0/passlist.
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Pass List /usr/local/etc/suricata/suricata_5401_igb0/passlist processed: Total entries parsed: 20, IP addresses/netblocks/aliases added to No Block list: 18, IP addresses/netblocks ignored because they were covered by existing entries: 2.
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Created Firewall Interface IP Change monitor thread for auto-whitelisting of firewall interface IP addresses.
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: pfSense Suricata Custom Blocking Module initialized: pf-table=snort2c block-ip=src kill-state=yes block-drops-only=yes passlist-debugging=no
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: fast output device (regular) initialized: alerts.log
                      [209444 - ] 2023-11-25 12:48:26 Info: alert-pf: Firewall Interface IP Address Change monitoring thread IM#01 has successfully started.
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: http-log output device (regular) initialized: http.log
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: stats output device (regular) initialized: stats.log
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-syslog: Syslog output initialized
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: eve-log output device (regular) initialized: eve.json
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Warning: output-json-alert: HTTP body logging has been configured, however, metadata logging has not been enabled. HTTP body logging will be disabled.
                      [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: output-json-email-common: Going to log the md5 sum of email subject
                      [100456 - Suricata-Main] 2023-11-25 12:48:43 Info: detect: 2 rule files processed. 44008 rules successfully loaded, 0 rules failed
                      [100456 - Suricata-Main] 2023-11-25 12:48:43 Info: threshold-config: Threshold config parsed: 0 rule(s) found
                      [100456 - Suricata-Main] 2023-11-25 12:48:43 Info: detect: 44017 signatures processed. 1282 are IP-only rules, 6728 are inspecting packet payload, 35719 inspect application layer, 109 are decoder event only
                      [100456 - Suricata-Main] 2023-11-25 12:48:43 Warning: detect-flowbits: flowbit 'ET.GenericPhish_Adobe' is checked but not set. Checked in 2023048 and 0 other sigs
                      [100456 - Suricata-Main] 2023-11-25 12:48:43 Warning: detect-flowbits: flowbit 'is_ssh_client_kex' is checked but not set. Checked in 2001977 and 1 other sigs
                      [100456 - Suricata-Main] 2023-11-25 12:49:09 Info: runmodes: Using 1 live device(s).
                      [209486 - RX#01-igb0] 2023-11-25 12:49:09 Info: pcap: igb0: running in 'auto' checksum mode. Detection of interface state will require 1000 packets
                      [209486 - RX#01-igb0] 2023-11-25 12:49:09 Info: pcap: igb0: snaplen set to 14180
                      [100456 - Suricata-Main] 2023-11-25 12:49:10 Notice: threads: Threads created -> RX: 1 W: 12 FM: 1 FR: 1 Engine started.
                      [209486 - RX#01-igb0] 2023-11-25 12:49:14 Info: checksum: No packets with invalid checksum, assuming checksum offloading is NOT used
                      [209498 - W#12] 2023-11-25 20:21:08 Error: spm-hs: Hyperscan returned fatal error -1.

                      I'm running Suricata only on the WAN interface, in IPS Mode (Legacy), the Pattern Matcher Algorithm set to Auto. Do you need any other information about configuration? Maybe the part from the backup file?

                      bmeeksB 1 Reply Last reply Reply Quote 0
                      • bmeeksB
                        bmeeks @chrysmon
                        last edited by bmeeks

                        @chrysmon said in Netgate hardware:

                        @bmeeks Yes, The last log entry in the STATUS > SYSTEM LOGS is:

                        suricata 17648 [209498] <Error> -- Hyperscan returned fatal error -1.

                        This is from the suricata.log:

                        [107176 - Suricata-Main] 2023-11-25 12:48:26 Notice: suricata: This is Suricata version 7.0.2 RELEASE running in SYSTEM mode
                        [107176 - Suricata-Main] 2023-11-25 12:48:26 Info: cpu: CPUs/cores online: 12
                        [107176 - Suricata-Main] 2023-11-25 12:48:26 Info: suricata: Setting engine mode to IDS mode by default
                        [107176 - Suricata-Main] 2023-11-25 12:48:26 Info: app-layer-htp-mem: HTTP memcap: 671088640
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Creating automatic firewall interface IP address Pass List.
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: alert-pf output device (regular) initialized: block.log
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Loading and parsing Pass List from: /usr/local/etc/suricata/suricata_5401_igb0/passlist.
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Pass List /usr/local/etc/suricata/suricata_5401_igb0/passlist processed: Total entries parsed: 20, IP addresses/netblocks/aliases added to No Block list: 18, IP addresses/netblocks ignored because they were covered by existing entries: 2.
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: Created Firewall Interface IP Change monitor thread for auto-whitelisting of firewall interface IP addresses.
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-pf: pfSense Suricata Custom Blocking Module initialized: pf-table=snort2c block-ip=src kill-state=yes block-drops-only=yes passlist-debugging=no
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: fast output device (regular) initialized: alerts.log
                        [209444 - ] 2023-11-25 12:48:26 Info: alert-pf: Firewall Interface IP Address Change monitoring thread IM#01 has successfully started.
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: http-log output device (regular) initialized: http.log
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: stats output device (regular) initialized: stats.log
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: alert-syslog: Syslog output initialized
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: logopenfile: eve-log output device (regular) initialized: eve.json
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Warning: output-json-alert: HTTP body logging has been configured, however, metadata logging has not been enabled. HTTP body logging will be disabled.
                        [100456 - Suricata-Main] 2023-11-25 12:48:26 Info: output-json-email-common: Going to log the md5 sum of email subject
                        [100456 - Suricata-Main] 2023-11-25 12:48:43 Info: detect: 2 rule files processed. 44008 rules successfully loaded, 0 rules failed
                        [100456 - Suricata-Main] 2023-11-25 12:48:43 Info: threshold-config: Threshold config parsed: 0 rule(s) found
                        [100456 - Suricata-Main] 2023-11-25 12:48:43 Info: detect: 44017 signatures processed. 1282 are IP-only rules, 6728 are inspecting packet payload, 35719 inspect application layer, 109 are decoder event only
                        [100456 - Suricata-Main] 2023-11-25 12:48:43 Warning: detect-flowbits: flowbit 'ET.GenericPhish_Adobe' is checked but not set. Checked in 2023048 and 0 other sigs
                        [100456 - Suricata-Main] 2023-11-25 12:48:43 Warning: detect-flowbits: flowbit 'is_ssh_client_kex' is checked but not set. Checked in 2001977 and 1 other sigs
                        [100456 - Suricata-Main] 2023-11-25 12:49:09 Info: runmodes: Using 1 live device(s).
                        [209486 - RX#01-igb0] 2023-11-25 12:49:09 Info: pcap: igb0: running in 'auto' checksum mode. Detection of interface state will require 1000 packets
                        [209486 - RX#01-igb0] 2023-11-25 12:49:09 Info: pcap: igb0: snaplen set to 14180
                        [100456 - Suricata-Main] 2023-11-25 12:49:10 Notice: threads: Threads created -> RX: 1 W: 12 FM: 1 FR: 1 Engine started.
                        [209486 - RX#01-igb0] 2023-11-25 12:49:14 Info: checksum: No packets with invalid checksum, assuming checksum offloading is NOT used
                        [209498 - W#12] 2023-11-25 20:21:08 Error: spm-hs: Hyperscan returned fatal error -1.

                        I'm running Suricata only on the WAN interface, in IPS Mode (Legacy), the Pattern Matcher Algorithm set to Auto. Do you need any other information about configuration? Maybe the part from the backup file?

                        No, this is obviously the HyperScan issue described in the thread I linked. It's right here in the log:

                        suricata 17648 [209498] <Error> -- Hyperscan returned fatal error -1.

                        But I was wanting information about this bug posted in the other thread I linked so that there aren't half a dozen other threads scattered around the forum about the same issue. It makes it hard for me to track who has what problem and what shared information they might have if there are lots of different threads all about the same basic issue. Much easier to keep track when all the comments and reports about a given issue are in the same thread.

                        Please post anything else you have about this issue in the thread I linked earlier. Here is the direct link again: https://forum.netgate.com/topic/184101/suricata-process-dying-due-to-hyperscan-problem.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post