Using Inline mode with vmx interfaces.

Marc05 · Dec 23, 2022, 6:29 AM

I've been trying to get Suricata working in Inline mode on ESXi using VMXNET3 with pfSense 2.7/23.01. The VM is configured with a single interface (vmx0) which carries multiple VLANs (tag 4095 on ESXi), 4 CPU sockets, and all memory is reserved.

Unfortunately after Suricata starts, all traffic is dropped and the firewall must be accessed via console.

System logs show this right before traffic stops:

Dec 22 23:59:05 	kernel 		145.637963 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048
Dec 22 23:59:05 	kernel 		145.637946 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048
Dec 22 23:59:05 	kernel 		145.637902 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048
Dec 22 23:59:05 	kernel 		144.941099 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048
Dec 22 23:59:05 	kernel 		144.912824 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048
Dec 22 23:59:05 	kernel 		144.912809 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048
Dec 22 23:59:05 	kernel 		144.912763 [ 851] iflib_netmap_config txr 4 rxr 4 txd 512 rxd 512 rbufsz 2048

Suricata logs show the following (note the processed being manually killed to restore connectivity):

22/12/2022 -- 23:58:17 - <Notice> -- This is Suricata version 6.0.8 RELEASE running in SYSTEM mode
22/12/2022 -- 23:58:17 - <Info> -- CPUs/cores online: 4
22/12/2022 -- 23:58:17 - <Info> -- HTTP memcap: 67108864
22/12/2022 -- 23:58:17 - <Info> -- Netmap: Setting IPS mode
22/12/2022 -- 23:58:17 - <Info> -- fast output device (regular) initialized: alerts.log
22/12/2022 -- 23:58:17 - <Info> -- http-log output device (regular) initialized: http.log
22/12/2022 -- 23:58:21 - <Info> -- Rule with ID 2026440 is bidirectional, but source and destination are the same, treating the rule as unidirectional
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 invalid frame header"; flow:established; app-layer-event:http2.invalid_frame_header; classtype:protocol-command-decode; sid:2290000; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26763
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 invalid client magic"; flow:established; app-layer-event:http2.invalid_client_magic; classtype:protocol-command-decode; sid:2290001; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26764
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 invalid frame data"; flow:established; app-layer-event:http2.invalid_frame_data; classtype:protocol-command-decode; sid:2290002; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26765
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 invalid header"; flow:established; app-layer-event:http2.invalid_header; classtype:protocol-command-decode; sid:2290003; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26766
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 invalid frame length"; flow:established; app-layer-event:http2.invalid_frame_length; classtype:protocol-command-decode; sid:2290004; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26767
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 header frame with extra data"; flow:established; app-layer-event:http2.extra_header_data; classtype:protocol-command-decode; sid:2290005; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26768
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 too long frame data"; flow:established; app-layer-event:http2.long_frame_data; classtype:protocol-command-decode; sid:2290006; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26769
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 stream identifier reuse"; flow:established; app-layer-event:http2.stream_id_reuse; classtype:protocol-command-decode; sid:2290007; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26770
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 invalid HTTP1 settings during upgrade"; flow:established; app-layer-event:http2.invalid_http1_settings; classtype:protocol-command-decode; sid:2290008; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26771
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_UNKNOWN_PROTOCOL(124)] - protocol "http2" cannot be used in a signature.  Either detection for this protocol is not yet supported OR detection has been disabled for protocol through the yaml option app-layer.protocols.http2.detection-enabled
22/12/2022 -- 23:58:37 - <Error> -- [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert http2 any any -> any any (msg:"SURICATA HTTP2 failed decompression"; flow:established; app-layer-event:http2.failed_decompression; classtype:protocol-command-decode; sid:2290009; rev:1;)" from file /usr/local/etc/suricata/suricata_41734_vmx0.99/rules/suricata.rules at line 26772
22/12/2022 -- 23:58:38 - <Info> -- 1 rule files processed. 31993 rules successfully loaded, 10 rules failed
22/12/2022 -- 23:58:38 - <Warning> -- [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - can't suppress sid 2012252, gid 1: unknown rule
22/12/2022 -- 23:58:38 - <Info> -- Threshold config parsed: 1 rule(s) found
22/12/2022 -- 23:58:39 - <Info> -- 31993 signatures processed. 1200 are IP-only rules, 4619 are inspecting packet payload, 26015 inspect application layer, 108 are decoder event only
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.http.javaclient.vulnerable' is checked but not set. Checked in 2014750 and 10 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.pdf.in.http' is checked but not set. Checked in 2015866 and 6 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.JS.Obfus.Func' is checked but not set. Checked in 2017246 and 1 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ms.rdp.synack' is checked but not set. Checked in 2014384 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.http2' is checked but not set. Checked in 2034095 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.http.javaclient' is checked but not set. Checked in 2016113 and 30 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.IE7.NoRef.NoCookie' is checked but not set. Checked in 2023671 and 10 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.F5AuthBypass' is checked but not set. Checked in 2036547 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.JavaArchiveOrClass' is checked but not set. Checked in 2017756 and 3 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.WinHttpRequest' is checked but not set. Checked in 2019823 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ETPRO.RTF' is checked but not set. Checked in 2020700 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'HTTP.UncompressedFlash' is checked but not set. Checked in 2016396 and 2 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'exe.no.referer' is checked but not set. Checked in 2020500 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'is_proto_irc' is checked but not set. Checked in 2002029 and 10 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.gadu.loggedin' is checked but not set. Checked in 2008320 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.ButterflyJoin' is checked but not set. Checked in 2011296 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.MSSQL' is checked but not set. Checked in 2013411 and 1 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.ms08067_header' is checked but not set. Checked in 2008739 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.http.binary' is checked but not set. Checked in 2018103 and 6 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.ELFDownload' is checked but not set. Checked in 2019896 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.DocVBAProject' is checked but not set. Checked in 2020170 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.wininet.UA' is checked but not set. Checked in 2021312 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.MS.XMLHTTP.ip.request' is checked but not set. Checked in 2022050 and 1 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.MS.XMLHTTP.no.exe.request' is checked but not set. Checked in 2022053 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.MCOFF' is checked but not set. Checked in 2022303 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'et.MS.WinHttpRequest.no.exe.request' is checked but not set. Checked in 2022653 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.armwget' is checked but not set. Checked in 2024242 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Netwire.HB.1' is checked but not set. Checked in 2018282 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.smb.binary' is checked but not set. Checked in 2027402 and 4 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.Socks5.OnionReq' is checked but not set. Checked in 2027704 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.autoit.ua' is checked but not set. Checked in 2019165 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'min.gethttp' is checked but not set. Checked in 2023711 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.tcpraw.png' is checked but not set. Checked in 2035477 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.GenericPhish_Adobe' is checked but not set. Checked in 2023048 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.phpBB3_test' is checked but not set. Checked in 2010894 and 3 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.phpBB3_register_stage4' is checked but not set. Checked in 2010897 and 0 other sigs
22/12/2022 -- 23:58:39 - <Warning> -- [ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit 'ET.phpBB3_register_stage2' is checked but not set. Checked in 2010896 and 0 other sigs
22/12/2022 -- 23:59:04 - <Info> -- Disabling promiscuous mode on iface vmx0
22/12/2022 -- 23:59:04 - <Info> -- Disabling promiscuous mode on iface vmx0^
22/12/2022 -- 23:59:04 - <Info> -- Going to use 4 thread(s) for device vmx0
22/12/2022 -- 23:59:04 - <Info> -- devname [fd: 8] netmap:vmx0-0/R@conf:host-rings=4 vmx0 opened
22/12/2022 -- 23:59:04 - <Info> -- devname [fd: 12] netmap:vmx0^0/T@conf:host-rings=4 vmx0^ opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 13] netmap:vmx0-1/R vmx0 opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 15] netmap:vmx0^1/T vmx0^ opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 16] netmap:vmx0-2/R vmx0 opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 17] netmap:vmx0^2/T vmx0^ opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 18] netmap:vmx0-3/R vmx0 opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 19] netmap:vmx0^3/T vmx0^ opened
22/12/2022 -- 23:59:05 - <Info> -- Disabling promiscuous mode on iface vmx0^
22/12/2022 -- 23:59:05 - <Info> -- Disabling promiscuous mode on iface vmx0
22/12/2022 -- 23:59:05 - <Info> -- Going to use 4 thread(s) for device vmx0^
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 20] netmap:vmx0^0/R@conf:host-rings=4 vmx0^ opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 21] netmap:vmx0-0/T@conf:host-rings=4 vmx0 opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 22] netmap:vmx0^1/R vmx0^ opened
22/12/2022 -- 23:59:05 - <Info> -- devname [fd: 23] netmap:vmx0-1/T vmx0 opened
22/12/2022 -- 23:59:06 - <Info> -- devname [fd: 24] netmap:vmx0^2/R vmx0^ opened
22/12/2022 -- 23:59:06 - <Info> -- devname [fd: 25] netmap:vmx0-2/T vmx0 opened
22/12/2022 -- 23:59:06 - <Info> -- devname [fd: 26] netmap:vmx0^3/R vmx0^ opened
22/12/2022 -- 23:59:06 - <Info> -- devname [fd: 27] netmap:vmx0-3/T vmx0 opened
22/12/2022 -- 23:59:06 - <Notice> -- all 8 packet processing threads, 2 management threads initialized, engine started.
22/12/2022 -- 23:59:52 - <Notice> -- Signal Received.  Stopping engine.
22/12/2022 -- 23:59:52 - <Info> -- time elapsed 47.968s
22/12/2022 -- 23:59:54 - <Info> -- cleaning up signature grouping structure... complete

bmeeks · Dec 23, 2022, 3:29 PM

What version of ESXi are you running?

Some quick Google searches turned up some issues with the VMXNET3 driver and FreeBSD in certain setups.

I don't currently have an ESXi pfSense VM, but I do have a couple of pfSense VMs running under the latest VMware Workstation version. Those are using the VMXNET3 driver without an issue.

Could you try changing the virtual NIC to be an e1000 emulation as a test?

Marc05 · Dec 26, 2022, 4:04 PM

It's ESXi 7. If it's a must, I can test E1000, but that lacks so many things (most importantly multiple NIC queues) that it's not worth it to me. FWIW, it used to work to an extent when I was doing PCI passthrough for the NIC.

Cool_Corona · Dec 26, 2022, 7:13 PM

Causing no issues here :) Everything is logging as it should.

Cool_Corona · Dec 26, 2022, 7:56 PM

Only issue is the traffic graphs are not working when Suricata is in inline mode.

And thats a PITA.................

Cool_Corona · Dec 26, 2022, 8:08 PM

AND there is a major performance hit using inline vs legacy mode

Same server

Inline:

Legacy:

Marc05 · Dec 26, 2022, 8:08 PM

I'm wondering if the fact that it's a trunk port is related to the issue - I have not tried it without VLANs on it (aka access port).

Cool_Corona · Dec 26, 2022, 8:23 PM

@Marc05 Here is my test Suricata Config.

suricataconfig.txt

Marc05 · Dec 27, 2022, 3:56 AM

Thank you for posting it. I don't see inline being used on there, and it's an older package version. I don't see anything else that stands out that might explain the issue I'm running into.

Cool_Corona · Dec 27, 2022, 7:03 AM

@marc05 Still running 2.5.2 since 2.6 is unstable and VLANs are not working as it should