Throughput drop on Netgate 8200 MAX LAN/VLAN (ix1) with Suricata inline mode
-
Good day!
Device: Netgate 8200 MAX
Interface: LAN/VLAN (ix1)With Suricata in inline mode, throughput falls from ~1 Gbps to ~300 Mbps.
With Suricata in legacy mode, throughput remains ~1 Gbps.
What might be causing this?
iperf results;
-
You show a VLAN configured on the LAN physical interface. VLANs and netmap (the underlying FreeBSD kernel device used to support inline IPS mode operation) are not great friends
. While it can work, a VLAN interface requires the use of an emulated netmap adapter which is a software construct that is much less efficient than the hardware adapter netmap interfaces. Another thing that limits IPS performance on pfSense is the fact IPS mode uses the host rings netmap interface. That interface is virtual and is much slower than the hardware rings associated with the NIC.Another issue that can severely affect throughput is the number of enabled rules. More rules means more CPU work and less throughput.
Lastly, you may need to fine-tune settings for the NIC adapter using sysctl variables. You would need to perform your own research for that. I have no experience with that and thus no tips to offer.
Legacy Mode uses the PCAP library to simply grab copies of packets traversing an interface. Suricata is then fed those copied packets to digest while the original packets continue on to the host. That means Legacy Mode will leak the initial packets and let the connection be made. Then, after Suricata has time to compare the packet or packets to the signatures and if there is a match, a pfctl firewall API call is made to place the offending IP address into a pf table for subsequent blocking. Another API call is then made to flush any active states that are associated with the blocked IP.
Also noticed that you posted this same issue on the upstream Suricata forum. That will not help. The Suricata package on pfSense is highly customized and the developers upstream are not privy to the inner workings of the Suricata setup used in pfSense (nor in OPNsense, for that matter). Both *Sense products use a GUI front-end for managing Suricata. Suricata itself (the binary used to do the actual inspecting of traffic) has no GUI. It is managed completely at the command line level. But that is not true on pfSense as the GUI code manages the underlying binary and controls the creation of the suricata.yaml file.
If you want optimum wire-speed IPS performance with Suricata, then you should install it on a separate hardware platform with at least three NIC ports running Linux. Suricata is optimized for Linux and no so much for FreeBSD. Three NIC ports allow one for a management address and then the other two for a netmap or AF_PACKET bridge handled by Suricata. You would need to install the binary package for the Linux distro you chose and configure Suricata by hand using the CLI. In your use case, this new hardware box would go inline between the LAN port of your 8200 MAX and the LAN switch in your network.
-
Thank you for the reply.
Switching Suricata’s Run Mode to Workers resolved the throughput drop. Really appreciate the help!
-
@smsigroupit said in Throughput drop on Netgate 8200 MAX LAN/VLAN (ix1) with Suricata inline mode:
Thank you for the reply.
Switching Suricata’s Run Mode to Workers resolved the throughput drop. Really appreciate the help!
Ah --- yes. I forgot to mention trying Workers Run Mode in my previous post. Workers mode allows the netmap packet handling module to avoid the need to lock the netmap rings during critical functions.
