Snort Inline - Horrible Performance?
-
I have all offloading disabled and have tried a few things and no matter what with legacy mode I get my full 2Gbps up/down but with inline mode it craters to ~55Mbps up/down and I'm not really sure what I'm missing or if it's just a compatibility thing with the card I'm using. I haven't done any advanced tweaking or changes so if those are things I should make let me know. I've reloaded between swapping from Legacy to Inline mode and following disabling all offloading to be triple sure (checksum wasn't disabled initially).
IPS Policy is set to Balanced for both legacy and inline and Policy mode for Inline.Specs:
pfSense version 22.05
Intel Xeon CPU D-1537
Intel X710-T2L NIC -
You will never see 2 Gbps performance with Snort inline on most hardware. The Snort version in use on pfSense is single-threaded, so it needs raw CPU clock speed and not a high core count to run better. Single-threaded operation means it will only ever use a single CPU core. But the faster the clock speed, the better the single core performance will be.
I suspect that using Legacy Blocking Mode you are actually dropping packets in the inspection engine at that 2 Gigabit/sec rate, but you would never know that because Legacy Mode works with copies of the packets. And if a packet is not copied over for inspection, you won't see any interruption in the original path.
With all the above said, I still would expect several hundred megabits/second of throughput with Inline IPS Mode. But a 1.7 GHz base clock with a single-threaded application like Snort is not going to hit lightspeed. Perhaps some tuning of interrupt loads may help ??
The Netgate folks have tested Snort before with Inline IPS Mode and were able to achieve throughput near 1 Gigabit/sec. Note that the number of enabled rules plays a huge role. The Balanced IPS policy is going to bring in quite a few rules -- perhaps many of which you don't actually need in your environment. Part of the "art" of managing an IDS/IPS setup is matching enabled rules to the vulnerabilities that may exist in your protected network.
-
One thing you can try- if you are willing- is to switch over to Suricata. It is multi-threaded and has some patches to its netmap code to improve performance. Not saying it will make it to 2 Gbps either with your CPU, but it might substantially improve throughput as compared to the single-threaded Snort instance.
Do NOT run both at the same time, though! Remove Snort first before installing Suricata (your settings will stay in the pfSense configuration, so you can easily install the package again and all your settings will come right back).
Both packages have near identical GUIs, and operate if much the same way. It should be easy to find your way around in Suricata if you are accustomed to the Snort package.
-
@bmeeks I did just give Suricata a shot, inline mode seemed to work great until after about 5-10 minutes it just stopped passing traffic with no log messages or crashes to indicate what was wrong I just stopped being able to access anything outside the network. Snort was uninstalled when I was testing so not sure what was going on there. I've switched back to snort legacy mode for now although I might look at the connectivity rule set later on and see if that brings any changes or improvements with inline mode just for giggles.
-
@lurick said in Snort Inline - Horrible Performance?:
@bmeeks I did just give Suricata a shot, inline mode seemed to work great until after about 5-10 minutes it just stopped passing traffic with no log messages or crashes to indicate what was wrong I just stopped being able to access anything outside the network. Snort was uninstalled when I was testing so not sure what was going on there. I've switched back to snort legacy mode for now although I might look at the connectivity rule set later on and see if that brings any changes or improvements with inline mode just for giggles.
Having Suricata unexpectedly stop passing traffic is strange. There was a bug a few versions back in the flow manager code that caused that, but it was fixed upstream. I am not aware of any other reports of that behavior, and we never actually deployed that broken version of Suricata on pfSense. We stayed on the 5.x branch until that flow manager bug was fixed upstream.
It may be a NIC driver thing, though. I believe there are still a few bugs with NIC drivers in FreeBSD as a result of porting most of them over to use the
iflibwrapper system. There are some really good things that came from theiflibmigration, but it does still have a few warts in the implementation for some drivers. Don't know if your card may be one of those or not. -
@bmeeks Yah, was running version 6.0.4_1 but I'll dig some more into the NIC and iflib stuff and see if there is a known incompatibility or other issue that I might have missed. Just for posterity currently running the following firmware on the X710 NIC:
sysctl dev.ixl.1.fw_version
dev.ixl.1.fw_version: fw 8.1.63299 api 1.12 nvm 8.10 etid 800093ea oem 1.267.0
