Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0
-
Hi @bmeeks -
Just went through the upgrade here and everything went fine. I wish, however, I could say the same for using inline mode instead of legacy mode:
- My system has 4 port Chelsio T540 10Gbit SFP+ network expansion card (for the LAN interfaces) and an Intel i210 based 1Gbit WAN interface. I was able to enable inline mode on all these interfaces (including those with VLAN's) without issues.
- Unfortunately with inline mode enabled performance really nose-dived. I used to be able to push 4Gbit/s+ between two different (LAN) network segments (both with Snort's Balanced policy enabled), using iperf3 in legacy mode. However, with inline mode enabled instead (and still using the Balanced policy) that throughput number reduced sharply to around 200Mbit/s.
- I also saw a large number of netmap transmit errors during the upload portion of WAN speed test (I have a symmetric 1Gbit fiber connection), as described here: https://forum.netgate.com/topic/138613/configuring-pfsense-netmap-for-suricata-inline-ips-mode-on-em-igb-interfaces Perhaps my system needs to have a faster CPU - it is Xeon D-1518 based which has worked quite well up now.
Ultimately, I decided to go back to legacy mode for now while I investigate these netmap performance issues further. Do you have any ideas why performance might be so bad? I do have all the offloading and flow control options disabled, but it didn't really seem to help.
I also have one quick question for you: While Snort is in legacy mode, does it matter what the IPS Policy Mode is set to under interface categories (i.e. either Alert or Policy)? Am I correct in thinking that it actually does not matter in legacy mode as long as Block Offenders is checked?
Thanks in advance for your help, I really appreciate it.
-
@tman222 said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
Hi @bmeeks -
Just went through the upgrade here and everything went fine. I wish, however, I could say the same for using inline mode instead of legacy mode:
- My system has 4 port Chelsio T540 10Gbit SFP+ network expansion card (for the LAN interfaces) and an Intel i210 based 1Gbit WAN interface. I was able to enable inline mode on all these interfaces (including those with VLAN's) without issues.
- Unfortunately with inline mode enabled performance really nose-dived. I used to be able to push 4Gbit/s+ between two different (LAN) network segments (both with Snort's Balanced policy enabled), using iperf3 in legacy mode. However, with inline mode enabled instead (and still using the Balanced policy) that throughput number reduced sharply to around 200Mbit/s.
- I also saw a large number of netmap transmit errors during the upload portion of WAN speed test (I have a symmetric 1Gbit fiber connection), as described here: https://forum.netgate.com/topic/138613/configuring-pfsense-netmap-for-suricata-inline-ips-mode-on-em-igb-interfaces Perhaps my system needs to have a faster CPU - it is Xeon D-1518 based which has worked quite well up now.
Ultimately, I decided to go back to legacy mode for now while I investigate these netmap performance issues further. Do you have any ideas why performance might be so bad? I do have all the offloading and flow control options disabled, but it didn't really seem to help.
I also have one quick question for you: While Snort is in legacy mode, does it matter what the IPS Policy Mode is set to under interface categories (i.e. either Alert or Policy)? Am I correct in thinking that it actually does not matter in legacy mode as long as Block Offenders is checked?
Thanks in advance for your help, I really appreciate it.
The netmap device is a complicated beast and has gone through several generations of tweaks on FreeBSD. There has also just been a big change in the overall NIC hardware driver interface introduced in FreeBSD-12. That new interface is called iflib. So in FreeBSD-12, netmap functionality is handled within iflib, and NIC manufacturers create their drivers for FreeBSD-12 to use iflib. So netmap interfacing comes without any extra work by the NIC hardware drive author. None of that is the case on FreeBSD-11. There, each hardware NIC driver vendor is responsible for their own netmap intefacing. Some do it better than others.
Another complicating issue is that netmap is not designed to work well with VLANs. So when you have VLANs, netmap is not going to be a good idea for you in terms of performance. You also will need to disable any and all hardware-level VLAN options on the NIC.
Lastly, there is a different netmap interface API version in FreeBSD-11 versus FreeBSD-12. Only FreeBSD-12 supports multiple host rings in netmap. FreeBSD-11 exposes only a single pair of Tx/Rx host rings.
Netmap is really designed to rapidly move data from one physical NIC to another physical NIC, but it does not work as well when exchanging data between a physical NIC and the host stack (or the kernel itself). The implementation on pfSense needs netmap to exchange data between a NIC and the host stack.
There have been some recent commits (software updates) made to the netmap portion of the iflib code in FreeBSD-12. Some of those changes explicitly addressed netmap transmit errors. Apparently there is a bug in the iflib code related to them. However, since pfSense-2.4.5 is based on FreeBSD-11, those iflib updates will not get into pfSense-2.4.5.
So short answer to the long story above is netmap is going to be selective feature that works well for some, but not for others. And on FreeBSD-11 it is also going to be hardware dependent. And on both pfSense-2.4.5 and pfSense-2.5, if you are using VLANs, then netmap is not an ideal choice for you with the IDS/IPS.
-
@tman222 said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
I also have one quick question for you: While Snort is in legacy mode, does it matter what the IPS Policy Mode is set to under interface categories (i.e. either Alert or Policy)? Am I correct in thinking that it actually does not matter in legacy mode as long as Block Offenders is checked?
Thanks in advance for your help, I really appreciate it.
No, that setting has no influence when the operational mode is Legacy Blocking. In fact, I really meant for that option to be hidden when an interface is running in Legacy Mode. I will need to fix that so it does not even display when using Legacy Mode on an interface.
-
@tman222
I found the change that is causing your problem. However, that code has been running over in pfSense-2.5 DEVEL for over a year, so it's weird that nobody running Snort 4.x on the DEVELOPMENT snapshots has encountered this. It's caused by the extra long name of your real interfaces with the VLAN ID tacked on the end.A typical NIC real interface name is three or four characters such as em0, bce1, etc. An interface with that name length won't hit the suffix name limit defined by the Snort binary, even when a VLAN ID is tacked on (unless the VLAN ID is really long). Your NIC real interface shows up as vtnet2, so that is a bit longer than normal. That, plus your three digit VLAN ID, is what puts you over the limit when the underscore character is included.
The code change was made in the 4.x branch of Snort to address some other issues with the older technique used in the 3.x Snort branch of passing a randomly-generated GUID as part of the PID file name. I can possibly revert back to that method, but I need to carefully review all of the 4.x PHP code to be sure I find and fix all the places that reference the PID filename.
The only thing you can do in the interim is shorten that real interface name. The only method I think you have is to shorten the VLAN ID, but that of course likely requires other changes in your system that you may not wish to undertake.
-
Hi there,
I just upgraded from 2.4.4.3 to newest build, prior to the upgrade I upgraded snort and pfblocker.
After the upgrade snort would not start and I received a similar error, Ialso noticed that the snort rules failed to download at first, but i managed to download them manually.
Sep 19 03:20:10 php /tmp/snort_bge0_startcmd.php: The command '/usr/local/bin/snort -R _bge0 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert -l /var/log/snort/snort_bge022424 --pid-path /var/run --nolock-pidfile --no-interface-pidfile -G 22424 -c /usr/local/etc/snort/snort_22424_bge0/snort.conf -i bge0' returned exit code '1', the output was 'ERROR size 1464 != 1448'
my WAN adapter is named bge0 so nothing unusual and I do not have VLAN tagging enabled in pfsense as that is handled on the switch.
The only other thing i have changed apart from the upgrade was that I disabled PXE boot rom from the NIC at server boot.
This the first integrated adapter out of two on a HP dl385 G4 server dual Xeon CPU with 6gb Mem, it has run for years and never failed an upgrade.
Please let me know if you want any other info from the system.Kind Regards
Sebastian
-
@mikrosec said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
Hi there,
I just upgraded from 2.4.4.3 to newest build, prior to the upgrade I upgraded snort and pfblocker.
After the upgrade snort would not start and I received a similar error, Ialso noticed that the snort rules failed to download at first, but i managed to download them manually.
Sep 19 03:20:10 php /tmp/snort_bge0_startcmd.php: The command '/usr/local/bin/snort -R _bge0 -D -q --suppress-config-log --daq pcap --daq-mode passive --treat-drop-as-alert -l /var/log/snort/snort_bge022424 --pid-path /var/run --nolock-pidfile --no-interface-pidfile -G 22424 -c /usr/local/etc/snort/snort_22424_bge0/snort.conf -i bge0' returned exit code '1', the output was 'ERROR size 1464 != 1448'
my WAN adapter is named bge0 so nothing unusual and I do not have VLAN tagging enabled in pfsense as that is handled on the switch.
The only other thing i have changed apart from the upgrade was that I disabled PXE boot rom from the NIC at server boot.
This the first integrated adapter out of two on a HP dl385 G4 server dual Xeon CPU with 6gb Mem, it has run for years and never failed an upgrade.
Please let me know if you want any other info from the system.Kind Regards
Sebastian
That error is not related the other poster's error in this thread. His error message clearly stated the PID suffix was too long.
While it does not say so, your error seems like it might be an MTU or snaplen thing. Is this interface by chance a PPPoE interface?
LATER FOLLOW-UP:
Okay, did some Google research and found this error can be caused by having incorrect versions of some shared libraries. If you upgraded from pfSense-2.4.4_p3, that is clearly possible.
Go to the GLOBAL SETTINGS tab and verify that the "Keep Settings After Deinstall" setting is enabled (box checked). That is the default, but check to be sure. Then go to the SYSTEM > PACKAGE MANAGER tab and then Installed Packages and delete the Snort package. Let the package removal complete. Then go to the Available Packages tab and install it again. Report back on how that works.
-
It's a normal Ethernet Link nothing special, but you are absolutely right it is not related I just found this error:
98858 FATAL ERROR: Failed to initialize dynamic preprocessor: SF_MODBUS version 1.1.1 (-2)
I will investigate and open another thread if necessary, sorry for barging in.
KInd Regards
-
@mikrosec said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
It's a normal Ethernet Link nothing special, but you are absolutely right it is not related I just found this error:
98858 FATAL ERROR: Failed to initialize dynamic preprocessor: SF_MODBUS version 1.1.1 (-2)
I will investigate and open another thread if necessary, sorry for barging in.
KInd Regards
Follow my instructions in the "LATER FOLLOW-UP" edit I made to my previous post above. That should correct your problem.
-
@bmeeks Thank you for that!
I checked that "Click to retain Snort settings after package removal." was enabled and deinstalled and reinstalled smoothly without any errors. And it started up smoothly, like it has for 8 years now :-)
Saved my night
Cheers.
-
@mikrosec said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
@bmeeks Thank you for that!
I checked that "Click to retain Snort settings after package removal." was enabled and deinstalled and reinstalled smoothly without any errors. And it started up smoothly, like it has for 8 years now :-)
Saved my night
Cheers.
The
pkg
utilty that FreeBSD uses to install and update packages sometimes misses updating certain shared libraries, particularly when changing base OS versions such as when updating from pfSense-2.4.4 to 2.4.5 (that was a FreeBSD operating system update). In this case, some of the Snort preprocessor libs did not get deleted and the new versions installed. -
@bmeeks said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
I found the change that is causing your problem. However, that code has been running over in pfSense-2.5 DEVEL for over a year, so it's weird that nobody running Snort 4.x on the DEVELOPMENT snapshots has encountered this. It's caused by the extra long name of your real interfaces with the VLAN ID tacked on the end.
A typical NIC real interface name is three or four characters such as em0, bce1, etc. An interface with that name length won't hit the suffix name limit defined by the Snort binary, even when a VLAN ID is tacked on (unless the VLAN ID is really long). Your NIC real interface shows up as vtnet2, so that is a bit longer than normal. That, plus your three digit VLAN ID, is what puts you over the limit when the underscore character is included.
The code change was made in the 4.x branch of Snort to address some other issues with the older technique used in the 3.x Snort branch of passing a randomly-generated GUID as part of the PID file name. I can possibly revert back to that method, but I need to carefully review all of the 4.x PHP code to be sure I find and fix all the places that reference the PID filename.
The only thing you can do in the interim is shorten that real interface name. The only method I think you have is to shorten the VLAN ID, but that of course likely requires other changes in your system that you may not wish to undertake.
Thanks for the quick check! :)
Can you estimate how long you will need for the fixing this?
-
After upgrade to the new version, I am unsable to perform a "rule action" to modify rules within the Snort alerts. Happens with Firefox and Safari.
Edit: Happens also directly within the rules editor of the interfaces.
Is there any way to roll back to the old package version?
-
@dotsch said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
After upgrade to the new version, I am unsable to perform a "rule action" to modify rules within the Snort alerts. Happens with Firefox and Safari.
Edit: Happens also directly within the rules editor of the interfaces.
I am unable to reproduce your problem. I changed a rule from DROP to ALERT and it showed up as "user-forced Alert" on the RULES tab.
You do understand that when making a change on the ALERTS tab that the icon for the past alert will not change. If the rule was DROP when the rule was triggered, that is the action written to the log file and thus will be the action displayed. However, it will be different on the RULES tab if you force an action change on the ALERTS tab and then go check the same rule on the RULES tab.
Lastly, the action change dialog pops up as what Bootstrap calls a Modal Dialog. It's possible that some security settings in your browser might be preventing that from displaying. Are you saying you never see the Modal Dialog box, or you see the box and make a change but it does not "stick"?
Is there any way to roll back to the old package version?
No, there is no way to roll back. The old package is removed from the repository.
-
@Beerman said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
Can you estimate how long you will need for the fixing this?
It will be several days. Perhaps as long as a week. Once I make the change, the update has to be reviewed and approved by the pfSense team before it gets posted in the package repositories.
-
@bmeeks said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
@dotsch said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
After upgrade to the new version, I am unsable to perform a "rule action" to modify rules within the Snort alerts. Happens with Firefox and Safari.
Edit: Happens also directly within the rules editor of the interfaces.
I am unable to reproduce your problem. I changed a rule from DROP to ALERT and it showed up as "user-forced Alert" on the RULES tab.
You do understand that when making a change on the ALERTS tab that the icon for the past alert will not change. If the rule was DROP when the rule was triggered, that is the action written to the log file and thus will be the action displayed. However, it will be different on the RULES tab if you force an action change on the ALERTS tab and then go check the same rule on the RULES tab.
Lastly, the action change dialog pops up as what Bootstrap calls a Modal Dialog. It's possible that some security settings in your browser might be preventing that from displaying. Are you saying you never see the Modal Dialog box, or you see the box and make a change but it does not "stick"?
There is nothing happens, when I click to any icon...
What for security settings do you mean?Is there any way to roll back to the old package version?
No, there is no way to roll back. The old package is removed from the repository.
Is there any plan to implement this. I had a lot of issues in the past years, and think it would help to make pfsense more professional. -
@dotsch said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
@bmeeks said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
@dotsch said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
After upgrade to the new version, I am unsable to perform a "rule action" to modify rules within the Snort alerts. Happens with Firefox and Safari.
Edit: Happens also directly within the rules editor of the interfaces.
I am unable to reproduce your problem. I changed a rule from DROP to ALERT and it showed up as "user-forced Alert" on the RULES tab.
You do understand that when making a change on the ALERTS tab that the icon for the past alert will not change. If the rule was DROP when the rule was triggered, that is the action written to the log file and thus will be the action displayed. However, it will be different on the RULES tab if you force an action change on the ALERTS tab and then go check the same rule on the RULES tab.
Lastly, the action change dialog pops up as what Bootstrap calls a Modal Dialog. It's possible that some security settings in your browser might be preventing that from displaying. Are you saying you never see the Modal Dialog box, or you see the box and make a change but it does not "stick"?
There is nothing happens, when I click to any icon...
What for security settings do you mean?Is there any way to roll back to the old package version?
No, there is no way to roll back. The old package is removed from the repository.
Is there any plan to implement this. I had a lot of issues in the past years, and think it would help to make pfsense more professional.I assume that you do know you can't modify the action of a rule when you are using Legacy Mode Blocking. The option to modify a rule's action is only enabled when you use Inline IPS Mode blocking. When using Legacy Mode Blocking, you can't change the action of rules and thus the icons are not "clickable". The Inline IPS Mode option is new to this version of Snort on pfSense-2.4.5 RELEASE. Previously this option was only available in pfSense-2.5 DEVEL.
-
Thanks @bmeeks for all this information, I really appreciate it!
I was able to make some progress after coming across these couple interesting links last night:
https://lists.freebsd.org/pipermail/freebsd-net/2017-March/047384.html
https://lists.freebsd.org/pipermail/svn-src-all/2016-June/126504.htmlIt turns out that Chelsio offers direct netmap support via virtual interfaces which need to be enabled separately with a sysctl tunable. The regular interfaces don't seem to support netmap natively so when I enabled inline mode in Snort, netmap was actually being emulated, which explains the poor performance I was seeing. Unfortunately I missed this key log entry the first time around, which offered a clue:
Sep 19 12:47:08 kernel 028.141828 [ 318] generic_netmap_register Emulated adapter for cxl1 activated
In any case, I went ahead and enabled the virtual inferfaces by setting the sysctl tunable
hw.cxgbe.num_vis=2
and then confirmed with dmesg that I now had netmap support for the Chelsio card:vcxl1: netmap queues/slots: TX 8/1023, RX 8/1024 vcxl1: 8 txq, 8 rxq (NIC); 8 txq, 8 rxq (netmap)
I then reassigned interface to use the virtual interface instead (i.e. instead of using cxl1 for the interface I assigned vcxl1). Everything was working well at this point, traffic was passing fine, and performance was as expected.
However, as soon I switched over from legacy mode to inline mode in Snort and started using netmap, all traffic ceased on the vcxl1 interface. If I turn Snort off or go back to legacy mode, everything works fine and traffic passes. At this point I'm stumped - I still have offloading, checksumming, flow control disabled. It does appear like all traffic is being dropped for some reason, but I'm unsure why. @bmeeks are you aware of any additional logging facilities in Snort or otherwise I could look into to try to investigate this further? Or would it maybe make sense to give Suricatta and its inline mode a try instead (if for some reason there is a problem with the Snort netmap implementation)?
Thanks again for all your help, I really appreciate it.
-
@tman222 said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
However, as soon I switched over from legacy mode to inline mode in Snort and started using netmap, all traffic ceased on the vcxl1 interface. If I turn Snort off or go back to legacy mode, everything works fine and traffic passes. At this point I'm stumped - I still have offloading, checksumming, flow control disabled. It does appear like all traffic is being dropped for some reason, but I'm unsure why. @bmeeks are you aware of any additional logging facilities in Snort or otherwise I could look into to try to investigate this further? Or would it maybe make sense to give Suricatta and its inline mode a try instead (if for some reason there is a problem with the Snort netmap implementation)?
Thanks again for all your help, I really appreciate it.
Unfortunately there is no additional logging within Snort relative to netmap operation. Netmap is a kernel device, and Snort is basically ignorant of what happens internally within the device. All the Snort binary does is call an API to open an interface in netmap device mode. It gets a "success" or "failure" message, and that's about it.
Having said that, there are continuing changes happening within the netmap code in FreeBSD. This is particularly true in the new iflib wrapper API found in FreeBSD-12 and up (which is what pfSense-2.5 DEVELOPMENT is based upon). And I do know that in the not too distant past a bug was fixed in the code where the netmap device interacts with virtual devices. Perhaps that bug (which I assume still exists in the legacy FreeBSD code base) is what is biting your Chelsio virtual configuration ???
Netmap has proven to be pretty problematic with both Snort and Suricata. And it is not just on pfSense, either. The "other" firewall distro based on FreeBSD has also had the same types of netmap problems with Inline IPS Mode. Things are getting better, but the code in FreeBSD is not yet 100% there. As a result, you will find some NICs that work fine while others won't work at all. If your NIC hardware won't work with Inline IPS Mode, then you will just have to switch over to Legacy Mode Blocking. The situation is marginally better on FreeBSD-12 with the new iflib wrapper, but even there it is not yet perfected.
-
Thanks @bmeeks for this additional info - it's very helpful!
I do have some good news! I was able to make some more progress and got everything working in native netmap mode on the Chelsio interfaces. It turns out the that the missing tunable was
hw.cxgbe.fl_pktshift
which needs to be set to 0:https://lists.freebsd.org/pipermail/freebsd-net/2016-January/044433.html
I had considered setting this tunable before but didn't change it because I thought the default value was already 0. Turns out I was reading the wrong version of the FreeBSD documentation and in 11.3 it is actually set to 2 by default (Doh! It's always something simple isn't it? :)). In any case, once I set the pktshift value to 0, traffic started passing fine.
So to summarize, to use native netmap support with Chelsio cards set the following tunables in your
loader.conf.local
:hw.cxgbe.num_vis=2 hw.cxgbe.fl_pktshift=0
This will create a virtual interface for each physical interface, e.g.
cxl0
will havevcxl0
and so on. These virtual interfaces support netmap natively. You can confirm this by running dmesg and looking for netmap in the output, e.g.:dmesg | grep netmap
You should see netmap support for the Chelsio virtual interfaces, including any other interfaces in your firewall that support netmap natively. For sample output, please see my previous post above. Finally you need to assign the virtual interface(s) to your network segment(s) and enable Snort Inline mode.
==================
Unfortunately, performance is not much better for me. Even when using netmap natively on the Chelsio card (as opposed to emulated mode) and enabling Snort Inline (netmap) mode, I was still only able to pass about 200-300Mbit/s between two different 10Gbit LAN interfaces (separate physical subnets, no VLAN's) using an iperf3 test. This is very similar to what I saw in emulated netmap mode. With Snort legacy (pcap) mode, I usually get around 4Gbit/s. WAN performance is even worse and limited to 50-80 Mbit/s down/up on a symmetric 1Gbit fiber connection. I do not think the issue lies with the Chelsio netmap support because if I enable Snort inline IPS mode solely on the WAN interface which is Intel
igb
, and not on any of the Chelsio interfaces, I see the exact same WAN down/up speed limitation.@bmeeks you had mentioned this above:
Lastly, there is a different netmap interface API version in FreeBSD-11 versus FreeBSD-12. Only FreeBSD-12 supports multiple host rings in netmap. FreeBSD-11 exposes only a single pair of Tx/Rx host rings.
Do you think I may be hitting this (single host ring) limitation? Is there any way to tune Snort? Also, do you think it might be worth trying Suricata instead if they handle this interfacing differently (better?) Or would I be better off trying pfSense 2.5.0 at this point?
Thanks again for your help and insight I really appreciate it.
-
@tman222 said in Upcoming Snort Package Updates for pfSense-2.4.5 and pfSense-2.5.0:
@bmeeks you had mentioned this above:
Lastly, there is a different netmap interface API version in FreeBSD-11 versus FreeBSD-12. Only FreeBSD-12 supports multiple host rings in netmap. FreeBSD-11 exposes only a single pair of Tx/Rx host rings.
Do you think I may be hitting this (single host ring) limitation? Is there any way to tune Snort? Also, do you think it might be worth trying Suricata instead if they handle this interfacing differently (better?) Or would I be better off trying pfSense 2.5.0 at this point?
Thanks again for your help and insight I really appreciate it.
Perhaps, but there is also the issue of Snort being single-threaded. I'm not a kernel threading guru, so I'm not sure whether multi-threaded operation would help. My gut says "yeah", but there are other places where bottlenecks can occur besides just with the netmap packet processing. I don't know if you have duplicate hardware, but if you did, you could test using pfSense-2.5 snapshots. They are based on FreeBSD-12 and have the multiple host ring support (but only if the native NIC also exposes multiple rings).
Also, with Legacy Mode Blocking using PCAP, you would need to enable all the Snort stats to see if any packets are in fact getting dropped by Snort. You would not notice this on your speed test throughput because Snort is working on copies of the packets while the originals have gone on to the kernel when using Legacy Blocking. So the PCAP engine might be dropping some of the copied packets at high line rates. I think there are some stats for that when Statistics Mode is enabled (on the PREPROCESSORS tab). But I might be confusing some Suricata features here, so I don't want to be quoted on that.
When I first deployed the Inline IPS option with netmap on pfSense-2.5, the Netgate team did some testing of their own. If I recall correctly, they were able to get close to Gigabit line rates, but I don't know how many rules they had enabled. I'm thinking probably not very many. The Snort binary is likely not optimized for inline operation. Remember that Snort's history has been IDS first and foremost. The inline mode was added later as I understand it. And it's likely that code is not streamlined. Snort natively, through its DAQ library, supports two different inline modes on FreeBSD. One uses
ipfw
while the other usesnetmap
. The change I made to DAQ on pfSense was to allow thenetmap
mode to support host stack interfaces. Natively, DAQ only supported physical interfaces, so setting up an inline IPS mode between a NIC and the OS itself was not possible. You could only specify two different physical NIC ports as thenetmap
endpoints. This was not optimum for pfSense, so I added the ability to open and use host stack rings in DAQ.