Suricata & P2P Blocking - Working but would like to fine tune.
-
@bmeeks thanks. I played around with that today with some not great results. I adjusted our testing vlan and enabled Inline IPS. (we have Intel x540-at2 NICs in the server which are on the supported list and the GUI didn't complain) Within seconds all clients across all vlans on the server stopped functioning. All our lan segments are vlans off one of the x540 NICs and after digging a little I found a couple of posts saying that inline mode doesn't work well with single NICs that have multiple vlans assigned. Is that the case and the issue that I likely ran into? Or was it that there was a mix of Legacy and Inline on the same NIC, but diff vlans?
-
To expand a bit... all offloading settings for the NIC are disabled per recommendations and Run Mode was set to Workers. New clients could not get a DHCP address, those with existing or even fixed addresses could not ping the gateway across all vlans. As soon as I disabled the interface with Inline enabled, things came back to life on the other interfaces within a minute or so.
-
Yes, VLANs and Inline IPS Mode do not play well due to limitations with how the kernel's netmap device works with VLANs. You can have some luck by running a single Suricata instance on the physical parent interface instead of multiple instances on each VLAN.
So, for example, if your VLANs were
ix0.100, ix0.200, ix0.300
, then you would configure a single Suricata instance onix0
. -
@bmeeks Unless I'm missing something, it does not appear that you can assign Suricata to the parent interface when things are vlan’d. It's not an option in the drop down list as things are normally configured with vlans.
If I add the parent ix1 interface as an assignment within pfSense itself (with or without giving it an actual IP address), then all the vlans on that interface stop functioning. Doesn’t matter if the interface is enabled or disabled in the pfSense UI. As soon as I delete the assignment on the parent interface, all the vlans start working again.
-
@PalisadesTahoe said in Suricata & P2P Blocking - Working but would like to fine tune.:
@bmeeks Unless I'm missing something, it does not appear that you can assign Suricata to the parent interface when things are vlan’d. It's not an option in the drop down list as things are normally configured with vlans.
If I add the parent ix1 interface as an assignment within pfSense itself (with or without giving it an actual IP address), then all the vlans on that interface stop functioning. Doesn’t matter if the interface is enabled or disabled in the pfSense UI. As soon as I delete the assignment on the parent interface, all the vlans start working again.
Hmm... currently I do not have an environment where I can test VLAN connectivity with Suricata.
One issue that currently exists in FreeBSD is that the netmap kernel device cannot properly parse interface names with the VLAN ID suffix added. So, for example, the call to open a netmap instance on an interface named
ix1.897
would fail because the netmap parser treats the period as a reserved special character and rejects it as part of a valid interface name. There was an update to the netmap device upstream about two weeks ago that fixes this, but that update is not in either of the current pfSense RELEASE editions (it's not in CE nor Plus). It should be in the next pfSense release (which currently looks like it will be 24.03 for Plus).When I first replied to your post I was remembering more how Snort works than Suricata. After going back and refreshing my memory by examining the Suricata GUI package code, I remembered that the GUI code automatically strips out the VLAN IDs and attempts to run the instance on the physical parent instead. That worked when I tested it quite some time in the past. But it appears to be failing in your case -- likely because there is no instance of the physical interface active.
VLANs and Inline IPS Mode are more or less incompatible. While it can work in some limited ways sometimes with certain NIC drivers, it is not reliable. Inline IPS Mode works best on plain vanilla interfaces. That means no VLANs, no Bridges, and no LAGGs. VLANs, Bridges, and LAGGs are all types of virtual interfaces in FreeBSD. Netmap wants to work with physical interfaces only. It is very cantankerous with virtual interfaces.
-
I have a DMZ interface thats on a trunk(tagged) port on the firewall. I cant use Inline for this but i do want that additional visability. Can i run Suricata on IDS mode then on this trunked interface? What about IPS? Can i use legacy and it work ?
-
@michmoor said in Suricata & P2P Blocking - Working but would like to fine tune.:
I have a DMZ interface thats on a trunk(tagged) port on the firewall. I cant use Inline for this but i do want that additional visability. Can i run Suricata on IDS mode then on this trunked interface? What about IPS? Can i use legacy and it work ?
Legacy Mode should work fine. It uses the PCAP library, and that library is happy with virtual interfaces (or physical ones). It is only netmap that is picky, but Inline IPS requires netmap to function. Legacy Mode does not need netmap as it uses
pf
itself to implement blocks by adding IP addresses to a pre-existingpf
table.The trade-off is Legacy Mode blocks an IP entirely (once blocked, all traffic to/from that IP is blocked). Inline IPS Mode can selectively drop individual packets without needing to block the IP entirely.
-
@bmeeks
Thanks for clearing that up. Im bookmarking your comment.
This has been a confusing one for me. What works what doesnt and when should you apply, etc..
Really appreciate it -
@bmeeks said in Suricata & P2P Blocking - Working but would like to fine tune.:
VLANs and Inline IPS Mode are more or less incompatible.
That seals the deal then. Until we put 8 more nic's into this server, I guess we're blocking hosts and not packets. Thanks again for the detailed info and prompt replies.
-
@PalisadesTahoe said in Suricata & P2P Blocking - Working but would like to fine tune.:
@bmeeks said in Suricata & P2P Blocking - Working but would like to fine tune.:
VLANs and Inline IPS Mode are more or less incompatible.
That seals the deal then. Until we put 8 more nic's into this server, I guess we're blocking hosts and not packets. Thanks again for the detailed info and prompt replies.
That is likely the best option for now - at least until the netmap device can at least open VLAN interfaces properly. Here is the upstream FreeBSD commit I referred to earlier: https://github.com/pfsense/FreeBSD-src/commit/ad874544d9f018bf8eef4053b5ca7b856c4674cb. This went into FreeBSD about two weeks ago, and recently got merged into the development branch of pfSense. That's why it should be available in the next production release.
A good bit of netmap device improvement has come from the OPNsense team with the assistance of the Sunny Valley and Klara folks as well. As they roll fixes into FreeBSD, they get inherited into pfSense when the Netgate team merges the updated kernel code.
-
Hi @bmeeks,
Sorry to hijack an old thread, but quick question. We're about to build a new pfSense server for similar use and plan on using Suricata again. I'm curious what 10G network adapters you would recommend so that we could enable inline IPS? Are we OK sticking with the Intel x540-AT2 adapters that we have in our current server now that the netmap patch mentioned has been committed into the FreeBSD code?
Thanks! -
@PalisadesTahoe said in Suricata & P2P Blocking - Working but would like to fine tune.:
Hi @bmeeks,
Sorry to hijack an old thread, but quick question. We're about to build a new pfSense server for similar use and plan on using Suricata again. I'm curious what 10G network adapters you would recommend so that we could enable inline IPS? Are we OK sticking with the Intel x540-AT2 adapters that we have in our current server now that the netmap patch mentioned has been committed into the FreeBSD code?
Thanks!Sorry, but I have no knowledge of 10G NICs. My guess is that support in FreeBSD will be hit or miss, especially where netmap is concerned.
There are far more high-speed Suricata deployments on Linux platforms than there are on FreeBSD. The upstream Suricata team does rather extensive testing and benchmarking on Linux. All they do for FreeBSD is make sure the code compiles. They do no performance testing/optimization on FreeBSD that I am aware of.
If I were considering a true 10G deployment of IDS/IPS, then I would use a separate hardware platform running a current Linux variant and configure Suricata using the applicable Deb/RPM package for the chosen Linux distro. Suricata on pfSense is going to always use the host rings interface, and that is inherently slower than direct hardware interfaces. When you run separate dedicated IDS/IPS hardware, you can utilize two separate NICs for input and output traffic without need of host rings.
-
@bmeeks Thanks. We're kind of stuck on fBSD since it's the backend of pfSense at the moment.
-
@PalisadesTahoe said in Suricata & P2P Blocking - Working but would like to fine tune.:
@bmeeks Thanks. We're kind of stuck on fBSD since it's the backend of pfSense at the moment.
I don't think you will get anywhere near 10G performance using IDS/IPS with pfSense right now. The use of the host rings interface with netmap is just not capable of that.
What I'm talking about is a completely separate box sitting between your LAN and the pfSense interface, or between pfSense and your WAN connection. A true dual-port NIC hardware device similar to the old Sourcefire IPS appliance.
Something similar to this, but built with your own hardware: https://www.cisco.com/c/en/us/support/security/firepower-8000-series-appliances/series.html. The network traffic goes in one NIC port and comes out of another (matching NIC port pairs for each traffic path) similar to a bridge. With enough CPU and RAM, that type of arrangement can easily hit 10G throughput.