Blog suricata-vs-snort - Snort 3.0?
-
@musicwizard said in Blog suricata-vs-snort - Snort 3.0?:
Kind of strange you say that. Im a IT admin
I probably phrased what I said poorly. It wasn't meant to be an insult, just that programming often takes a lot of time.
@michmoor said in Blog suricata-vs-snort - Snort 3.0?:
What would it take for packages to be moved to the 'Maintained by Netgate' status
No idea. They do help on pfBlocker occasionally, I think I've seen, not sure about others. One might think a popular package without a maintainer would be to Netgate's advantage to keep running.
-
@steveits said in Blog suricata-vs-snort - Snort 3.0?:
@michmoor said in Blog suricata-vs-snort - Snort 3.0?:
What would it take for packages to be moved to the 'Maintained by Netgate' status
No idea. They do help on pfBlocker occasionally, I think I've seen, not sure about others. One might think a popular package without a maintainer would be to Netgate's advantage to keep running.
Netgate is probably not eager to step in and take over a package mainly because they already have plenty of work on their plates maintaining TNSR, CE and now Plus core pfSense software.
They have stepped up and made quick fixes to popular packages when some pretty significant show-stopper issue surfaces -- as in the package won't install or run at all, or has some other major defect. I think this has mostly happened when the normal package maintainer is not immediately available.
And there are some packages that were actually created by Netgate staff. Service Watchdog is one that I recall, and I think Acme is another. Those packages were created to fill specific niches. OpenVPN Client Export is another such package.
-
@steveits said in Blog suricata-vs-snort - Snort 3.0?:
One might think a popular package without a maintainer would be to Netgate's advantage to keep running.
Agreed. Better to have it written on 'paper' for it to be official.
-
@bmeeks said in Blog suricata-vs-snort - Snort 3.0?:
Netgate is probably not eager to step in and take over a package mainly because they already have plenty of work on their plates maintaining TNSR, CE and now Plus core pfSense software.
This is one of those areas where I understand a bit. I agree with what you are saying completely. With limited time and resources its all about where to throw the bodies to fix or improve core features.
Users want more apps on top of what's available and i get it. At the end of the day, the devs and maintainers are doing the best they can. -
@michmoor said in Blog suricata-vs-snort - Snort 3.0?:
@bmeeks said in Blog suricata-vs-snort - Snort 3.0?:
Netgate is probably not eager to step in and take over a package mainly because they already have plenty of work on their plates maintaining TNSR, CE and now Plus core pfSense software.
This is one of those areas where I understand a bit. I agree with what you are saying completely. With limited time and resources its all about where to throw the bodies to fix or improve core features.
Users want more apps on top of what's available and i get it. At the end of the day, the devs and maintainers are doing the best they can.And certain features can actually work better not as add-on packages but as core constituents of the system. That's how Suricata is handled on OPNsense. It is always there as a core component, and the user has the option of enabling it or not. But the code is a part of the base system install. It's similar to how the DNS Resolver and DNS Forwarder are handled today in pfSense.
So I would expect that if the features of a particular package were very popular and almost all users installed that package, and it became associated with "pfSense" itself, the functionality of the package would be absorbed into the base system.
-
@bmeeks said in Blog suricata-vs-snort - Snort 3.0?:
So I would expect that if the features of a particular package were very popular and almost all users installed that package, and it became associated with "pfSense" itself, the functionality of the package would be absorbed into the base system.
How difficult is it getting a package intergrated? Id imagine Suricata and pfblocker are the most important and popular package to date.
-
@michmoor said in Blog suricata-vs-snort - Snort 3.0?:
@bmeeks said in Blog suricata-vs-snort - Snort 3.0?:
So I would expect that if the features of a particular package were very popular and almost all users installed that package, and it became associated with "pfSense" itself, the functionality of the package would be absorbed into the base system.
How difficult is it getting a package intergrated? Id imagine Suricata and pfblocker are the most important and popular package to date.
It's not a terribly difficult task to actually integrate, but testing everything to be sure hidden bugs did not break something huge would be time consuming.
While Suricata and Snort both seem popular now, to be honest I really don't understand why that is so on pfSense. The vast majority of network traffic today is encrypted. That means without MITM the IDS/IPS is almost blind. It can see source and destination IP addresses and ports, and it can see a little bit of what's going on via the SNI header when that is present, but that's about all it can see for encrypted traffic. There is currently no built-in pathway for the IDS/IPS to see unencrypted data even when MITM is in place. Creating such a pathway would require some major reworking of the internal network plumbing of FreeBSD. If I were Netgate, I would not really consider the effort involved in bringing an IDS/IPS into the core system worth it over the long term. To make it really functional, you would also need to rework the network stack AND offer a ready-to-go MITM setup.
The features of pfBlockerNG-devel would be a more natural fit.
I personally believe that most protections are going to have to move more to the endpoints and be used less (and eventually almost not all) on the network perimeter.
-
@bmeeks said in Blog suricata-vs-snort - Snort 3.0?:
There is currently no built-in pathway for the IDS/IPS to see unencrypted data even when MITM is in place. Creating such a pathway would require some major reworking of the internal network plumbing of FreeBSD.
I think we discussed this on a previous thread as well.
The integration with FreeBSD is the issue. If you take the landscape of how other firewalls are doing IPS(Threat Prevention), they break into TLS and then inspect traffic. Those other firewalls have a very customizable OS so thats likely why such an integration is possible.
The function is separated right now within PF or OPN and so the integration perhaps with Squid and Suricata would need to happen within a combined binary perhaps? Please correct me if I'm wrong with my analysis. -
@michmoor said in Blog suricata-vs-snort - Snort 3.0?:
@bmeeks said in Blog suricata-vs-snort - Snort 3.0?:
There is currently no built-in pathway for the IDS/IPS to see unencrypted data even when MITM is in place. Creating such a pathway would require some major reworking of the internal network plumbing of FreeBSD.
I think we discussed this on a previous thread as well.
The integration with FreeBSD is the issue. If you take the landscape of how other firewalls are doing IPS(Threat Prevention), they break into TLS and then inspect traffic. Those other firewalls have a very customizable OS so thats likely why such an integration is possible.
The function is separated right now within PF or OPN and so the integration perhaps with Squid and Suricata would need to happen within a combined binary perhaps? Please correct me if I'm wrong with my analysis.No, it's a bit more complicated than just a simple change to a couple of binaries. It has to do with the physical path packets travel from NIC to kernel and back. This gets into the User Space versus Kernel Space problem, too. The current IDS/IPS packages run in User Space. For Inline IPS Mode, they use the special netmap kernel device. What netmap does is set up two common memory buffers that both the hardware NIC driver, Suricata (or Snort) running in User Space, and the kernel network stack share. The NIC copies incoming data to the common memory area designated for the NIC and then tells the netmap device the data is waiting there. The netmap device tells Suricata that data is present in the shared memory area (in the netmap rings) from the NIC driver. Suricata reads and examines the data and compares it against the rule signatures. For traffic that does not trigger a DROP rule, Suricata copies the packet data from the NIC common data area to the separate common data area it shares with the kernel stack (called the Host Stack in netmap). For traffic that triggers a DROP rule, the packet data is simply not copied to the kernel shared memory area. Therefore we call the packet "dropped" because it was not transferred from the NIC driver memory buffer over to the kernel network stack memory buffer. Netmap, by creating and exposing those two shared memory areas, allows a User Space application to process network packets as fast as possible (almost line rate if the netmap pipe is between two physical interfaces as opposed to one physical interface and the Host Stack).
The problem here is that in FreeBSD all this netmap stuff lives immediately after the NIC driver (when talking about inbound traffic). At this point all of the payload data is still encrypted as it is passed to the kernel. It will stay encrypted until the kernel stack delivers it to the MITM application (typically a proxy server app running also in User Space). The reverse is also true. Data outbound from the system is already encrypted by the time it reaches the last step of the process where netmap is used to copy it from the kernel space buffer over to the NIC driver buffer (via Suricata again where Suricata inspects and does the actual copy of data).
To make this work you will need to rewrite how some of the kernel network stack works so that your IDS/IPS sees inbound data AFTER the MITM has decrypted it, and it sees outbound data BEFORE the MITM has encrypted it. That's a tall order unless you are creating a proprietary MITM and IDS/IPS configuration by radically altering the operating system's network stack and how data packets travel around.
There are other ways perhaps to do this via sockets where apps swap data with each other, but that would be terribly slow. You could also perhaps do some creative routing with several physical interfaces to send traffic to and from a proxy and Suricata, but it would use several physical interfaces and be difficult to set up.
Not saying this is impossible - just that it's a tall order. The few big boy firewall vendors that offer this have a highly customized OS on their box. Creation and maintenance of that is one of the reasons their cost is so astronomical, too.
-
@bmeeks This is extremely insightful here. Thanks for the added detail that im glad i know about now.
So the way i see it, the IPS is only 'useless' on pf from the standpoint of lack of integration with any MITM process. In a perfect world with unlimited resources if there can be some integration then the IPS component would be of more value.
As it stands, its pretty useless (no disrespect to you of course), and this is more in line with what you have been saying for quite some time about its usefulness. -
@michmoor said in Blog suricata-vs-snort - Snort 3.0?:
@bmeeks This is extremely insightful here. Thanks for the added detail that im glad i know about now.
So the way i see it, the IPS is only 'useless' on pf from the standpoint of lack of integration with any MITM process. In a perfect world with unlimited resources if there can be some integration then the IPS component would be of more value.
As it stands, its pretty useless (no disrespect to you of course), and this is more in line with what you have been saying for quite some time about its usefulness.Correct. But it's not just
pf
. The same is true with many other operating system configurations.My point in my other postings about the "usefulness" of IPS is that on the perimeter it is becoming less and less effective due to the encryption. Thus having it on the firewall inspecting traffic flowing between hosts is not really doing a whole lot unless the payload data is cleartext.