Suricata on pfSense 3 starts and kills the WAN
-
Hi Bill,
Makes sense. Anyways, I am going back to Snort which has been working well for years now.
I just wanted to see where things were evolving… ;-)Again, thanks for your help and reactivity!
Cheers,
Eric
-
Snort does not currently use Netmap, and the fact it works but Suricata does not with inline mode (which does use Netmap) seems to me at least to point the finger at Netmap itself.
I'm sure this will get fixed. Netmap has great promise for providing really high performance operations for packet inspection and forwarding. The pfSense team will likely be working with the FreeBSD folks to get this figured out and fixed.
Bill
-
That is fantastic news. I would certainly benefit from highspeed as right now I am on 1Gbps FFTH and snort caps me to 350Mbps… ;-)
i'll definitely keep an eye on it.
Keep up the great work!
-
Hello,
I read your discussion about wan kill connection problem / Suricata inline mode. I have the same problem and cannot reach solution.
Have you found any solution? I would like to use inline mode (legacy mode working OK). Propably is problem in netmap HW. Same configuraction, but in vmware host is working well.Thank you
Martin Pouzar
-
Snort does not currently use Netmap, and the fact it works but Suricata does not with inline mode (which does use Netmap) seems to me at least to point the finger at Netmap itself.
I'm sure this will get fixed. Netmap has great promise for providing really high performance operations for packet inspection and forwarding. The pfSense team will likely be working with the FreeBSD folks to get this figured out and fixed.
Bill
Any progress on the netmap update? How would we know of a resolution other than the system pulling an updated file? TIA!
-
Snort does not currently use Netmap, and the fact it works but Suricata does not with inline mode (which does use Netmap) seems to me at least to point the finger at Netmap itself.
I'm sure this will get fixed. Netmap has great promise for providing really high performance operations for packet inspection and forwarding. The pfSense team will likely be working with the FreeBSD folks to get this figured out and fixed.
Bill
Any progress on the netmap update? How would we know of a resolution other than the system pulling an updated file? TIA!
You will need to follow the bug reports on Redmine. I am not a kernel developer and thus am not working on the Netmap problem. That will be up to the pfSense developers and/or the FreeBSD folks. From what I can tell looking at the various issues mentioned here on the forums, there are issues with Netmap and with IPSEC traffic on some NIC drivers.
I suspect for sure nothing would be fixed with Netmpa until at least the 2.3.1 version of pfSense goes to release.
Bill
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
-
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
No, 2.3_1 (2.3.0_1 really, we'll be more clear on the assumed 0 in the future). The only change is upgrading NTP, it won't have any impact on anything to do with the kernel.
From what I can tell looking at the various issues mentioned here on the forums, there are issues with Netmap and with IPSEC traffic on some NIC drivers.
I suspect for sure nothing would be fixed with Netmpa until at least the 2.3.1 version of pfSense goes to release.
That was just a working theory, that problem ended up having no relation to netmap. Though the issues that exist with it and inline IPS need to be narrowed down and have their own bug(s) opened. That's low on my priority list at the moment.
Bill if you have any specifics on what the issues with it are, no need to identify a root cause, go ahead and open a bug ticket.
-
@cmb:
Bill if you have any specifics on what the issues with it are, no need to identify a root cause, go ahead and open a bug ticket.
There is a bug already open on the incompatibility with Netmap (in Suricata) and the limiter in the traffic shaper. Apparently traffic shaping does not work with inline IPS mode enabled in Suricata. The only change on the Suricata side would be the loading of the Netmap device driver when using inline IPS mode.
I don't know what might be causing the original issue this thread was about, but just made a guess that it might be related to the other interface hanging/freezing problems that appeared to be NIC-specific.
Bill
-
@cmb:
Bill if you have any specifics on what the issues with it are, no need to identify a root cause, go ahead and open a bug ticket.
There is a bug already open on the incompatibility with Netmap (in Suricata) and the limiter in the traffic shaper. Apparently traffic shaping does not work with inline IPS mode enabled in Suricata. The only change on the Suricata side would be the loading of the Netmap device driver when using inline IPS mode.
I don't know what might be causing the original issue this thread was about, but just made a guess that it might be related to the other interface hanging/freezing problems that appeared to be NIC-specific.
Bill
Hi Bill,
FYI I don't have traffic shaping enabled here…
-
Maybe it should be split into two issue.
-
Maybe it should be split into two issue.
Probably so. There was already a thread someplace about the shaper/limiter and Netmap.
Suricata itself really does not do things much differently for inline mode other than enable the Netmap driver. There are issues with the driver and certain NICs (there are unsupported NICs, for example). I have also seen a bug report or two posted on the Suricata Redmine site and on the Netmap Git repository related to Netmap operation in Suricata.
Nevertheless, inline mode appears to be working for some folks. Anecdotal evidence based on the posts here would indicate it is working for more folks than it is not working on. At least that is my reading of the limited data.
The original poster indicated it failed on one piece of physical hardware, but the same configuration was working in a VM. That would indicate to me a NIC driver problem with Netmap. As was mentioned when I first posted the new inline version, some NIC hardware just plain won't work with Netmap for now.
Bill
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
I've been out for a week traveling - death in the family.
The 2.3.1 release didn't seem to fix things. As noted in the latest postings.
Bill, is there a post listing which cards/drivers do work? I'm currently running through an em quad card, but am willing to pick up another to check it out.
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
I've been out for a week traveling - death in the family.
The 2.3.1 release didn't seem to fix things. As noted in the latest postings.
Bill, is there a post listing which cards/drivers do work? I'm currently running through an em quad card, but am willing to pick up another to check it out.
I don't have one handy, but someone posted a link in the old 2.3-BETA forum several months ago. Have a search through that archive and see if you can find it. Just go to that archived sub-forum and search for "Suricata". The post should turn up.
Bill
-
OK, thanks Bill…
I found this thread: https://forum.pfsense.org/index.php?topic=107847.msg600868#msg600868
containing this information:
"netmap natively supports the following devices:
On FreeBSD: em(4), igb(4), ixgbe(4), lem(4), re(4).
On Linux e1000(4), e1000e(4), igb(4), ixgbe(4), mlx4(4), forcedeth(4),
r8169(4).NICs without native support can still be used in netmap mode through emu-
lation. Performance is inferior to native netmap mode but still signifi-
cantly higher than sockets, and approaching that of in-kernel solutions
such as Linux's pktgen.Emulation is also available for devices with native netmap support, which
can be used for testing or performance comparison. The sysctl variable
dev.netmap.admode globally controls how netmap mode is implemented."Source: https://www.freebsd.org/cgi/man.cgi?query=netmap&apropos=0&sektion=4&manpath=FreeBSD+10.2-RELEASE&arch=default&format=html#SUPPORTED_DEVICES
I have been using the "em" quad NIC, which should have functioned properly per above. I have a "re" dual NIC that I've installed and will be setting up WAN on it to test.
-
Same exact driver and nic here…
OK, thanks Bill…
I found this thread: https://forum.pfsense.org/index.php?topic=107847.msg600868#msg600868
containing this information:
"netmap natively supports the following devices:
On FreeBSD: em(4), igb(4), ixgbe(4), lem(4), re(4).
On Linux e1000(4), e1000e(4), igb(4), ixgbe(4), mlx4(4), forcedeth(4),
r8169(4).NICs without native support can still be used in netmap mode through emu-
lation. Performance is inferior to native netmap mode but still signifi-
cantly higher than sockets, and approaching that of in-kernel solutions
such as Linux's pktgen.Emulation is also available for devices with native netmap support, which
can be used for testing or performance comparison. The sysctl variable
dev.netmap.admode globally controls how netmap mode is implemented."Source: https://www.freebsd.org/cgi/man.cgi?query=netmap&apropos=0&sektion=4&manpath=FreeBSD+10.2-RELEASE&arch=default&format=html#SUPPORTED_DEVICES
I have been using the "em" quad NIC, which should have functioned properly per above. I have a "re" dual NIC that I've installed and will be setting up WAN on it to test.
-
OK, thanks Bill…
I found this thread: https://forum.pfsense.org/index.php?topic=107847.msg600868#msg600868
containing this information:
"netmap natively supports the following devices:
On FreeBSD: em(4), igb(4), ixgbe(4), lem(4), re(4).
On Linux e1000(4), e1000e(4), igb(4), ixgbe(4), mlx4(4), forcedeth(4),
r8169(4).I have been using the "em" quad NIC, which should have functioned properly per above. I have a "re" dual NIC that I've installed and will be setting up WAN on it to test.
Using the "re" NIC hasn't solved things for me. Still looking for the netmap solution to drop.
-
I have a quad igb Inteface and Suricata inline does not work for me. locks up the interface on the first rule match. The only rules I use are custom rules and I only have a handful of those. Using pfsense 2.3.2 I have to restart the pfsense box when I get this lock up.
Using snort now, and suricata works in legacy mode.
One thing I do notice after turning on the netmap feature is netmap errors on the console. Not a lot of them, but pops up a new one every few minutes with a bad pkt. Is this normal, or is this an indication of a NIC mismatch with netmap?
I really need this inline feature, so I hope these issues get resolved soon.
-
Is it possible that the inline feature is blocking the src and dst. This would kill the WAN for sure. I would assume that the inline and legacy would treat the rules in the same manor. I do have the WAN and local IP's in the pass list.
When this issue occurs in inline mode. I can no longer access the GUI, but the console still works.
What can I run in the console to test the interfaces when this occurs?