Suricata on pfSense 3 starts and kills the WAN
-
I thought you had copied some messages out of the system log from the GUI. There is a setting for the GUI display of the system log to sort events either most recent at the top or oldest at the top. That's the setting I was referring to.
I suspect Netmap might be the culprit here. If you want to test some more, do this –
1. Install Suricata. Go to the INTERFACE SETTINGS tab for the WAN in Suricata and change the blocking mode to Legacy Mode and save the change. Now immediately go back and disable blocking for the WAN. Save that change. Now start/restart Suricata and see what happens.
Maybe you have already done this, but the sequence of steps above should result in the Netmap module not loading in Suricata.
Bill
-
You are right, doing it this way it works.
An,ything I can do to make it work inline ?
-
Tried re-enabling inline mode. No go.
I tend to have the feeling that suricata is just blocking WAN DHCP request…
I don't see any kernel errors anymore...
-
Tried re-enabling inline mode. No go.
I tend to have the feeling that suricata is just blocking WAN DHCP request…
I don't see any kernel errors anymore...
I think there are some issues with Netmap (the new technology used for inline mode). Some other users are having issues with certain NICs even with Suricata not installed. One user also uncovered some bug reports to the Netmap developer on Github, so I'm thinking Netmap itself is the issue. When it malfunctions, because of where it lies in the packet chain, it can kill connectivity dead. pfSense 2.3 has Netmap compiled in by default now (this is new, the 2.2.x line did not use Netmap). So we may be seeing some growing pains from the new technology.
You may be stuck with using Legacy Mode blocking for the near term while any potential Netmap bugs are sorted out.
Bill
-
Hi Bill,
Makes sense. Anyways, I am going back to Snort which has been working well for years now.
I just wanted to see where things were evolving… ;-)Again, thanks for your help and reactivity!
Cheers,
Eric
-
Snort does not currently use Netmap, and the fact it works but Suricata does not with inline mode (which does use Netmap) seems to me at least to point the finger at Netmap itself.
I'm sure this will get fixed. Netmap has great promise for providing really high performance operations for packet inspection and forwarding. The pfSense team will likely be working with the FreeBSD folks to get this figured out and fixed.
Bill
-
That is fantastic news. I would certainly benefit from highspeed as right now I am on 1Gbps FFTH and snort caps me to 350Mbps… ;-)
i'll definitely keep an eye on it.
Keep up the great work!
-
Hello,
I read your discussion about wan kill connection problem / Suricata inline mode. I have the same problem and cannot reach solution.
Have you found any solution? I would like to use inline mode (legacy mode working OK). Propably is problem in netmap HW. Same configuraction, but in vmware host is working well.Thank you
Martin Pouzar
-
Snort does not currently use Netmap, and the fact it works but Suricata does not with inline mode (which does use Netmap) seems to me at least to point the finger at Netmap itself.
I'm sure this will get fixed. Netmap has great promise for providing really high performance operations for packet inspection and forwarding. The pfSense team will likely be working with the FreeBSD folks to get this figured out and fixed.
Bill
Any progress on the netmap update? How would we know of a resolution other than the system pulling an updated file? TIA!
-
Snort does not currently use Netmap, and the fact it works but Suricata does not with inline mode (which does use Netmap) seems to me at least to point the finger at Netmap itself.
I'm sure this will get fixed. Netmap has great promise for providing really high performance operations for packet inspection and forwarding. The pfSense team will likely be working with the FreeBSD folks to get this figured out and fixed.
Bill
Any progress on the netmap update? How would we know of a resolution other than the system pulling an updated file? TIA!
You will need to follow the bug reports on Redmine. I am not a kernel developer and thus am not working on the Netmap problem. That will be up to the pfSense developers and/or the FreeBSD folks. From what I can tell looking at the various issues mentioned here on the forums, there are issues with Netmap and with IPSEC traffic on some NIC drivers.
I suspect for sure nothing would be fixed with Netmpa until at least the 2.3.1 version of pfSense goes to release.
Bill
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
-
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
No, 2.3_1 (2.3.0_1 really, we'll be more clear on the assumed 0 in the future). The only change is upgrading NTP, it won't have any impact on anything to do with the kernel.
From what I can tell looking at the various issues mentioned here on the forums, there are issues with Netmap and with IPSEC traffic on some NIC drivers.
I suspect for sure nothing would be fixed with Netmpa until at least the 2.3.1 version of pfSense goes to release.
That was just a working theory, that problem ended up having no relation to netmap. Though the issues that exist with it and inline IPS need to be narrowed down and have their own bug(s) opened. That's low on my priority list at the moment.
Bill if you have any specifics on what the issues with it are, no need to identify a root cause, go ahead and open a bug ticket.
-
@cmb:
Bill if you have any specifics on what the issues with it are, no need to identify a root cause, go ahead and open a bug ticket.
There is a bug already open on the incompatibility with Netmap (in Suricata) and the limiter in the traffic shaper. Apparently traffic shaping does not work with inline IPS mode enabled in Suricata. The only change on the Suricata side would be the loading of the Netmap device driver when using inline IPS mode.
I don't know what might be causing the original issue this thread was about, but just made a guess that it might be related to the other interface hanging/freezing problems that appeared to be NIC-specific.
Bill
-
@cmb:
Bill if you have any specifics on what the issues with it are, no need to identify a root cause, go ahead and open a bug ticket.
There is a bug already open on the incompatibility with Netmap (in Suricata) and the limiter in the traffic shaper. Apparently traffic shaping does not work with inline IPS mode enabled in Suricata. The only change on the Suricata side would be the loading of the Netmap device driver when using inline IPS mode.
I don't know what might be causing the original issue this thread was about, but just made a guess that it might be related to the other interface hanging/freezing problems that appeared to be NIC-specific.
Bill
Hi Bill,
FYI I don't have traffic shaping enabled here…
-
Maybe it should be split into two issue.
-
Maybe it should be split into two issue.
Probably so. There was already a thread someplace about the shaper/limiter and Netmap.
Suricata itself really does not do things much differently for inline mode other than enable the Netmap driver. There are issues with the driver and certain NICs (there are unsupported NICs, for example). I have also seen a bug report or two posted on the Suricata Redmine site and on the Netmap Git repository related to Netmap operation in Suricata.
Nevertheless, inline mode appears to be working for some folks. Anecdotal evidence based on the posts here would indicate it is working for more folks than it is not working on. At least that is my reading of the limited data.
The original poster indicated it failed on one piece of physical hardware, but the same configuration was working in a VM. That would indicate to me a NIC driver problem with Netmap. As was mentioned when I first posted the new inline version, some NIC hardware just plain won't work with Netmap for now.
Bill
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
I've been out for a week traveling - death in the family.
The 2.3.1 release didn't seem to fix things. As noted in the latest postings.
Bill, is there a post listing which cards/drivers do work? I'm currently running through an em quad card, but am willing to pick up another to check it out.
-
2.3.1 Release out today…
I've loaded it up to see if it corrects things.
I've been out for a week traveling - death in the family.
The 2.3.1 release didn't seem to fix things. As noted in the latest postings.
Bill, is there a post listing which cards/drivers do work? I'm currently running through an em quad card, but am willing to pick up another to check it out.
I don't have one handy, but someone posted a link in the old 2.3-BETA forum several months ago. Have a search through that archive and see if you can find it. Just go to that archived sub-forum and search for "Suricata". The post should turn up.
Bill
-
OK, thanks Bill…
I found this thread: https://forum.pfsense.org/index.php?topic=107847.msg600868#msg600868
containing this information:
"netmap natively supports the following devices:
On FreeBSD: em(4), igb(4), ixgbe(4), lem(4), re(4).
On Linux e1000(4), e1000e(4), igb(4), ixgbe(4), mlx4(4), forcedeth(4),
r8169(4).NICs without native support can still be used in netmap mode through emu-
lation. Performance is inferior to native netmap mode but still signifi-
cantly higher than sockets, and approaching that of in-kernel solutions
such as Linux's pktgen.Emulation is also available for devices with native netmap support, which
can be used for testing or performance comparison. The sysctl variable
dev.netmap.admode globally controls how netmap mode is implemented."Source: https://www.freebsd.org/cgi/man.cgi?query=netmap&apropos=0&sektion=4&manpath=FreeBSD+10.2-RELEASE&arch=default&format=html#SUPPORTED_DEVICES
I have been using the "em" quad NIC, which should have functioned properly per above. I have a "re" dual NIC that I've installed and will be setting up WAN on it to test.
