2.3.b.20160324.0516 + Suricata IPS eventually kills internet traffic issue…

gsiemon

You may want to review this post from Bill as he says you need to disable all hardware acceleration on nics running with Suricata IPS.

https://forum.pfsense.org/index.php?topic=108068.msg601891#msg601891

bmeeks

This –

@gsiemon:

You may want to review this post from Bill as he says you need to disable all hardware acceleration on nics running with Suricata IPS.

https://forum.pfsense.org/index.php?topic=108068.msg601891#msg601891

Netmap (used for inline IPS mode) requires both TCP segmentation offloading and hardware checksum calcs be disabled as illustrated in the linked post.

Bill

Tantamount

Thanks for the confirmation.

Were the warnings that already exist for hardware acceleration supposed to give fair warning that dragons be here?

I'm just trying to figure out if I should feature request some type of settings conflict check for the user or not. Does pfsense do any type of sanity check elsewhere for the user?

bmeeks

@Tantamount:

Thanks for the confirmation.

Were the warnings that already exist for hardware acceleration supposed to give fair warning that dragons be here?

I'm just trying to figure out if I should feature request some type of settings conflict check for the user or not. Does pfsense do any type of sanity check elsewhere for the user?

No, the package does not currently check those settings. But it is a valid suggestion that it should. I will add that in a coming update. Working on a few other things, too, so give me a little while to get everything done. A small update to the package was posted today, but that only fixes a Barnyard2 run dependency that was missing and causing the Barnyard2 binary to not get installed along with Suricata.

Bill

Tantamount

I added Suricata to my LAN interface, and am running into the same problem as above:

It works for about half a minute, and then all communication on my LAN interface goes dead.

/var/log/system.log is mostly silent when this happens:

Apr 1 16:03:17 pfSense kernel: em0: link state changed to DOWN
Apr 1 16:03:17 pfSense check_reload_status: Linkup starting em0

The hardware acceleration settings under System->Advanced->Networking are supposed to be global, correct? All the hardware choices are disabled.

The nic hardware is fairly recently supported I think? Perhaps there are still some bugs to work out?

em0@pci0:0:31:6: class=0x020000 card=0xe0001458 chip=0x15b88086 rev=0x31 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection (2) I219-V'
class = network
subclass = ethernet

Oddly I had enabled Suricata on the LAN a number of days back, but forgot to enable the ET Pro classifications. Once I did that, about a few hours later I started running into this trouble. Hopefully I just need to allocate more memory somewhere?

Edit:

Am running the latest version of 2.3 beta:

2.3-RC (amd64)
built on Thu Mar 31 23:48:37 CDT 2016

Tantamount

Apparently Suricata 3.0.1 was released today. There's mention of lots of stability and memory fixes… perhaps this will help my situation?

http://suricata-ids.org/news/

bmeeks

@Tantamount:

I added Suricata to my LAN interface, and am running into the same problem as above:

It works for about half a minute, and then all communication on my LAN interface goes dead.

/var/log/system.log is mostly silent when this happens:

Apr 1 16:03:17 pfSense kernel: em0: link state changed to DOWN
Apr 1 16:03:17 pfSense check_reload_status: Linkup starting em0

The hardware acceleration settings under System->Advanced->Networking are supposed to be global, correct? All the hardware choices are disabled.

The nic hardware is fairly recently supported I think? Perhaps there are still some bugs to work out?

em0@pci0:0:31:6: class=0x020000 card=0xe0001458 chip=0x15b88086 rev=0x31 hdr=0x00
vendor = 'Intel Corporation'
device = 'Ethernet Connection (2) I219-V'
class = network
subclass = ethernet

Oddly I had enabled Suricata on the LAN a number of days back, but forgot to enable the ET Pro classifications. Once I did that, about a few hours later I started running into this trouble. Hopefully I just need to allocate more memory somewhere?

Edit:

Am running the latest version of 2.3 beta:

2.3-RC (amd64)
built on Thu Mar 31 23:48:37 CDT 2016

Were you running Suricata on just the LAN interface or both LAN and WAN at the same time?

Bill

Tantamount

Both.

bmeeks

@Tantamount:

Both.

Thanks for the info. Just looking for some baseline data to start with troubleshooting/reproducing the problem. I admit to not having run Suricata for an extended period with inline IPS mode enabled.

Bill

Tantamount

Well, I'm not certain if it was due to prolonged use, or the more recent enabling of the et pro rules.

Also, bad things happen even when I disable the "block" setting (Which I assume disables the inline stuff?).

I've just unchecked all of the LAN categories and attempted to re-enable, but the same problem happened.

If I use the console and kill the suricata process attached to the lan nic, the lan traffic immediately starts to work again.