Snort Inline Mode caused WAN to drop every few minutes

[promo76]

After following the Netgate Guide(https://forum.netgate.com/topic/143812/snort-package-4-0-inline-ips-mode-introduction-and-configuration-instructions/43) to configure Inline Mode for Snort it caused major issues with my WAN connection and Internal VLANs. I switched back to Legacy Mode for the internal LAN and even then the WAN would drop the connection every few minutes. I am just wondering if I missed something. I could not get it to work until I switched all my interfaces back to Legacy Mode.
I am running PFSense 2.4.5 and SNORT 4.1.2_3. The NICs on the appliance are from the igb family.

Any help is appreciated! Thank you!

[bmeeks]

I am using Snort with Inline IPS Mode enabled on a Netgate SG-5100 appliance without issue. The NICs on my WAN and LAN are both igb chipsets.

Inline IPS Mode uses the FreeBSD kernel netmap device. There are some quirks with that device. One is that "attaching" and "detaching" from it via a software application triggers the netmap device and kernel to perform a "down then up" physical cycle of the interface. So the same basic thing as doing an "ifconfig down" and "ifconfig up" sequence.

The following things might make Snort restart on an interface and thus trigger the down/up sequence:

1. Scheduled rules updates when new rules are actually available.
2. Receipt of a "restart all packages" command from pfSense itself. The firewall may issue this command in response to several things.

Improper settings for certain hardware tunables can cause problems with netmap operation.

FreeBSD-11.3/STABLE (which pfSense-2.4.5 is based on) uses an older API version for the netmap device interface. There are perhaps new netmap bug fixes from upstream that have not been backported to FreeBSD-11.3/STABLE.

If Inline IPS Mode is unstable for you on your hardware, switch to Legacy Blocking Mode. That does not use the netmap device. A reboot of the box after switching would not be a bad idea either if you had substantial issues with Inline IPS Mode.

To see if something else is really at fault, disable the IDS/IPS completely for a period to see if the interfaces become stable then. Perhaps something else is causing the interface cycling??

[promo76]

@bmeeks
Confession Time! I managed to get NTOPNG 4.2 installed and was working. When I enabled the option to create VLAN Timeseries it broke my config. I had to unistall NTOPNG and then switch back to legacy mode on all interfaces. I just enabled Inline Mode on the WAN again. I will see how it goes.
Do you think NTOPNG might have broken the Inline Mode config for SNORT?

[bmeeks]

@promo76 said in Snort Inline Mode caused WAN to drop every few minutes:

@bmeeks
Confession Time! I managed to get NTOPNG 4.2 installed and was working. When I enabled the option to create VALN Timeseries it broke my config. I had to unistall NTOPNG and then switch back to legacy mode on all interfaces. I just enabled Inline Mode on the WAN again. I will see how it goes.
Do you think NTOPNG might have broken the Inline Mode config for SNORT?

Yes, they do not like each other. Inline IPS Mode, because of the kernel netmap device, is incompatible with many things. Limiters, sometimes Traffic Graph will malfunction, and ntopNG. There are probably others. You need a plain-vanilla firewall in terms of extra packages to use Inline IPS Mode effectively.

[promo76]

@bmeeks
Thank you!

[tsmalmbe]

I am facing some similar issues on my Watchguards. I turned every interface (some native and some VLAN's) INLINE last night, and today I had issues connecting to here and there and nothing made any sense at all. I was able to ping a server in another network, but Samba was not working. The only change was going INLINE at this point. Turning them back to LEGACY did not work until I rebooted the whole damn thing (just like Brian here said). After making the interfaces LEGACY and rebooting, things are normal again. Maybe turning into INLINE and then rebooting would work?

Would bandwidthd, haproxy, softflowd or status_traffic_totals play a role in this?

[tsmalmbe]

@bmeeks A minute to review this question of mine and comment? Highly appreciated as always.

[bmeeks]

@tsmalmbe said in Snort Inline Mode caused WAN to drop every few minutes:

@bmeeks A minute to review this question of mine and comment? Highly appreciated as always.

The answer is very simple: switch to Legacy Mode if you want to use blocking with Snort (or Suricata) on your hardware.

I've said on this board innumerable times that Inline IPS Mode relies on the kernel netmap device, and the kernel netmap device relies on well-written support within the hardware NIC driver. If that support is not well-written (meaning bug free), then netmap does not work reliably. That in turn means Inline IPS Mode does not work reliably. "Not working reliably" can manifest in ways from simple disruption of traffic on the configured interface to potentially a complete lockup of the firewall. There is a warning dialog that is displayed at the top of the INTERFACE SETTINGS page when you switch an interface to Inline IPS Mode and save the change. The message clearly says you may experience difficulties.

You also have two installed packages that are likely not going to cooperate with the netmap kernel device: bandwidthd and softflowd. That's because when an interface is placed into netmap operation, it is disconnected from the kernel's control and placed under the the control of the app initiating the netmap connection. For the IPS/IDS packages, that would be Snort or Suricata.