Odd Suricata Inline IPS behavior

[bawoodruff]

pfSense 2.6
Supermicro C2758 8c/8t 2.4ghz, 16gb ram
WAN: Intel 1gbit "igb0"
LAN: Intel X710-DA2 10gbit "ixl0"
No VLANs present

If I enable "Inline Mode" after about 5 minutes, I get the below errors and the interface stops responding compeltely unles I reboot, then it starts all over again.

[NollipfSense]

@bawoodruff You need to increase buffer size System >Advanced > System Tunables

[bawoodruff]

@nollipfsense Do you happen to know the name of the ellement I need to change? there are a couple of items denoted as being related to a buffer.

[NollipfSense]

@bawoodruff See here: https://docs.netgate.com/pfsense/en/latest/hardware/tune.html

You may need to add it.

[bmeeks]

@bawoodruff said in Odd Suricata Inline IPS behavior:

pfSense 2.6
Supermicro C2758 8c/8t 2.4ghz, 16gb ram
WAN: Intel 1gbit "igb0"
LAN: Intel X710-DA2 10gbit "ixl0"
No VLANs present

If I enable "Inline Mode" after about 5 minutes, I get the below errors and the interface stops responding compeltely unles I reboot, then it starts all over again.

The iflib_netmap_config messages are normal. They are just informational in nature describing the number of NIC queues (or netmap rings) the physical NIC driver is offering.

The other log entries are actual errors and appear to be coming from the physical NIC driver itself. A Google search only turned up two results with similar wording, and neither of them actually appeared to apply here. So I'm not able to help with the true errors. Could be helped by a tunable, but also could very well be a problem with your specific NIC's firmware not being handled correctly by the FreeBSD ixl driver code.

I'll ping @stephenw10 by mentioning him here in this reply. Let's see if he has seen this error before. He is a great hardware troubleshooting guy.

[bawoodruff]

Just out of curiousity, I changed the settings within Suricata to use the WAN interface instead (igb) instead of the LAN (ixl) interface and it seems to be stable, but I don't have as much insight as to what was going where.

[NollipfSense]

@bawoodruff That means Bill's insight might have been correct that it was some driver issue.

[bawoodruff]

This may be an issue with the firmware version of the card.

dev.ixl.0.fw_version: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002899 oem 17.4352.12

I wouldn't think that it would be the driver, since it's one that is explicitly documented as one being compatible with Inline mode unless it's some new bug that I haven't been able to find any documentation on... But if the card's firmware is too far out of date, it could cause issues for the driver I suppose.

[bmeeks]

@bawoodruff said in Odd Suricata Inline IPS behavior:

This may be an issue with the firmware version of the card.

dev.ixl.0.fw_version: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002899 oem 17.4352.12

I wouldn't think that it would be the driver, since it's one that is explicitly documented as one being compatible with Inline mode unless it's some new bug that I haven't been able to find any documentation on... But if the card's firmware is too far out of date, it could cause issues for the driver I suppose.

The driver names in FreeBSD can be thought of as somewhat generic. They refer to a class of NICs, but within the class there can be subtle hardware/firmware variations. I suspect that's what is going on in your case, since moving to a different interface with a different physical NIC eliminates the problem. The same Suricata and configuration is running in both cases, so that would eliminate Suricata as the source of the problem.

[brtan]

brtan 3 days ago
@bmeeks , thanks for your detailed instructions. This is what I have done:
a) I have pfsense 2.6
b) snort 4.1.6 on my LAN interface
c) Applied "Inline Policy" to block offenders
d) I used "Connectivity" category
d) My connection to my home devices via Anydesk is blocked as expected

Questions:

I want to disable the rule that blocked/dropped/rejected the connection but I can't find any such rule in the alert logs via Alert Log View Filter. How can I find the rule that's blocking my connection to Anydesk?

If I do find such rule, I believe I should be doing the following steps in SID management
a) Create a dropSID.conf
b) Add in the the pair of GEN_ID:SID in that file
c) Place in under the "Disable SID" List
d) Rebuild the LAN

Thanks !

[bmeeks]

@brtan said in Odd Suricata Inline IPS behavior:

brtan 3 days ago
@bmeeks , thanks for your detailed instructions. This is what I have done:
a) I have pfsense 2.6
b) snort 4.1.6 on my LAN interface
c) Applied "Inline Policy" to block offenders
d) I used "Connectivity" category
d) My connection to my home devices via Anydesk is blocked as expected

Questions:

I want to disable the rule that blocked/dropped/rejected the connection but I can't find any such rule in the alert logs via Alert Log View Filter. How can I find the rule that's blocking my connection to Anydesk?

If I do find such rule, I believe I should be doing the following steps in SID management
a) Create a dropSID.conf
b) Add in the the pair of GEN_ID:SID in that file
c) Place in under the "Disable SID" List
d) Rebuild the LAN

Thanks !

If Snort drops traffic using Inline IPS Mode, the alert that triggered the drop should be showing on the ALERTS tab with that line printed in red text. If you have a very busy network with lots of Snort alerts being triggered, it is possible the alert log was rotated before you had checked for the Anydesk alert. The ALERTS tab only displays alerts from the currently active log -- it does not parse the rotated logs.

Try an Anydesk connection again and then immediately open and view the ALERTS tab. The default sort is by date and time, so any recent alerts will be at the top of the list.

Do you have any other packages installed on the firewall? Other things can block besides Snort. Examples are pfBlockerNG-devel and DNSBL.

You have outlined the correct steps for disabling a rule using SID MGMT features. You can also simply click the appropriate icon on the ALERTS tab to force-disable a rule as well.

[brtan]

@bmeeks said in Odd Suricata Inline IPS behavior:

pfBlockerNG-devel and DNSBL

I can access Anydesk with these 2 plug-ins

I disabled all in individual rules on Snort and start Snort

Anydesk was blocked right away and there was no alert at all