iflib_netmap_config causing interface bounce and convergence

Lurick

I've actually noticed this for a while but figured I'd at least ask. I'll get the following messages (4 times and then 4 times a few seconds later) almost exactly 5 minutes after a pfsense reboot:

This will cause the following on the connected device resulting in OSPFv2 and OSPFv3 convergence due to the interface reset:

Mar  9 07:25:04.827 EST: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/48, changed state to down
Mar  9 07:25:05.825 EST: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/48, changed state to down
Mar  9 07:25:05.829 EST: %OSPFv3-5-ADJCHG: Process 1, IPv6, Nbr 192.168.2.1 on TenGigabitEthernet1/0/48 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar  9 07:25:05.832 EST: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.2.1 on TenGigabitEthernet1/0/48 from FULL to DOWN, Neighbor Down: Interface down or detached
Mar  9 07:25:19.281 EST: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/48, changed state to up
Mar  9 07:25:20.282 EST: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/48, changed state to up
Mar  9 07:25:28.148 EST: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.2.1 on TenGigabitEthernet1/0/48 from LOADING to FULL, Loading Done
Mar  9 07:25:28.824 EST: %OSPFv3-5-ADJCHG: Process 1, IPv6, Nbr 192.168.2.1 on TenGigabitEthernet1/0/48 from LOADING to FULL, Loading Done

The only thing I can think of is maybe this has to do with suricata being in Inline mode but I'm not 100% sure.
I've seen this as far back as 22.xx builds but usually it's not an issue since on stable builds I'm not really rebooting my pfsense VM all that often.
Perhaps this is expected but I'd rather make sure than just leave it just in case.

bmeeks

@Lurick:
This is expected behavior in terms of the netmap log messages and Suricata.

The Inline IPS Mode for the Suricata package uses the native netmap device in FreeBSD. That device is initialized when Suricata is started on an interface with Inline IPS Mode operation enabled. FreeBSD responds to the netmap initialization by cycling the chosen interface (in effect, taking it offline and bring it back up again). This is done because during netmap operation the interface is disconnected from the kernel network stack and instead routed through the netmap device.

The "down then back up" behavior shows up in the pfSense logs as an interface "flapping". This can kick off the various "restart packages" activities built into pfSense.

There is no way around this when using Inline IPS Mode. That's one of the reasons Suricata is not recommended for HA setups, especially when using Inline IPS Mode. It's also why I don't recommend running Suricata on the WAN. Run it on the LAN. It is actually much better on the LAN because then NAT will not obfuscate local addresses by making every local host appear to be using the WAN's public IP. Search for a second here on the Netgate forum and you can find a number of posts I've created explaining why running on the LAN is much preferred to running on the LAN.

Lurick

@bmeeks Awesome, thank you for that explanation!
It makes sense for it to cycle the interface like that then, I wanted to make 100% sure this was supposed to happen :)