Suricata inline with VLANs

lcs

Hello,
pfSense HW : CPU Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz, 8024 GB RAM ; Current utilization ~21%
2.6.0-RELEASE (amd64)
Suricata v6.0.4_1

I have the following familiar problem: Suricata is running in inline mode on my parent LAN interface (igb0) only. When it's running, all my VLANs are dead.

I've read a lot of threads back from ~2017 and this is related to netmap stripping VLAN tags when processing the stream.
A assume it's the same issue, because when suricata is running I don't see any incomming traffic on my vlan interfaces and all DHCP requests are comming in my VLAN1-LAN.

I've read "Configuring pfSense/netmap for Suricata Inline IPS mode on em/igb interfaces" and did the tweaks described, I have all kind of offloading disabled and I still experience this issue.

Can someone shed some light on the current status of this issues? Or point me to some reading material if I missed something.

I'm attaching some output, as requested in the thread :

[2.6.0-RELEASE][admin@pfSense]/root: ifconfig igb0
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: LAN
	options=9120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NETMAP>
	ether a0:36:9f:70:17:66
	inet6 fe80::a236:9fff:fe70:1766%igb0 prefixlen 64 scopeid 0x1
	inet 192.168.0.1 netmask 0xffffffc0 broadcast 192.168.0.63
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

[2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep netmap
device	netmap
hw.cxgbe.native_netmap: 2
dev.netmap.iflib_rx_miss_bufs: 6100
dev.netmap.iflib_rx_miss: 4057
dev.netmap.iflib_crcstrip: 1
dev.netmap.bridge_batch: 1024
dev.netmap.default_pipes: 0
dev.netmap.priv_buf_num: 4098
dev.netmap.priv_buf_size: 2048
dev.netmap.buf_curr_num: 163840
dev.netmap.buf_num: 163840
dev.netmap.buf_curr_size: 2048
dev.netmap.buf_size: 2048
dev.netmap.priv_ring_num: 4
dev.netmap.priv_ring_size: 20480
dev.netmap.ring_curr_num: 200
dev.netmap.ring_num: 200
dev.netmap.ring_curr_size: 36864
dev.netmap.ring_size: 36864
dev.netmap.priv_if_num: 2
dev.netmap.priv_if_size: 1024
dev.netmap.if_curr_num: 100
dev.netmap.if_num: 100
dev.netmap.if_curr_size: 1024
dev.netmap.if_size: 1024
dev.netmap.ptnet_vnet_hdr: 1
dev.netmap.generic_rings: 1
dev.netmap.generic_ringsize: 1024
dev.netmap.generic_mit: 100000
dev.netmap.generic_hwcsum: 0
dev.netmap.admode: 0
dev.netmap.fwd: 0
dev.netmap.txsync_retry: 2
dev.netmap.no_pendintr: 1
dev.netmap.no_timestamp: 0
dev.netmap.verbose: 0

[2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep msi
hw.sdhci.enable_msi: 1
hw.puc.msi_disable: 0
hw.pci.honor_msi_blacklist: 1
hw.pci.msix_rewrite_table: 0
hw.pci.enable_msix: 1
hw.pci.enable_msi: 1
hw.mfi.msi: 1
hw.malo.pci.msi_disable: 0
hw.ix.enable_msix: 1
hw.cxgb.msi_allowed: 2
hw.bce.msi_enable: 1
hw.aac.enable_msi: 1
machdep.disable_msix_migration: 0
machdep.num_msi_irqs: 2048
dev.igb.1.iflib.disable_msix: 0
dev.igb.0.iflib.disable_msix: 0
compat.linuxkpi.mlx4_msi_x: 1

[2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep igb
irq264: igb0:rxq0:259 @cpu0(domain0): 3406975
irq265: igb0:rxq1:261 @cpu1(domain0): 449176
irq266: igb0:rxq2:263 @cpu2(domain0): 177558
irq267: igb0:rxq3:265 @cpu3(domain0): 196477
irq268: igb0:aq:267 @cpu0(domain0): 19
irq271: igb1:rxq0:273 @cpu0(domain0): 3562102
irq272: igb1:rxq1:275 @cpu1(domain0): 525689
irq273: igb1:rxq2:277 @cpu2(domain0): 109028
irq274: igb1:rxq3:279 @cpu3(domain0): 182179
irq275: igb1:aq:281 @cpu0(domain0): 2
dev.igb.1.wake: 0
dev.igb.1.interrupts.rx_overrun: 0
dev.igb.1.interrupts.rx_desc_min_thresh: 0
dev.igb.1.interrupts.tx_queue_min_thresh: 3428702
dev.igb.1.interrupts.tx_queue_empty: 1162993
dev.igb.1.interrupts.tx_abs_timer: 0
dev.igb.1.interrupts.tx_pkt_timer: 0
dev.igb.1.interrupts.rx_abs_timer: 0
dev.igb.1.interrupts.rx_pkt_timer: 3428702
dev.igb.1.interrupts.asserts: 4378990
dev.igb.1.mac_stats.tso_ctx_fail: 0
dev.igb.1.mac_stats.tso_txd: 0
dev.igb.1.mac_stats.tx_frames_1024_1522: 64042
dev.igb.1.mac_stats.tx_frames_512_1023: 11526
dev.igb.1.mac_stats.tx_frames_256_511: 10145
dev.igb.1.mac_stats.tx_frames_128_255: 195493
dev.igb.1.mac_stats.tx_frames_65_127: 849639
dev.igb.1.mac_stats.tx_frames_64: 32148
dev.igb.1.mac_stats.mcast_pkts_txd: 1
dev.igb.1.mac_stats.bcast_pkts_txd: 1
dev.igb.1.mac_stats.good_pkts_txd: 1162993
dev.igb.1.mac_stats.total_pkts_txd: 1162993
dev.igb.1.mac_stats.good_octets_txd: 199531351
dev.igb.1.mac_stats.good_octets_recvd: 3824914432
dev.igb.1.mac_stats.rx_frames_1024_1522: 2478913
dev.igb.1.mac_stats.rx_frames_512_1023: 212806
dev.igb.1.mac_stats.rx_frames_256_511: 47827
dev.igb.1.mac_stats.rx_frames_128_255: 297392
dev.igb.1.mac_stats.rx_frames_65_127: 201838
dev.igb.1.mac_stats.rx_frames_64: 189926
dev.igb.1.mac_stats.mcast_pkts_recvd: 273
dev.igb.1.mac_stats.bcast_pkts_recvd: 7386
dev.igb.1.mac_stats.good_pkts_recvd: 3428702
dev.igb.1.mac_stats.total_pkts_recvd: 3454915
dev.igb.1.mac_stats.xoff_txd: 0
dev.igb.1.mac_stats.xoff_recvd: 0
dev.igb.1.mac_stats.xon_txd: 0
dev.igb.1.mac_stats.xon_recvd: 0
dev.igb.1.mac_stats.coll_ext_errs: 0
dev.igb.1.mac_stats.alignment_errs: 0
dev.igb.1.mac_stats.crc_errs: 0
dev.igb.1.mac_stats.recv_errs: 0
dev.igb.1.mac_stats.recv_jabber: 0
dev.igb.1.mac_stats.recv_oversize: 0
dev.igb.1.mac_stats.recv_fragmented: 0
dev.igb.1.mac_stats.recv_undersize: 0
dev.igb.1.mac_stats.recv_no_buff: 0
dev.igb.1.mac_stats.missed_packets: 0
dev.igb.1.mac_stats.defer_count: 0
dev.igb.1.mac_stats.sequence_errors: 0
dev.igb.1.mac_stats.symbol_errors: 0
dev.igb.1.mac_stats.collision_count: 0
dev.igb.1.mac_stats.late_coll: 0
dev.igb.1.mac_stats.multiple_coll: 0
dev.igb.1.mac_stats.single_coll: 0
dev.igb.1.mac_stats.excess_coll: 0
dev.igb.1.queue_rx_3.rx_irq: 0
dev.igb.1.queue_rx_3.rxd_tail: 202
dev.igb.1.queue_rx_3.rxd_head: 203
dev.igb.1.queue_rx_2.rx_irq: 0
dev.igb.1.queue_rx_2.rxd_tail: 299
dev.igb.1.queue_rx_2.rxd_head: 300
dev.igb.1.queue_rx_1.rx_irq: 0
dev.igb.1.queue_rx_1.rxd_tail: 768
dev.igb.1.queue_rx_1.rxd_head: 769
dev.igb.1.queue_rx_0.rx_irq: 0
dev.igb.1.queue_rx_0.rxd_tail: 107
dev.igb.1.queue_rx_0.rxd_head: 108
dev.igb.1.queue_tx_3.tx_irq: 0
dev.igb.1.queue_tx_3.txd_tail: 380
dev.igb.1.queue_tx_3.txd_head: 380
dev.igb.1.queue_tx_2.tx_irq: 0
dev.igb.1.queue_tx_2.txd_tail: 1006
dev.igb.1.queue_tx_2.txd_head: 1006
dev.igb.1.queue_tx_1.tx_irq: 0
dev.igb.1.queue_tx_1.txd_tail: 849
dev.igb.1.queue_tx_1.txd_head: 849
dev.igb.1.queue_tx_0.tx_irq: 0
dev.igb.1.queue_tx_0.txd_tail: 176
dev.igb.1.queue_tx_0.txd_head: 176
dev.igb.1.fc_low_water: 32752
dev.igb.1.fc_high_water: 32768
dev.igb.1.rx_control: 71598082
dev.igb.1.device_control: 1477968449
dev.igb.1.watchdog_timeouts: 0
dev.igb.1.rx_overruns: 0
dev.igb.1.link_irq: 2
dev.igb.1.dropped: 0
dev.igb.1.eee_control: 1
dev.igb.1.itr: 488
dev.igb.1.tx_abs_int_delay: 66
dev.igb.1.rx_abs_int_delay: 66
dev.igb.1.tx_int_delay: 66
dev.igb.1.rx_int_delay: 0
dev.igb.1.rs_dump: 0
dev.igb.1.reg_dump: General Registers
dev.igb.1.fc: 0
dev.igb.1.debug: -1
dev.igb.1.fw_version: EEPROM V3.16-0 eTrack 0x800004ff
dev.igb.1.nvm: -1
dev.igb.1.iflib.rxq3.rxq_fl0.buf_size: 2048
dev.igb.1.iflib.rxq3.rxq_fl0.credits: 1023
dev.igb.1.iflib.rxq3.rxq_fl0.cidx: 203
dev.igb.1.iflib.rxq3.rxq_fl0.pidx: 202
dev.igb.1.iflib.rxq3.cpu: 3
dev.igb.1.iflib.rxq2.rxq_fl0.buf_size: 2048
dev.igb.1.iflib.rxq2.rxq_fl0.credits: 1023
dev.igb.1.iflib.rxq2.rxq_fl0.cidx: 300
dev.igb.1.iflib.rxq2.rxq_fl0.pidx: 299
dev.igb.1.iflib.rxq2.cpu: 2
dev.igb.1.iflib.rxq1.rxq_fl0.buf_size: 2048
dev.igb.1.iflib.rxq1.rxq_fl0.credits: 1023
dev.igb.1.iflib.rxq1.rxq_fl0.cidx: 769
dev.igb.1.iflib.rxq1.rxq_fl0.pidx: 768
dev.igb.1.iflib.rxq1.cpu: 1
dev.igb.1.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.igb.1.iflib.rxq0.rxq_fl0.credits: 1023
dev.igb.1.iflib.rxq0.rxq_fl0.cidx: 108
dev.igb.1.iflib.rxq0.rxq_fl0.pidx: 107
dev.igb.1.iflib.rxq0.cpu: 0
dev.igb.1.iflib.txq3.r_abdications: 0
dev.igb.1.iflib.txq3.r_restarts: 0
dev.igb.1.iflib.txq3.r_stalls: 0
dev.igb.1.iflib.txq3.r_starts: 117438
dev.igb.1.iflib.txq3.r_drops: 0
dev.igb.1.iflib.txq3.r_enqueues: 117438
dev.igb.1.iflib.txq3.ring_state: pidx_head: 0702 pidx_tail: 0702 cidx: 0702 state: IDLE
dev.igb.1.iflib.txq3.txq_cleaned: 234834
dev.igb.1.iflib.txq3.txq_processed: 234874
dev.igb.1.iflib.txq3.txq_in_use: 42
dev.igb.1.iflib.txq3.txq_cidx_processed: 378
dev.igb.1.iflib.txq3.txq_cidx: 338
dev.igb.1.iflib.txq3.txq_pidx: 380
dev.igb.1.iflib.txq3.no_tx_dma_setup: 0
dev.igb.1.iflib.txq3.txd_encap_efbig: 0
dev.igb.1.iflib.txq3.tx_map_failed: 0
dev.igb.1.iflib.txq3.no_desc_avail: 0
dev.igb.1.iflib.txq3.mbuf_defrag_failed: 0
dev.igb.1.iflib.txq3.m_pullups: 0
dev.igb.1.iflib.txq3.mbuf_defrag: 0
dev.igb.1.iflib.txq3.cpu: 3
dev.igb.1.iflib.txq2.r_abdications: 0
dev.igb.1.iflib.txq2.r_restarts: 0
dev.igb.1.iflib.txq2.r_stalls: 0
dev.igb.1.iflib.txq2.r_starts: 67575
dev.igb.1.iflib.txq2.r_drops: 0
dev.igb.1.iflib.txq2.r_enqueues: 67575
dev.igb.1.iflib.txq2.ring_state: pidx_head: 2039 pidx_tail: 2039 cidx: 2039 state: IDLE
dev.igb.1.iflib.txq2.txq_cleaned: 135108
dev.igb.1.iflib.txq2.txq_processed: 135148
dev.igb.1.iflib.txq2.txq_in_use: 42
dev.igb.1.iflib.txq2.txq_cidx_processed: 1004
dev.igb.1.iflib.txq2.txq_cidx: 964
dev.igb.1.iflib.txq2.txq_pidx: 1006
dev.igb.1.iflib.txq2.no_tx_dma_setup: 0
dev.igb.1.iflib.txq2.txd_encap_efbig: 0
dev.igb.1.iflib.txq2.tx_map_failed: 0
dev.igb.1.iflib.txq2.no_desc_avail: 0
dev.igb.1.iflib.txq2.mbuf_defrag_failed: 0
dev.igb.1.iflib.txq2.m_pullups: 0
dev.igb.1.iflib.txq2.mbuf_defrag: 0
dev.igb.1.iflib.txq2.cpu: 2
dev.igb.1.iflib.txq1.r_abdications: 0
dev.igb.1.iflib.txq1.r_restarts: 0
dev.igb.1.iflib.txq1.r_stalls: 0
dev.igb.1.iflib.txq1.r_starts: 297430
dev.igb.1.iflib.txq1.r_drops: 0
dev.igb.1.iflib.txq1.r_enqueues: 297430
dev.igb.1.iflib.txq1.ring_state: pidx_head: 0470 pidx_tail: 0470 cidx: 0470 state: IDLE
dev.igb.1.iflib.txq1.txq_cleaned: 594727
dev.igb.1.iflib.txq1.txq_processed: 594767
dev.igb.1.iflib.txq1.txq_in_use: 42
dev.igb.1.iflib.txq1.txq_cidx_processed: 847
dev.igb.1.iflib.txq1.txq_cidx: 807
dev.igb.1.iflib.txq1.txq_pidx: 849
dev.igb.1.iflib.txq1.no_tx_dma_setup: 0
dev.igb.1.iflib.txq1.txd_encap_efbig: 0
dev.igb.1.iflib.txq1.tx_map_failed: 0
dev.igb.1.iflib.txq1.no_desc_avail: 0
dev.igb.1.iflib.txq1.mbuf_defrag_failed: 0
dev.igb.1.iflib.txq1.m_pullups: 0
dev.igb.1.iflib.txq1.mbuf_defrag: 0
dev.igb.1.iflib.txq1.cpu: 1
dev.igb.1.iflib.txq0.r_abdications: 0
dev.igb.1.iflib.txq0.r_restarts: 0
dev.igb.1.iflib.txq0.r_stalls: 0
dev.igb.1.iflib.txq0.r_starts: 680464
dev.igb.1.iflib.txq0.r_drops: 0
dev.igb.1.iflib.txq0.r_enqueues: 680605
dev.igb.1.iflib.txq0.ring_state: pidx_head: 0669 pidx_tail: 0669 cidx: 0669 state: IDLE
dev.igb.1.iflib.txq0.txq_cleaned: 912519
dev.igb.1.iflib.txq0.txq_processed: 912559
dev.igb.1.iflib.txq0.txq_in_use: 41
dev.igb.1.iflib.txq0.txq_cidx_processed: 175
dev.igb.1.iflib.txq0.txq_cidx: 135
dev.igb.1.iflib.txq0.txq_pidx: 176
dev.igb.1.iflib.txq0.no_tx_dma_setup: 0
dev.igb.1.iflib.txq0.txd_encap_efbig: 0
dev.igb.1.iflib.txq0.tx_map_failed: 0
dev.igb.1.iflib.txq0.no_desc_avail: 0
dev.igb.1.iflib.txq0.mbuf_defrag_failed: 0
dev.igb.1.iflib.txq0.m_pullups: 0
dev.igb.1.iflib.txq0.mbuf_defrag: 0
dev.igb.1.iflib.txq0.cpu: 0
dev.igb.1.iflib.override_nrxds: 0
dev.igb.1.iflib.override_ntxds: 0
dev.igb.1.iflib.use_logical_cores: 0
dev.igb.1.iflib.separate_txrx: 0
dev.igb.1.iflib.core_offset: 0
dev.igb.1.iflib.tx_abdicate: 0
dev.igb.1.iflib.rx_budget: 0
dev.igb.1.iflib.disable_msix: 0
dev.igb.1.iflib.override_qs_enable: 0
dev.igb.1.iflib.override_nrxqs: 0
dev.igb.1.iflib.override_ntxqs: 0
dev.igb.1.iflib.driver_version: 7.6.1-k
dev.igb.1.%parent: pci2
dev.igb.1.%pnpinfo: vendor=0x8086 device=0x1533 subvendor=0x103c subdevice=0x0003 class=0x020000
dev.igb.1.%location: slot=0 function=0 dbsf=pci0:2:0:0 handle=\_SB_.PCI0.RP06.PXSX
dev.igb.1.%driver: igb
dev.igb.1.%desc: Intel(R) I210 (Copper)
dev.igb.0.wake: 0
dev.igb.0.interrupts.rx_overrun: 0
dev.igb.0.interrupts.rx_desc_min_thresh: 0
dev.igb.0.interrupts.tx_queue_min_thresh: 1373925
dev.igb.0.interrupts.tx_queue_empty: 3414605
dev.igb.0.interrupts.tx_abs_timer: 0
dev.igb.0.interrupts.tx_pkt_timer: 0
dev.igb.0.interrupts.rx_abs_timer: 0
dev.igb.0.interrupts.rx_pkt_timer: 1373925
dev.igb.0.interrupts.asserts: 4230086
dev.igb.0.mac_stats.tso_ctx_fail: 0
dev.igb.0.mac_stats.tso_txd: 0
dev.igb.0.mac_stats.tx_frames_1024_1522: 2475823
dev.igb.0.mac_stats.tx_frames_512_1023: 223599
dev.igb.0.mac_stats.tx_frames_256_511: 60037
dev.igb.0.mac_stats.tx_frames_128_255: 288133
dev.igb.0.mac_stats.tx_frames_65_127: 357046
dev.igb.0.mac_stats.tx_frames_64: 9967
dev.igb.0.mac_stats.mcast_pkts_txd: 22
dev.igb.0.mac_stats.bcast_pkts_txd: 96
dev.igb.0.mac_stats.good_pkts_txd: 3414605
dev.igb.0.mac_stats.total_pkts_txd: 3414605
dev.igb.0.mac_stats.good_octets_txd: 3814231622
dev.igb.0.mac_stats.good_octets_recvd: 213341125
dev.igb.0.mac_stats.rx_frames_1024_1522: 64022
dev.igb.0.mac_stats.rx_frames_512_1023: 11854
dev.igb.0.mac_stats.rx_frames_256_511: 12436
dev.igb.0.mac_stats.rx_frames_128_255: 225885
dev.igb.0.mac_stats.rx_frames_65_127: 859524
dev.igb.0.mac_stats.rx_frames_64: 200204
dev.igb.0.mac_stats.mcast_pkts_recvd: 3632
dev.igb.0.mac_stats.bcast_pkts_recvd: 14745
dev.igb.0.mac_stats.good_pkts_recvd: 1373925
dev.igb.0.mac_stats.total_pkts_recvd: 1373933
dev.igb.0.mac_stats.xoff_txd: 0
dev.igb.0.mac_stats.xoff_recvd: 0
dev.igb.0.mac_stats.xon_txd: 0
dev.igb.0.mac_stats.xon_recvd: 0
dev.igb.0.mac_stats.coll_ext_errs: 0
dev.igb.0.mac_stats.alignment_errs: 0
dev.igb.0.mac_stats.crc_errs: 0
dev.igb.0.mac_stats.recv_errs: 0
dev.igb.0.mac_stats.recv_jabber: 0
dev.igb.0.mac_stats.recv_oversize: 0
dev.igb.0.mac_stats.recv_fragmented: 0
dev.igb.0.mac_stats.recv_undersize: 0
dev.igb.0.mac_stats.recv_no_buff: 0
dev.igb.0.mac_stats.missed_packets: 0
dev.igb.0.mac_stats.defer_count: 0
dev.igb.0.mac_stats.sequence_errors: 0
dev.igb.0.mac_stats.symbol_errors: 0
dev.igb.0.mac_stats.collision_count: 0
dev.igb.0.mac_stats.late_coll: 0
dev.igb.0.mac_stats.multiple_coll: 0
dev.igb.0.mac_stats.single_coll: 0
dev.igb.0.mac_stats.excess_coll: 0
dev.igb.0.queue_rx_3.rx_irq: 0
dev.igb.0.queue_rx_3.rxd_tail: 631
dev.igb.0.queue_rx_3.rxd_head: 636
dev.igb.0.queue_rx_2.rx_irq: 0
dev.igb.0.queue_rx_2.rxd_tail: 551
dev.igb.0.queue_rx_2.rxd_head: 552
dev.igb.0.queue_rx_1.rx_irq: 0
dev.igb.0.queue_rx_1.rxd_tail: 970
dev.igb.0.queue_rx_1.rxd_head: 971
dev.igb.0.queue_rx_0.rx_irq: 0
dev.igb.0.queue_rx_0.rxd_tail: 355
dev.igb.0.queue_rx_0.rxd_head: 356
dev.igb.0.queue_tx_3.tx_irq: 0
dev.igb.0.queue_tx_3.txd_tail: 10
dev.igb.0.queue_tx_3.txd_head: 10
dev.igb.0.queue_tx_2.tx_irq: 0
dev.igb.0.queue_tx_2.txd_tail: 14
dev.igb.0.queue_tx_2.txd_head: 14
dev.igb.0.queue_tx_1.tx_irq: 0
dev.igb.0.queue_tx_1.txd_tail: 12
dev.igb.0.queue_tx_1.txd_head: 12
dev.igb.0.queue_tx_0.tx_irq: 0
dev.igb.0.queue_tx_0.txd_tail: 455
dev.igb.0.queue_tx_0.txd_head: 456
dev.igb.0.fc_low_water: 32752
dev.igb.0.fc_high_water: 32768
dev.igb.0.rx_control: 71598106
dev.igb.0.device_control: 1075577409
dev.igb.0.watchdog_timeouts: 0
dev.igb.0.rx_overruns: 0
dev.igb.0.link_irq: 19
dev.igb.0.dropped: 0
dev.igb.0.eee_control: 1
dev.igb.0.itr: 488
dev.igb.0.tx_abs_int_delay: 66
dev.igb.0.rx_abs_int_delay: 66
dev.igb.0.tx_int_delay: 66
dev.igb.0.rx_int_delay: 0
dev.igb.0.rs_dump: 0
dev.igb.0.reg_dump: General Registers
dev.igb.0.fc: 0
dev.igb.0.debug: -1
dev.igb.0.fw_version: EEPROM V3.16-0 eTrack 0x800004ff
dev.igb.0.nvm: -1
dev.igb.0.iflib.rxq3.rxq_fl0.buf_size: 2048
dev.igb.0.iflib.rxq3.rxq_fl0.credits: 0
dev.igb.0.iflib.rxq3.rxq_fl0.cidx: 647
dev.igb.0.iflib.rxq3.rxq_fl0.pidx: 647
dev.igb.0.iflib.rxq3.cpu: 3
dev.igb.0.iflib.rxq2.rxq_fl0.buf_size: 2048
dev.igb.0.iflib.rxq2.rxq_fl0.credits: 0
dev.igb.0.iflib.rxq2.rxq_fl0.cidx: 552
dev.igb.0.iflib.rxq2.rxq_fl0.pidx: 552
dev.igb.0.iflib.rxq2.cpu: 2
dev.igb.0.iflib.rxq1.rxq_fl0.buf_size: 2048
dev.igb.0.iflib.rxq1.rxq_fl0.credits: 0
dev.igb.0.iflib.rxq1.rxq_fl0.cidx: 971
dev.igb.0.iflib.rxq1.rxq_fl0.pidx: 971
dev.igb.0.iflib.rxq1.cpu: 1
dev.igb.0.iflib.rxq0.rxq_fl0.buf_size: 2048
dev.igb.0.iflib.rxq0.rxq_fl0.credits: 0
dev.igb.0.iflib.rxq0.rxq_fl0.cidx: 356
dev.igb.0.iflib.rxq0.rxq_fl0.pidx: 356
dev.igb.0.iflib.rxq0.cpu: 0
dev.igb.0.iflib.txq3.r_abdications: 0
dev.igb.0.iflib.txq3.r_restarts: 0
dev.igb.0.iflib.txq3.r_stalls: 0
dev.igb.0.iflib.txq3.r_starts: 0
dev.igb.0.iflib.txq3.r_drops: 0
dev.igb.0.iflib.txq3.r_enqueues: 0
dev.igb.0.iflib.txq3.ring_state: pidx_head: 1464 pidx_tail: 1464 cidx: 1464 state: IDLE
dev.igb.0.iflib.txq3.txq_cleaned: 0
dev.igb.0.iflib.txq3.txq_processed: 1
dev.igb.0.iflib.txq3.txq_in_use: 0
dev.igb.0.iflib.txq3.txq_cidx_processed: 1
dev.igb.0.iflib.txq3.txq_cidx: 0
dev.igb.0.iflib.txq3.txq_pidx: 0
dev.igb.0.iflib.txq3.no_tx_dma_setup: 0
dev.igb.0.iflib.txq3.txd_encap_efbig: 0
dev.igb.0.iflib.txq3.tx_map_failed: 0
dev.igb.0.iflib.txq3.no_desc_avail: 0
dev.igb.0.iflib.txq3.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq3.m_pullups: 0
dev.igb.0.iflib.txq3.mbuf_defrag: 0
dev.igb.0.iflib.txq3.cpu: 3
dev.igb.0.iflib.txq2.r_abdications: 0
dev.igb.0.iflib.txq2.r_restarts: 0
dev.igb.0.iflib.txq2.r_stalls: 0
dev.igb.0.iflib.txq2.r_starts: 0
dev.igb.0.iflib.txq2.r_drops: 0
dev.igb.0.iflib.txq2.r_enqueues: 0
dev.igb.0.iflib.txq2.ring_state: pidx_head: 1825 pidx_tail: 1825 cidx: 1825 state: IDLE
dev.igb.0.iflib.txq2.txq_cleaned: 0
dev.igb.0.iflib.txq2.txq_processed: 1
dev.igb.0.iflib.txq2.txq_in_use: 0
dev.igb.0.iflib.txq2.txq_cidx_processed: 1
dev.igb.0.iflib.txq2.txq_cidx: 0
dev.igb.0.iflib.txq2.txq_pidx: 0
dev.igb.0.iflib.txq2.no_tx_dma_setup: 0
dev.igb.0.iflib.txq2.txd_encap_efbig: 0
dev.igb.0.iflib.txq2.tx_map_failed: 0
dev.igb.0.iflib.txq2.no_desc_avail: 0
dev.igb.0.iflib.txq2.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq2.m_pullups: 0
dev.igb.0.iflib.txq2.mbuf_defrag: 0
dev.igb.0.iflib.txq2.cpu: 2
dev.igb.0.iflib.txq1.r_abdications: 0
dev.igb.0.iflib.txq1.r_restarts: 0
dev.igb.0.iflib.txq1.r_stalls: 0
dev.igb.0.iflib.txq1.r_starts: 0
dev.igb.0.iflib.txq1.r_drops: 0
dev.igb.0.iflib.txq1.r_enqueues: 0
dev.igb.0.iflib.txq1.ring_state: pidx_head: 0949 pidx_tail: 0949 cidx: 0949 state: IDLE
dev.igb.0.iflib.txq1.txq_cleaned: 0
dev.igb.0.iflib.txq1.txq_processed: 1
dev.igb.0.iflib.txq1.txq_in_use: 0
dev.igb.0.iflib.txq1.txq_cidx_processed: 1
dev.igb.0.iflib.txq1.txq_cidx: 0
dev.igb.0.iflib.txq1.txq_pidx: 0
dev.igb.0.iflib.txq1.no_tx_dma_setup: 0
dev.igb.0.iflib.txq1.txd_encap_efbig: 0
dev.igb.0.iflib.txq1.tx_map_failed: 0
dev.igb.0.iflib.txq1.no_desc_avail: 0
dev.igb.0.iflib.txq1.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq1.m_pullups: 0
dev.igb.0.iflib.txq1.mbuf_defrag: 0
dev.igb.0.iflib.txq1.cpu: 1
dev.igb.0.iflib.txq0.r_abdications: 0
dev.igb.0.iflib.txq0.r_restarts: 0
dev.igb.0.iflib.txq0.r_stalls: 0
dev.igb.0.iflib.txq0.r_starts: 0
dev.igb.0.iflib.txq0.r_drops: 0
dev.igb.0.iflib.txq0.r_enqueues: 0
dev.igb.0.iflib.txq0.ring_state: pidx_head: 0418 pidx_tail: 0418 cidx: 0418 state: IDLE
dev.igb.0.iflib.txq0.txq_cleaned: 0
dev.igb.0.iflib.txq0.txq_processed: 1270273
dev.igb.0.iflib.txq0.txq_in_use: 0
dev.igb.0.iflib.txq0.txq_cidx_processed: 513
dev.igb.0.iflib.txq0.txq_cidx: 0
dev.igb.0.iflib.txq0.txq_pidx: 0
dev.igb.0.iflib.txq0.no_tx_dma_setup: 0
dev.igb.0.iflib.txq0.txd_encap_efbig: 0
dev.igb.0.iflib.txq0.tx_map_failed: 0
dev.igb.0.iflib.txq0.no_desc_avail: 0
dev.igb.0.iflib.txq0.mbuf_defrag_failed: 0
dev.igb.0.iflib.txq0.m_pullups: 0
dev.igb.0.iflib.txq0.mbuf_defrag: 0
dev.igb.0.iflib.txq0.cpu: 0
dev.igb.0.iflib.override_nrxds: 0
dev.igb.0.iflib.override_ntxds: 0
dev.igb.0.iflib.use_logical_cores: 0
dev.igb.0.iflib.separate_txrx: 0
dev.igb.0.iflib.core_offset: 0
dev.igb.0.iflib.tx_abdicate: 0
dev.igb.0.iflib.rx_budget: 0
dev.igb.0.iflib.disable_msix: 0
dev.igb.0.iflib.override_qs_enable: 0
dev.igb.0.iflib.override_nrxqs: 0
dev.igb.0.iflib.override_ntxqs: 0
dev.igb.0.iflib.driver_version: 7.6.1-k
dev.igb.0.%parent: pci1
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x1533 subvendor=0x103c subdevice=0x0003 class=0x020000
dev.igb.0.%location: slot=0 function=0 dbsf=pci0:1:0:0 handle=\_SB_.PCI0.PEG0.PEGP
dev.igb.0.%driver: igb
dev.igb.0.%desc: Intel(R) I210 (Copper)
dev.igb.%parent: 

[2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep rss
device	wlan_rssadapt
hw.bxe.udp_rss: 0
hw.ix.enable_rss: 1
hw.cxgbe.nm_split_rss: 0
compat.linuxkpi.mlx4_udp_rss: 1

cat /var/log/system.log | grep netmap
empty

[2.6.0-RELEASE][admin@pfSense]/root: cat /var/log/system.log | grep sig
Feb 21 18:53:16 pfSense php-fpm[370]: [Suricata] Suricata signalled with SIGHUP for LAN (igb0)...

[2.6.0-RELEASE][admin@pfSense]/root: cat /var/log/suricata/suricata_*/suricata.log | grep -m 1 "signatures processed"
21/2/2022 -- 18:13:25 - <Info> -- 34459 signatures processed. 1214 are IP-only rules, 8885 are inspecting packet payload, 23804 inspect application layer, 107 are decoder event only

bmeeks

The short answer, and you won't like it, is that Suricata and VLANs and Inline IPS Mode (which uses netmap) hate each other. This has, in some ways, actually gotten worse as FreeBSD migrated to the iflib network driver wrapper library. But it really never did work correctly anyway. And it's not just on pfSense. If you cruise over to the forum for the "other Sense" product, you will find tons of Suricata and VLAN issues reported there as well.

If you must use VLANs, you should switch to Legacy Blocking Mode.

lcs

@bmeeks You've guessed that right.
I'm not too happy about the legacy mode. Sometimes when I have a false positive hosts are blocked and the min time is 15 min which is causing some issues.
Maybe I can run suricata on the WAN and use VLANs on LAN.

bmeeks

@lcs said in Suricata inline with VLANs:

@bmeeks You've guessed that right.
I'm not too happy about the legacy mode. Sometimes when I have a false positive hosts are blocked and the min time is 15 min which is causing some issues.
Maybe I can run suricata on the WAN and use VLANs on LAN.

I wish I had happier news. I've done quite a bit of research about netmap, especially over the early summer last year when I worked with the Suricata team on implementing multiple host rings support in Suricata. The way the netmap device is plumbed into the FreeBSD network stack makes working with VLANs natively and transparently not really possible. This is especially true if the hardware NIC driver does hardware VLAN tagging. The tags get copied by the driver into a part of kernel space that netmap does not see.

Add to that the fact FreeBSD moved NIC drivers over to a new wrapper API library called iflib. That required rewriting many drivers. And during the rewrite phase some bugs were introduced, including some regressions. Those are still being worked out. The bugs affected things other than just netmap, though.

Netmap was really designed for a slightly different use case than what is currently being done in Suricata and Snort on pfSense (and on the "other Sense" product, too). On the two firewall distros netmap is used to intercept traffic between the NIC driver and the kernel network stack. That is called host stack mode. That mode is where the VLAN troubles live. The way netmap was originally conceived was to simply route traffic between two physical NIC ports at super high speed bypassing the kernel network stack completely. It would essentially just bridge two NIC ports. But on a typical firewall appliance that is wasteful of valuable NIC ports.