Suricata inline with VLANs
-
Hello,
pfSense HW : CPU Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz, 8024 GB RAM ; Current utilization ~21%
2.6.0-RELEASE (amd64)
Suricata v6.0.4_1I have the following familiar problem: Suricata is running in inline mode on my parent LAN interface (igb0) only. When it's running, all my VLANs are dead.
I've read a lot of threads back from ~2017 and this is related to netmap stripping VLAN tags when processing the stream.
A assume it's the same issue, because when suricata is running I don't see any incomming traffic on my vlan interfaces and all DHCP requests are comming in my VLAN1-LAN.I've read "Configuring pfSense/netmap for Suricata Inline IPS mode on em/igb interfaces" and did the tweaks described, I have all kind of offloading disabled and I still experience this issue.
Can someone shed some light on the current status of this issues? Or point me to some reading material if I missed something.
I'm attaching some output, as requested in the thread :
[2.6.0-RELEASE][admin@pfSense]/root: ifconfig igb0 igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: LAN options=9120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NETMAP> ether a0:36:9f:70:17:66 inet6 fe80::a236:9fff:fe70:1766%igb0 prefixlen 64 scopeid 0x1 inet 192.168.0.1 netmask 0xffffffc0 broadcast 192.168.0.63 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> [2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep netmap device netmap hw.cxgbe.native_netmap: 2 dev.netmap.iflib_rx_miss_bufs: 6100 dev.netmap.iflib_rx_miss: 4057 dev.netmap.iflib_crcstrip: 1 dev.netmap.bridge_batch: 1024 dev.netmap.default_pipes: 0 dev.netmap.priv_buf_num: 4098 dev.netmap.priv_buf_size: 2048 dev.netmap.buf_curr_num: 163840 dev.netmap.buf_num: 163840 dev.netmap.buf_curr_size: 2048 dev.netmap.buf_size: 2048 dev.netmap.priv_ring_num: 4 dev.netmap.priv_ring_size: 20480 dev.netmap.ring_curr_num: 200 dev.netmap.ring_num: 200 dev.netmap.ring_curr_size: 36864 dev.netmap.ring_size: 36864 dev.netmap.priv_if_num: 2 dev.netmap.priv_if_size: 1024 dev.netmap.if_curr_num: 100 dev.netmap.if_num: 100 dev.netmap.if_curr_size: 1024 dev.netmap.if_size: 1024 dev.netmap.ptnet_vnet_hdr: 1 dev.netmap.generic_rings: 1 dev.netmap.generic_ringsize: 1024 dev.netmap.generic_mit: 100000 dev.netmap.generic_hwcsum: 0 dev.netmap.admode: 0 dev.netmap.fwd: 0 dev.netmap.txsync_retry: 2 dev.netmap.no_pendintr: 1 dev.netmap.no_timestamp: 0 dev.netmap.verbose: 0 [2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep msi hw.sdhci.enable_msi: 1 hw.puc.msi_disable: 0 hw.pci.honor_msi_blacklist: 1 hw.pci.msix_rewrite_table: 0 hw.pci.enable_msix: 1 hw.pci.enable_msi: 1 hw.mfi.msi: 1 hw.malo.pci.msi_disable: 0 hw.ix.enable_msix: 1 hw.cxgb.msi_allowed: 2 hw.bce.msi_enable: 1 hw.aac.enable_msi: 1 machdep.disable_msix_migration: 0 machdep.num_msi_irqs: 2048 dev.igb.1.iflib.disable_msix: 0 dev.igb.0.iflib.disable_msix: 0 compat.linuxkpi.mlx4_msi_x: 1 [2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep igb irq264: igb0:rxq0:259 @cpu0(domain0): 3406975 irq265: igb0:rxq1:261 @cpu1(domain0): 449176 irq266: igb0:rxq2:263 @cpu2(domain0): 177558 irq267: igb0:rxq3:265 @cpu3(domain0): 196477 irq268: igb0:aq:267 @cpu0(domain0): 19 irq271: igb1:rxq0:273 @cpu0(domain0): 3562102 irq272: igb1:rxq1:275 @cpu1(domain0): 525689 irq273: igb1:rxq2:277 @cpu2(domain0): 109028 irq274: igb1:rxq3:279 @cpu3(domain0): 182179 irq275: igb1:aq:281 @cpu0(domain0): 2 dev.igb.1.wake: 0 dev.igb.1.interrupts.rx_overrun: 0 dev.igb.1.interrupts.rx_desc_min_thresh: 0 dev.igb.1.interrupts.tx_queue_min_thresh: 3428702 dev.igb.1.interrupts.tx_queue_empty: 1162993 dev.igb.1.interrupts.tx_abs_timer: 0 dev.igb.1.interrupts.tx_pkt_timer: 0 dev.igb.1.interrupts.rx_abs_timer: 0 dev.igb.1.interrupts.rx_pkt_timer: 3428702 dev.igb.1.interrupts.asserts: 4378990 dev.igb.1.mac_stats.tso_ctx_fail: 0 dev.igb.1.mac_stats.tso_txd: 0 dev.igb.1.mac_stats.tx_frames_1024_1522: 64042 dev.igb.1.mac_stats.tx_frames_512_1023: 11526 dev.igb.1.mac_stats.tx_frames_256_511: 10145 dev.igb.1.mac_stats.tx_frames_128_255: 195493 dev.igb.1.mac_stats.tx_frames_65_127: 849639 dev.igb.1.mac_stats.tx_frames_64: 32148 dev.igb.1.mac_stats.mcast_pkts_txd: 1 dev.igb.1.mac_stats.bcast_pkts_txd: 1 dev.igb.1.mac_stats.good_pkts_txd: 1162993 dev.igb.1.mac_stats.total_pkts_txd: 1162993 dev.igb.1.mac_stats.good_octets_txd: 199531351 dev.igb.1.mac_stats.good_octets_recvd: 3824914432 dev.igb.1.mac_stats.rx_frames_1024_1522: 2478913 dev.igb.1.mac_stats.rx_frames_512_1023: 212806 dev.igb.1.mac_stats.rx_frames_256_511: 47827 dev.igb.1.mac_stats.rx_frames_128_255: 297392 dev.igb.1.mac_stats.rx_frames_65_127: 201838 dev.igb.1.mac_stats.rx_frames_64: 189926 dev.igb.1.mac_stats.mcast_pkts_recvd: 273 dev.igb.1.mac_stats.bcast_pkts_recvd: 7386 dev.igb.1.mac_stats.good_pkts_recvd: 3428702 dev.igb.1.mac_stats.total_pkts_recvd: 3454915 dev.igb.1.mac_stats.xoff_txd: 0 dev.igb.1.mac_stats.xoff_recvd: 0 dev.igb.1.mac_stats.xon_txd: 0 dev.igb.1.mac_stats.xon_recvd: 0 dev.igb.1.mac_stats.coll_ext_errs: 0 dev.igb.1.mac_stats.alignment_errs: 0 dev.igb.1.mac_stats.crc_errs: 0 dev.igb.1.mac_stats.recv_errs: 0 dev.igb.1.mac_stats.recv_jabber: 0 dev.igb.1.mac_stats.recv_oversize: 0 dev.igb.1.mac_stats.recv_fragmented: 0 dev.igb.1.mac_stats.recv_undersize: 0 dev.igb.1.mac_stats.recv_no_buff: 0 dev.igb.1.mac_stats.missed_packets: 0 dev.igb.1.mac_stats.defer_count: 0 dev.igb.1.mac_stats.sequence_errors: 0 dev.igb.1.mac_stats.symbol_errors: 0 dev.igb.1.mac_stats.collision_count: 0 dev.igb.1.mac_stats.late_coll: 0 dev.igb.1.mac_stats.multiple_coll: 0 dev.igb.1.mac_stats.single_coll: 0 dev.igb.1.mac_stats.excess_coll: 0 dev.igb.1.queue_rx_3.rx_irq: 0 dev.igb.1.queue_rx_3.rxd_tail: 202 dev.igb.1.queue_rx_3.rxd_head: 203 dev.igb.1.queue_rx_2.rx_irq: 0 dev.igb.1.queue_rx_2.rxd_tail: 299 dev.igb.1.queue_rx_2.rxd_head: 300 dev.igb.1.queue_rx_1.rx_irq: 0 dev.igb.1.queue_rx_1.rxd_tail: 768 dev.igb.1.queue_rx_1.rxd_head: 769 dev.igb.1.queue_rx_0.rx_irq: 0 dev.igb.1.queue_rx_0.rxd_tail: 107 dev.igb.1.queue_rx_0.rxd_head: 108 dev.igb.1.queue_tx_3.tx_irq: 0 dev.igb.1.queue_tx_3.txd_tail: 380 dev.igb.1.queue_tx_3.txd_head: 380 dev.igb.1.queue_tx_2.tx_irq: 0 dev.igb.1.queue_tx_2.txd_tail: 1006 dev.igb.1.queue_tx_2.txd_head: 1006 dev.igb.1.queue_tx_1.tx_irq: 0 dev.igb.1.queue_tx_1.txd_tail: 849 dev.igb.1.queue_tx_1.txd_head: 849 dev.igb.1.queue_tx_0.tx_irq: 0 dev.igb.1.queue_tx_0.txd_tail: 176 dev.igb.1.queue_tx_0.txd_head: 176 dev.igb.1.fc_low_water: 32752 dev.igb.1.fc_high_water: 32768 dev.igb.1.rx_control: 71598082 dev.igb.1.device_control: 1477968449 dev.igb.1.watchdog_timeouts: 0 dev.igb.1.rx_overruns: 0 dev.igb.1.link_irq: 2 dev.igb.1.dropped: 0 dev.igb.1.eee_control: 1 dev.igb.1.itr: 488 dev.igb.1.tx_abs_int_delay: 66 dev.igb.1.rx_abs_int_delay: 66 dev.igb.1.tx_int_delay: 66 dev.igb.1.rx_int_delay: 0 dev.igb.1.rs_dump: 0 dev.igb.1.reg_dump: General Registers dev.igb.1.fc: 0 dev.igb.1.debug: -1 dev.igb.1.fw_version: EEPROM V3.16-0 eTrack 0x800004ff dev.igb.1.nvm: -1 dev.igb.1.iflib.rxq3.rxq_fl0.buf_size: 2048 dev.igb.1.iflib.rxq3.rxq_fl0.credits: 1023 dev.igb.1.iflib.rxq3.rxq_fl0.cidx: 203 dev.igb.1.iflib.rxq3.rxq_fl0.pidx: 202 dev.igb.1.iflib.rxq3.cpu: 3 dev.igb.1.iflib.rxq2.rxq_fl0.buf_size: 2048 dev.igb.1.iflib.rxq2.rxq_fl0.credits: 1023 dev.igb.1.iflib.rxq2.rxq_fl0.cidx: 300 dev.igb.1.iflib.rxq2.rxq_fl0.pidx: 299 dev.igb.1.iflib.rxq2.cpu: 2 dev.igb.1.iflib.rxq1.rxq_fl0.buf_size: 2048 dev.igb.1.iflib.rxq1.rxq_fl0.credits: 1023 dev.igb.1.iflib.rxq1.rxq_fl0.cidx: 769 dev.igb.1.iflib.rxq1.rxq_fl0.pidx: 768 dev.igb.1.iflib.rxq1.cpu: 1 dev.igb.1.iflib.rxq0.rxq_fl0.buf_size: 2048 dev.igb.1.iflib.rxq0.rxq_fl0.credits: 1023 dev.igb.1.iflib.rxq0.rxq_fl0.cidx: 108 dev.igb.1.iflib.rxq0.rxq_fl0.pidx: 107 dev.igb.1.iflib.rxq0.cpu: 0 dev.igb.1.iflib.txq3.r_abdications: 0 dev.igb.1.iflib.txq3.r_restarts: 0 dev.igb.1.iflib.txq3.r_stalls: 0 dev.igb.1.iflib.txq3.r_starts: 117438 dev.igb.1.iflib.txq3.r_drops: 0 dev.igb.1.iflib.txq3.r_enqueues: 117438 dev.igb.1.iflib.txq3.ring_state: pidx_head: 0702 pidx_tail: 0702 cidx: 0702 state: IDLE dev.igb.1.iflib.txq3.txq_cleaned: 234834 dev.igb.1.iflib.txq3.txq_processed: 234874 dev.igb.1.iflib.txq3.txq_in_use: 42 dev.igb.1.iflib.txq3.txq_cidx_processed: 378 dev.igb.1.iflib.txq3.txq_cidx: 338 dev.igb.1.iflib.txq3.txq_pidx: 380 dev.igb.1.iflib.txq3.no_tx_dma_setup: 0 dev.igb.1.iflib.txq3.txd_encap_efbig: 0 dev.igb.1.iflib.txq3.tx_map_failed: 0 dev.igb.1.iflib.txq3.no_desc_avail: 0 dev.igb.1.iflib.txq3.mbuf_defrag_failed: 0 dev.igb.1.iflib.txq3.m_pullups: 0 dev.igb.1.iflib.txq3.mbuf_defrag: 0 dev.igb.1.iflib.txq3.cpu: 3 dev.igb.1.iflib.txq2.r_abdications: 0 dev.igb.1.iflib.txq2.r_restarts: 0 dev.igb.1.iflib.txq2.r_stalls: 0 dev.igb.1.iflib.txq2.r_starts: 67575 dev.igb.1.iflib.txq2.r_drops: 0 dev.igb.1.iflib.txq2.r_enqueues: 67575 dev.igb.1.iflib.txq2.ring_state: pidx_head: 2039 pidx_tail: 2039 cidx: 2039 state: IDLE dev.igb.1.iflib.txq2.txq_cleaned: 135108 dev.igb.1.iflib.txq2.txq_processed: 135148 dev.igb.1.iflib.txq2.txq_in_use: 42 dev.igb.1.iflib.txq2.txq_cidx_processed: 1004 dev.igb.1.iflib.txq2.txq_cidx: 964 dev.igb.1.iflib.txq2.txq_pidx: 1006 dev.igb.1.iflib.txq2.no_tx_dma_setup: 0 dev.igb.1.iflib.txq2.txd_encap_efbig: 0 dev.igb.1.iflib.txq2.tx_map_failed: 0 dev.igb.1.iflib.txq2.no_desc_avail: 0 dev.igb.1.iflib.txq2.mbuf_defrag_failed: 0 dev.igb.1.iflib.txq2.m_pullups: 0 dev.igb.1.iflib.txq2.mbuf_defrag: 0 dev.igb.1.iflib.txq2.cpu: 2 dev.igb.1.iflib.txq1.r_abdications: 0 dev.igb.1.iflib.txq1.r_restarts: 0 dev.igb.1.iflib.txq1.r_stalls: 0 dev.igb.1.iflib.txq1.r_starts: 297430 dev.igb.1.iflib.txq1.r_drops: 0 dev.igb.1.iflib.txq1.r_enqueues: 297430 dev.igb.1.iflib.txq1.ring_state: pidx_head: 0470 pidx_tail: 0470 cidx: 0470 state: IDLE dev.igb.1.iflib.txq1.txq_cleaned: 594727 dev.igb.1.iflib.txq1.txq_processed: 594767 dev.igb.1.iflib.txq1.txq_in_use: 42 dev.igb.1.iflib.txq1.txq_cidx_processed: 847 dev.igb.1.iflib.txq1.txq_cidx: 807 dev.igb.1.iflib.txq1.txq_pidx: 849 dev.igb.1.iflib.txq1.no_tx_dma_setup: 0 dev.igb.1.iflib.txq1.txd_encap_efbig: 0 dev.igb.1.iflib.txq1.tx_map_failed: 0 dev.igb.1.iflib.txq1.no_desc_avail: 0 dev.igb.1.iflib.txq1.mbuf_defrag_failed: 0 dev.igb.1.iflib.txq1.m_pullups: 0 dev.igb.1.iflib.txq1.mbuf_defrag: 0 dev.igb.1.iflib.txq1.cpu: 1 dev.igb.1.iflib.txq0.r_abdications: 0 dev.igb.1.iflib.txq0.r_restarts: 0 dev.igb.1.iflib.txq0.r_stalls: 0 dev.igb.1.iflib.txq0.r_starts: 680464 dev.igb.1.iflib.txq0.r_drops: 0 dev.igb.1.iflib.txq0.r_enqueues: 680605 dev.igb.1.iflib.txq0.ring_state: pidx_head: 0669 pidx_tail: 0669 cidx: 0669 state: IDLE dev.igb.1.iflib.txq0.txq_cleaned: 912519 dev.igb.1.iflib.txq0.txq_processed: 912559 dev.igb.1.iflib.txq0.txq_in_use: 41 dev.igb.1.iflib.txq0.txq_cidx_processed: 175 dev.igb.1.iflib.txq0.txq_cidx: 135 dev.igb.1.iflib.txq0.txq_pidx: 176 dev.igb.1.iflib.txq0.no_tx_dma_setup: 0 dev.igb.1.iflib.txq0.txd_encap_efbig: 0 dev.igb.1.iflib.txq0.tx_map_failed: 0 dev.igb.1.iflib.txq0.no_desc_avail: 0 dev.igb.1.iflib.txq0.mbuf_defrag_failed: 0 dev.igb.1.iflib.txq0.m_pullups: 0 dev.igb.1.iflib.txq0.mbuf_defrag: 0 dev.igb.1.iflib.txq0.cpu: 0 dev.igb.1.iflib.override_nrxds: 0 dev.igb.1.iflib.override_ntxds: 0 dev.igb.1.iflib.use_logical_cores: 0 dev.igb.1.iflib.separate_txrx: 0 dev.igb.1.iflib.core_offset: 0 dev.igb.1.iflib.tx_abdicate: 0 dev.igb.1.iflib.rx_budget: 0 dev.igb.1.iflib.disable_msix: 0 dev.igb.1.iflib.override_qs_enable: 0 dev.igb.1.iflib.override_nrxqs: 0 dev.igb.1.iflib.override_ntxqs: 0 dev.igb.1.iflib.driver_version: 7.6.1-k dev.igb.1.%parent: pci2 dev.igb.1.%pnpinfo: vendor=0x8086 device=0x1533 subvendor=0x103c subdevice=0x0003 class=0x020000 dev.igb.1.%location: slot=0 function=0 dbsf=pci0:2:0:0 handle=\_SB_.PCI0.RP06.PXSX dev.igb.1.%driver: igb dev.igb.1.%desc: Intel(R) I210 (Copper) dev.igb.0.wake: 0 dev.igb.0.interrupts.rx_overrun: 0 dev.igb.0.interrupts.rx_desc_min_thresh: 0 dev.igb.0.interrupts.tx_queue_min_thresh: 1373925 dev.igb.0.interrupts.tx_queue_empty: 3414605 dev.igb.0.interrupts.tx_abs_timer: 0 dev.igb.0.interrupts.tx_pkt_timer: 0 dev.igb.0.interrupts.rx_abs_timer: 0 dev.igb.0.interrupts.rx_pkt_timer: 1373925 dev.igb.0.interrupts.asserts: 4230086 dev.igb.0.mac_stats.tso_ctx_fail: 0 dev.igb.0.mac_stats.tso_txd: 0 dev.igb.0.mac_stats.tx_frames_1024_1522: 2475823 dev.igb.0.mac_stats.tx_frames_512_1023: 223599 dev.igb.0.mac_stats.tx_frames_256_511: 60037 dev.igb.0.mac_stats.tx_frames_128_255: 288133 dev.igb.0.mac_stats.tx_frames_65_127: 357046 dev.igb.0.mac_stats.tx_frames_64: 9967 dev.igb.0.mac_stats.mcast_pkts_txd: 22 dev.igb.0.mac_stats.bcast_pkts_txd: 96 dev.igb.0.mac_stats.good_pkts_txd: 3414605 dev.igb.0.mac_stats.total_pkts_txd: 3414605 dev.igb.0.mac_stats.good_octets_txd: 3814231622 dev.igb.0.mac_stats.good_octets_recvd: 213341125 dev.igb.0.mac_stats.rx_frames_1024_1522: 64022 dev.igb.0.mac_stats.rx_frames_512_1023: 11854 dev.igb.0.mac_stats.rx_frames_256_511: 12436 dev.igb.0.mac_stats.rx_frames_128_255: 225885 dev.igb.0.mac_stats.rx_frames_65_127: 859524 dev.igb.0.mac_stats.rx_frames_64: 200204 dev.igb.0.mac_stats.mcast_pkts_recvd: 3632 dev.igb.0.mac_stats.bcast_pkts_recvd: 14745 dev.igb.0.mac_stats.good_pkts_recvd: 1373925 dev.igb.0.mac_stats.total_pkts_recvd: 1373933 dev.igb.0.mac_stats.xoff_txd: 0 dev.igb.0.mac_stats.xoff_recvd: 0 dev.igb.0.mac_stats.xon_txd: 0 dev.igb.0.mac_stats.xon_recvd: 0 dev.igb.0.mac_stats.coll_ext_errs: 0 dev.igb.0.mac_stats.alignment_errs: 0 dev.igb.0.mac_stats.crc_errs: 0 dev.igb.0.mac_stats.recv_errs: 0 dev.igb.0.mac_stats.recv_jabber: 0 dev.igb.0.mac_stats.recv_oversize: 0 dev.igb.0.mac_stats.recv_fragmented: 0 dev.igb.0.mac_stats.recv_undersize: 0 dev.igb.0.mac_stats.recv_no_buff: 0 dev.igb.0.mac_stats.missed_packets: 0 dev.igb.0.mac_stats.defer_count: 0 dev.igb.0.mac_stats.sequence_errors: 0 dev.igb.0.mac_stats.symbol_errors: 0 dev.igb.0.mac_stats.collision_count: 0 dev.igb.0.mac_stats.late_coll: 0 dev.igb.0.mac_stats.multiple_coll: 0 dev.igb.0.mac_stats.single_coll: 0 dev.igb.0.mac_stats.excess_coll: 0 dev.igb.0.queue_rx_3.rx_irq: 0 dev.igb.0.queue_rx_3.rxd_tail: 631 dev.igb.0.queue_rx_3.rxd_head: 636 dev.igb.0.queue_rx_2.rx_irq: 0 dev.igb.0.queue_rx_2.rxd_tail: 551 dev.igb.0.queue_rx_2.rxd_head: 552 dev.igb.0.queue_rx_1.rx_irq: 0 dev.igb.0.queue_rx_1.rxd_tail: 970 dev.igb.0.queue_rx_1.rxd_head: 971 dev.igb.0.queue_rx_0.rx_irq: 0 dev.igb.0.queue_rx_0.rxd_tail: 355 dev.igb.0.queue_rx_0.rxd_head: 356 dev.igb.0.queue_tx_3.tx_irq: 0 dev.igb.0.queue_tx_3.txd_tail: 10 dev.igb.0.queue_tx_3.txd_head: 10 dev.igb.0.queue_tx_2.tx_irq: 0 dev.igb.0.queue_tx_2.txd_tail: 14 dev.igb.0.queue_tx_2.txd_head: 14 dev.igb.0.queue_tx_1.tx_irq: 0 dev.igb.0.queue_tx_1.txd_tail: 12 dev.igb.0.queue_tx_1.txd_head: 12 dev.igb.0.queue_tx_0.tx_irq: 0 dev.igb.0.queue_tx_0.txd_tail: 455 dev.igb.0.queue_tx_0.txd_head: 456 dev.igb.0.fc_low_water: 32752 dev.igb.0.fc_high_water: 32768 dev.igb.0.rx_control: 71598106 dev.igb.0.device_control: 1075577409 dev.igb.0.watchdog_timeouts: 0 dev.igb.0.rx_overruns: 0 dev.igb.0.link_irq: 19 dev.igb.0.dropped: 0 dev.igb.0.eee_control: 1 dev.igb.0.itr: 488 dev.igb.0.tx_abs_int_delay: 66 dev.igb.0.rx_abs_int_delay: 66 dev.igb.0.tx_int_delay: 66 dev.igb.0.rx_int_delay: 0 dev.igb.0.rs_dump: 0 dev.igb.0.reg_dump: General Registers dev.igb.0.fc: 0 dev.igb.0.debug: -1 dev.igb.0.fw_version: EEPROM V3.16-0 eTrack 0x800004ff dev.igb.0.nvm: -1 dev.igb.0.iflib.rxq3.rxq_fl0.buf_size: 2048 dev.igb.0.iflib.rxq3.rxq_fl0.credits: 0 dev.igb.0.iflib.rxq3.rxq_fl0.cidx: 647 dev.igb.0.iflib.rxq3.rxq_fl0.pidx: 647 dev.igb.0.iflib.rxq3.cpu: 3 dev.igb.0.iflib.rxq2.rxq_fl0.buf_size: 2048 dev.igb.0.iflib.rxq2.rxq_fl0.credits: 0 dev.igb.0.iflib.rxq2.rxq_fl0.cidx: 552 dev.igb.0.iflib.rxq2.rxq_fl0.pidx: 552 dev.igb.0.iflib.rxq2.cpu: 2 dev.igb.0.iflib.rxq1.rxq_fl0.buf_size: 2048 dev.igb.0.iflib.rxq1.rxq_fl0.credits: 0 dev.igb.0.iflib.rxq1.rxq_fl0.cidx: 971 dev.igb.0.iflib.rxq1.rxq_fl0.pidx: 971 dev.igb.0.iflib.rxq1.cpu: 1 dev.igb.0.iflib.rxq0.rxq_fl0.buf_size: 2048 dev.igb.0.iflib.rxq0.rxq_fl0.credits: 0 dev.igb.0.iflib.rxq0.rxq_fl0.cidx: 356 dev.igb.0.iflib.rxq0.rxq_fl0.pidx: 356 dev.igb.0.iflib.rxq0.cpu: 0 dev.igb.0.iflib.txq3.r_abdications: 0 dev.igb.0.iflib.txq3.r_restarts: 0 dev.igb.0.iflib.txq3.r_stalls: 0 dev.igb.0.iflib.txq3.r_starts: 0 dev.igb.0.iflib.txq3.r_drops: 0 dev.igb.0.iflib.txq3.r_enqueues: 0 dev.igb.0.iflib.txq3.ring_state: pidx_head: 1464 pidx_tail: 1464 cidx: 1464 state: IDLE dev.igb.0.iflib.txq3.txq_cleaned: 0 dev.igb.0.iflib.txq3.txq_processed: 1 dev.igb.0.iflib.txq3.txq_in_use: 0 dev.igb.0.iflib.txq3.txq_cidx_processed: 1 dev.igb.0.iflib.txq3.txq_cidx: 0 dev.igb.0.iflib.txq3.txq_pidx: 0 dev.igb.0.iflib.txq3.no_tx_dma_setup: 0 dev.igb.0.iflib.txq3.txd_encap_efbig: 0 dev.igb.0.iflib.txq3.tx_map_failed: 0 dev.igb.0.iflib.txq3.no_desc_avail: 0 dev.igb.0.iflib.txq3.mbuf_defrag_failed: 0 dev.igb.0.iflib.txq3.m_pullups: 0 dev.igb.0.iflib.txq3.mbuf_defrag: 0 dev.igb.0.iflib.txq3.cpu: 3 dev.igb.0.iflib.txq2.r_abdications: 0 dev.igb.0.iflib.txq2.r_restarts: 0 dev.igb.0.iflib.txq2.r_stalls: 0 dev.igb.0.iflib.txq2.r_starts: 0 dev.igb.0.iflib.txq2.r_drops: 0 dev.igb.0.iflib.txq2.r_enqueues: 0 dev.igb.0.iflib.txq2.ring_state: pidx_head: 1825 pidx_tail: 1825 cidx: 1825 state: IDLE dev.igb.0.iflib.txq2.txq_cleaned: 0 dev.igb.0.iflib.txq2.txq_processed: 1 dev.igb.0.iflib.txq2.txq_in_use: 0 dev.igb.0.iflib.txq2.txq_cidx_processed: 1 dev.igb.0.iflib.txq2.txq_cidx: 0 dev.igb.0.iflib.txq2.txq_pidx: 0 dev.igb.0.iflib.txq2.no_tx_dma_setup: 0 dev.igb.0.iflib.txq2.txd_encap_efbig: 0 dev.igb.0.iflib.txq2.tx_map_failed: 0 dev.igb.0.iflib.txq2.no_desc_avail: 0 dev.igb.0.iflib.txq2.mbuf_defrag_failed: 0 dev.igb.0.iflib.txq2.m_pullups: 0 dev.igb.0.iflib.txq2.mbuf_defrag: 0 dev.igb.0.iflib.txq2.cpu: 2 dev.igb.0.iflib.txq1.r_abdications: 0 dev.igb.0.iflib.txq1.r_restarts: 0 dev.igb.0.iflib.txq1.r_stalls: 0 dev.igb.0.iflib.txq1.r_starts: 0 dev.igb.0.iflib.txq1.r_drops: 0 dev.igb.0.iflib.txq1.r_enqueues: 0 dev.igb.0.iflib.txq1.ring_state: pidx_head: 0949 pidx_tail: 0949 cidx: 0949 state: IDLE dev.igb.0.iflib.txq1.txq_cleaned: 0 dev.igb.0.iflib.txq1.txq_processed: 1 dev.igb.0.iflib.txq1.txq_in_use: 0 dev.igb.0.iflib.txq1.txq_cidx_processed: 1 dev.igb.0.iflib.txq1.txq_cidx: 0 dev.igb.0.iflib.txq1.txq_pidx: 0 dev.igb.0.iflib.txq1.no_tx_dma_setup: 0 dev.igb.0.iflib.txq1.txd_encap_efbig: 0 dev.igb.0.iflib.txq1.tx_map_failed: 0 dev.igb.0.iflib.txq1.no_desc_avail: 0 dev.igb.0.iflib.txq1.mbuf_defrag_failed: 0 dev.igb.0.iflib.txq1.m_pullups: 0 dev.igb.0.iflib.txq1.mbuf_defrag: 0 dev.igb.0.iflib.txq1.cpu: 1 dev.igb.0.iflib.txq0.r_abdications: 0 dev.igb.0.iflib.txq0.r_restarts: 0 dev.igb.0.iflib.txq0.r_stalls: 0 dev.igb.0.iflib.txq0.r_starts: 0 dev.igb.0.iflib.txq0.r_drops: 0 dev.igb.0.iflib.txq0.r_enqueues: 0 dev.igb.0.iflib.txq0.ring_state: pidx_head: 0418 pidx_tail: 0418 cidx: 0418 state: IDLE dev.igb.0.iflib.txq0.txq_cleaned: 0 dev.igb.0.iflib.txq0.txq_processed: 1270273 dev.igb.0.iflib.txq0.txq_in_use: 0 dev.igb.0.iflib.txq0.txq_cidx_processed: 513 dev.igb.0.iflib.txq0.txq_cidx: 0 dev.igb.0.iflib.txq0.txq_pidx: 0 dev.igb.0.iflib.txq0.no_tx_dma_setup: 0 dev.igb.0.iflib.txq0.txd_encap_efbig: 0 dev.igb.0.iflib.txq0.tx_map_failed: 0 dev.igb.0.iflib.txq0.no_desc_avail: 0 dev.igb.0.iflib.txq0.mbuf_defrag_failed: 0 dev.igb.0.iflib.txq0.m_pullups: 0 dev.igb.0.iflib.txq0.mbuf_defrag: 0 dev.igb.0.iflib.txq0.cpu: 0 dev.igb.0.iflib.override_nrxds: 0 dev.igb.0.iflib.override_ntxds: 0 dev.igb.0.iflib.use_logical_cores: 0 dev.igb.0.iflib.separate_txrx: 0 dev.igb.0.iflib.core_offset: 0 dev.igb.0.iflib.tx_abdicate: 0 dev.igb.0.iflib.rx_budget: 0 dev.igb.0.iflib.disable_msix: 0 dev.igb.0.iflib.override_qs_enable: 0 dev.igb.0.iflib.override_nrxqs: 0 dev.igb.0.iflib.override_ntxqs: 0 dev.igb.0.iflib.driver_version: 7.6.1-k dev.igb.0.%parent: pci1 dev.igb.0.%pnpinfo: vendor=0x8086 device=0x1533 subvendor=0x103c subdevice=0x0003 class=0x020000 dev.igb.0.%location: slot=0 function=0 dbsf=pci0:1:0:0 handle=\_SB_.PCI0.PEG0.PEGP dev.igb.0.%driver: igb dev.igb.0.%desc: Intel(R) I210 (Copper) dev.igb.%parent: [2.6.0-RELEASE][admin@pfSense]/root: sysctl -a | grep rss device wlan_rssadapt hw.bxe.udp_rss: 0 hw.ix.enable_rss: 1 hw.cxgbe.nm_split_rss: 0 compat.linuxkpi.mlx4_udp_rss: 1 cat /var/log/system.log | grep netmap empty [2.6.0-RELEASE][admin@pfSense]/root: cat /var/log/system.log | grep sig Feb 21 18:53:16 pfSense php-fpm[370]: [Suricata] Suricata signalled with SIGHUP for LAN (igb0)... [2.6.0-RELEASE][admin@pfSense]/root: cat /var/log/suricata/suricata_*/suricata.log | grep -m 1 "signatures processed" 21/2/2022 -- 18:13:25 - <Info> -- 34459 signatures processed. 1214 are IP-only rules, 8885 are inspecting packet payload, 23804 inspect application layer, 107 are decoder event only
-
The short answer, and you won't like it, is that Suricata and VLANs and Inline IPS Mode (which uses netmap) hate each other. This has, in some ways, actually gotten worse as FreeBSD migrated to the iflib network driver wrapper library. But it really never did work correctly anyway. And it's not just on pfSense. If you cruise over to the forum for the "other Sense" product, you will find tons of Suricata and VLAN issues reported there as well.
If you must use VLANs, you should switch to Legacy Blocking Mode.
-
@bmeeks You've guessed that right.
I'm not too happy about the legacy mode. Sometimes when I have a false positive hosts are blocked and the min time is 15 min which is causing some issues.
Maybe I can run suricata on the WAN and use VLANs on LAN. -
@lcs said in Suricata inline with VLANs:
@bmeeks You've guessed that right.
I'm not too happy about the legacy mode. Sometimes when I have a false positive hosts are blocked and the min time is 15 min which is causing some issues.
Maybe I can run suricata on the WAN and use VLANs on LAN.I wish I had happier news. I've done quite a bit of research about netmap, especially over the early summer last year when I worked with the Suricata team on implementing multiple host rings support in Suricata. The way the netmap device is plumbed into the FreeBSD network stack makes working with VLANs natively and transparently not really possible. This is especially true if the hardware NIC driver does hardware VLAN tagging. The tags get copied by the driver into a part of kernel space that netmap does not see.
Add to that the fact FreeBSD moved NIC drivers over to a new wrapper API library called iflib. That required rewriting many drivers. And during the rewrite phase some bugs were introduced, including some regressions. Those are still being worked out. The bugs affected things other than just netmap, though.
Netmap was really designed for a slightly different use case than what is currently being done in Suricata and Snort on pfSense (and on the "other Sense" product, too). On the two firewall distros netmap is used to intercept traffic between the NIC driver and the kernel network stack. That is called host stack mode. That mode is where the VLAN troubles live. The way netmap was originally conceived was to simply route traffic between two physical NIC ports at super high speed bypassing the kernel network stack completely. It would essentially just bridge two NIC ports. But on a typical firewall appliance that is wasteful of valuable NIC ports.