Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login
    Introducing Netgate Nexus: Multi-Instance Management at Your Fingertips.

    pfSense CE 2.9.0: pfsync import after reboot causes unsigned timestamp underflow and excessive local state retention

    Scheduled Pinned Locked Moved HA/CARP/VIPs
    1 Posts 1 Posters 48 Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I Offline
      infinity2011
      last edited by

      Hello,

      We reproduced a PF/pfsync state-expiration issue in a two-node pfSense HA test cluster. I would appreciate help identifying whether this is a known issue and whether there's a supported fix.

      Environment:

      • Both nodes: pfSense CE 2.9.0-RELEASE.
      • Kernel: FreeBSD 16.0-CURRENT #12, built 2026-08-17.
      • Build identifier: RELENG_2_9_0-n256132-d8e3138ecf52.
      • Identical kernel SHA256 on both nodes:
        f09fefb2e5061ce217b17519fb8205b5a837562a72f9c3e170b1d3f7633ba510
      • VMware ESXi 8.0 Update 3, VMXNET3 interfaces.
      • Identical WAN/LAN/SYNC assignments on both nodes.
      • Dedicated SYNC network, directed pfSync peers, explicit Filter Host IDs 1 and 2.
      • CARP, XMLRPC, and Kea DHCP HA configured.
      • Test cluster only, no production traffic.
      • No unofficial kernel or patch installed.

      Observed problem:

      After rebooting the secondary, importing an existing TCP state can produce a very large expiration timestamp. This affects the kernel's local expiration decision, not only the GUI display.

      Controlled reproduction:

      1. Establish a TCP connection through the primary.
      2. Close that connection with a controlled RST, leaving its PF entries in TIME_WAIT:TIME_WAIT with the existing 90-second timeout.
      3. Reboot the secondary while those entries still exist on the primary.
      4. Observe their bulk import on the secondary.

      We correlated the pfsync records, state IDs and creator ID with instruction-level DTrace kinst probes and FBT probes in the installed kernel.

      For both entries of the selected closed connection:

      • Canonical state IDs: 6ac0918c and 6ac0918d.
      • Creator ID: 1.
      • T, selected receiver timeout: 90 seconds.
      • R, remaining lifetime received through pfsync: 54 seconds.
      • U, actual millisecond uptime operand used at import: 11833 ms.
      • D, measured intermediate result: 36000 ms.
      • Result after subtraction, measured in a register and confirmed in inserted state memory:
        18446744073709527449.

      The arithmetic is:

      D = ((90 - 54) * 1000) modulo 2^32
      = 36000 ms

      U - D = 11833 - 36000
      = -24167 ms

      Stored unsigned 64-bit result:
      (-24167) modulo 2^64 = 18446744073709527449

      There is NO intermediate 32-bit wrap in this reproduction. The demonstrated underflow occurs at U-D and its unsigned 64-bit representation.

      The effective timeouts matched. Source raw/scaled timeout was 90 seconds, with 54 states and adaptive thresholds 1962000/3924000. No adaptive reduction applied. Different timeouts or adaptive scaling are not required to reproduce
      this case.

      Local expiration test:

      After successful bulk import, we temporarily disabled outbound pfsync on the primary using "ifconfig pfsync0 -syncdev". This isolated propagation of primary deletions; the physical SYNC interface and Kea communication remained
      available.

      We then temporarily disabled CARP on the primary so the secondary took over.

      The primary removed the closed-flow entries. The secondary still retained them approximately 169 seconds after closure, despite the unchanged 90-second timeout.

      Tracing the actual "pf purge" thread showed pf_state_expires() returning an absolute deadline of 1271310385 uptime seconds, including an evaluation approximately 187 seconds after closure.

      The roughly 40-year deadline is a calculated result, not an observed 40-year retention period.

      This experiment demonstrates defective LOCAL expiration on the secondary. It does NOT demonstrate failure to process a valid received pfsync DELETE, because primary deletion propagation was deliberately isolated.

      Other results:

      • A separate established TCP connection remained on the same socket and passed echo checks after failover, failback, during a primary reboot, and after recovery.
      • Traffic on that established connection restored a normal expiration interval, while its displayed age remained abnormal.
      • No traffic loss or memory exhaustion was demonstrated.
      • Both guests rebooted successfully.
      • Original configurations and temporary runtime settings were restored; diagnostic instrumentation was removed.
      • Timeouts were not changed during the tests.

      For source comparison only, this FreeBSD snapshot contains the corresponding import arithmetic. It is not claimed to be the exact source of our installed Netgate kernel:

      https://github.com/freebsd/freebsd-src/blob/765615997a3a3ab649505d58087b7375736fdff7/sys/netpfil/pf/if_pfsync.c#L861

      Questions:

      1. Is this imported-timestamp underflow already tracked by Netgate or FreeBSD? If so, could you point us to the issue and corrective commit?
      2. Which supported pfSense CE release or official hotfix includes the correction, and is it applicable to RELENG_2_9_0-n256132-d8e3138ecf52?
      3. Is there a supported workaround that preserves stateful failover?
      4. What additional sanitized tracing or state output would help confirm the appropriate fix?

      Thank you.

      1 Reply Last reply Reply Quote 0
      • First post
        Last post
      Copyright 2026 Rubicon Communications LLC (Netgate). All rights reserved.
      Privacy Policy · Cookie Policy