Dummynet Limiter Bypasses with Multi-WAN Policy Routing & Chained Limiters

DarkFnh

Hi everyone,

I've been testing the latest 26.07 beta and am running into severe state-tracking issues with dummynet limiters bypassing rules under specific Multi-WAN routing scenarios.

I've narrowed this down to two distinct behaviors.

Issue 1: Upload Limiter Bypassed by Gateway Groups

My upload limiter only functions correctly when the rule's Gateway is set to Default. The moment I assign my Multi-WAN Gateway Group to the LAN pass rule, the upload limit completely fails, allowing traffic to pass unshaped.

Based on my testing:
When policy routing is active, the route-to directive forces a secondary evaluation (pf_test()) on the egress interface.
This second state fails to inherit the dummynet pipe assigned during the initial LAN evaluation.
The packet effectively escapes the queue before the traffic shaper can clamp down on the bandwidth.

Issue 2: Download Limiter Ignored (Chained WAN/LAN Conflict)

Unlike the upload, the download limiter fails entirely—regardless of whether I use the default route or the Multi-WAN gateway group.

I traced this back to a conflict with having global WAN limiters active concurrently. I have WANDown limiters applied to the WAN interfaces to manage bufferbloat. It appears that if the incoming download stream hits the WAN limiter first, the firewall engine completely ignores the secondary LAN limiter downstream.

If I disable the WAN limiters and flush states, the LAN 100 Mbit/s download limit immediately starts working.
pfSense seems incapable of natively chaining these limiters; the pf engine sees the packet as "already shaped" and refuses to honor the internal LAN cap.

Steps to Reproduce:
1- Configure independent root limiters (no child queues) for LAN DHCP limits.
2- Assign the limiters to a LAN pass rule (In/Out) with a Multi-WAN Gateway Group applied.

Upload fails entirely due to policy routing stripping the state.
Configure global WAN limiters for bufferbloat on the incoming interfaces.
Download fails entirely because the WAN limiter overrides the LAN limiter.

Are there any active patches to address the route-to state stripping? Additionally, is the chained limiter bypass an expected behavior of dummynet, or is there a proper workaround (aside from floating rules, which have proven highly unreliable in my multi-WAN setup)?

Thanks in advance for any insights.

marcosm

Did your setup work in a previous version? Does it still work if you revert to that version (e.g. using Boot Environments)? You can also share a status report (go to /status.php) from a working and nonworking setup and share them here for review: https://nc.netgate.com/nextcloud/s/To7TYXzNktZ2bqW

DarkFnh

Hi @marcosm

Sadly I deleted the old boot environments after I noticed that the beta is stable.

A. The current working scenario for both upload and download limiting is:

1- disabling all the floating WAN bufferbloat rules.
2- configuring the low bandwidth LAN rules to utilize the default gateway instead of the MultiWAN I configured.

This allows both Upload and download limiting to work.

B. The current working scenario for only upload limiting is:

1- configuring the low bandwidth LAN rules to utilize the default gateway instead of the MultiWAN I configured.

This allows only Upload limiting to work.

otherwise both dont work.

I uploaded the log file for the current beta.

marcosm

Did your setup work in a previous version? I tested a similar setup - same AQM and schedulers - and it's working for me. The main difference in the config being that I'm not using PPPoE interfaces. Have you tried setting the pipe/limiter delay to 1?

DarkFnh

@marcosm

Yup it was working and I didnt notice it breaking till after I updated to the beta.

I tested the delay and it didnt work.

To be honest its not really a critical issue for me but I would love if it got fixed.

marcosm

What version was it working on?

DarkFnh

@marcosm

To be honest I am not sure when it broke as I just noticed that it was broken yesterday when a guest connected to the network and did a speed test and I had already deleted the old version snapshots.

Just to note disabling the WAN bufferbloat floating rules makes the LAN limiters work on the beta.