LAN limiter: pfsync_undefer_state: unable to find deferred state

Derelict

Again, still working on bringing up a 2.2.1 HA pair. Everything going along fine until I enabled a limiter on LAN. It is a simple 3x3Mbit/s pipe per IP address. As soon as I enable the limiter on the final pass any any any rule on LAN and open a new state (just a ping to an address outside WAN) I immediately get logs and console messages "pfsync_undefer_state: unable to find deferred state."

I have tried it both with and without captive portal enabled for what that's worth.

The traffic seems to make it through.

Don't know if it's related to or another symptom of this: https://redmine.pfsense.org/issues/4326

salmonbaytech

I have lots of the same error message in my logs also.

I have a pair of APU running 2.1.5, and everything was working great.

Upgraded the secondary APU to 2.2.1 and that did not go well, had to reflash and resetup. after that, I upgraded the primary and its "broken" had to take it offline.

primary router was having latency issues, could not login to web interface or ssh. pulled the network cables.

I also had a captive portal on the primary, it was not working after the upgrade.

Derelict

This is essentially fixed in the latest 2.2.2 snapshots but there is still something wonky with HA/pfsync+limiters.

I can imagine that putting significant traffic through 2.2.1 with those log messages (one for every packet, it looked like) would bring a node to its knees.

sepp_huber

The problem is still there in "2.3.4-RELEASE-p1" with our HA/CARP setup.
A few days ago I have configured LIMITERs everything is working fine … until today.
Both firewalls are flooded with the message "pfsync_undefer_state: unable to find deferred statepfsync_undefer_state: unable to find deferred state".
Only removing the limiters and reboot helps that both firewalls are quiet again.

Because of this problem i cannot use limiters.
There is also another problem with LAGGs and the altq traffic shaper...
https://forum.pfsense.org/index.php?topic=93718.0

So at the moment I cannot use any traffic shaping or limiting >:(

sepp_huber

I found this solution here and will try it out for the next days.
The symptom is not exactly the same, but it deals with Limiters and HA and is not solved.
Btw. I had also a crash of the master node after those flooding messages.
https://redmine.pfsense.org/issues/4310#note-44

For those still with problems you can use limiters in HA with any version w/out kernel panic but for that you need additional configuration.

1. Create a new limiter for both upload and download with the bandwidth limit. Name it with the name you want and _donotuse at the end (just for safety)
2. Create a new Queue inside of each limiter (When inside of the limiter "Add New Queue" green button)
3. Name the queues with the vlan/rule name and the bandwidth you set in the limit and with _up or _down (for reference) and set the weight to 100 for that queue to use 100% of the limiter
4. Assign the queues you created to the rules you want to limit the bandwidth. MAKE SURE YOU ASSIGN THE QUEUE AND NOT THE LIMITER, IT YOU CHOOSE THE LIMITER YOU WILL HAVE THE KERNEL PANIC IN THE 2nd MEMBER. That's why it's a better practice to use the name _donotuse in the limiters.

Notes:
You still need to create 1xlimiter + 1xqueue per each flow per rule
If you assign the same queues to multiple rules they will share the same "roof" defined in the limiter
You can create multiple queues for one limiter with different weight, very useful if you want to have, for example, a top limit of 400Mbit and give rule1 guaranteed 10% of those, rule2 50% and rule3 40%. If all of the rules/queues are being maxed out you will have a perfect bandwidth balance. If for example rule 2 and 3 don't have any traffic rule1 will be able to use the 400Mbit since we only define a minimum guaranteed.

Derelict

There is not a problem with LAGGs and altq. The raw LAGG interface does not support altq. Tag a VLAN on the LAGG and that will support altq.

An alternative is to disable pfsync and use limiters. You will lose state sync but that is often a better compromise than not using limiters.

sepp_huber

I found this solution here and will try it out for the next days.
The symptom is not exactly the same, but it deals with Limiters and HA and is not solved.
Btw. I had also a crash of the master node after those flooding messages.
https://redmine.pfsense.org/issues/4310#note-44

After a few days operating in production, the solution above is working with pfsync and limiters… perfect.

Tag a VLAN on the LAGG and that will support altq.

OK, thanks for your advice! At the moment we do not use any VLANs…