Traffic not going to Limiters after 2.4.4
-
@jacotec WOW WOW WOW!!!! We have something!!! Changing Limiter to Codel/QFQ as suggested made it work.
In my case I just changed the Download Limiter, and u can see how dynamic Download queues get filled in:
Limiters: 00001: 9.500 Mbit/s 0 ms burst 0 q131073 50 sl. 0 flows (1 buckets) sched 65537 weight 0 lmax 0 pri 0 AQM CoDel target 5ms interval 100ms NoECN sched 65537 type FIFO flags 0x0 0 buckets 0 active 00002: 950.000 Kbit/s 0 ms burst 0 q131074 50 sl. 0 flows (1 buckets) sched 65538 weight 0 lmax 0 pri 0 droptail sched 65538 type FIFO flags 0x0 0 buckets 0 active Queues: q00001 50 sl. 5 flows (256 buckets) sched 1 weight 20 lmax 1500 pri 0 droptail mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x0000 BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp 90 ip 0.0.0.0/0 192.168.211.11/0 9 4635 0 0 0 94 ip 0.0.0.0/0 192.168.211.15/0 1 330 0 0 0 99 ip 0.0.0.0/0 192.168.211.50/0 112 20579 0 0 0 108 ip 0.0.0.0/0 192.168.211.61/0 5 260 0 0 0 109 ip 0.0.0.0/0 192.168.211.60/0 94 22025 0 0 0 q00002 50 sl. 0 flows (256 buckets) sched 1 weight 1 lmax 1500 pri 0 droptail mask: 0x00 0x00000000/0x0000 -> 0xffffffff/0x0000 q00003 50 sl. 0 flows (256 buckets) sched 2 weight 20 lmax 0 pri 0 droptail mask: 0x00 0xffffffff/0x0000 -> 0x00000000/0x0000 q00004 50 sl. 0 flows (256 buckets) sched 2 weight 1 lmax 0 pri 0 droptail mask: 0x00 0xffffffff/0x0000 -> 0x00000000/0x0000
I wouldn't imagine that choosing a more complex scheduler would make it work...
Anyway, I see this as a "Yellow alarm", because there should be no reason for Taildrop/FIFO aqm/sched to fail.PS: I didn't need to delete /tmp/rules.limiters. As soon as I applied Limiters config, it began to work.
-
@grimson Nope, I've never used that. I was on a regular 2.4.3 installation.
-
@vpreatoni Good to hear it works for you now as well!
-
Can see that our traffic shaper is nonfunctional now as of 2.4.4 in terms of per-host dynamic bandwidth shaping.
Work around for the missing queues/inability to create queues in 2.4.4 was to delete all limiters & queues, then recreate them, then reassign queues to firewall rules. This part worked eventually. Then, found out that the limiter diagnostic info was not functional with Taildrop/FIFO (ipfw pipe show and ipfw queue show). Switching to Codel/QFQ allowed monitoring queues using ipfw pipe show and ipfw queue show, but while the overall bandwidth limiting was working (capping to the max allocated bandwidth), the shaper was NOT actually shaping, either with Taildrop/FIFO or with Codel/QFQ. Codel/QFQ caused system to crash eventually and had to be manually restarted.
Whole point of the setup was to do the amazing per-host dynamic bandwidth dividing that pfsense was so good with. Can confirm now that although the limiters/queues are recreated and working to limit the maximum aggregate bandwidth, the mask by destination (Down_LAN) or sources addresses (Up_LAN) does not seem to work. These queues are under the DownLimiter and UpLimiter limiters. Up_LAN is assigned to In pipe on a LAN interface firewall rule and Down_LAN is assigned to Out pipe. All was working before 2.4.4! The hosts always showed identical traffic during peak usage, dividing the total bandwidth evenly. This is nonfunctional now.
Is there anyway I can directly verify if traffic is actually going through per-host queues when using Taildrop/FIFO though ipfw pipe show and ipfw queue show do not show that happening?
-
Update: Have now switched to Codel/Round Robin. This combination seems to work -- traffic goes to child queues as expected and we can achieve the per-host dynamic bandwidth allocation. Would be nice if any other combinations including Taildrop with FIFO or QFQ would also work in the future to try and find optimal settings.
-
After some testing, I can confirm CoDel/QFQ is pretty fucked up!. My server restarts evry 2 or 3 days use, and I get flooded with:
qfq_dequeue BUG/* non-workconserving leaf */
I'm attaching my debug info here: 0_1539455038230_textdump.tar.0
Please any dev reply to this post. I'm happy to provide any debugging information or do some testing, but IT IS NOT SERIOUS TO RELEASE A VERSION SO FUCKED UP WITH BUGS, and there was no rush to do it. 2.4.3_1 was working fine. Our servers are production machines. -
@vpreatoni said in Traffic not going to Limiters after 2.4.4:
After some testing, I can confirm CoDel/QFQ is pretty fucked up!. My server restarts evry 2 or 3 days use, and I get flooded with:
qfq_dequeue BUG/* non-workconserving leaf */
This problem was driving me crazy, but I fix it by doing this:
- Removed every limiter on traffic shaper page.
- Full reboot of pfsense box
- Recreated the limiters and I even used different names for them, just in case...
- Assigned the new limiters to the relevant firewall rules.
That fixed my problem.
-
I'm also having problems with limiters and shaper in general.
I have upgraded to 2.4.4, and limiters dissapeared. I recreated them, but there was no incoming traffic at all. Upload traffic seemed ok. So, i reverted to 2.4.3 (it's a VM), but then i realized that in 2.4.3 i cannot edit the limiters, or the shaper, or bad things happen. Just by modifying the bandwidth on the limiters resulted in very low incoming bandwidth (something like 700 kbps instead of 3.5 mbps). So, i decided to delete all limiters and re-create them, and i got the same problem than with the upgrade to 2.4.4: no incoming traffic. Removing the traffic shaper (CBQ) also resulted on the same problem. Fortunately, i just reverted to a previous snapshot, and restored normal functionality. But it seems like i cannot modify anything at all on the shapers/limiters, or bad things happen. Not good.
Maybe part of the issue was already on 2.4.3?
Adding new functionality is nice, but lots of things can go wrong with every change. To me, reliability is much more important than new stuff, and lately i have second thoughts about clicking the update button. -
Howdy Folks, I don't know if this relative but I will just throw it out here. I have traffic shaper using codel as per the Netgate vid and started to get a log of (config_aqm unable to configure flow set). Also the WAN connection would break.
This all started when I made a change to the interfaces settings, From "default" to "autoselect" .After making the change to autoselect everything ran fine for 8 hours with no indicating problems. Reboots of Pfsense and modem changed nothing.
I then changed the interfaces back to default and all problems evaporated.
When I made the changes, reboots and state table reset were done per best practice.
Using: 2.4.4
Supermicro C2558
cable
I hope this is helpful. -
I see that there's a 2.4.4-p1 version now. Has this problem been fixed there?
-
@fsr said in Traffic not going to Limiters after 2.4.4:
I see that there's a 2.4.4-p1 version now. Has this problem been fixed there?
Yes, assuming it was this: https://redmine.pfsense.org/issues/8973
-
@fsr Yes, it's fixed!. Now it is clear which scheduler is the default one (WF2Q+), and works perfect.
Haven't tested QFQ yet, but I'm pretty happy with Codel ACM/WF2Q+ sched behavior.Some other issues have been solved in 2.4.4_1 too, like the unbound memory leak.
-
Excellent!! Thanks a lot!!