@snitem

Limit every vlan (each for 1 appartement)
To 60 down and 9 up

Set the bloat limiter
On your wan with floating rule to the exact up / down you get for your wan connection

A fair method for all users
They have a 60/9 connection protected by a pfS

And you can also tweak the limiters with a time based scheduler

But be aware limiters on 2.6 CE
Are a bit well as far as I know not working

BR NP

@michmoor Normally that faster connection is the primary WAN (its failover, so it only uses the cable until it goes down) but it's still crap because the upload is so low. 50Mbps doesn't go very far when you have multiple machines fighting for it.

And then when my cable ISP (annoyingly often) goes out and it fails over to running off Starlink, and in some cases (like my work laptop) I can't control it to separate the backup traffic from "needs to work all the time" traffic because I'm not an admin and they set it up to run everything over VPN that pfsense can't see the content of, so I need to find some alternative way to prioritize per-host.

@abel406 said in Limiter not work:

guess that's one of the handycaps of using it for home or lab

That's not a thing. I suggest starting a new thread about what isn't working, if it's not limiter related?

@steveits said in Wizard causes bug on VOiP/SIP:

There is this bug which is marked as fixed in 22.05 and the upcoming 2.7

I have read it. That seems exactly like this bug.

Strangely, I had the VOIP prior to 2.6.0 so this bug was not in 2.5.x and was reintroduced in 2.6.0

Limiter:
WAN Down
cfa1be8b-07b8-42e5-bca8-75536c23c63e-image.png
71d93f82-ec99-4638-b410-1e1a3a3f2ea8-image.png
WAN Down Queue
8207ed47-31c6-4d06-b248-139cf2a2aee8-image.png
WAN Up
6abd8191-66bf-40fa-8140-b7c9f5801489-image.png
7b94ffcd-b857-48cb-88d8-719a524ef471-image.png
WAN Up Queue
552ee1ae-3ee0-4429-bdff-65130be02653-image.png
Floating Rule:
c1e705c4-ae8c-4a8b-9ac9-1cd89f303655-image.png
3a563a94-1400-4c29-98de-7779e30632b7-image.png
f19765d6-71bc-41ec-b5fc-d64b21a0d300-image.png

@steveits

I think I got it. I will try. Thank you very much.

@svaldes Yes. The easiest way is probably to run the wizard and put that server IP in as a VoIP server. Note you can always rerun the wizard, or create your own floating rules, rename rules/queues created, etc.

@racing_shadows Thanks for sharing. I did not find anywhere to set bandwidth for a que, just the interface. Also, I use PRIQ.

Confused about this one thing, with a 290 main queue and a 225 queue for everything else, why do I still get latency? Even if the 225 is saturated, there should be enough bandwidth where the main queue isn’t jammed no?

Or another way to put it, shouldn’t the queue only take into effect if the threshold is exceeded? I mean that literally and not literally, I don’t know how it technically works, but I would assume the queue would flow naturally, if I have 10 checkout lanes in a supermarket, and I have 8 people waiting, that shouldn’t cause a jam?

@lonnie said in Ackqueue versus Queue:

downloading a little faster if I set Ackqueue to a higher priority queue than the regular queue

Basically, yes. The idea is the ACKs get sent out as fast as possible so the web server streams the download more consistently. The shaping wizard does this by default.

This may help:
https://www.slideshare.net/NetgateUSA/traffic-shaping-basics-with-priq-pfsense-hangout-february-2016

I think that's from this?
https://www.youtube.com/watch?v=it_5xvC28vs&ab_channel=Netgate

well good news and bad news, it seems everything is looking good in terms of FQ_CODEL for all real physical clients, and for virtual machines but not for docker containers.

I run Pfsense in a virtual machine, on my esxi host as well as unraid in a virutal machine (both on the same physical host) and the unraid virtual machine runs docker containers.

esxi host
pfsense vm || unraid
docker
containers

I have 2 physical ethernet ports on the esxi host, so I dedicated one for WAN and one for LAN for easy segregation. pfsense being the sole VM with access to WAN vswitch

In order for docker to use subnet addresses for containers (hosting them at different addresses than the host address) requires macvlan or ipvlan network settings in docker. I opted for macvlan as some random internet article mentioned that cpu overhead was higher using ipvlan than macvlan.

Now the way in which macvlan is done is through mac for forging the mac address. The security settings in esxi virtual networking permit this so I was forced to enable promiscuous mode and forged transmits, placing the docker host on its own port group within the same vswitch.

This fixed the docker address issue but created problems in relation to FQ_CODEL as now when traffic was coming in destined for docker, it didn't appear to journey through the pfsense vm first. I thought it was strange that a rule which usually has 2k+ states created, only had 7 but didn't really think much of it at the time.

i've now changed to ipvlan docker network setting but this hasn't resolved the issue. I have disabled promiscuous mode and forged transmits, and placed the docker host back on the same port group within the same vswitch. Traffic rules seem to all properly be flowing through pfsense now but I still have it bypassing FQ_CODEL for some reason.

At this point i'm not really sure where to go from here, it could be pfsense it could be docker, it could be my own user error. What is a strange observation is that it sometimes works.

For example starting a large downstream file transfer (for now I have set up quite low artificial limits of 50mbps down and 15 mbps up so I can easily observe if its clamping appropriately)

alt text
for a time it did clamp appropriately (although at a lower rate than it should) then it progressively gets higher. If the rules didn't exist then it would have started off at 600 mbps. During the times where its clamped at 25mbps (even though it should be 50) the traffic was visible and updating in the firewall rules. After it went beyond that the left hand counter stopped going up.

Okay returned to this after a couple of hours of head scratching and found the culprit. The half speed is a result of lan side rules + floating rules, which in effect double up and cause half bandwidth. Disabling one of them resolved that part.

Since this is a service which required port forwarding I had an additional rule (i don't use floating rules, i prefer the old rule style of applying them to lan interfaces with a single lan to lan rule at the top which doesn't use fq_codel). It appears that a racing condition was happening, in that traffic meant for the VPN was split between 2 different rules (the LAN rule and the port forwarding rule on the VPN interface). This was what resulted in the weird behaviour of it clamping for a time and then not. Once a significant amount of clamping had occurred, traffic was flowing through the VPN interface rule which didn't have FQ_CODEL limiters placed on it.

Why this behaviour occurs I have no clue, its a fun little racing condition. When you add them both together, it results in the correct amount of traffic flowing through the interface too.

alt text
Lan side
alt text
VPN side

This all occurred on PFsense CE 2.6 and now at this point I think I can comfortably say everything is working as normal.

In my foolishness I sidegraded / upgraded to pfsense plus 22.05 and this caused the behaviour to perform differently / poorly.

In CE 2.6 FQ_CODEL clamping happens instantly. In 22.05 it was delayed, which caused latency spikes until it got it under wraps.

alt text

Might not be clearly visible, but you can kind of see the sawtoothing occurring in 22.05.
Its more clearly visible on the 1h resolution but only shows 1 instance of it occurring.
alt text

So thats it! i've now got a working setup with pfsense 2.6.0 and i'll stick to this now until something major comes down the pipelines or some bugs exist.