Limiters and floating rules
-
I have a multi-wan setup and would like to enable FQ_CODEL on the WAN interfaces.
To achieve this I have created limiters with child queues for each upload/download limit for each WAN. I have then created a number of MATCH floating rules on each WAN interface to assign the upload/download queues to traffic on those WAN interfaces.
My problem is that the queuing of inbound (download) packets on the WAN does not seem to function correctly for packets matched by floating rules. This shows up as dropped ping packets when the download speed reaches the limiter speed. I know it is the limiter dropping the packets as I can see the ping responses come in on the WAN interface using packet capture, but they are not forwarded to the LAN interface. Upload or outbound traffic on the WAN seems to work closer to what is expected. Ping times are more in line with what I would expect for FQ_CODEL, however there are still one or two dropped ping packets during the upload speedtests.
I have tried different variations of the above setup, such as using PASS floating rules with QUICK flag set, etc. Any floating rule variation I have tried has displayed the same characteristics.
However, if I assign the traffic shaping queues to the Allow all from LAN rule in the LAN zone, the traffic shaping works much more like I would expect for FQ_CODEL. There are no dropped ping packets when the download speed reaches the limit, the download speed is capped at the limiter speed, and there are no dropped pings during upload either.
My problem is due to the mulit-wan aspect of the setup, I cannot have the traffic shaping assigned on the Allow from LAN rule as that rule will automatically fail between two gateways with different link properties.
Is anyone seeing similar issues with limiters?
The best I think I seem to be able to achieve using limiters is only having the primary WAN link working correctly with shaping, or not being able to shape download traffic and flaky shaping of upload traffic. Both present their own issues with our desired setup.
-
Your best bet is to establish a gateway group. Put the WAN gateways in the group and and give them the correct Tier. You can now set a gateway group as a default gateway in 2.4.4. Then under your LAN rule you can change the gateway to the group and let it do the failover for you that way.
So if you have say 3 WAN's - WAN1 WAN2 and WAN3 . Make the following groups:
WAN_GW_ALL - WAN1-3 Tier1 on all with Packet Loss / High LatencyPFSense will round robin traffic now across all 3 WAN's unless packet loss / high latency or a down event removes a GW.
Make sure under General Setup you assign a DNS server to each WAN. Typically I do this:
WAN 1 - 8.8.8.8 / 4.2.2.2
WAN 2 - 8.8.4.4 / 4.2.2.3
WAN 3 - 4.2.2.4 / OpenDNS server or ISPThis is similar to how I set it up for my LAN parties but I create a few different groups and rules to send traffic to a specific WAN / modem.
-
I think you've misunderstood the problem. I don't have any problem with the gateway failover setup. This is working without issue.
My issue is I can only seem to get the traffic shaping limiters to work as expected if I apply it to the LAN interface allow rule that uses the failover gateway. The issue with this is my traffic shaping limiters are specific for each WAN upload/download speed, but I can only apply a single upload/download queue pair on this rule. When the gateway switches, the traffic shaping limits are no longer correct for the other WAN gateways as the gateways all run at different speeds.
To apply the correct traffic shapers to each WAN I created floating rules tied to each specific WAN interface. As mentioned in my post though, this doesn't seem to work correctly as it will drop pings completely if the download limit is reached and intermittently drop pings if the upload limit is reached, which is not how I would expect FQ_CODEL to react and not how it reacts if the queues are applied to the LAN interface rule.
-
That's not how it's supposed to react; some discussion is going on in the main/gigantic fq_codel thread in this sub-forum, and a bug report was put in as well. FYI, I have seen this in 2.4.3 testing this patch as well. For now, I've made my rules not classify ICMP traffic, and reset states prior to testing again.
-
@mattund said in Limiters and floating rules:
For now, I've made my rules not classify ICMP traffic,....
Can you explain (or show gui snapshot) how you did that? I have a web-based power strip that pings connected equipment and power cycles them if they don't respond. With FQ_CODEL pings don't come through when pfsense is under heavy load and the equipment is power cycled unnecessarily. It sounds like you have worked out the solution to my problem, but I am not sure how to implement your suggestion.
Thank you in advance!
-
@revengineer said in Limiters and floating rules:
@mattund said in Limiters and floating rules:
For now, I've made my rules not classify ICMP traffic,....
Can you explain (or show gui snapshot) how you did that? I have a web-based power strip that pings connected equipment and power cycles them if they don't respond. With FQ_CODEL pings don't come through when pfsense is under heavy load and the equipment is power cycled unnecessarily. It sounds like you have worked out the solution to my problem, but I am not sure how to implement your suggestion.
Thank you in advance!
+1
-
I've made my floating rule classify only TCP/UDP traffic, and not "All" protocols
This excludes things like ICMP from being touched by FQ_CODEL. If traffic other than TCP/UDP, but not ICMP is a concern to you, you may want other floating rules for those protocols as well.
-
@mattund said in Limiters and floating rules:
I've made my floating rule classify only TCP/UDP traffic, and not "All" protocols
This excludes things like ICMP from being touched by FQ_CODEL. If traffic other than TCP/UDP, but not ICMP is a concern to you, you may want other floating rules for those protocols as well.
Got it, that makes sense. I thought there was a way to specify all minus ICMP. Follow up question: Did you reduced the bandwidth for the CoDel traffic to reserve bandwidth for the non TCP/UDP traffic? Or is this typically in the noise and does not matter?
-
I count it as "in the noise" per-say, and let it pass my mind. I find ICMP traverses the network leisurely anyway, and besides, I haven't found a way to NOT drop the traffic -- if ICMP is getting dropped a lot of other problems can seep in...
-
@mattund said in Limiters and floating rules:
I count it as "in the noise" per-say, and let it pass my mind. I find ICMP traverses the network leisurely anyway, and besides, I haven't found a way to NOT drop the traffic -- if ICMP is getting dropped a lot of other problems can seep in...
Great, thanks. I made the change and confirm that I can ping my cable modem connected to WAN even under full bandwidth load.
