Limiters cutting off all WAN traffic (internet) while LAN traffic piles up

smicschool

Help! I've been struggling for weeks with getting limiters to work in my school, thank you in advance for helping:

We have a 20Mbps and 4Mbps WANs that are getting bogged down due to no bandwidth management.

I'm using pfSense as a transparent bridge to limit traffic, whenever I enable limiters, virtually all my WAN traffic gets cut off (internet becomes very slow, but I can still ping) and LAN traffic climbs immediately to 20 Mbps. What would cause this?

Here is my setup:

I'm using version 2.1 (same problem in 2.0.3), my layout: WAN1, WAN2 –- Cisco 2900 --- pfSense --- LAN

net.link.bridge.pfil_member = 1
net.link.bridge.pfil_bridge = 0
Manual Outbound NAT rule generation = checked
DHCP off
bridged WAN-LAN
static IP address on LAN
add standard allow rules on WAN and LAN (DNS, HTTP, HTTPS, SSH etc), no rules on bridge OPT1
setup two limiters: UploadLimit 400kbit/s with source mask and DownloadLimit 500kbit/s with destination mask
put limiters on LAN as the top rule above "Default allow LAN to any" (In/Out = UploadLimit/DownloadLimit)

Things I've tried to no avail:

using either the limiter itself or its child queue in the rule (should I be putting the limiter or the child queue?)
adding either two or four child queues (eg http://forum.pfsense.org/index.php?topic=40542.0)
setting limiter queue size to 1000 or 10000
permutations of net.link.bridge.pfil_member/net.link.bridge.pfil_bridge
using traffic shaping wizard

I've read hundreds of posts/docs and tried hundreds of permutations without success, please help!

Thank you!!

smicschool

OK, I'm narrowing down the problem:

It turns out that when I turn on the limiter rule, the upload traffic gets completely cut off. A speedtest will show the correct download speed, but the upload test will always fail.

However, if I set my upload limiter to a very exagerated amount, such as 10 Mbps, I can squeeze about 0.5 Mbps through in speedtest.

So my pfSense can limp through the day with this wrong setting, but I hope to get to the root of the problem.

Any thoughts for this newbie?

Thank you!

supermega

Hi smicschool

The problem is that the upload speedtest runs into a timeout because the upload speed is extremly slow. Thats the reason because you can ping

I opend up a thread with the same problem but I have this issue only with ipv6 based rules.
http://forum.pfsense.org/index.php/topic,66828.0.html

But it doesn't matter if IPv4 or IPv6. I think there is a problem in generaly with the limiters.

regards

supermega

smicschool

Thank you for the suggestion!

I tried to google the slow upload reason and came up with a few things: duplex mismatch, bad cables, MTU mismatch etc.

But nothing seems to suggest that there's a problem with upload limiter (dummynet) itself.

I hope to try to change NIC cards and cables and troubleshoot more.

Anyone else has suggestions?

supermega

Here are some good informations and tuning options for network cards.

https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

shinzo

While i am no expert too much queue is not a good thing, packets need to be dropped from time to time. I have one rule that gives everyone on the LAN x amount of bytes for down and up.

The setup for 1mb_UPLOAD limiter is bandwith 1 mbit/s burst 0 with mask source addresss
and the setup for 7mb_DOWNLOAD is bandwith 7 mbits/ burst 0 with mask destination address

then The floating rule i have is like this
Interface is LAN, Direction is IN, Source is lan subnet, destination is any. In/out = 1mb_up / 7mb_down
With this set up all LAN connections made to WAN are limited to x bytes per second and cannot go over no matter what.

Burst cannot be blank in 2.1 so you can choose 0 or any number.

IF this setup doesnt work, set net.link.bridge.pfil_bridge=1

Also from the looks of it you aren't putting the rule as a floating one. Remove the LAN rule you created and create it in floating rules and report back. The limiter info will show if the limiter is working or not.

kejianshi

I agree with shinzo - I don't use traffic shaping, but I've seen this complaint. Dropping packets properly is a big help if thats not being done correctly. Otherwise you just end up with an enormous back log of forgotten irrelevant packets.

smicschool

Thank you all supermega, shinzo and kejianshi for your kind suggestions, but I haven't solved the problem.

I looked at the tuning cards link but it didn't have the cards I'm using (re and msk cards).

I also tried shinzo's suggestions but it wasn't able to limit. It went wide open to 20Mbps/6Mbps. (I tried both net.link.bridge.pfil_bridge=1 and 0). I've tried different permutations of putting limiters on LAN/WAN/OPT1 or pairs of child queues on both LAN/WAN to no avail.

I also changed cables and added another brand new NIC card and tried different permutations of interface assignments to no avail.

If I disable all the rules (pfctl -d), the upload speed becomes normal (6Mbps), so I think it might a problem with my rules/settings/pfSense (probably not hardware).

I'm open to more suggestions, thank you all again, much appreciated!!