Limiters cutting off all WAN traffic (internet) while LAN traffic piles up

[smicschool]

Help! I've been struggling for weeks with getting limiters to work in my school, thank you in advance for helping:

We have a 20Mbps and 4Mbps WANs that are getting bogged down due to no bandwidth management.

I'm using pfSense as a transparent bridge to limit traffic, whenever I enable limiters, virtually all my WAN traffic gets cut off (internet becomes very slow, but I can still ping) and LAN traffic climbs immediately to 20 Mbps. What would cause this?

Here is my setup:

I'm using version 2.1 (same problem in 2.0.3), my layout:  WAN1, WAN2 –- Cisco 2900 --- pfSense --- LAN

net.link.bridge.pfil_member = 1
    net.link.bridge.pfil_bridge = 0
    Manual Outbound NAT rule generation = checked
    DHCP off
    bridged WAN-LAN
    static IP address on LAN
    add standard allow rules on WAN and LAN (DNS, HTTP, HTTPS, SSH etc), no rules on bridge OPT1
    setup two limiters: UploadLimit 400kbit/s with source mask and DownloadLimit 500kbit/s with destination mask
    put limiters on LAN as the top rule above "Default allow LAN to any" (In/Out = UploadLimit/DownloadLimit)

Things I've tried to no avail:

using either the limiter itself or its child queue in the rule (should I be putting the limiter or the child queue?)
    adding either two or four child queues (eg http://forum.pfsense.org/index.php?topic=40542.0)
    setting limiter queue size to 1000 or 10000
    permutations of net.link.bridge.pfil_member/net.link.bridge.pfil_bridge
    using traffic shaping wizard

I've read hundreds of posts/docs and tried hundreds of permutations without success, please help!

Thank you!!

[smicschool]

OK, I'm narrowing down the problem:

It turns out that when I turn on the limiter rule, the upload traffic gets completely cut off. A speedtest will show the correct download speed, but the upload test will always fail.

However, if I set my upload limiter to a very exagerated amount, such as 10 Mbps, I can squeeze about 0.5 Mbps through in speedtest.

So my pfSense can limp through the day with this wrong setting, but I hope to get to the root of the problem.

Any thoughts for this newbie?

Thank you!

[supermega]

Hi smicschool

The problem is that the upload speedtest runs into a timeout because the upload speed is extremly slow. Thats the reason because you can ping

I opend up a thread with the same problem but I have this issue only with ipv6 based rules.
http://forum.pfsense.org/index.php/topic,66828.0.html

But it doesn't matter if IPv4 or IPv6. I think there is a problem in generaly with the limiters.

regards

supermega

[smicschool]

Thank you for the suggestion!

I tried to google the slow upload reason and came up with a few things: duplex mismatch, bad cables, MTU mismatch etc.

But nothing seems to suggest that there's a problem with upload limiter (dummynet) itself.

I hope to try to change NIC cards and cables and troubleshoot more.

Anyone else has suggestions?

[supermega]

Here are some good informations and tuning options for network cards.

https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

[shinzo]

While i am no expert too much queue is not a good thing, packets need to be dropped from time to time.  I have one rule that gives everyone on the LAN x amount of bytes for down and up.

The setup for 1mb_UPLOAD limiter is bandwith 1 mbit/s burst 0 with mask source addresss
and the setup for 7mb_DOWNLOAD is bandwith 7 mbits/ burst 0 with mask destination address

then The floating rule i have is like this
Interface is LAN, Direction is IN, Source is lan subnet, destination is any.  In/out = 1mb_up / 7mb_down
With this set up all LAN connections made to WAN are limited to x bytes per second and cannot go over no matter what.

Burst cannot be blank in 2.1 so you can choose 0 or any number.

IF this setup doesnt work, set net.link.bridge.pfil_bridge=1

Also from the looks of it you aren't putting the rule as a floating one. Remove the LAN rule you created and create it in floating rules and report back.  The limiter info will show if the limiter is working or not.

[kejianshi]

I agree with shinzo - I don't use traffic shaping, but I've seen this complaint.  Dropping packets properly is a big help if thats not being done correctly.  Otherwise you just end up with an enormous back log of forgotten irrelevant packets.

[smicschool]

Thank you all supermega, shinzo and kejianshi for your kind suggestions, but I haven't solved the problem.

I looked at the tuning cards link but it didn't have the cards I'm using (re and msk cards).

I also tried shinzo's suggestions but it wasn't able to limit. It went wide open to 20Mbps/6Mbps. (I tried both net.link.bridge.pfil_bridge=1 and 0). I've tried different permutations of putting limiters on LAN/WAN/OPT1 or pairs of child queues on both LAN/WAN to no avail.

I also changed cables and added another brand new NIC card and tried different permutations of interface assignments to no avail.

If I disable all the rules (pfctl -d), the upload speed becomes normal (6Mbps), so I think it might a problem with my rules/settings/pfSense (probably not hardware).

I'm open to more suggestions, thank you all again, much appreciated!!