Limiter blocks internet access (Squid transparent proxy)

  • Banned

    @gringo13:

    What is problem

    It is broken! Did you read the thread?

    @gringo13:

    and what can i do?

    Ditch the proxy, or wait, or get debugging and coding.


  • @doktornotor:

    @gringo13:

    What is problem

    It is broken! Did you read the thread?

    @gringo13:

    and what can i do?

    Ditch the proxy, or wait, or get debugging and coding.

    Problem is at the same time transparent mode and traffic shapper doesnt works.
    If i disable limiter then no block internet. But i enable limiter block internet.
    Or i disable transparent mode and enable limiter then works fine but doesnt work filter.

    What do I need to work both at the same time?

  • Banned

    @gringo13:

    What do I need to work both at the same time?

    Go re-read the previous reply a couple of times.


  • This issue persists on 2.2.2? Oh Crap :(


  • I also noticed this yesterday. After limiters added to pass all rule and logging enabled, the rule blocks all traffic for that interface and fills up the System logs.

  • Banned

    Your "fills up the System logs" non-issue has nothing to do with the topic here. When you log ALL passed traffic, then yeah, your logs are going to fill up, limiters or not.


  • @doktornotor:

    Hmmm? Not really sure how's this related to unbound, or even any resolver at all? When I put limiters on a NAT firewall rule, the traffic stop flowing. As simple as that.

    This should be fixed in 2.2.3 snapshots.

  • Banned

    Thanks, will test as soon as nanobsd becomes usable again…  :D


  • @ermal:

    @doktornotor:

    Hmmm? Not really sure how's this related to unbound, or even any resolver at all? When I put limiters on a NAT firewall rule, the traffic stop flowing. As simple as that.

    This should be fixed in 2.2.3 snapshots.

    I am seeing this problem on 2.2.3-DEVELOPMENT (amd64) built on Fri Jun 19 14:25:29 CDT 2015 FreeBSD 10.1-RELEASE-p13.  No traffic with limiter and transparent proxy.

  • Banned

    Yeah this is still broken. Don't use limiters on NAT.

    https://redmine.pfsense.org/issues/4596
    https://redmine.pfsense.org/issues/4590


  • Still not working on Pfsense 2.2.3 final release. I need both, limiter and  transparent squid proxy to work together for my scenario.

    Regards,

    Nabeel


  • I have been having this problem also. It's a BIG problem actually for me. Does anyone know if it's been fixed yet, and if not if it's been brought to the developers attention ?

  • LAYER 8 Netgate

    Apparently the changes to fix this are significant so they have pushed it to 2.3.  I see they're planning a 2.2.5 first so you're looking at months (at least) before limiters are usable again. Use 2.1.5 and hope no significant vulnerabilities appear since they have stated they will not be patched.  Or evaluate other options, as I am.

    2.2 is, for the most part, useless if you rely on dummynet limiters.


  • @doktornotor:

    Well then stick with 2.1.5 until fixed.

    Can any1 share 2.1.5 v pfsense usb image ?

  • LAYER 8 Netgate

    That's a pretty good question.

    I just clicked around and couldn't find a 2.1.5 download.

    You might want to start thinking about other products/distros if you can't wait months for the functionality you need.

    I <3 pfSense but this limiter shit is getting old.

  • Banned

    @Derelict:

    That's a pretty good question.

    I just clicked around and couldn't find a 2.1.5 download.

    You clicking skills suck.  ;D :P

    Just click on the "Just show me the mirrors" on the download page. Select one, and go to "old" dir.

  • LAYER 8 Netgate

    Didn't see the old dir.  Knew it was there somewhere.  Thanks.


  • SOLVED*

    I managed to find a simple fix. All I needed to do was create a pass all firewall rule on the (LAN) interface for port 3128 (my proxy port).

    IPv4 TCP * * * 3128 * none   Rule to allow transparent proxy to work

    It worked and the speed limiter still works also.


  • @Alfanetindo:

    SOLVED*

    I managed to find a simple fix. All I needed to do was create a pass all firewall rule on the (LAN) interface for port 3128 (my proxy port).

    IPv4 TCP * * * 3128 * none   Rule to allow transparent proxy to work

    It worked and the speed limiter still works also.

    anyone else tested this ?


  • @Abhishek:

    @Alfanetindo:

    SOLVED*

    I managed to find a simple fix. All I needed to do was create a pass all firewall rule on the (LAN) interface for port 3128 (my proxy port).

    IPv4 TCP * * * 3128 * none   Rule to allow transparent proxy to work

    It worked and the speed limiter still works also.

    anyone else tested this ?

    Limiter still not working!


  • I can confirm that the issue solved 100 %

    My configuration :

    1.  Pfsense Version :  2.2.4-RELEASE (amd64)
    built on Sat Jul 25 19:57:37 CDT 2015

    2.  Packages Installed :  A.  Squid :  2.7.9 pkg v.4.3.6  ( Do not install squid3 – its very buggy )
    b.  Squidguard : 1.9.14  -- squid configured as a transparent proxy on lan interface  - rest are default settings.

    3.  Memory : 1 GB

    4.  Bandwidth Available :  4 MB

    5.  Limiter applies for testing :  only to 1 ip  ( 256 kb download and 1 mb upload )

    6.  Result  tested with speed.net  (  Worked exactly as expected )

    7.  All test carried when no one else using internet ( doubly confirmed )

    Please mark that this issue is fully resolved.

    Kudos and special thanks to  Alfanetindo  for a simple but a great solution.

    Steps need to be taken...

    1.  Following rule must be first rule

    IPv4 TCP    *    *    *    3128    *    none        Rule to allow transparent proxy to work

    2.  Then you can apply the limiter rule.







  • the order of the actual pf rules must be the issue then, perhaps someone can post the pf rules of working 2.1.5 and not working 2.2.x


  • Not "solved" and the rule change does not "solve" it. Looks like it just bypasses the limiter.

    Tried on 2.2.4, squid 3 (what was installed, has not been transparent since I decided that limiter fairness beat the heck out of squid caching if I had to pick only one of those) - traffic limited at 10 and running 10.6 shot above 12, quality shot from 40 to 1500 ms.

    Uninstalled squid 3, installed 2.7.9.

    Traffic again shot above 12, quality went to 400, then 1200 ms.

    Turned off transparent and disabled firewall rule. Traffic remained high, quality low, so I reset states as well to flush it out.

    Back to 10.4 and 27 ms.

    Guess I'll have to find a second box to run an independent squid instance between pfSense and the rest of the LAN, since this is not remotely working (on older versions I could have both work, but only when cache hits were shaped, which was NOT the point, and the workarounds some claimed to work for that always left me with a locked up system and no network access.

    I have been running the limiter (and basically no squid, or only non-transparent squid which is functionally like no squid) since last spring with excellent results on getting fairness while allowing most of the BW to be used (one user gets it all (minus limiter overhead to make the limiter work at all), two users share evenly, 80 users share evenly) and holding quality to a reasonable level.

    "Quality to a reasonable level" is basically tuning the main limiters' in/out values that are then divided among users.


  • Finally the only way to fix this was installing the old version of pfsense 2.1.5. I tested with squid transparent mode, dansguardian and Limiters and everything works fine. I was reading the pfsense Digest and there are many security issues and bugs from the old version 2.1.5 to the last version 2.2.4, like a multiple Cross-Site Scripting (XSS) vulnerabilities were found in the pfSense WebGUI, and OpenSSL “FREAK” vulnerability (If packages include a web server or similar component, such as a proxy, an improper user configuration may be affected. Consult the package documentation or forum for details.)

    My question, is there any secure way to keep this old version for remote access?

    Regards!


  • That's a pretty strange technical debate here about rule handling access to port 3128 while idea is to use transparent proxy which is, by design, implemented in such a way that proxy port is unknown browser side and accessed only internally.

    Not reading even further, when I saw such proposal in term of FW rule associated with transparent proxy, I was…  :o ???....  ::)

    If issue is with transparent proxy only, why don't you move to explicit proxy with is definitely far better, in any case?


  • Explicit proxy is fine for my fixed machines that won't be on another network; and it's set up on them, in fact.

    Setting up explicit proxy on mobile machines tends to break them when they go elsewhere. The user base not being all that savvy, various possible schemes of network settings to implement explicit proxy here that they would change away from when elsewhere might work for 2% of them. And it would be a pain for that 2%, even - Oh, I switched networks. Now I need to switch network settings. Oh, Joy.

    Auto Proxy discovery is a delightfully kludgy old process (netscape - that brings back memories) and not turned on by default for most systems.

    So, for effective proxy that actually works for the majority of a mobile user-base, transparent is useful (when it works.)

    Your environment may differ.


  • Also just want to point out that limiter also break NAT Reflection mode for port forwards  :-[


  • Has it been solved for the new version 2.2.4?


  • nah not sure  maybe for 2.2.5 :)

    I would love to have limiter to work with NAT reflection

  • LAYER 8 Netgate

    As far as I know this problem is punted to 2.3, unfortunately.


  • So on 2.2.2 Limiter does not have any issue with NAT reflection? on 2.2.4 still theres issues

  • LAYER 8 Netgate

    I think it's 2.2.X.


  • @JDvD:

    Has it been solved for the new version 2.2.4?

    I'm having the problem on 2.2.4, so, no.


  • Hi there,

    I have got the same problem. Version 2.2.4 (64Bit) does not work with transparent proxy anymore. In version 2.1.5 it worked fine. In that version (2.1.5) it was also possible to change the port of squid to a port beneeth 100. This is not working in 2.2.4 aswell.

    I guess this must be a bug.  ??? :-\


  • 7 months later and this issue has not been addressed yet?…not complaining tho', SmallWall has kept me happy so far.
    I hope this issue will be addressed tho' would like to use pf.


  • Hello,

    after updating to 2.2.5 the bug ist still there. traffic-shaping does not work with proxy in transparent mode.

    :-[


  • Ok, I have not tried it with the new version (2.2.5).
    I also see in several post that there is a confusion, let's clarify this, the Limiter + Transparent Proxy not work, but, Limiter + Proxy NO-Transparent, work?

    I think it's the same problem for all Traffic Shaper.

  • Banned

    This entire topic has nothing to do with proxy. Limiters are (still) broken when applied to any NAT firewall rules; this is nothing specific to transparent Squid. On 2.2.x, and I cannot see any difference on 2.3 either. Broken as in dropping traffic -> unusable.

    https://redmine.pfsense.org/issues/4326


  • I meant to the subject of title (Limiter + Proxy), but, you have made it clear that it is a generalized problem from the NAT firewall rules. Thank you doktornotor


  • JAJAJA NO SOLUTION … back to 2.0.3 and fix it