Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Haproxy - Traffic Limiting

    Scheduled Pinned Locked Moved pfSense Packages
    2 Posts 1 Posters 2.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      treuss
      last edited by

      Hi,

      we're currently setting up haproxy for traffic limiting. Our main motivation is to slow down bots/crawlers that stress our servers exceedingly.

      Our haproxy looks like this:

      
      frontend p2-web-80
        # Table definition  
        stick-table type ip size 512k expire 30s store conn_cur,conn_rate(3s),http_req_rate(5s)
      
        # Allow clean known IPs to bypass the filter
        #tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }
      
        # Shut the new connection as long as the client has already XX opened
        tcp-request connection reject if { src_conn_cur ge 10 }
        tcp-request connection track-sc1 src
      
        # Shut the new connection as long as the client has already 10 opened
        tcp-request connection reject if { src_conn_rate ge 10 }
        tcp-request connection track-sc1 src
      
        # Use General Purpose Couter (gpc) 0 in SC1 as a global abuse counter
        # Monitors the number of request sent by an IP over a period of 10 seconds
        tcp-request connection track-sc1 src
        tcp-request connection reject if { src_get_gpc0 gt 15 }
      
      backend
        # If the source IP generated 10 or more http request over the defined period,
        # flag the IP as abuser on the frontend
        acl abuse src_http_err_rate(p2-web-80) ge 10
        acl flag_abuser src_inc_gpc0(p2-web-80)
        tcp-request content reject if abuse flag_abuser
      
      

      We've tested these settings using siege, which worked pretty well, however, there are some open questions we haven't been able to find answers for:

      • Although Firefox uses only 6 concurrent connections, some GET-requests on images will be blocked by haproxy, if Firefox is forced to reload the page via CTRL+SHIFT+R. Why is that?

      • We need to maintain a whitelist (tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }) of corporate customers with clients hidden via NAT. Otherwise, obviously, their multiple requests will be blocked. Is it possible to maintain such a list on pfsense's GUI?

      • In our backend, we would like to handle abusers, tracked by the frontend p2-web-80. Unfortunately, when I activate the backend rules, I'm getting this error:

        
            [WARNING] 050/104454 (87620) : parsing acl keyword 'src_inc_gpc0(p2-web-80)' :
            no pattern to match against were provided, so this ACL will never match.
            If this is what you intended, please add '–' to get rid of this warning.
            If you intended to match only for existence, please use '-m found'.
            If you wanted to force an int to match as a bool, please use '-m bool'. 
        
        

        Is there a way to fix this?

      Best regards
      Thomas

      1 Reply Last reply Reply Quote 0
      • T
        treuss
        last edited by

        bump

        1 Reply Last reply Reply Quote 0
        • First post
          Last post
        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.