Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Pfsense freeze at DDoS attack - Tuning?

    Scheduled Pinned Locked Moved Firewalling
    68 Posts 10 Posters 23.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      lowprofile
      last edited by

      Hi

      When i got hit by various ddos attack types, e.x SSDP at 400mbit (have 1gbit connection) the firewall looses packets.
      A constant ping shows every 5th ping is getting timeout. Only for a second, hence packetloss. I am having ddos service from providor but they only guaranteed 100mbit clean traffic.

      Why does pfsense freezes/doesn't respond for an attack which is half the bandwidth? It is struggling at 4-500mbit incoming traffic. I think i need to tune somewhere…

      Some good hints and advice? I would appreciate it so much!

      My box:

      Supermicro
      i3, 2.8ghz 2/4 core.
      16GB ram
      120 SSd disk
      1Gbit NIC (quad)
      1Gbit uplink

      thanks in advance!

      1 Reply Last reply Reply Quote 0
      • S
        Supermule Banned
        last edited by

        Skift udbyder Anders og det har ikke noget med tuning at gøre ;)

        1 Reply Last reply Reply Quote 0
        • L
          lowprofile
          last edited by

          I have new information now…

          It is now confirmed that the bottleneck is PFsense in this case. It seems like 1 special attack does freeze the box. I made a packet capture and got this:

          84 38.733142 119.246.140.4 76.28.11.29 TCP 60 46288→80 [SYN, ECN, CWR] Seq=0 Win=0 Len=0
          88 38.733361 76.28.11.29 119.246.140.4 TCP 58 80→46288 [SYN, ACK, ECN] Seq=0 Ack=1 Win=14600 Len=0 MSS=1460

          What kind of attack is this? and how to stop it? I smell some SYN attack? It seems like pfsense tries to reply the "request", hence the freeze. (many of these lines)

          My "ip": 76.28.11.29 (faked)

          1 Reply Last reply Reply Quote 0
          • H
            Harvy66
            last edited by

            What are your WAN firewall rules? By default, PFSense should be dropping incoming data, not responding to it.

            1 Reply Last reply Reply Quote 0
            • L
              lowprofile
              last edited by

              WAN rules are mostly open with all proto and "any" "any" to several servers. Some customers are running "unmanaged" so everything is opened on their IP. They got hit on port 80, webservers  or my dns servers on port 53 (no open resolvers)
              I will try to play with synproxy, i have to get it fixed since i do have anti-ddos by providor but pfsense can't even handle small syn flood. I will try this guide. http://pfsensesetup.com/syn-flood-prevention-in-pfsense/

              1 Reply Last reply Reply Quote 0
              • S
                Supermule Banned
                last edited by

                When tested yesterday on a 100mbit homelink the load looked like this on pfsense.

                It dropped packages (ping) but remained online and did not hang at any moment.

                In production, I dont have a netgear CG3000 infront and I believe some of the loss could be accounted for by that.

                I dont get why you run any/any inbound since it opens up for a wide range of ways to hammer the backend servers.

                DDoS.PNG
                DDoS.PNG_thumb

                1 Reply Last reply Reply Quote 0
                • K
                  kejianshi
                  last edited by

                  You are running with your firewall wide open on the WAN?  Why would you do such a thing?

                  1 Reply Last reply Reply Quote 0
                  • D
                    doktornotor Banned
                    last edited by

                    @kejianshi:

                    You are running with your firewall wide open on the WAN?  Why would you do such a thing?

                    Honeypot? Works well, apparently. :D :D :D

                    1 Reply Last reply Reply Quote 0
                    • L
                      lowprofile
                      last edited by

                      @kejianshi:

                      You are running with your firewall wide open on the WAN?  Why would you do such a thing?

                      It is not wide open. It is open according to the need from customers. I have some virtual servers running (hosting)
                      Open or not. It should not be the reason :-) You can give me any ip (send me PM) where pfsense is in front and i would be able to make it freeze by sending 1 special attack. If anyone would like to try, send me pm.

                      1 Reply Last reply Reply Quote 0
                      • K
                        kejianshi
                        last edited by

                        Sure - Try pfsense.org

                        1 Reply Last reply Reply Quote 0
                        • L
                          lowprofile
                          last edited by

                          @kejianshi:

                          Sure - Try pfsense.org

                          down  ;)

                          1 Reply Last reply Reply Quote 0
                          • K
                            kejianshi
                            last edited by

                            Well - If its truly an issue, I'm sure they will be the ones who want to know, so seems like a logical choice.

                            1 Reply Last reply Reply Quote 0
                            • L
                              lowprofile
                              last edited by

                              Supermule will confirm…. when he is online.... He also wanted to test  :)

                              1 Reply Last reply Reply Quote 0
                              • K
                                kejianshi
                                last edited by

                                I need mine up - So I'd rather not be the test subject.

                                1 Reply Last reply Reply Quote 0
                                • L
                                  lowprofile
                                  last edited by

                                  @kejianshi:

                                  I need mine up - So I'd rather not be the test subject.

                                  Trust me. We all want.  :) Though i only did it for some seconds. i will try different scenarios and return

                                  1 Reply Last reply Reply Quote 0
                                  • T
                                    Tikimotel
                                    last edited by

                                    Have you looked at Calomel.org for tuning tips?

                                    https://calomel.org/freebsd_network_tuning.html

                                    Sysctl.conf (so most option can be set by adding them to the system tunables menu option)

                                    # General Security and DoS mitigation
                                    #net.bpf.optimize_writers=0           # bpf are write-only unless program explicitly specifies the read filter (default 0)
                                    #net.bpf.zerocopy_enable=0            # zero-copy BPF buffers, breaks dhcpd ! (default 0)
                                    net.inet.ip.check_interface=1         # verify packet arrives on correct interface (default 0)
                                    #net.inet.ip.portrange.randomized=1   # randomize outgoing upper ports (default 1)
                                    net.inet.ip.process_options=0         # ignore IP options in the incoming packets (default 1)
                                    #net.inet.ip.random_id=1              # assign a random IP_ID to each packet leaving the system (default 0)
                                    net.inet.ip.redirect=0                # do not send IP redirects (default 1)
                                    #net.inet.ip.accept_sourceroute=0     # drop source routed packets since they can not be trusted (default 0)
                                    #net.inet.ip.sourceroute=0            # if source routed packets are accepted the route data is ignored (default 0)
                                    net.inet.ip.stealth=1                 # do not reduce the TTL by one(1) when a packets goes through the firewall (default 0)
                                    #net.inet.icmp.bmcastecho=0           # do not respond to ICMP packets sent to IP broadcast addresses (default 0)
                                    #net.inet.icmp.maskfake=0             # do not fake reply to ICMP Address Mask Request packets (default 0)
                                    #net.inet.icmp.maskrepl=0             # replies are not sent for ICMP address mask requests (default 0)
                                    #net.inet.icmp.log_redirect=0         # do not log redirected ICMP packet attempts (default 0)
                                    net.inet.icmp.drop_redirect=1         # no redirected ICMP packets (default 0)
                                    #net.inet.icmp.icmplim=500            # number of ICMP/TCP RST packets/sec, increase for bittorrent or many clients. (default 200)
                                    #net.inet.icmp.icmplim_output=1       # show "Limiting open port RST response" messages (default 1)
                                    #net.inet.tcp.always_keepalive=0      # tcp keep alive detection for dead peers, can be spoofed (default 1)
                                    net.inet.tcp.drop_synfin=1            # SYN/FIN packets get dropped on initial connection (default 0)
                                    #net.inet.tcp.ecn.enable=1            # explicit congestion notification (ecn) warning: some ISP routers abuse ECN (default 0)
                                    net.inet.tcp.fast_finwait2_recycle=1  # recycle FIN/WAIT states quickly (helps against DoS, but may cause false RST) (default 0)
                                    net.inet.tcp.icmp_may_rst=0           # icmp may not send RST to avoid spoofed icmp/udp floods (default 1)
                                    #net.inet.tcp.maxtcptw=15000          # max number of tcp time_wait states for closing connections (default 5120)
                                    net.inet.tcp.msl=5000                 # 5s maximum segment life waiting for an ACK in reply to a SYN-ACK or FIN-ACK (default 30000)
                                    net.inet.tcp.path_mtu_discovery=0     # disable MTU discovery since most ICMP type 3 packets are dropped by others (default 1)
                                    #net.inet.tcp.rfc3042=1               # on packet loss trigger the fast retransmit algorithm instead of tcp timeout (default 1)
                                    net.inet.udp.blackhole=1              # drop udp packets destined for closed sockets (default 0)
                                    net.inet.tcp.blackhole=2              # drop tcp packets destined for closed ports (default 0)
                                    #net.route.netisr_maxqlen=2048        # route queue length (rtsock using "netstat -Q") (default 256)
                                    security.bsd.see_other_uids=0         # users only see their own processes. root can see all (default 1)
                                    
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • S
                                      Supermule Banned
                                      last edited by

                                      Problem is that enterprise boxes should be able to handle that amount of traffic.

                                      It times out at 75mbit using af specific sort of DDoS packages despite tuning.

                                      1 Reply Last reply Reply Quote 0
                                      • H
                                        Harvy66
                                        last edited by

                                        1. Careful with SYN proxy, it breaks  the window sizing, which means your max window is 64KB. High latency links will be crazy slow.

                                        2. Someone needs to test loading PFSense? I got a 50Mb connection, soon to be 100Mb.

                                        1 Reply Last reply Reply Quote 0
                                        • C
                                          cmb
                                          last edited by

                                          @Harvy66:

                                          1. Careful with SYN proxy, it breaks  the window sizing, which means your max window is 64KB. High latency links will be crazy slow.

                                          Yes don't use SYN proxy, that's a solution to a 1990s problem. Unless you have a device opened that's susceptible to 1990s problems of course (things with shitty IP stacks that don't have modern SYN flood mitigation built-in, talking '90s Linux versions, Windows NT 4 and earlier, etc).

                                          @Harvy66:

                                          1. Someone needs to test loading PFSense? I got a 50Mb connection, soon to be 100Mb.

                                          We've done and continue to do load testing. Things in 2.2 are faster than ever. Not as fast as we'd like, as we'd like to see 10 Gbps wire speed at 64 byte frames and will be doing work towards that goal, but there isn't a firewall in existence today that'll do that.

                                          @Harvy66:

                                          I got a 50Mb connection, soon to be 100Mb.

                                          That's all? :) We have two gigabit drops at our primary datacenter (and redundant 10G fiber run from there to our office next door), and max that out with ease. I have 300 Mb service at home, which an APU can max out with room to spare, much less anything faster. With any normal usage, multi-Gbps is achievable.

                                          @Supermule:

                                          Problem is that enterprise boxes should be able to handle that amount of traffic.

                                          A 400 Mb flood of 64 byte packets all opening new connections? That's hell on any firewall if you're passing the traffic. The lowest limit of any firewall is in its ability to handle new connections. 400 Mbps of SYN flood traffic is over 800,000 new connections/sec. Not sure about OP's specific combination of hardware but generally speaking that's probably close to double the most you're going to get through pf. The most expensive Cisco ASA maxes out at 350,000 new connections/sec, and costs $150K-300K+ USD per box depending on which features you license.

                                          So OP's effectively dealing with an attack that's more than twice as big as the biggest Cisco ASA that costs as much as a house could handle. Cisco's as "enterprise" as it gets. The fact pf can't deal with it either shouldn't surprise you.

                                          Firewalls in general are the wrong answer for DDoS. No matter what firewall it is, it's going to have a hell of a hard time dealing with huge floods of new connections.

                                          Block the flooded traffic and you'll be in as good of shape as you can be if it's hitting your firewall. Better when you're under an attack that big to have your upstream null route the IP that's being attacked. Or if you absolutely have to keep it up, send that through a DDoS mitigation provider first.

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            cmb
                                            last edited by

                                            @Supermule:

                                            It times out at 75mbit using af specific sort of DDoS packages despite tuning.

                                            On what hardware? What specifically are you running to throw traffic at it?

                                            75 Mbps of 64 byte frames is over 150,000 pps (which if it weren't all new connections wouldn't be a big deal with fast enough hardware), and 150,000 new connections/sec of traffic you're passing with any firewall requires some serious horsepower.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.