pfSense CPU Interrupts bottleneck during SYN FLOOD DDOS



  • Hello,

    I work for a hosting company and we run a Dell Poweredge with..

    Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz
    16 CPUs: 2 package(s) x 8 core(s)
    AES-NI CPU Crypto: Yes (active)

    On average, we have a constant 800k states, but when we get a Syn Flood DDOS attack we have 8 million. (10 million max)

    We are hitting a CPU Interrupts bottleneck, which puts our CPU at 99% and we start getting packet loss. Bandwidth is not a factor.

    We can block, no log, kill states on the subnets responsible and that brings it back down.

    We are implementing a DDOS solution, but in the meantime we want to throw more horsepower at the problem. I have seen debates about More Ghz vs more cores.

    But for the sake of the CPU interrupts bottleneck, would it help to have more cores? 32 or 64 with lower Ghz, or 16 Cores with 4.8Ghz (with Turbo)

    Thanks,

    Dan


  • Netgate Administrator

    Depends what limit you're hitting now. Is 99% shown on the dashboard? Try running top -aSH at the command line to see how that load is spread across the cores.

    If it is showing 100% on the dash though it's probably using 100% on all CPU cores in which case anything will help.

    Steve



  • That's correct, 99% on the dashboard. I don't think I kept any screenshots of top, but I will grab one the next time it happens.

    Thanks for your input.

    Thanks,

    Dan


  • LAYER 8 Netgate

    What kind of NICs?



  • Chelsio 10Gb SFP+ T520.

    Here's my thoughts. And they do get out there sometimes so correct me if I am wrong.

    To me, it doesn't matter how WIDE (cores) we go, because the interrupts are going to keep building until the IPFW can process the packets that the NIC is receiving. All cores become consumed with the t5nex0.

    I have heard that the IPFW operates best with higher clock speeds, like it may be somewhat single threaded.

    Please confirm.

    Thanks,

    Dan


  • LAYER 8 Netgate

    Why are you talking about ipfw? Are you running Captive Portal?



  • Sorry, I was trying to edit my post, I meant the Firewall packet filer.

    Thanks,

    Dan


  • Netgate Administrator

    It doesn't scale linearly certainly. More cores will help as long as you have queues provided by the driver for them to service. The Chelsio driver provides 16 Tx queues and 8 Rx queues so it's unlikely more Cores than that will help.
    16 faster cores will probably be able to provide more throughput. That's speculating somewhat though.

    Steve



  • Thanks you,

    I assume you meant 16Rx and 8Tx queues?

    It sounds like you are saying that each queue is limited to one core, and if so, could it help to have at least 24 cores? (16+8).

    Thanks,

    Dan


  • Netgate Administrator

    Sorry, yeah, typo'd that. It's 8 Rx cores.

    I'm not sure how that load would spread across 24 cores. There may be some work required to get the appropriate core affinity.

    Steve


Log in to reply