Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    What is the biggest attack in GBPS you stopped

    Scheduled Pinned Locked Moved General pfSense Questions
    737 Posts 33 Posters 603.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Supermule Banned
      last edited by

      But CPU load is not the issue??

      Unless it consumes the cache and cannot process the IO cue.

      But it should handle 3Gbyte/S no problem. So I dont understand why it should be CPU related at all.

      1 Reply Last reply Reply Quote 0
      • S
        Supermule Banned
        last edited by

        I am more into the way PF handles the packets and what it does with it internally.

        pf reads copies of packets and inspects them. Maybe its in that regard something (buffer) could be to small.

        So when we reach x number of PPS, then it falls apart and cant keep up.

        1 Reply Last reply Reply Quote 0
        • H
          Harvy66
          last edited by

          In the case of that one attack kernel-que was using nearly 100% of my CPU. If it was less thread friendly, it probably would have been closer to 25% cpu. The old PF could not consume 100% of a multi-core CPU, the new PF potentially could.

          1 Reply Last reply Reply Quote 0
          • S
            Supermule Banned
            last edited by

            How do you inspect the kernel queue in pf??

            I could then compare the 2.1.5 vs 2.2.2 and see if there are any difference?

            @Harvy66:

            In the case of that one attack kernel-que was using nearly 100% of my CPU. If it was less thread friendly, it probably would have been closer to 25% cpu. The old PF could not consume 100% of a multi-core CPU, the new PF potentially could.

            1 Reply Last reply Reply Quote 0
            • M
              mer
              last edited by

              As FreeBSD grew from 4.x to 5 and up to 10 I believe they've been pushing finer and finer grained locking into the kernel (like Solaris).  This means that there can be more kernel preemption happening.  I'd have to dig into code to verify but I'm guessing the "kernel igb0 que"  thread is the one sitting between the bottom half of the interrupt handler and the next layer up (basically holding packets).  If PF is looking at packets, modifying (or not) and putting them back, there may be a lot of queue manipulation going on (grab lock, deque/enque packet, release lock) so that may cause the que threads to start sucking CPU.

              This is speculation, generalities;  I have not really looked at this portion of the FBSD kernel so I could be totally off base.

              1 Reply Last reply Reply Quote 0
              • S
                Supermule Banned
                last edited by

                Is there any way to inspect time used by process or cpu cycles used in pfSense???

                1 Reply Last reply Reply Quote 0
                • H
                  Harvy66
                  last edited by

                  System Activity or "ps" will tell you total CPU time consumed. Just remember, a quad core can consume 4 CPU seconds per second.

                  1 Reply Last reply Reply Quote 0
                  • F
                    firewalluser
                    last edited by

                    @kroberts:

                    @firewalluser:

                    I'm sure you are familiar with the wide ranging contracts that exist in the world today beit Non Disclosure Agreements, or even the more common non compete agreements as exampled here: http://pando.com/2014/03/22/revealed-apple-and-googles-wage-fixing-cartel-involved-dozens-more-companies-over-one-million-employees/
                    http://www.businessinsider.com/emails-eric-schmidt-sergey-brin-hiring-apple-2014-3?IR=T
                    http://venturebeat.com/2014/05/23/4-tech-companies-are-paying-a-325m-fine-for-their-illegal-non-compete-pact/

                    Put simply, you are not in a position to prove your innocence, because

                    1. To adhere to the terms of any NDA contract you may have been forced/coerced to sign would mean any disclosure would render you in breach of your contract and liable to whatever penalties may have been included in any agreement and who in their "right" mind would put themselves at a disadvantage?

                    2. Even if you have not signed any NDA contract you still cant prove your innocence, ergo the spooks & govt(s) still win, its classic divide and conquer techniques, which then begs the question why trust military & govt(s) or banks who carry out activities in secret?

                    What I can say is trust can take ages to build up, but can be destroyed in seconds.

                    On the point of passing enough traffic through pfsense, this has happened with less than 1mbits of traffic, a simple web page loading can trigger the OS cores to hang. Volume is irrelevant in the example I mention, but in relation to this thread and amounts of data, I wondered two things, exploiting the CPU designs namely cache and/or something network related as also mentioned by Kom here
                    https://forum.pfsense.org/index.php?topic=91856.msg517296#msg517296

                    I'm inclined at this stage to err towards something nic related but I will examine the zip posted by supermule to see if I can see anything untoward, but this could be a variation on the heap spraying exploit http://en.wikipedia.org/wiki/Heap_spraying

                    I wonder if those affected are running snort and if so do the problems still exist, assuming snort is already aware of the problem much like AV software need to have found a virus before it can protect against it?

                    All of the above is said with the best of intentions and for it to be educational to those who might not be aware of the deceit and duplicity in the world today.

                    Edit.

                    Has anyone tried an earlier version of pfsense like a 1.x version by any chance? ;)

                    And likewise by your logic you can't prove you haven't signed a contract with some organization bent on sowing distrust in pfSense in general, or cmb specifically.  Your post here looks exactly like what I would expect such an attack to look like.  Don't bother denying it, you're obviously under and NDA or non-compete.

                    Conspiracy theories are great entertainment, but don't get carried away by it.

                    I can but it will only come out in court if it ever gets to court. The Rabbit Warren runs deeper than you may want to believe!

                    Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                    Asch Conformity, mainly the blind leading the blind.

                    1 Reply Last reply Reply Quote 0
                    • F
                      firewalluser
                      last edited by

                      @Harvy66:

                      System Activity or "ps" will tell you total CPU time consumed. Just remember, a quad core can consume 4 CPU seconds per second.

                      Not always, you need to understand how the L2 cache works, ie its shared between cores on Intel, but AMD tend to have a cache amount per core, ie AMD would be less prone to cache collisions unlike Intel cpu's.

                      Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                      Asch Conformity, mainly the blind leading the blind.

                      1 Reply Last reply Reply Quote 0
                      • F
                        firewalluser
                        last edited by

                        @mer:

                        As FreeBSD grew from 4.x to 5 and up to 10 I believe they've been pushing finer and finer grained locking into the kernel (like Solaris).  This means that there can be more kernel preemption happening.  I'd have to dig into code to verify but I'm guessing the "kernel igb0 que"  thread is the one sitting between the bottom half of the interrupt handler and the next layer up (basically holding packets).  If PF is looking at packets, modifying (or not) and putting them back, there may be a lot of queue manipulation going on (grab lock, deque/enque packet, release lock) so that may cause the que threads to start sucking CPU.

                        This is speculation, generalities;  I have not really looked at this portion of the FBSD kernel so I could be totally off base.

                        I suspect its like KOM suggested with the driver nic integrating into the OS, thats why I have suggested to others where possible to go back to a 1.x pf version to test. The earlier OS's wont be so bogged down with "new" features", but your idea of locking is on the ball in terms of how OS's handle multi threading as against preemptive threading.

                        Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                        Asch Conformity, mainly the blind leading the blind.

                        1 Reply Last reply Reply Quote 0
                        • S
                          Supermule Banned
                          last edited by

                          1st one is idle and 2nd is DoS.

                          Load is approx. 20mbit and maybe 50k PPS

                          SA_idle.PNG
                          SA_idle.PNG_thumb
                          SA_DoS.PNG
                          SA_DoS.PNG_thumb

                          1 Reply Last reply Reply Quote 0
                          • M
                            mer
                            last edited by

                            Out of curiosity, has any posted about this over in FreeBSD mailing lists?

                            1 Reply Last reply Reply Quote 0
                            • H
                              Harvy66
                              last edited by

                              @firewalluser:

                              @Harvy66:

                              System Activity or "ps" will tell you total CPU time consumed. Just remember, a quad core can consume 4 CPU seconds per second.

                              Not always, you need to understand how the L2 cache works, ie its shared between cores on Intel, but AMD tend to have a cache amount per core, ie AMD would be less prone to cache collisions unlike Intel cpu's.

                              Cache misses counts as CPU time. If it takes an extra 250 cycles because of a cache miss, well, that's counting against you. CPU time is the amount of time a process has been scheduled. What it does during that time is irrelevant from the schedulers's standpoint.

                              1 Reply Last reply Reply Quote 0
                              • dennypageD
                                dennypage
                                last edited by

                                250?

                                @Harvy66:

                                [Cache misses counts as CPU time. If it takes an extra 250 cycles because of a cache miss, well, that's counting against you.
                                [/quote]

                                1 Reply Last reply Reply Quote 0
                                • H
                                  Harvy66
                                  last edited by

                                  I'm not sure I understand the question. 250 cycles is a number I just pulled from thin air for going to main memory. I was talking about reading CPU time from "ps" or "top" and I said on a quad core, every second of real time is 4 seconds of CPU time. firewalluser then mentioned "not always" and said some stuff about L2 cache. I was responding by saying that cache does not affect CPU time.

                                  1 Reply Last reply Reply Quote 0
                                  • dennypageD
                                    dennypage
                                    last edited by

                                    Sorry, I didn't understand the thin air part. :)

                                    @Harvy66:

                                    I'm not sure I understand the question. 250 cycles is a number I just pulled from thin air for going to main memory.

                                    1 Reply Last reply Reply Quote 0
                                    • K
                                      kroberts
                                      last edited by

                                      @firewalluser:

                                      @kroberts:

                                      @firewalluser:

                                      I'm sure you are familiar with the wide ranging contracts that exist in the world today beit Non Disclosure Agreements, or even the more common non compete agreements as exampled here: http://pando.com/2014/03/22/revealed-apple-and-googles-wage-fixing-cartel-involved-dozens-more-companies-over-one-million-employees/
                                      http://www.businessinsider.com/emails-eric-schmidt-sergey-brin-hiring-apple-2014-3?IR=T
                                      http://venturebeat.com/2014/05/23/4-tech-companies-are-paying-a-325m-fine-for-their-illegal-non-compete-pact/

                                      Put simply, you are not in a position to prove your innocence, because

                                      1. To adhere to the terms of any NDA contract you may have been forced/coerced to sign would mean any disclosure would render you in breach of your contract and liable to whatever penalties may have been included in any agreement and who in their "right" mind would put themselves at a disadvantage?

                                      2. Even if you have not signed any NDA contract you still cant prove your innocence, ergo the spooks & govt(s) still win, its classic divide and conquer techniques, which then begs the question why trust military & govt(s) or banks who carry out activities in secret?

                                      What I can say is trust can take ages to build up, but can be destroyed in seconds.

                                      On the point of passing enough traffic through pfsense, this has happened with less than 1mbits of traffic, a simple web page loading can trigger the OS cores to hang. Volume is irrelevant in the example I mention, but in relation to this thread and amounts of data, I wondered two things, exploiting the CPU designs namely cache and/or something network related as also mentioned by Kom here
                                      https://forum.pfsense.org/index.php?topic=91856.msg517296#msg517296

                                      I'm inclined at this stage to err towards something nic related but I will examine the zip posted by supermule to see if I can see anything untoward, but this could be a variation on the heap spraying exploit http://en.wikipedia.org/wiki/Heap_spraying

                                      I wonder if those affected are running snort and if so do the problems still exist, assuming snort is already aware of the problem much like AV software need to have found a virus before it can protect against it?

                                      All of the above is said with the best of intentions and for it to be educational to those who might not be aware of the deceit and duplicity in the world today.

                                      Edit.

                                      Has anyone tried an earlier version of pfsense like a 1.x version by any chance? ;)

                                      And likewise by your logic you can't prove you haven't signed a contract with some organization bent on sowing distrust in pfSense in general, or cmb specifically.  Your post here looks exactly like what I would expect such an attack to look like.  Don't bother denying it, you're obviously under and NDA or non-compete.

                                      Conspiracy theories are great entertainment, but don't get carried away by it.

                                      I can but it will only come out in court if it ever gets to court. The Rabbit Warren runs deeper than you may want to believe!

                                      Come on. Conspiracy theory is self proved and self destructive by its very nature. My sole point was that babbling about it here is in no way helpful to the original topic. Please stop.

                                      1 Reply Last reply Reply Quote 0
                                      • D
                                        doktornotor Banned
                                        last edited by

                                        @kroberts:

                                        Come on. Conspiracy theory is self proved and self destructive by its very nature. My sole point was that babbling about it here is in no way helpful to the original topic. Please stop.

                                        +1. This topic and the way of "disclosing" the issue is already shitty enough – even without this conspiracy junk.

                                        1 Reply Last reply Reply Quote 0
                                        • F
                                          firewalluser
                                          last edited by

                                          @Harvy66:

                                          @firewalluser:

                                          @Harvy66:

                                          System Activity or "ps" will tell you total CPU time consumed. Just remember, a quad core can consume 4 CPU seconds per second.

                                          Not always, you need to understand how the L2 cache works, ie its shared between cores on Intel, but AMD tend to have a cache amount per core, ie AMD would be less prone to cache collisions unlike Intel cpu's.

                                          Cache misses counts as CPU time. If it takes an extra 250 cycles because of a cache miss, well, that's counting against you. CPU time is the amount of time a process has been scheduled. What it does during that time is irrelevant from the schedulers's standpoint.

                                          @Harvy66:

                                          System Activity or "ps" will tell you total CPU time consumed. Just remember, a quad core can consume 4 CPU seconds per second.

                                          Yes

                                          Yes & No

                                          If no cache collisions occur then yes your "4 CPU seconds per second" would be right but when a cache collision occurs then its a matter of debate whether the cpu is giving you any cpu time useful to the task being asked of it by said software because a cache collision by definition is a failure of the cpu/core depending on where the cache collision occurs ie L1,2,3 which means no cpu processing useful to the task being asked of it as it backs out and resolves the cache collision.

                                          To then make it a little more complicated or simpler depending on perspective, if the cache collision occurs on cache shared across all the cores then no you dont get your 4 cpu seconds per second as the CPU backs out and resolves the cache collision which holds up one or more other cores.

                                          If the cache collision occurs on cache available only to a single core like L1 and some L2 (L2 on some chips is shared and on others its a small % of the total L2 but unique to each core), then you could consider it in your 4 cpu seconds per second statement but then there is still the matter of whether the CPU is giving you any "useful" processing time whilst it resolves the collision. Technically the time spent/clock cycles filling the cache having a collision and then resolving the collision is time wasted but it could still show as 100% core or CPU activity depending on the cache affected. So Yes when you see CPU activity at 100%, that would be correct but its not the whole picture as its hiding the cock ups of the CPU cache and the bus waits that are occuring.

                                          Now even if we dont have any cache collisions, on a multi core cpu, time is then further spent wasted as the individual cores spend time waiting to access ram or the disk depending on bus architecture.

                                          I've got software here which I have written which can run mulithreaded and multi cored, but its also capable of running on a single thread on a single core or x threads on a single core or x threads on x cores.

                                          Guess which one runs the fastest?

                                          The single threaded single core version.

                                          Why is this?

                                          Its because there is no time wasted handshaking between threads at the OS level and cores at the HW to access the ram and disk. Disk activity shows this up the most as disk/permanent storage is an order of magnitude slower to access even SSD's when compared to ram.

                                          In some respects even though Arm chips are RISC ie dont have as many common tasks normally carried out by OS software functions which have made it into the cpu architecture unlike say Intels AES-NI to pick a relevant example http://en.wikipedia.org/wiki/AES_instruction_set
                                          of where some common software functions have made it into the cpu architecture, they generally but not always tend to speed up the software but all of this ultimately depends on how the software is written and to a lessor extent the language and compiler used as optimising compilers like cache can work for you and against you as well depending on the chip used to run the software.

                                          This is why I suggested right back at the beginning to try a 1.x version of pfsense. Considering the new features and improvements to functionality made to OS's over time, not only can code be compared easily, it will be possible to workout by elimination and some observations where the problem lies. I suspect knowing how HW drivers used to be for printers especially HP printers in the Win3.1,W95,W98, NT3.5, NT4 days that the drivers have not been updated enough to keep pace with OS developments, hence why I agree with KOM and suspect its a NIC hook issue in the OS, but it will also be compounded by the multi core's seen in cpu's today which is why I also suggested for those running it virtualised like on ESXI, to restrict the core's available to 1.

                                          Apologies if this making you suck eggs, but due to limited data ie not knowing you or your past I dont know how much you know or dont know, hence the explaination above.  :)

                                          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                                          Asch Conformity, mainly the blind leading the blind.

                                          1 Reply Last reply Reply Quote 0
                                          • K
                                            kejianshi
                                            last edited by

                                            Its not that I don't believe hardware is intercepted and modified for some people or that state level agencies don't hack and compromise target systems.  Its just that unless there is some reason I doubt seriously they are doing it to you.

                                            Are you someone worth targeting?

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.