Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    What is the biggest attack in GBPS you stopped

    Scheduled Pinned Locked Moved General pfSense Questions
    737 Posts 33 Posters 600.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      Supermule Banned
      last edited by

      Mine is 80.197.148.74 right now :)

      I am pinging 92.28.255.154 and requests is timing out

      1 Reply Last reply Reply Quote 0
      • F
        firewalluser
        last edited by

        92.28.252.160

        Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

        Asch Conformity, mainly the blind leading the blind.

        1 Reply Last reply Reply Quote 0
        • S
          Supermule Banned
          last edited by

          Reply :)

          Should I hit that one?

          1 Reply Last reply Reply Quote 0
          • F
            firewalluser
            last edited by

            Hold off for a moment as something screwing is going on with the fw rules (PF) at the moment, but I got a brief DDOS if that was you but no ICMP showed up in the fw logs from your ip address.

            I need to check out whats going on with the fw rules as something isnt right this end so I'm going to reboot as resetting all states didnt reset the rules surprisingly.

            edit.
            The logs run about 2mins behind on this fw so any change I make and reset of states takes a few minutes longer before I can confirm its ok in the log.

            Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

            Asch Conformity, mainly the blind leading the blind.

            1 Reply Last reply Reply Quote 0
            • S
              Supermule Banned
              last edited by

              Ready?

              1 Reply Last reply Reply Quote 0
              • F
                firewalluser
                last edited by

                I am now but had to reboot as the fw went slow when I reassigned the nics, I could still navigate around but it was sooo slow so I did a reboot and unbound takes about 10mins and increasing all the time, after a reboot before I can get back out and all the while the cpu is maxed out.

                Anyway new ip is 92.24.143.49

                DDOS away….

                Edit. Ping rule is still in effect on the wan interface so you should be able to ping away on the ip address as well.

                Edit2. This is the EMx Intel nic we are testing now as well, still dusting off the old single core amd laptop atm.

                Edit2. CPU is currently around 38% ticking over with just me accessing this site at the moment and I've got the console visible so I can see if there any Interrupt Storms showing up with this nic.

                Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                Asch Conformity, mainly the blind leading the blind.

                1 Reply Last reply Reply Quote 0
                • F
                  firewalluser
                  last edited by

                  Well I sat through that one no problem as it finished just a moment ago.

                  I didnt touch the RRD graph though, snort was slower but generally all ok and responsive. What I noticed with the EMx Intel nic is I did edge up a bit on bandwidth. Where as the usb nics saw a consistent 2.42Mbps this morning, the Intel nic varied between 2.42-2.45Mbps not alot more but as its got a faster connection to the MB and CPU unlike my usb nics sharing bandwith over usb hubs, again thats to be expected.

                  Other thing I noticed and wonder about is in the system activity for the memory was the Wired & Buf MB's increased over time (100x above avg) and this was with high latency throughout for the state time out (system:advanced:Firewall & NAT, Firewall Optimisation Option). This is with no tuning ie its pretty much out of the box pfsense so the states might be an issue if someone has a faster net connection than I do and dont tweak this.

                  Unbound continued to send more requests out over time (to be expected) but as the inbound was swamped so the unbound states timed out as genuine comms couldnt get through again as its a pot luck what does get though.

                  When I was fiddling with snort I did notice zombie processes pop up, most of the time it was 1 or 2 zombie processes but I did see once 8 zombie processes.

                  I will be increasing the browser timeouts so they stay trying to connect before the pf states time out as this might also make it more likely I can stay connected to the web throughout such an event.

                  Its certainly an interesting test.

                  I think next time I will have the RRD graphs running as this took the fw down on the first test this morning but I dont know why yet.

                  Thanks SM its been educational!  :D

                  Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                  Asch Conformity, mainly the blind leading the blind.

                  1 Reply Last reply Reply Quote 0
                  • S
                    Supermule Banned
                    last edited by

                    Youre welcome mate :)

                    1 Reply Last reply Reply Quote 0
                    • T
                      tim.mcmanus
                      last edited by

                      RRD graphs use a process called php-fpm.  You'll see it in top.

                      Also, when you run top, use the following arguments:  top -HSP

                      It'll look like the attachment, which gives you some good resolution into what's running on the system.

                      ![Screen Shot 2015-06-01 at 10.49.39 PM.png](/public/imported_attachments/1/Screen Shot 2015-06-01 at 10.49.39 PM.png)
                      ![Screen Shot 2015-06-01 at 10.49.39 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2015-06-01 at 10.49.39 PM.png_thumb)

                      1 Reply Last reply Reply Quote 0
                      • S
                        Supermule Banned
                        last edited by

                        How do you get a list that long Tim??

                        Mine has maybe 10 processes listed on the page

                        I am beginning to see this in top

                        0 root     -92    0     0K   176K CPU4    4   1:17  98.97% kernel{em0 taskq}
                        
                        
                        1 Reply Last reply Reply Quote 0
                        • F
                          firewalluser
                          last edited by

                          It looks like the same output seen in the Diagnostics: System Activity webpage and on mine even throughout the ddos I had no more than 20 processes running unlike your screen shot which has a few more running.

                          I wonder if I load up my fw with more services to increase the processes running might increase the system exhaustion.

                          I'm still undecided whether this is an SMP issue or an exhaustion of some system resources, the reason I say the latter is I'm reminded of MS Small Business Server which I'd maintained since 2000, as windows has grown so has the hw requirements to the point in 2008R2 and later you needed MS SQL server running on its own box as the software components have grown in size and functionality. My favourite was SBS2003 as I had those running better than MS could, ie they could stay up for months if it wasnt the need to reboot after some sort of windows update that needed a reboot. I did have to reset the odd counter but they ran sweet, however when underload you could get those machines to bog down eventually grinding to a halt and thus a reboot, so I wonder if something similar is happening.

                          I'm gonna try and find something other than syslog to report back the state of the machine, perhaps security onion which I need to check out still, but I'm keen to find something that can control parts of the fw depending on some situations occurring, ie if I get a ddos, I can resave the WAN interface and get assigned a new ip address as one measure I'd like to implement. I could knock something up but its also a case of getting the data out of pfsense over and above syslog data that would make it even more useful.

                          Still lots to learn.

                          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                          Asch Conformity, mainly the blind leading the blind.

                          1 Reply Last reply Reply Quote 0
                          • T
                            tim.mcmanus
                            last edited by

                            I can ssh into the box from another source, and I have a 27" screen, so that gives me better screen output on the CLI.

                            1 Reply Last reply Reply Quote 0
                            • S
                              Supermule Banned
                              last edited by

                              Didnt think of that. :D

                              ![top -HSP.PNG](/public/imported_attachments/1/top -HSP.PNG)
                              ![top -HSP.PNG_thumb](/public/imported_attachments/1/top -HSP.PNG_thumb)

                              1 Reply Last reply Reply Quote 0
                              • F
                                firewalluser
                                last edited by

                                Whilst I remember the crash reports.

                                Crash report begins.  Anonymous machine information:

                                amd64
                                10.1-RELEASE-p9
                                FreeBSD 10.1-RELEASE-p9 #0 57b23e7(releng/10.1)-dirty: Mon Apr 13 20:30:25 CDT 2015    root@pfs22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10

                                Crash report details:

                                PHP Errors:
                                [02-Jun-2015 12:13:04 Europe/London] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 265551872 bytes) in /usr/local/www/diag_packet_capture.php on line 456

                                Filename: /var/crash/minfree
                                2048

                                and

                                Crash report begins.  Anonymous machine information:

                                amd64
                                10.1-RELEASE-p9
                                FreeBSD 10.1-RELEASE-p9 #0 57b23e7(releng/10.1)-dirty: Mon Apr 13 20:30:25 CDT 2015    root@pfs22-amd64-builder:/usr/obj.amd64/usr/pfSensesrc/src/sys/pfSense_SMP.10

                                Crash report details:

                                PHP Errors:
                                [02-Jun-2015 17:32:09 UTC] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 265551872 bytes) in /usr/local/www/diag_packet_capture.php on line 456

                                Looks like pfsense doesnt like packet capturing a ddos  ;D, havent checked the line out yet but I couldnt find anything in /var/crash/, although having the packet capture shut down nicely would be ideal.

                                It might be useful if the crash reports werent deleted after a reboot and I dont know if this got sent through to ESF or not and only one file mentioned in the first report, no additional file in the 2nd crash report.

                                I also dont think the Wired & Buf MB's increases were anything to do with the packet capture as it looks the packet cpature crashed well before the end of the test.

                                Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                                Asch Conformity, mainly the blind leading the blind.

                                1 Reply Last reply Reply Quote 0
                                • S
                                  Supermule Banned
                                  last edited by

                                  After fiddling with some settings under system ->tunables i got it to distribute the load better so to speak.

                                  No 100% cpu core and then it works fine.

                                  ![top -HSP_1.PNG](/public/imported_attachments/1/top -HSP_1.PNG)
                                  ![top -HSP_1.PNG_thumb](/public/imported_attachments/1/top -HSP_1.PNG_thumb)

                                  1 Reply Last reply Reply Quote 0
                                  • M
                                    mer
                                    last edited by

                                    @firewalluser:

                                    I'm gonna try and find something other than syslog to report back the state of the machine, perhaps security onion which I need to check out still, but I'm keen to find something that can control parts of the fw depending on some situations occurring, ie if I get a ddos, I can resave the WAN interface and get assigned a new ip address as one measure I'd like to implement. I could knock something up but its also a case of getting the data out of pfsense over and above syslog data that would make it even more useful.

                                    Still lots to learn.

                                    For quick simple stats, man vmstat  man procstat

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      tim.mcmanus
                                      last edited by

                                      @firewalluser:

                                      PHP Errors:
                                      [02-Jun-2015 12:13:04 Europe/London] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 265551872 bytes) in /usr/local/www/diag_packet_capture.php on line 456

                                      PHP Errors:
                                      [02-Jun-2015 17:32:09 UTC] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 265551872 bytes) in /usr/local/www/diag_packet_capture.php on line 456

                                      Those are php crashes.  Don't run the webUI and see if it works.  Odd that ph would crash the machine.  I would have expected it to die as a process and then respawn.

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        tim.mcmanus
                                        last edited by

                                        @Supermule:

                                        After fiddling with some settings under system ->tunables i got it to distribute the load better so to speak.

                                        No 100% cpu core and then it works fine.

                                        snort and the two kernel processes {em0, task}, {em1, task} are still nearly the same between the two screen shots.  Interesting that one of those kernel processes isn't sitting on a CPU in the second screen shot.

                                        1 Reply Last reply Reply Quote 0
                                        • F
                                          firewalluser
                                          last edited by

                                          @tim.mcmanus:

                                          @firewalluser:

                                          PHP Errors:
                                          [02-Jun-2015 12:13:04 Europe/London] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 265551872 bytes) in /usr/local/www/diag_packet_capture.php on line 456

                                          PHP Errors:
                                          [02-Jun-2015 17:32:09 UTC] PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 265551872 bytes) in /usr/local/www/diag_packet_capture.php on line 456

                                          Those are php crashes.  Don't run the webUI and see if it works.  Odd that ph would crash the machine.  I would have expected it to die as a process and then respawn.

                                          I've got a permanent packet capture going in so I have records for posterity as the police wont do anything in some circumstances if you get hacked over here, so the more data the better as it will be possible to track down those behind hack attempts with more data, whilst also making it easier to incriminate myself as well I'm sure.

                                          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                                          Asch Conformity, mainly the blind leading the blind.

                                          1 Reply Last reply Reply Quote 0
                                          • F
                                            firewalluser
                                            last edited by

                                            @Supermule:

                                            After fiddling with some settings under system ->tunables i got it to distribute the load better so to speak.

                                            No 100% cpu core and then it works fine.

                                            So what settings did you add or change?

                                            @mer:

                                            @firewalluser:

                                            I'm gonna try and find something other than syslog to report back the state of the machine, perhaps security onion which I need to check out still, but I'm keen to find something that can control parts of the fw depending on some situations occurring, ie if I get a ddos, I can resave the WAN interface and get assigned a new ip address as one measure I'd like to implement. I could knock something up but its also a case of getting the data out of pfsense over and above syslog data that would make it even more useful.

                                            Still lots to learn.

                                            For quick simple stats, man vmstat  man procstat

                                            Thanks, however whats the best way to get all/as much data as possible including rule matches out of pfsense, I've got to check out security onion but if there are other ways I'm all ears. I saw nagios earlier which might be interesting to see what and how it gets data out of pfsense and other platforms, but theres things it doesnt do which I'd like.

                                            Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                                            Asch Conformity, mainly the blind leading the blind.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.