Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    What is the biggest attack in GBPS you stopped

    General pfSense Questions
    33
    737
    458.7k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • F
      firewalluser
      last edited by

      I think the inevitable use of Dtrace is upon us.  :)

      I dont know the status of FreeBSD 11, but if its at least in a Release Candidate status, is it worth trying to port pfsense onto a FreeBSD 11 build?

      Although even if we could, if the problems still showed up, we'd still need Dtrace….

      Is there really no debugging facility built into pfsense?

      Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

      Asch Conformity, mainly the blind leading the blind.

      1 Reply Last reply Reply Quote 0
      • S
        Supermule Banned
        last edited by

        I get this in the system logs as of today…

        php-fpm[60486]: /interfaces.php: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em1' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.6 Copyright 2004-2014 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 13 leases to leases file. Can't install new lease database /var/db/dhcpd.leases.1433496021 to /var/db/dhcpd.leases: No such file or directory Listening on BPF/em1/00:50:56:a5:77:03/192.168.10.0/24 Sending on BPF/em1/00:50:56:a5:77:03/192.168.10.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that befo

        And this

        php-fpm[85395]: /rc.newwanip: The command '/usr/local/sbin/dhcpd -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpd.conf -pf /var/run/dhcpd.pid em1' returned exit code '1', the output was 'Internet Systems Consortium DHCP Server 4.2.6 Copyright 2004-2014 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Wrote 13 leases to leases file. Listening on BPF/em1/00:50:56:a5:77:03/192.168.10.0/24 Sending on BPF/em1/00:50:56:a5:77:03/192.168.10.0/24 Can't bind to dhcp address: Address already in use Please make sure there is no other dhcp server running and that there's no entry for dhcp or bootp in /etc/inetd.conf. Also make sure you are not running HP JetAdmin software, which includes a bootp server. If you did not get this software from ftp.isc.org, please get the latest from ftp.isc.org and install that before requesting help. If you did get this software from ftp.isc.org and have not yet read the README, please read it bef

        Comes right after this

        | Jun 5 11:20:28 | check_reload_status: Reloading filter |
        | Jun 5 11:20:28 | check_reload_status: Restarting OpenVPN tunnels/interfaces |
        | Jun 5 11:20:28 | check_reload_status: Restarting ipsec tunnels |
        | Jun 5 11:20:28 | check_reload_status: updating dyndns Yousee |
        | Jun 5 11:20:28 | check_reload_status: Reloading filter |
        | Jun 5 11:20:28 | check_reload_status: Restarting OpenVPN tunnels/interfaces |
        | Jun 5 11:20:28 | check_reload_status: Restarting ipsec tunnels |
        | Jun 5 11:20:28 | check_reload_status: updating dyndns Yousee |
        | Jun 5 11:20:26 | check_reload_status: Syncing firewall |
        | Jun 5 11:20:26 | check_reload_status: Syncing firewall |
        | Jun 5 11:20:25 | check_reload_status: Reloading filter |
        | Jun 5 11:20:25 | php-fpm[9696]: /rc.start_packages: Reloading Squid for configuration sync |
        | Jun 5 11:20:25 | php-fpm[9696]: /rc.start_packages: Restarting/Starting all packages. |
        | Jun 5 11:20:23 | lighttpd[33558]: (connections.c.1692) SSL (error): 5 -1 1 Operation not permitted |
        | Jun 5 11:20:23 | lighttpd[33558]: (connections.c.619) connection closed: write failed on fd 20 |
        | Jun 5 11:20:23 | lighttpd[33558]: (network_openssl.c.118) SSL: 5 -1 1 Operation not permitted |
        | Jun 5 11:20:23 | check_reload_status: Reloading filter |
        | Jun 5 11:20:23 | check_reload_status: Starting packages |
        | Jun 5 11:20:23 | php-fpm[85395]: /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 80.197.148.74 ->  80.197.148.74 - Restarting packages. |
        | Jun 5 11:20:23 | php-fpm[60486]: /interfaces.php: Removing static route for monitor 81.19.224.67 and adding a new route through 80.197.148.1 |
        | Jun 5 11:20:21 | php-fpm[85395]: /rc.newwanip: Resyncing OpenVPN instances for interface WAN. |
        | Jun 5 11:20:21 | check_reload_status: updating dyndns wan |

        1 Reply Last reply Reply Quote 0
        • F
          firewalluser
          last edited by

          Did you set an IP address in ESXi for that nic?

          Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

          Asch Conformity, mainly the blind leading the blind.

          1 Reply Last reply Reply Quote 0
          • S
            Supermule Banned
            last edited by

            No it gets that from DHCP on WAN from my ISP. Hypervisor mgmt network is on the LAN.

            DHCP is running on LAN issuing IP's to a Linksys wireless AP in bridge mode.

            I do not run HP Jetadmin software on LAN.

            log1.PNG
            log1.PNG_thumb
            log2.PNG
            log2.PNG_thumb

            1 Reply Last reply Reply Quote 0
            • S
              Supermule Banned
              last edited by

              Disabled Apinger…

              All I get in the logs is this then...

              traffic_drop.PNG
              traffic_drop.PNG_thumb
              log3.PNG
              log3.PNG_thumb

              1 Reply Last reply Reply Quote 0
              • F
                firewalluser
                last edited by

                This might be an approach for getting Dtrace working on pfsense.

                https://forum.pfsense.org/index.php?topic=94838.msg527131#msg527131

                Different package, but might work, bit like the only way to get Rpi's working virtually is on Ubuntu with QEMU installed to emulate the ARM's, at which point it becomes alot quicker by virtual of stonking fast hw setting up freebsd and pfsense on RPi's, but I digress.

                Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                Asch Conformity, mainly the blind leading the blind.

                1 Reply Last reply Reply Quote 0
                • S
                  Supermule Banned
                  last edited by

                  Thanks man!

                  Over my head…. I need others to chime in on this. Spent the last mth using all my spare time to test this frigging thing.

                  After I did the cron job, then I can make it come alive again.

                  I have to settle with that right now...........

                  Currently migrating all the pf's in the datacenter to Mikrotik since I cant have this hosting customers. At home I dont care.

                  1 Reply Last reply Reply Quote 0
                  • F
                    firewalluser
                    last edited by

                    So is it worth starting a Getting Dtrace to work on pfsense thread?

                    We only have to ask the right questions, document everything we do so that others can learn from it as well if they have the thirst for knowledge.

                    There's plenty of examples online which show how to compile code onto platforms where a package doesnt exist for downloading.

                    There's generally parallels everywhere if you look.  ;)

                    Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                    Asch Conformity, mainly the blind leading the blind.

                    1 Reply Last reply Reply Quote 0
                    • S
                      Supermule Banned
                      last edited by

                      Posted a bounty for Dtrace package.

                      https://forum.pfsense.org/index.php?topic=94846.0

                      1 Reply Last reply Reply Quote 0
                      • T
                        tim.mcmanus
                        last edited by

                        dtrace is installed already on pfSense.

                        Go to the FreeBSD forums now.  You're not dealing with pfSense anymore when you start working with dtrace.  This is the wrong place to look for that kind of guidance.

                        I think I'm having déjà vu….

                        1 Reply Last reply Reply Quote 0
                        • S
                          Supermule Banned
                          last edited by

                          OK.

                          But it still leaves me with no options to monitor pfsense deep under the hood with timestamps.

                          https://forum.pfsense.org/index.php?topic=94260.0

                          Any ideas?

                          1 Reply Last reply Reply Quote 0
                          • F
                            firewalluser
                            last edited by

                            You need to be more specific with what you want exactly, plus programmers are very specific and you might find Dtrace does what you want, so would it be fair for anyone to just compile and get Dtrace running?

                            Besides when you say what you say, have you considered the other app/conversation thread here https://forum.pfsense.org/index.php?topic=94843.msg527152#msg527152 or is your browser running single tab/single core/single thread and you have not locked/mutexed this thread and swapped to the other thread yet? Or if your browser is running multitabbed/multi core/multi threaded, have you locked this thread in your browser by switching tabs to read the other thread/multi core/multi thread if you catch my drift.  ;D

                            There's parallels everywhere if you have an abstract look on life.

                            Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                            Asch Conformity, mainly the blind leading the blind.

                            1 Reply Last reply Reply Quote 0
                            • F
                              firewalluser
                              last edited by

                              @tim.mcmanus:

                              dtrace is installed already on pfSense.

                              Go to the FreeBSD forums now.  You're not dealing with pfSense anymore when you start working with dtrace.  This is the wrong place to look for that kind of guidance.
                              I think I'm having déjà vu….

                              Significance might not have registered. Apologies.  :o

                              Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                              Asch Conformity, mainly the blind leading the blind.

                              1 Reply Last reply Reply Quote 0
                              • T
                                tim.mcmanus
                                last edited by

                                dtrace stuff.  Remember, dtrace is a component of the FreeBSD distribution, so you need to go there to understand it and get support from the community.  These forums are more product-specific for pfSense–mostly dealing with networking versus deep coding.  So you're questions are best answered in the FreeBSD forums.

                                https://wiki.freebsd.org/DTrace

                                https://wiki.freebsd.org/DTrace/Tutorial

                                http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/dtrace.html

                                http://www.brendangregg.com/USEmethod/use-freebsd.html

                                1 Reply Last reply Reply Quote 0
                                • T
                                  tim.mcmanus
                                  last edited by

                                  Smoking gun?

                                  https://forums.freebsd.org/threads/freebsd-10-1-network-unaccessible-after-high-traffic.51743/

                                  Potential fix? (synproxy state)

                                  https://forums.freebsd.org/threads/enabling-ipv6-eventually-freezes-the-gateway.47959/#post-267820

                                  See #4 from this link:

                                  http://www.onlamp.com/pub/a/bsd/2004/04/15/pf_developers.html

                                  https://forums.freebsd.org/threads/freebsd-9-0-release-syn-flood.29071/#post-163533

                                  Notice how all of this freakin' awesome information is coming from the FreeBSD forums.  I strongly suggest you consolidate your findings in this thread and move the discussion over to the FreeBSD networking forums.  They have dozens of discussions on SYN floods with some excellent guidance given.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    Supermule Banned
                                    last edited by

                                    Yes. It just doesnt work as expected.

                                    Until the drop in traffic occurs. So I need to find a way to see whats initiated when the traffic drops.

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      tim.mcmanus
                                      last edited by

                                      @Supermule:

                                      Yes. It just doesnt work as expected.

                                      Until the drop in traffic occurs. So I need to find a way to see whats initiated when the traffic drops.

                                      https://forums.freebsd.org/forums/networking.7/

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        Supermule Banned
                                        last edited by

                                        Yes….one more forum to watch....................... :D

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          Supermule Banned
                                          last edited by

                                          Just a small update..

                                          Changed NIC's to vmxnet3 in vmware and this happened.

                                          Youtube Video

                                          Same traffic drop but curves are different. On E1000 they follow each other. On vmxnet3 they are totally seperated. Red is 15mbit/s and black is almost zero.

                                          Suddenly then traffic drops on incoming WAN and it begins to flow traffic and respond to ping for Google DNS.

                                          No change what so ever to anything.

                                          vmxnet3 drivers are much worse handling this kind of attack than the E1000 drivers.

                                          EDIT: When Core nr 3 drops, the FW comes alive and begins to route packets. Overall load is cut in half.

                                          EDIT EDIT:  Getting 100% interrupt in the top -HSP on the console. Again the E1000 almost had zero.

                                          Logs are flooded with this.

                                          | Jun 7 19:54:14 | check_reload_status: Reloading filter |
                                          | Jun 7 19:54:14 | check_reload_status: Restarting OpenVPN tunnels/interfaces |
                                          | Jun 7 19:54:14 | check_reload_status: Restarting ipsec tunnels |
                                          | Jun 7 19:54:14 | check_reload_status: updating dyndns Yousee |
                                          | Jun 7 19:54:01 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:49 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:43 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:38 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:33 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:28 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:22 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:15 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:06 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:53:01 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:56 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:51 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:45 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:40 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:35 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:28 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:23 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:18 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:13 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:07 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:52:02 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:56 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:50 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:46 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:40 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:34 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:29 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:24 | check_reload_status: Reloading filter |
                                          | Jun 7 19:51:24 | check_reload_status: Restarting OpenVPN tunnels/interfaces |
                                          | Jun 7 19:51:24 | check_reload_status: Restarting ipsec tunnels |
                                          | Jun 7 19:51:24 | check_reload_status: updating dyndns Yousee |
                                          | Jun 7 19:51:22 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:16 | kernel: vmx0: watchdog timeout on queue 0 |
                                          | Jun 7 19:51:11 | kernel: vmx0: watchdog timeout on queue 0 |

                                          traffic_drop3.PNG
                                          traffic_drop3.PNG_thumb
                                          vmxnet3_vmwareload.png
                                          vmxnet3_vmwareload.png_thumb
                                          interrupts_vmxnet.PNG
                                          interrupts_vmxnet.PNG_thumb

                                          1 Reply Last reply Reply Quote 0
                                          • F
                                            firewalluser
                                            last edited by

                                            And an update from me.

                                            Currently downloading and getting 5.42Mbps according to the dashboard, yet when you did the ddos the otherday that was only giving showing 2.42Mbps.

                                            So why does a ddos cause my bandwidth to drop down? Could it be related to this post https://forum.pfsense.org/index.php?topic=94903.0 ie a hw issue, or is freebsd or pfsense.

                                            I have a huwei supplied router dont know if it can go into modem only mode, but if it can I will check the speeds again with this device if you dont mind doing a Britney Spears and hit me one more time?  ;)

                                            Capitalism, currently The World's best Entertainment Control System and YOU cant buy it! But you can buy this, or some of this or some of these

                                            Asch Conformity, mainly the blind leading the blind.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post